Toad World® Forums

DDL script encoding ?


#1

Hi,

I’m evaluating TDM 3.0.11.11.

I’m really disapppointed by the DDL script generation for MySQL.

There was the PrimaryKey/Auto_increment bug… already fixed.
Then the global before/after script doesn’t work, so I’ve done that in the script to correct it :

if (Instance.Model.BeforeScript != “”)
text += Instance.Model.BeforeScript+"\r\n";

and

if (Instance.Model.AfterScript != “”)
text += “\r\n\r\n”+Instance.Model.AfterScript;

Then I wonder why column comment are on a different line ?

But the most annoying problem is the script encoding !!!
I just cannot run it via the mysql command line…
Here’s the error :
mysql> source base.sql
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right synta
x to use near ’ ■/

Opening the file with an hexa editor show me some funny things :

  • the script starts with 2 non-existing characters
  • every character is coded on 2 bytes ! And I don’t know witch encoding can do that… certainly not UTF8…

(I’m using a French Windows XP Pro)

Thanks
Arnaud


#2

Windows programs typically use unicode. And unicode has two bytes for each character.
Windows can recognize encoding by adding additional bytes at the beginning of files. Specially for UTF-8. There’s byte order of utf-8 written in those bytes. Though common programs I work with do not recognize these bytes and even are not good when present.

You can copy the script content to notepad and save it there as UTF-8 (default is ANSI). That should work. But don’t forget to remove the beginning three chars in VI or other editor if you use the script in Linux/unix.


#3

First of all I want to say that I just want to improve the quality of this product by posting here.

If I’m right the encoding used by the DDL script is UCS-2 little endian.
The UCS-2 is quite like UTF16, but with fixed size (always 2 bytes… with is different of UTF16 wich not always uses 2 bytes)… and is known to be an obsolete charset (well wikipedia said so :wink: )

I’ve tried a lot of things to convert the files (notepad, ntepad++, ultraedit, php scripts, …), but none of them worked ! The latin chars (é, à, …) used in my table/column comments where not corretly converted… and mysql cannot read it properly.

As a result the DDL script is useless…

I will remove all of those (french) chars in TDM… or maybe try to force encoding in MySQL command line program.

But the need of conversion is not really convenient…

BUT I wonder one simple thing :
How can DDL script be encoding in UCS2 (=UTF16) while txp model file are encoded inUTF8 :
<?xml version="1.0" encoding="UTF-8"?>

The use of 2 different charset is a little bit weird. no ?

Thanks for the help…

Message was edited by: arnaud.lorenzi@neuros.fr


#4

Notepad did work for me, what I have written in my previous post, I had checked it first.
What I did: Edit with notepad (F4 in total commander)
Save as -> changed type (the bottom combo box) to UFT-8.
The resulting file was good.
I’m from slovakia, I tried latin2 chars as well.

For the reason of two encodings.
Windows internaly uses unicode. So when application opens file for writing a text, it chooses unicode. It’s a matter of programming language. To be able to save it as UTF-8 you have to convert it. For example in .Net there is one object that opens file, second that formats the source (unicode, ansi, utf-8) and the third object is data (again in memory in unicode). To summ up, the conversion must have been forgotton while programming the DDL script export.


#5

In fact I’ve found why the charset conversion “didn’t work” …

The conversion worked well… but it was the structure of the SQL script which was wrong.

I’ve created another post to sum up some bugs that I’ve found.

Thanx for your help arki… the use of notepad is quite simple.


#6

Arnaud,

thanks for your feedback. We will look into the matter. I created a new record in our system for this matter. CR# 38484.

Arki: thank you for your help. We really appreciate your activity!!!

Regards,

Vaclav