Grid Popup Editor for Clob Values displaying corrupted data

[megan.macfarlane]

Hi,
I'm on Beta version 14.1.73.858 64-bit and have come across a problem that doesn't occur in version 14.0.75.662.

When pasting a large chunk of text in a CLOB field via the Grid Popup Editor, the data appears corrupted from around row 1300 onwards . The Editor displays incorrect data in a single line from that row onwards. The data cannot be extracted correctly from that column using Export Dataset. The data is correct in the field, but can no longer be viewed or extracted via TOAD.

I've replicated the issue using Lorum Ipsom text on a test table using 2600 lines of text.

1. Open a Clob field using the popup editor
2. Paste in the text - in my case anything longer than1550 lines.
3. click the Post Edit tick.
4. Scroll back to the bottom of the text. The last few lines appears as this :

Praesent fringilla est tempor, bibendum libero id, bibendum diam.
Maecenas cursus odio et ligula ultrices tincidunt.
Proin rutrum metus id quam 涭愀氀攀猀甀愀搀愀 琀爀椀猀琀椀焀甀攀⸀ഀ਀ 唀琀 椀渀 搀椀愀洀 猀椀琀 愀洀攀琀 渀椀猀氀 挀漀渀猀攀焀甀愀琀 愀挀挀甀洀猀愀渀 渀攀挀 甀琀 搀漀氀漀爀⸀ഀ਀ 䄀攀渀攀愀渀 瘀攀猀琀椀戀甀氀甀洀 愀渀琀攀 猀攀搀 攀渀椀洀 挀甀爀猀甀猀 瀀漀爀琀愀⸀ഀ਀ 嘀攀猀
...

I've tried this on two different instances on different character sets WE8MSWIN1252 (19.0.0.0.0) and WE8ISO8859P1 (11.2.0.4), though both these are fine when accessed using TOAD 14.0 .

It looks to be a similar sort of problem to Data corruption in result of SELECT statement, beta14.0.51.637

[mathias.schroeder]

Hi,
i'm having the same problem. I discoverered the same chinese characters in the "Grid Popup Editor" of a clob column when opened from a read only Data Grid. There seems to be something fishy with this window/element. And i think i can reproduce the issue.

First of all the data is correct, there are no chinese characters in the clobs and SQL-Developer or an Export just shows everything fine. Here is what i can provide:

Client:

• Win10 64-bit with german env
• DB-Client is 19.10 (also tested with 19.3)
• Toad Version is 14.1.120.923 and issue is also in latest Beta 14.2.55.1008

Database is a 19.10 PDB on AIX with NLS_CHARACTERSET = WE8ISO8859P1

1. create table clob_test (id number, doc clob);
2. describe this table or open the data tab in SB
3. insert a record manually with ID = 1 and double click the clob-column
4. paste exactly 307081 bytes (eg from this ipsum pastebin text: https://pastebin.com/raw/n4AH5Wc2 or any other text with this length) in one line
5. click on "Post edit" and close this popup-editor
6. commit
7. now go to an editor window
8. select * from clob_test;
9. double click on the clob-column in the Data Grid (this is read only)
10. scroll all the way to the right (or keyboard End) and you see strange chinese characters like below

I can even copy the whole text and paste it to notepad++:

My colleague tested it with Toad 14.0 and there is no problem. He then installed 14.1 and got the issue.

I can also say that i cannot reproduce this on other databases on e.g. linux with NLS_CHARACTERSET = AL32UTF8

This is very strange and google translator is not helpful with this "nonsense":

椀渀 洀愀渀✀猀 栀椀猀琀漀爀礀⸀ 圀攀 栀愀瘀攀 昀攀氀琀 琀栀攀 最爀漀甀渀搀 猀栀愀欀攀 愀渀搀 琀栀攀 愀椀爀 猀栀愀琀琀攀爀攀搀 戀礀 琀栀攀

[JohnDorlon]

Sorry @megan.macfarlane I think I missed this when you posted it back in February.

I am able to reproduce this in a non-UTF8 characterset.

We'll have a fix for it in beta in a week or two. I'll post an update here when that happens.

-John

[dirk.mika1]

I was just about to post the same issue. When displayed as HEX, the content is shown correctly. BUT, it's not just shown wrong, it's copied with those "nonsense" to clipboard when using CTRL-C in the grid.

[JohnDorlon]

This is fixed in the beta that just posted.
Get it here.
-John

[mathias.schroeder]

Just tested it and it's fixed now.
Thank you

[nawiiwan]

Hi,
I have issue with "Popup Editor" for CLOBs.
DB: 19.0.0.0.0
NLS_CHARACTERSET: WE8MSWIN1252

Toad: Beta, 14.2.87.1051

Test case:
select TO_CLOB('a'||CHR(13)||CHR(10)) clv from dual
union all
select TO_CLOB('b'||CHR(10)) clv from dual;

Once I open Grid Popup Editor/Hex I see as last character "0A" instead of "0D0A" (top image)

But when I go to "Next record" and return "Prior record" then I can see "0D0A" (bottom image)

And after changing to "Text" and back to "Hex" "0D0A" is replaced by "0A".

Regards,
Maciej

[JohnDorlon]

which option do you have set here? I see as you describe with the 'b' row if I set this to "Fetch as windows, post as Unix". But I haven't been able to make any problem happen with the 'a' row.

I also don't see any problem if it's on "No Change" (I recommend this setting)

[nawiiwan]

Hi John.
I have enabled the same option which you highlight - "No change".

Try this test case:

1. select 'a'||CHR(13)||CHR(10) txt from dual;
2. in Data Grid do "double click" on cell with "a"
3. see Grid Popup/Hex - there is "61 0A" while I was expect to see in Hex exactly data which are in the table: "61 0D 0A".
   

Regards,
Maciej

[JohnDorlon]

I see as expected. I am not sure why you see different.

Using 14.2.87.1051, 64 bit,
Have tried both 12.1.0.2 Client, and 12.2.0.1 client
Have tried both 18c server (with WE8MSWIN1252) and 19c server (with UTF8 charset)
Have tried on both Windows 7 and Windows 10.

[nawiiwan]

I just check with Oracle XE, client 19.3.0.0, DB NLS_CHARACTERSET=AL32UTF8, Windows 10.

Toad 14.1.120.923 - works fine (like on your screenshot)
Toad Beta, 14.2.87.1051 - still wrong.

Then problem is not connected with OS, DB, Client.

Then I assume that I have problem with settings (or something else) in "AppData/Roaming/Quest Software/Toad for Oracle/14.2 Beta".
I did rename "14.2 Beta" to "14.2 Beta_old" and rename "SettingsLocations.ini" to "SettingsLocations.ini_tmp" and... now it looks fine.
It means that there is something in settings what causes problem.

John, have you got any suggestion which file(s) should provide to you to find root cause?

Regards,
Maciej

[JohnDorlon]

Zip up your 14.2 beta folder and I'll try it with your settings.

email it to john.dorlon@quest.com.

Thanks.

[nawiiwan]

Hi John.
Just send you zipped "Toad 14.2 Beta" folder.

Regards,
Maciej

[JohnDorlon]

I can reproduce the problem with your user files. I will post again when I know why this is happening.

[JohnDorlon]

This is a bug. It happens because in Options->Files->General, your setting for "Default EOL Mode" is set to "Unix". We should not be looking at that setting unless the contents are saved to a file.

Here's what's happening: When the popup editor is loaded, we put the data into the Text tab first. The EOL mode is applied to the text editor, so that when we ask it for its content, the line feeds will correspond to whatever the EOL mode is selected. When you switch to the Hex tab, we get the Hex content from the text editor on the other tab (so the two can stay in sync when you are making edits)

This will be fixed for next beta and 14.2 GA. The change will be: If Options -> Oracle -> General -> NewLine Format is set to "No Change", then the popup text editor will show whatever newline format is in the data, regardless of the setting of Options->Files->General->Default EOL Mode. And the "newline" message at the bottom of the window will reflect the correct setting.

[nawiiwan]

Hi John,
thank you for analyzing my case and feedback.
I'm not sure whether I got you right but I hope that Hex view will be independent of any settings - I mean will show always data as it is in the database.

I'll check beta once it is available and let know my feedback.

Regards,
Maciej

[JohnDorlon]

When you set Options -> Oracle -> General -> NewLine format to "Fetch as Windows, Post as Unix", the line feeds are converted as they are retrieved from and posted to the database. The Hex view just shows what data it receives . So, in 14.2 you must leave this set to "No Change" if you want to always see the actual line feed style that is in the database.

By the way, this change will be in the beta after today's beta. Sorry, I couldn't get it into this one in time.

And thank you for reporting the problem. Feedback like this is very valuable.

[megan.macfarlane]

Hi John,

After this issue last year I abandoned the beta and went back to 14.0. I just downloaded Toad 16.0.16.1406, and the problem is still occurring. Checking back to 15.1.113.1379, and it is still there too.

This time I've opened two nearly identical CLOBs and pasted them into a text editor. One is showing the invalid characters around row 1590, the other is fine.

Same DB, 11.2.0.4 WE8ISO8859P1, using 12c client.

OK clob:
..
"AsiaPacific":{
"label": "Asia Pacific",
353: {"":"Please Select"},
354: {"CentralAsia":"Central Asia"},
355: {"ChinaHongKongTaiwan":"China, Hong Kong & Taiwan"},
356: {"IndiaSouthAsia":"India & South Asia"},
357: {"JapanKorea":"Japan & Korea"},
358: {"PNGPacificIslands":"PNG & Pacific Islands"},
359: {"SouthEastAsia":"South-East Asia"}
},
..
Corrupted clob:
..
"AsiaPacific":{
"label": "Asia Pacific",
353: {"":"Please Select"},
354: {"CentralAsia":"Central Asia"},
355: {"ChinaHongKongTaiwan":"China, Hong Kong & Taiwan"},
356: {"IndiaSouthAsia":"India & South Asia"},
357: {"JapanKorea":"Japan & Korea"},
358: {"PNGPacific䤀猀氀愀渀搀猀∀㨀∀倀一䜀 倀愀挀椀昀椀 ....

Both clobs were added via the popup editor in Toad 14, work find in Toad 14.0, but one of them cannot be copied in Toad Beta 16 or Toad 15.

Thanks,
Megan

[JohnDorlon]

HI Megan,

I have tried and I cannot reproduce this problem in 15.1. I've tried with the settings as discussed above, and on a non-unicode database.

Can you please send me your user files folder to me so I can try with your settings? Instructions to do that and mail email address are in this post.

Thanks

John

[megan.macfarlane]

Hi John,
I've sent the user files.
Thanks,
Megan