Bug report: CLOBs: specified encoding (UTF-8) is ignored, system enc. used

Using DbVisualizer 9.0.5, Oracle 10g DB, table with CLOB column, database default character encoding is UTF-8 (AL32UTF8). Steps to reproduce the problem: # Create a file with UTF-8 encoding and with some national characters (e.g. cyrillic chars). For example: Датские кинематографисты (I have attached the example file ru.txt) # With DbVis connect to DB, open table data, double-click the cell (CLOB type), select "Open file", specify encoding UTF-8 (attached image img1.png), browse for file ru.txt # Observe, that even in edit window the encoding is lost (file is opened as Cp1257 or ASCII, cyrillic characters not shown) # Commit the record to database. # select the data with the help of: select DBMS_LOB.substr(COLUMN_NAME, 50) from TABLE_NAME; example is shown in attached image img2.png # Observe ASCII symbols, not unicode symbols. Example of correctly working program: NetBeans v7.3, attached image img3.png shows the result of the same query (DBMS_LOB.substr) when file was uploaded with NetBeans. Core problem: CLOB open file dialog does not care about specified encoding, file is always loaded using system default encoding.

Hello Hans, sorry, just found another follow-up-issue in this matter. I'm running DbVis Pro 9.1 now, the rest of my environment is still the same: Product: DbVisualizer Pro 9.1 Build: #2050 (2013/09/08 11:03) Java VM: Java HotSpot(TM) Client VM Java Version: 1.6.0_26 Java Vendor: Sun Microsystems Inc. OS Name: Linux OS Arch: i386 OS Version: 3.2.0-53-generic-pae The 'Encoding' setting in the connection properties is 'ISO-8859-15' (also tried 'windows-1252') The discribed bug in my step-by-step instruction (29.07.2013 09:31) is now fixed, concering the correct hex value of any inserted character. But after reopening the modified cell in the cell-editor, it will display a question mark '?' with a dark background for any non-ascii character in the 'text' tab (see file '1_nonascii_in_text.png'), while in the 'hex viewer' tab it's still the correct hex value(see file '2_nonascii_in_hex.png') . Well, this wouldn't be much of a big problem. If you edit this cell content now in the 'text' tab of the cell editor (e.g. add another character) and close it, and reopen it in cell editor again (see file '3_nonascii_in_text_after_modify.png'), then the formerly correct hex-value is now converted to '3F'' (see file '4_nonascii_in_hex_after_modify.png'), which is the hex-value of the question-mark. This is a problem, because all affected non-ascii characters will end up as question marks in the database. Still the same workaround prevents this behaviour: -Dfile.encoding=ISO-8859-15 By adding this line to the 'dbvis.vmoptions' file, all non-ascii characters are displayed well, also in cell editors 'text' tab, and editing the cell content no longer leads to converted question marks '3F' in the database. Hope this is reproducible for you and helps you getting rid of this. cheers roland

Hi Hans, i've been around with a small single engine aircraft and after flying alongside the norwegian coastline i entered sweden from Trondheim and have been to Älvdalen, Aleberg (Fälköping), Fjällbacka and Varberg. These are some rural spots with small natural airstrips. Run out of time to visit Stockholm Bromma airfield. Hope to make it next summer. While setting up new DbVis 9.1, i just realized, that my licence expired in april (great new feature to be prompted for that in the future!). So first i have to get a new one. At least i recognized the 'Encoding' setting in the properties tab. all the best Roland Edited by: Roland on 11.09.2013 10:15

Hi Roland, This is now also included in the just released DbVisualizer 9.1. I hope you had a great time in Sweden. Were did you go? Best Regards, Hans

Hello Roger, thanks for the fix, worked great. Sorry for being late with feedback, was on holidays ;-) (In beautiful Sweden, ...by the way) Cheers Roland Edited by: Roland on 09.09.2013 10:10

Hello Roger, thanks for your feedback. I'll try to provide you with some info to reproduce the issue. First some specs of the software involved Workstation with DbVisualizer 9.0.8: Ubuntu 12.04.2 LTS fully patched (Kernel 3.2.0-49-generic-pae) NLS_LANG=American_GERMANY.AL32UTF8 Database Driver: Oracle Thin (Oracle Instant Client 10.2) (http://docs.oracle.com/cd/B19306_01/appdev.102/b14294/install.htm) Database Server: Oracle8i Enterprise Edition Release 8.1.7.4.0 (Debian GNU/Linux 3.1 Kernel 2.4.34.2) NLS_LANGUAGE = AMERICAN NLS_CHARACTERSET = WE8ISO8859P15 NLS_NCHAR_CHARACTERSET = WE8ISO8859P15 So here's a short step-by-step instruction on how to reproduce the issue: 1. Have a table with a column of type 'LONG RAW' in an ISO encoded database. 2. Open this table in a new tab. 3. Choose an exisiting row, or create a new one, and edit the 'LONG RAW' field by choosing 'Edit cell in Window...' in the right-click- context-menue. 4. Insert a non-ascii character like the latin small letter 'a' with diaeresis: ä 5. In cell editor, switch to the tab 'Hex viewer' (if you've created a new row previously, then you may have to close and reopen the cell editor, to get the 'Hex Viewer' tab). If the hex value is 'c3 a4' (in case you inserted the 'ä'), then you have just inserted a multibyte UTF-8 character, which might not be the thing you expected ;-) Workaround: Make sure you have the following line in your 'dbvis.vmoptions' file: -Dfile.encoding=ISO-8859-15 Do step 1 to 5 again. Now the hex value should be 'e4' (in case you inserted the 'ä'), and everything looks well. So, hope this helps reproducing the issue. best regards roland

Hello Roger, i'm sorry, but this issue doesn't seem to be fixed in this Version: Product: DbVisualizer Pro 9.0.8 Build: #1940 (2013/07/16 15:16) Java VM: Java HotSpot(TM) Client VM Java Version: 1.6.0_26 Java Vendor: Sun Microsystems Inc. OS Name: Linux OS Arch: i386 OS Version: 3.2.0-49-generic-pae In fact it seems to be completely unchanged. The problem is still the same. Regarding your answer, posted on 03.07.2013, please keep in mind that this bug affects the linux version only. As i already pointed out in my initial posting on 03.07.2013 there's nothing wrong with the windows version 9.0.7. So your proposal to give the fix a try by using this windows version left some doubt wether this fact is clear. Hope this can be fixed in one of the next releases, to get rid of the static workaround in the dbvis.vmoptions file.

Hello Roger, this still seems to be an issue in the linux version of dbvis. I use this linux version: Product: DbVisualizer Pro 9.0.7 Build: #1935 (2013/06/04 10:02) Java VM: Java HotSpot(TM) Client VM Java Version: 1.6.0_26 Java Vendor: Sun Microsystems Inc. OS Name: Linux OS Arch: i386 OS Version: 3.2.0-48-generic-pae When i paste NON-ASCII characters into a Oracle 'long raw' field via the cell editor, then the system encoding (UTF-8) is used instead of the database encoding 'ISO-8859-15'. Just like Donatas described it previously. So under linux i still have to use this workaround: dbvis.vmoptions: -Dfile.encoding=ISO-8859-15 I double checked, that this is not true in this windows version of dbvis: Product: DbVisualizer Pro 9.0.7 Build: #1935 (2013/06/04 10:02) Java VM: Java HotSpot(TM) Client VM Java Version: 1.6.0_43 Java Vendor: Sun Microsystems Inc. OS Name: Windows XP OS Arch: x86 OS Version: 5.1

Yes, this build handles UTF-8 (and other encodings) correctly, big thanks! Donatas