IBM Support

Data corruption issue during migration from older server

Technote (troubleshooting)


When migrating from an older server that supports only the ISO-8859-1 locale to a newer server that uses UTF-8 locale to configure a database, the new database might not accept all characters without first using a conversion process.


When attempting to load data from a database created with ISO-8859-1 locale into a database using UTF-8 locale, reject files are created for rows containing unrecognizable characters even though CLIENT_LOCALE, DB_LOCALE and SERVER_LOCALE have all been set accordingly.


It is possible that setting CLIENT_LOCALE, DB_LOCALE and SERVER_LOCALE will still leave some characters unrecognizable to a database created with the UTF-8 locale.

Resolving the problem

Most of the time 8859-1 will load into a UTF-8 database without error, however if you have already attempted to load the data and failed, complete the following steps:

First, set the environment variables CLIENT_LOCALE, DB_LOCALE and SERVER_LOCALE so they have the appropriate settings. After migrating the old data onto the UTF-8 server, use a conversion process such as the iconv API to ensure all data will be accepted into the new database.

For example, once you have migrated the data file, you could run the command:

iconv -f ISO-8859-1 -t UTF-8 {filename} > {pipe, file}

Once the file has been converted, you can direct it to a new file or a pipe before loading it into the destination table.

*IMPORTANT* Note that any issues/defects with iconv or any other OS tool used for data conversion should be directed to the appropriate OS vendor and not to IBM. If an unfavorable action should occur during the use of an OS tool, IBM is not responsible.

Document information

More support for: Informix Servers

Software version: 11.5, 11.7, 12.1

Operating system(s): AIX, HP-UX, Linux, Solaris, Windows

Reference #: 1661529

Modified date: 04 May 2017

Translate this page: