Data corruption issue during migration from older server

Technote (troubleshooting)


Problem(Abstract)

When migrating from an older server that supports only the ISO-8859-1 locale to a newer server that uses UTF-8 locale to configure a database, the new database might not accept all characters without first using a conversion process.

Symptom

When attempting to load data from a database created with ISO-8859-1 locale into a database using UTF-8 locale, reject files are created for rows containing unrecognizable characters even though CLIENT_LOCALE, DB_LOCALE and SERVER_LOCALE have all been set accordingly.


Cause

It is possible that setting CLIENT_LOCALE, DB_LOCALE and SERVER_LOCALE will still leave some characters unrecognizable to a database created with the UTF-8 locale.

Resolving the problem

Most of the time 8859-1 will load into a UTF-8 database without error, however if you have already attempted to load the data and failed, complete the following steps:

First, set the environment variables CLIENT_LOCALE, DB_LOCALE and SERVER_LOCALE so they have the appropriate settings. After migrating the old data onto the UTF-8 server, use a conversion process such as the iconv API to ensure all data will be accepted into the new database.

For example, once you have migrated the data file, you could run the command:

iconv -f ISO-8859-1 -t UTF-8 {filename} > {pipe, file}

Once the file has been converted, you can direct it to a new file or a pipe before loading it into the destination table.

Rate this page:

(0 users)Average rating

Document information


More support for:

Informix Servers

Software version:

11.1, 11.5, 11.7, 12.1

Operating system(s):

AIX, HP-UX, Linux, Mac OS X, Solaris, Windows

Reference #:

1661529

Modified date:

2014-01-09

Translate my page

Machine Translation

Content navigation