What is the impact of changing QCCSID from shipped 65535 to another CCSID

Many systems can change the CCSID from 65535 to the CCSID associated with their language ID without experiencing adverse effects.
But "many" is definitely not the same as all.

If the system is configured with all I/O devices using the same language and the applications in use are home grown (or at least developed using the same language environment) then you are most likely safe. It however gets a lot more interesting when either or both of these assertions are not true.

To help understand this issue it is helpful to know that "#", "$" and "@ " that are allowed in names etc. have different hex values on different CCSIDs. Let's say you have a system in the UK, using UK terminals, running an application developed in the United States. Let us further assume that the application has a control field in one file where values such as $ and # might be used (not necessarily "seen" by an end user, but used to control processing within the application) and that this file is tagged with CCSID 37 (US). When the job (or system) is running with a CCSID of 65535 and the application program read the file, the "$" control value is read as x'5B' and the "#" as x'7B'. The application program compares the control field to a constant within the program (x'5B' and x'7B' respectively) and happily goes on its way.

Now lets change the job (or system) CCSID to 285 (UK). When the application opens the file data management will detect the difference between the file CCSID (37) and the job CCSID (285) and automatically convert the data from 37 to 285 on Reads and from 285 to 37 on Writes or Updates. This conversion will cause the "$" control value to be read into the application program as x'4A' (the code point for the "$" character in CCSID 285) and the poor application program (compiled using the CCSID 37 constant of x'5B' for the "$" sign) will no longer compare equal. This, in many cases, would be a definite problem :) The "#" however would be OK as it is x'7B' in both CCSIDs.

To further complicate this scenario, if the application reads a "#" and then writes a "$" due to a status change in the record (or just writes a "$" for some new record), data management will see the program constant x'5B' being written to the updated (or new) record, realize that x'5B' in CCSID 285 is the "£" and convert that to the "£" sign in CCSID 37 - x'B1'. This value will work OK on subsequent reads from the application program, but will look like the character "[" to anyone looking at the file using a command such as DSPPFM. If you now switch the job or system CCSID back to 65535 (due to the bad comparisons for existing records) you will find that now any records written or updated while you were in CCSID 37 are now messed up. You now have a very, very bad situation. I will point out though that this could have been avoided if the application file had had the decency to define the control field as Hex and not Character. Hex fields do not get converted by data management on IO operations. Most applications however, that I've seen anyway, do not utilize this capability.

Now one might say that the problem here is that we switched from CCSID 65535 to CCSID 285. What we "should" have done was switched from CCSID 65535 to CCSID 37. Work station data management, like data base data management, will also do CCSID conversions on your behalf. While the "control" field is now being correctly interpreted under CCSID 37, user data in the file is now being read into the application program under CCSID 37 even though it is really 285. WS data management can detect this difference and again provide automatic conversion on writes to the display file (or UIM panel) from 37 to 285, and on reads from the display from 285 to 37. So the "£" that the user entered (and saw) yesterday, today is displaying as a "$". This conversion can be turned off (and in fact the default is for it to be off), but if you decide in the future to install (or develop) an application that truly is CCSID aware (and utilizes the WS data
management capabilities to support a multi-lingual environment) you will find yourself with difficulties when trying to move into that environment -- as you've been somewhat lying to the system since you changed the CCSID to 37...

You really, really need to understand the application and system configuration environment in order to change the system CCSID from 65535 to a non-65535 value. Though as I mentioned at the start of this note, this can be a no-brainer for some environments. If it was simple to change for all systems IBM would have changed the default years ago. But IBM has no way of knowing what all mischief might be happening in that 65535 environment.

If you would like additional guidance on this issue please reach out to LabServices / Technology Services.     

There are simply too many factors involved for this TechNote to provide an answer that will work for every customer and every IBM i partition.                   

Acknowledgement: Mr. Bruce Vining for the content published in this TechNote.