Built-in number for XLATEMB is 390.
XLATEMB ( translation_template   : address )

ASCII

Abbreviation for American National Standard Code for Information Interchange.

bi-di (bidirectional language)

The ability to write and read a language in two directions, such as from left to right and from right to left.

Control map

A special layout of bytes used to control data conversion. The different types of control maps will be discussed later in this document.

Code page

A collection of characters assigned to code points.

Code points

A unique bit-pattern assigned to each graphic character, to be used by the computer when entering, storing, viewing, printing, or exchanging information.

Double Byte Character Set (DBCS)

A set of characters in which each character is represented by a 2-byte code.

Endian

The order of the bytes in memory. On big endian systems the most significant value is stored in the lowest address. On little endian systems the least significant value is stored in the lowest address. For example, take the integer value 13488. In big endian it is stored as hex '34B0' and on little endian it would be stored as hex 'B034'. The machine by default is big endian.

EBCDIC

Abbreviation for extended binary coded decimal interchange code.

Graphic

Term used to designate pure DBCS data.

ISO/IEC 10646

The international standard used to represent most of the world's written languages by assigning multiple bytes for each character. This standard was written by the International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC).

Mixed data

Data that contains single-byte and double-byte encoding.

Well formed mixed data

Mixed data where DBCS data is bracketed by shift-out (SO) and shift-in (SI) controls.

Octet

An ordered sequence of eight bits, considered as a unit.

Single Byte Character Set (SBCS)

A set of characters in which each character is represented by a 1-byte code.

String type

Used to determine the orientation and shaping of bi-di characters. The string types are defined in Table 7.

Substitution value

A single-byte or multiple-byte code to be output from a conversion when the input character is not found in a ward control block or ward.

UCS-2 Level 1

Defines the form and level of UCS. UCS-2 is a 16-bit form of UCS. Level 1 is an implementation level that does not support combining of characters. Every UCS-2 Level 1 character must be made up of only 2 bytes.

Unicode case level

The Unicode case level is the version of the Unicode standard used as the base for the casing function.

UTF-16

Defines a form and level of UCS. UTF-16 allows access to 63K characters as single UCS 16-bit units. It can access an additional 1 million characters by a mechanism known as surrogate pairs. Two ranges of UCS code values are reserved for the high (first) and low (second) values of these pairs. The high range is from hex D800 to hex DBFF and the low range is from hex DC00 to hex DFFF. A properly formed surrogate requires a high range code value followed by a low range value to form a valid character.

UTF-8

UTF-8 is the Unicode Transformation Format that serializes a Unicode code point as a sequence of one to four bytes. These sequences are mathematically equivalent to the set of UTF-16 characters.

Universal Multiple-Octet Coded Character Set (UCS)

Character set defined by IS0/IEC standard 10646.

Ward

A set of 256 single-byte or multiple-byte codes where all of the codes share a common first hex input byte when converting from a 2-byte code.

Ward control block

A set of 256 2-byte values within a control map. The 2-byte values provide the offsets from the start of the control map space to the beginning of all of the wards within the control map.

Function

The function selected determines the type of conversion to be performed. Table 1 outlines the types of conversions that may be performed and the operands required for each function. The Table 1 columns are defined as follows:

• Function - The function selected.

• Control map type - The type of control map required as input for the given function.

• Verification map allowed - A verification map is allowed for the given function. A verification map is never required.

• Source data type - The source data type required for the given function.

• Receiver data type - The receiver data type returned for the given function.

• Estimated required buffer size - Value used to determine the actual required receiver buffer length. Multiply the estimated required buffer size value by the source length to get the required receiver buffer length. If this value is less than the minimum buffer size, use the minimum buffer size value for the receiver buffer length.

Table 1. XLATEMB supported functions

Control flags

• Control map type: Determines which control map type will be supplied for the specified function. This field is verified against the function specified when a type D control map is specified. A control map type of D specified with an incorrect function results in the signaling of the template value invalid  (hex 3801) exception. Refer to Table 1 for details on which functions require this flag to be set.

• Substitution check: Check for substitution on conversion from UCS-2 Level 1 or UTF-16 or UTF-8 data. Substitution check is only supported for functions 0002, 0004, 0006, 0008, 000C, 0034, 0036, and 0038.

  When a substitution character is encountered, a substitution character used  (hex 0C20) exception is signaled at instruction completion. Complete results are placed in the receiver and receiver converted data length fields. The number of substitutions in the receiver data will be stored in the number of substitutions field of the exception data for the substitution character used  (hex 0C20) exception.

• Override default multiple-byte substitution value: Determines which multiple-byte substitution value will be placed into the receiver space when using a type C or type D control map and substitution is required on conversion to UCS-2 Level 1 data. Override default multiple-byte substitution value is only supported for functions 0003, 0005, and 0007.

  When substitution is required and the function is 0003, 0005, or 0007, the multiple-byte substitution value will be one of the following:

  • If the override default multiple-byte substitution value is 0, the default UCS-2 Level 1 substitution value of hex FFFD will be used.

  • If the override default multiple-byte substitution value is 1, the multiple-byte substitution value specified in the template will be used.

  Note:

  The override default multiple-byte substitution value field must be set to 0 for functions 0001, 0002, 0004, 0006, 0008, 0009, 000A, 000B, 000C, 0033, 0034, 0035, 0036, 0037, 0038, 003B, 003C, 00FE, 00FF and 0100 or a template value invalid  (hex 3801) exception will be signaled.

• Ward transparency: Determines whether source characters in an empty ward are converted using a multiple-byte substitution value, or moved to the receiver space transparently with no conversion taking place. Ward transparency is only supported for functions 0003, 0004, 0005, and 0007. If the ward transparency field is binary 1, then the multiple-byte substitution value is not used and the override default multiple-byte substitution value field is ignored.

  Note:

  The ward transparency field must be set to binary 0 for functions 0001, 0002, 0006, 0008, 0009, 000A, 000B, 000C,0033, 0034, 0035, 0036, 0037, 0038, 003B, 003C, 00FE, 00FF and 0100 or a template value invalid  (hex 3801) exception will be signaled. When using the ward transparency feature with functions 0005 and 0007, the first entry in the control map ward control block must be non-zero, or a template value invalid  (hex 3801) exception will be signaled.

• Well formed mixed data output: Determines whether a DBCS character in the last two bytes of the receiver space (as defined by receiver buffer size) should be replaced by a shift-in control.

  Note:

  The well formed mixed data output field must be set to 0 for function 0100 or a template value invalid  (hex 3801) exception will be signaled.

  Note:

  This control is only applied where data truncation is necessary. Mixed data output will always be well formed if the receiver space is large enough.

• Caching requested: Requests caching for the internal data structures used for a type E control map. The cache descriptor field in the control map must not be modified.

• endian mode: Requests that input and output of Unicode data be handled either as little endian or big endian.

  Note:

  The endian mode bit needs to be set to binary 0 for functions 0033, 0034, 0035, 0036, 0037, 0038, 003B, 003C, and 0100 or a template value invalid  (hex 3801) exception will be signaled.

• UTF-16 casing map: Requests that the UTF-16 casing map be used for casing functions.

  Note:

  The UTF-16 casing map bit needs to be set to binary 0 for functions 0001, 0002, 0003, 0004, 0005, 0006, 0007, 0008, 000B, 000C, 0033, 0034, 0035, 0036, 0037, 0038, 00FE, 00FF, and 0100 or a template value invalid  (hex 3801) exception will be signaled.

• Unicode bi-di formatting code: Requests that Unicode bi-di formatting codes be processed. This only applies to type D maps that have string types defined; otherwise it is ignored. For functions 0001 and 0033, bi-di formatting codes will be inserted. For functions 0002 and 0034, bi-di formatting codes will be removed.

  Note:

  The Unicode bi-di formatting code bit must be set to binary 0 for functions 0003, 0004, 0005, 0006, 0007, 0008, 0009, 000B, 000C, 0035, 0036, 0037, 0038, 00FE, 00FF, and 0100 or a template value invalid  (hex 3801) exception will be signaled.

Source length

The length of the source data contained in the space addressed by the source space pointer. A length value of less than 1 results in the signaling of the template value invalid  (hex 3801) exception.

Receiver buffer length

The length of the receiver space pointed to by the receiver space pointer. A length value of less than 1 results in the signaling of the template value invalid  (hex 3801) exception.

Receiver converted data length

The length of the data placed in the receiver space after conversion. This field is set by the machine and will always be less than or equal to the value specified for receiver buffer length.

Source range

The range of the double-byte content of the mixed ASCII source input data. Source range is only used with function 0007. The source range field is divided into 2 ranges, range 1 and range 2. Each range has a 1 byte lower and 1 byte upper limit. Some actual working examples of source ranges are defined below:

Single-byte substitution value

A single-byte value to be output from a conversion when the following occurs:

• A type C or type D control map ward control block entry is hex zeros and single-byte data is being processed.

• A type B control map ward control block entry is hex zeros.

Multiple-byte substitution value

A 2-byte value to be output from a conversion when a type C or type D control map ward control block entry is hex zeros and multiple-byte data is being processed. The multiple-byte substitution value is ignored if the ward transparency feature is being used.

Unicode case level version

Requests a specific Unicode case level version be used for the casing function. Allowable values are Hex 0000, Hex 0200 and Hex 0400.

• Hex 0000, 0200 are based on the Unicode standard 2.0

• Hex 0400 is based on the Unicode standard 4.0

Source string type

Overrides the default string type of the source field. This value only applies to type D maps that have string types defined, it is otherwise ignored.

Receiver string type

Overrides the default string type of the receiver field. This value only applies to type D maps that have string types defined, it is otherwise ignored.

Source

A space pointer to the source data buffer. The number of bytes available is specified by the source length field.

Receiver

A space pointer to the receiver data buffer. The number of bytes available is specified by the receiver buffer length field. If an error occurs during conversion this buffer will contain the data converted up to the point of the error. The length of the data converted is stored in the receiver converted data length field.

Verification pointer (optional)

A space pointer to a verification map to be used to verify UCS-2 Level 1 source data. The verification map has the following format:

The map size is a hex value that indicates the number of 2-byte verification map entries that exist in the map.

Each verification map entry contains one UCS-2 Level 1 code.

The following is an example of a verification map:

0011009A0100010101020103010401050106010701080109010A010B010C010D03B103B2

Table 2. Verification map layout

Control map

A space pointer to a control map to be used in the conversion of the source data. Refer to Table 1 for information on types of maps required for the various functions. If unused, the control map pointer must be a null pointer value.

Control map types

The following list explains the different types of control maps:

Type A -- Used to map SBCS data to UCS-2 Level 1 multiple-byte data. The type A control map has the following format:

The following is a partial example of a type A control map:

0100010101020103010401050106010701080109010A010B010C...01FE01FF

1. Use the first input byte, hex 0C, to index into the map.

2. At offset hex 0018, the control map value is hex 010C.

3. Now index into the space, using the second input byte of hex 08.

4. At offset hex 0010, the control map value is hex 0108.

5. Now index into the space, using the third input byte of hex 05.

6. At offset hex 000A, the control map value is hex 0105.

7. The result placed in the receiver data buffer is hex 010C01080105.

Table 3. Type A map layout

Type B -- Used to map UCS-2 Level 1 multiple-byte data to SBCS data. The type B control map has the following format:

1. Use the first input byte, hex 03, to index into the ward control block section.

2. The corresponding ward control block entry value, hex 0300, is used as an offset to the start of the ward detail for ward 03.

3. Using offset hex 0300 as a base, use the second input byte value, hex B3, to index into the ward detail for ward 03.

4. The corresponding entry in the ward 03 detail is the single-byte control map value of hex 8C.

5. The result placed in the receiver data buffer is hex 8C.

Table 4. Type B map layout

Type C -- The type C control map is used for conversion of the following:

• UCS-2 Level 1 to graphic

• Graphic to UCS-2 Level 1

• UCS-2 Level 1 to mixed EBCDIC

• Mixed EBCDIC to UCS-2 Level 1

• UCS-2 Level 1 to mixed ASCII

• Mixed ASCII to UCS-2 Level 1

The following is a partial example of a type C control map with simple offset values since the map is less than 64K bytes in size:

000000000200040000000000...000000000000000100020003000400050006
0007...3F3F3F3F0200020102020203020402050206...3F3F3F3F

The following is a partial example of a type C control map with offset values that are multiples of 512 bytes since the map is greater than 64K bytes in size:

000000000001000200000000...000000000000000100020003000400050006
0007...3F3F3F3F0200020102020203020402050206...3F3F3F3F....

1. Use the first byte of the graphic input value, hex 02, to index into the ward control block section of either example type C map.

2. The corresponding ward control block entry value, hex 0200 for the smaller map, is used as an offset to the start of the ward detail for ward 02, since the control map size is less than or equal to 64K. For control maps sizes greater than 64K the ward control block entry value would be 0001, indicating an offset of 512 bytes (offset of hex 0200), as shown in the example larger map.

3. Using offset hex 0200 as a base (regardless of which size map provided it), the second byte of the input value, hex 07, is used to index into the ward detail for ward 02.

4. The corresponding entry in the ward 02 detail is the double-byte control map value of hex 0007.

5. The result placed in the receiver data buffer is this hex 0007.

Table 5. Type C map layout

Type D -- A 4-byte selection code that indicates which predefined control map is to be used. The selection code is pointed to by the control map space pointer. Type D maps are only supported for functions 0001, 0002, 0003, 0004, 0005, 0006, 000B, 000C, 0033, 0034, 0035, 0036, 0037, and 0038. If zeros are encountered in the ward control block and the function is 0006, the specified single-byte substitution value or multiple-byte substitution value is placed into the receiver buffer.

If zeros are encountered and the function is 0005, the value output will be one of the following:

• If the ward transparency field is binary 0, then the appropriate multiple-byte substitution value will be output, the next input byte is skipped, and the conversion will continue with the next input byte.

• If the ward transparency field is binary 1, then the two source characters will be transparently moved to the receiver buffer, with no conversion taking place. The conversion will continue with the next input byte.

The type D control map selection code has the following format:

The predefined control maps are defined in Table 6.

Table 6. Supported predefined control maps

If the selection code field does not contain one of the supported values, a template value invalid  (hex 3801) exception is signaled.

The string types are defined in Table 7.

Table 7. String type definitions

Type E -- Used to map UCS-2 Level 1 multiple-byte data to its weights. The type E control map has two formats.

The first format, type E-1 has the following format. Type E-1 maps are only supported for function 0100

Table 8. Type E-1 map layout

The second format, type E-2 has the following format:

Type E-2 -- has a 4-byte selection code that indicates which predefined control map is to be used. The selection code is pointed to by the control map space pointer. Type E-2 maps are only supported for function 0100

Table 9. Type E-2 map layout

The predefined control maps are defined in Table 10.

Table 10. Supported predefined control maps

If the selection code field does not contain one of the supported values, a template value invalid  (hex 3801) exception is signaled.

• Range 1 is set to nulls.

• The upper limit is less than the lower limit for either range 1 or range 2.