The Unicode standard

The Unicode Standard is the specification of an encoding scheme for written characters and text. It is a universal standard that enables consistent encoding of multilingual text and allows text data to be interchanged internationally without conflict. The ISO standards for C and C++ refer to Information technology – Programming Languages – Universal Multiple-Octet Coded Character Set (UCS), ISO/IEC 10646:2003. (The term octet is used by ISO to refer to a byte.) The ISO/IEC 10646 standard is more restrictive than the Unicode Standard in the number of encoding forms: a character set that conforms to ISO/IEC 10646 is also conformant to the Unicode Standard.

The Unicode Standard specifies a unique numeric value and name for each character and defines three encoding forms for the bit representation of the numeric value. The name/value pair creates an identity for a character. The hexadecimal value representing a character is called a code point. The specification also describes overall character properties, such as case, directionality, alphabetic properties, and other semantic information for each character. Modeled on ASCII, the Unicode Standard treats alphabetic characters, ideographic characters, and symbols, and allows implementation-defined character codes in reserved code point ranges. According to the Unicode Standard, the encoding scheme of the standard is therefore sufficiently flexible to handle all known character encoding requirements, including coverage of all the world's historical scripts.

C99 and C++ allow the universal character name construct defined in ISO/IEC 10646 to represent characters outside the basic source character set. Both languages permit universal character names in identifiers, character constants, and string literals. C++ only In C++, you must compile with the -qlanglvl=ucs option for universal character name support.

The following table shows the generic universal character name construct and how it corresponds to the ISO/IEC 10646 short name.

Universal character name	ISO/IEC 10646 short name
where N is a hexadecimal digit
\UNNNNNNNN	NNNNNNNN
\uNNNN	0000NNNN

C99 and C++ disallow the hexadecimal values representing characters in the basic character set (base source code set) and the code points reserved by ISO/IEC 10646 for control characters.

The following characters are also disallowed:

Any character whose short identifier is less than 00A0. The exceptions are 0024 ($), 0040 (@), or 0060 (').
Any character whose short identifier is in the code point range D800 through DFFF inclusive.

UTF literals (IBM extension)

The ISO C and ISO C++ Committees have approved the implementation of u-literals and U-literals to support Unicode UTF-16 and UTF-32 character literals, respectively. This introduces new, 16-bit and 32-bit character types, char16_t and char32_t, as well as a new syntax for specifying UTF-16 and UTF-32 character and string literals. In C, the newly-introduced types are defined as typedefs inside the <uchar.h> header. In C++, the newly-introduced types are separate built-in types.

The following table shows the syntax for UTF literals.

Table 1. UTF literals
Syntax	Explanation
u'character'	Denotes a UTF-16 character.
u"character-sequence"	Denotes an array of UTF-16 characters.
U'character'	Denotes a UTF-32 character.
U"character-sequence"	Denotes an array of UTF-32 characters.

The following example shows how to declare a UTF-16 string and a UTF-32 string:

#include <uchar.h>

char16_t msg16[] = u"From \xbd to \u221e and beyond: \U0010ffff";
char32_t msg32[] = U"From \xbd to \u221e and beyond: \U0010ffff";

String concatenation of u-literals

The u-literals and U-literals follow the same concatenation rule as wide character literals: the normal character string is widened if they are present. The following shows the allowed combinations. All other combinations are invalid.

Combination	Result
u"a" u"b"	u"ab"
u"a" "b"	u"ab"
"a" u"b"	u"ab"

U"a" U"b"	U"ab"
U"a" "b"	U"ab"
"a" U"b"	U"ab"

Multiple concatentations are allowed, with these rules applied recursively.