IT05987: FOR JSON TYPE MESSAGES, ESCAPING IS INCORRECT. EXAMPLES ARE SURROGATE PAIRS, SUCH AS EMOJI, AND A BMP IN AN OBJECT NAME

Fix packs for DataPower B2B Appliance version 6.0
Fix packs for DataPower Integration Appliance version 6.0
Fix packs for DataPower Service Gateway version 6.0
Fix packs for DataPower Service Gateway version 6.0.1
Fix packs for DataPower B2B Appliance version 6.0.1
Fix packs for DataPower Integration Appliance version 6.0.1
Fix packs for DataPower Service Gateway version 7.0
Fix packs for DataPower B2B Appliance version 7.0
Fix packs for DataPower Integration Appliance version 7.0
Fix packs for DataPower Gateway version 7.1

APAR status

• Closed as program error.

Error description

• Processing JSON feeds (in UTF-8) through DataPower to convert
  them to JSONx.
  
  A customer might receive the below error log message when
  DataPower is converting the contained emoji characters to
  escaped characters like \uD83D\uDCAF\uD83D\uDCAF\uD83D\uDCAF:
  
  'Converting PIPE with map JSONInput results stored in PIPE'
  failed: Convert HTTP produced invalid XML: invalid character
  reference at offset ...'
  
  Also BMP (Basic Multilingual Plane) escapes might be handled
  incorrectly in object names.  Eg.  {"\u0040":...} resulting in
  <json:string name="&amp;#x0040;">...  .
  

Local fix

• See detailed workaround described in this DataPower forum
  posting:
  https://www.ibm.com/developerworks/community/forums/html/topic?i
  d=4c79ce53-ff6e-4ea4-b58f-97ec1690cd84#c784ebd8-ddf4-453e-8271-c
  77871da10bb
  

Problem summary

• Affected is processing of escaped characters in JSON strings for
  JSON request/response type services or services with "Convert
  Query Parameters to XML" action with JSON mapping.
  
  DataPower errors out for any escaped surrogate pair
  (\uXXXX\uYYYY), and 6 character escapes are handled wrong in
  object names ("\u0040" becomes "@".
  

Problem conclusion

• Fix is available in 6.0.0.14, 6.0.1.10, 7.0.0.7 and 7.1.0.5.
  
  For a list of the latest fix packs available, please see:
  http://www-01.ibm.com/support/docview.wss?uid=swg21237631
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  DATAPOWER

• Fixed component ID

  DP1234567

Applicable component levels

• R600 PSY

     UP

• R601 PSY

     UP

• R700 PSY

     UP

• R710 PSY

     UP