JR55416: IIS ON YA CORRECTLY USING HDFSRN JOBS NOT LOCALIZING(DISTRIBUTING) TRANSFORM LIBRARIES

APAR status

• Closed as program error.

Error description

• Build  a job which uses HDFS and which   contains  a transform
  and run it.
  Change the transform in some noticeable way, recompile and run.
  The changes will not be written to the compute nodes correctly.
  
  In the container localization log you will see this message:
  Warning: Cached HDFS library $TMP_BINARIES_LOC$trx_path doesn't
  exist
  
  This is because the call to check the existence of the transform
  library in HDFS is incorrect.
  

Local fix

• No workaround.
  

Problem summary

•  ERROR DESCRIPTION:
  Build  a job which uses HDFS and which   contains  a transform
  and run it.
  Change the transform in some noticeable way, recompile and run.
  The changes will not be written to the compute nodes correctly.
  
  In the container localization log you will see this message:
  Warning: Cached HDFS library $TMP_BINARIES_LOC$trx_path doesn't
  exist
  
  This is because the call to check the existence of the transform
  library in HDFS is incorrect.
  
  LOCAL FIX:
  No workaround.
  

Problem conclusion

• IMPORTANT:
  The patch for this issue contains several other important fixes
  for PX on Hadoop
     RTC249786  (04-Mar-16)IIS on YARN job not localizing
  transform libraries correctly using HDFS.
  RTC248975  (07-Mar-16)Failure to distribute binaries to compute
  nodes due to fs.permissions.umask-mode=007 being set in
  hdfs-site.xml
  JR55459    (08-Mar-16)Starvation kind of the conductor waiting
  to be assigned an available application master to run the job.
  RTC249367  (25-Mar-16)Hortonworks: Transformer job failed for
  non DS administrator user.
  JR55499    (25-Mar-16)Data locality appears to cause a bad
  container distribution which causes FC job failure.
  RTC249367  (08-Apr-16)Hortonworks: Transformer job failed for
  non DS administrator user.
  

Temporary fix

Comments

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

• Fixed component name

  WIS QUALITYSTAG

• Fixed component ID

  5724Q36QS

Applicable component levels

• RB50 PSY

     UP