DataStage job fails with the following error:
Internal Error: (!new_sc_op.availableNodes_.isEmpty()): sc/sc.C
The most common causes of the logged error:
"Internal Error: (!new_sc_op.availableNodes_.isEmpty()): sc/sc.C "
- Grid Toolkit is enabled, and project properties were recently updated to also enable the grid checkbox at the project level. The two types of grid support are not compatible.
- Grid toolkit is enabled (job log shows APT_GRID_ENABLE=YES), has recently installed a parallel engine patch which updated the osh program, but did then follow the special Grid Toolkit steps to move osh to osh.exe and substitute the osh script provided by the grid toolkit.
- Permission problems exist with the dataset, preventing update to one or more of its component files.
- Node configuration has changed between job runs such that a dataset the job is appending to or overwriting does not match node configuration of the prior successful job run.
Resolving the problem
Please check the following:
- First check to see if the failing job uses the Grid Toolkit. Do you see APT_GRID_ENABLE=YES in the job log environment variable list? If so, then open the DataStage administrator client, and then open the properties for the project that owns the failing job. On the Parallel tab, confirm that the grid checkbox is NOT selected. If it is, then uncheck it, save settings, and then start a new DataStage Designer or Director client session to run the job and confirm if it then works.
- If step 1 does not resolve the problem, but you are using the Grid Toolkit, then confirm if any DataStage maintenance was installed just prior to the problem (you can check the history section of the InformationServer/Version.xml file if not sure when last maintenance was applied). If maintenance was installed, then confirm if the file InformationServer/Server/PXEngine/bin/osh was updated recently and is now an executable program rather than a readable script. If so, then use the steps in the following technote to re-apply the changes the Grid Toolkit needs for osh processing:
Steps to re-enable the Grid Toolkit after InformationServer Patch overwrites osh and/or orchadmin executables
- If not using the Grid toolkit, or if the above 2 actions had no effect on problem, confirm if the failing job uses dataset stages. If so, confirm that the dataset file, directory, and all parent directories have read, write and execute permissions allowed for the userid running the job (this may be different than the login userid). Each event in the job log will list the userid that is running job. Also confirm if you are able to view the dataset using the Data Set Management option on the Designer tools menu. If errors occur, those errors will need to be resolved before running the job.
- If the job uses datasets and worked in the past but now fails, check the job logs for the older working job runs (if not yet purged) to see if the job ran on a different number of nodes when it ran successfully. Data set files are partitioned based upon number of nodes in the node configuration file when created, so if using datasets which are persistent (not re-created each job run), and the number of nodes changes, problems may occur. In that situation, see if switching job to original number of nodes resolves the problem.