What's the difference between a Compile and a Force Compile?
There are many stages in the compilation process for Parallel jobs:
1. The osh (Orchestrate) code is generated by DataStage client.
2. The trx (Transformer) code is generated by DataStage client. In addition, the associated osh (Orchestrate) code is generated for the next step.
3. Osh (PX Engine's binary) is called to run the transformer's osh code in order to translate the trx code to C++ code, and then compile it using the server's compiler to create object files.
This process can be time consuming, especially in jobs that have multiple transformer stages. In order to optimize the compilation process, by default, the GUI examines the differences in the internal logic to determine if step 2 and 3 for each transformer stage needs to be done. If no differences are detected then these steps are skipped to decrease the compilation time. However, this is not a perfect and exact science as external effects may take place that would influence how the transformer code is compiled.
One such example is enabling the following environment variable: APT_TRANSFORM_OPERATOR_DEBUG. This changes how we preserve the transformer code, and how we report the compilation.
Another such example is the use of external parallel routines. If a parallel routine was changed, it is highly recommended to use force compile to absorb the change into the transformer code that use the parallel routine.
In order to bypass the optimization effort the GUI provides, we offer the option to Force Compile the job. What this step does is make each and every piece of the job's code be rebuilt from scratch, and it makes step #2 and #3 above mandatory for the entire job. This naturally increases the amount of time it takes to compile an existing job, but it guarantees that any internal and external changes within the GUI will take effect for that job.