Question & Answer
Question
On some basic measure (no calculation, no weighted average rollup,...) an arithmetic overflow occurs during cube generation : The error message is : (TR1901) PDS-PPE-0104 A record was rejected. An arithmetic overflow occurred. (Data) in cube.mdc The underlying question is : what is the maximum value that Transformer can handle when measure type is "64-bit floating point" ?
Answer
What Transformer can handle is far less than what is theorically allowed by IEEE 754 standard for 64-bit floating point type (precision up to 308).
In Transformer, if Precision field is set to 0, the maximum [integer] value is :
9,223,372,036,854,775,679
[ and so the minimum value is -9,223,372,036.,854,775,679 ]
For information this corresponds to the following values :
7FFFFFFFFFFFFF7F in hexadecimal
111111111111111111111111111111111111111111111111111111101111111 in binary
which can also be calculated as 2^63 - 2^7 - 1
If Precision is more that 0, then just move the coma to the left. For example, if Precision is set to 9 (which is the maximum in Transformer), the maximum [decimal] value that can be stored is :
9,223,372,036.854775679
(about 9 billions with nine decimal digits)
Note that this limitation is for the incoming records from the data source ; in the final powercube, if this limitation is reach, the data in rounded to the last possible decimal and, for large numbers, the scientific syntax is applied (#.#########e18)
[{"Product":{"code":"SSEP7J","label":"Cognos Business Intelligence"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Transformer","Platform":[{"code":"PF033","label":"Windows"}],"Version":"10.2.2","Edition":"All Editions","Line of Business":{"code":"LOB10","label":"Data and AI"}}]
Was this topic helpful?
Document Information
Modified date:
15 June 2018
UID
swg21450594