Support Article
Decimal precision lost after calculations
SA-19040
Summary
Calculations using numbers with large number of digits to the right of the decimal indicator, are truncating the result values to fewer decimal digits.
And also loosing the result precision which is required by the application.
Error Messages
No error messages.
Steps to Reproduce
- Execute an activity which uses the @Divide(dividend,divisor) form of the function.
- Review the result returned by the function.
Root Cause
The Divide function determines a default precision of the function result. Sometimes, the default precision is not the precision which is required by the application.
Resolution
When the default is not satisfactory, use the @Divide(dividend, divisor, scale) form of the function where the scale specifies the desired result precision.
Tags:
Published January 31, 2016 - Updated December 2, 2021
Have a question? Get answers now.
Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.