Support Article

ExpressionEvaluators • sum returns inaccurate result

SA-31168


Summary



The evaluation of ExpressionEvaluators • sum--(ClipboardPropertyCollection) on a page list (referencing a Decimal property) is returning -12.950000000000001 when summing the following values,

-10.34
-0.89
-1.72


Error Messages



The validate returns false.


Steps to Reproduce



Use a validate rule to compare equality between one decimal number and a sum of other decimal numbers.


Root Cause



The issue described is one of the inherent problems with performing arithmetic on binary data which contain fractional values.

Very often, decimal numbers with fractional values will not convert evenly into a rational binary number.
Instead, it will convert to an irrational binary number.

For example, the decimal number 0.1 cannot be represented exactly, but is approximately 1.10011001100110011001101 × 2-4.

 

Resolution



One must approach equality comparisons of two non-integer numeric values carefully as often the values will have very slight differences.

The property qualifier pyDecimalPrecision can set a decimal precision when working with numeric properties.

Many articles and posts are available on the web regarding this issue and can be found with a web search for "binary rounding errors".

 

Published December 7, 2016 - Updated December 9, 2016


0% found this useful

Have a question? Get answers now.

Visit the Collaboration Center to ask questions, engage in discussions, share ideas, and help others.