ExpressionEvaluators • sum returns inaccurate result
SummaryThe evaluation of ExpressionEvaluators • sum--(ClipboardPropertyCollection) on a page list (referencing a Decimal property) is returning -12.950000000000001 when summing the following values,
Error MessagesThe validate returns false.
Steps to ReproduceUse a validate rule to compare equality between one decimal number and a sum of other decimal numbers.
The issue described is one of the inherent problems with performing arithmetic on binary data which contain fractional values.
Very often, decimal numbers with fractional values will not convert evenly into a rational binary number.
Instead, it will convert to an irrational binary number.
For example, the decimal number 0.1 cannot be represented exactly, but is approximately 1.10011001100110011001101 × 2-4.
One must approach equality comparisons of two non-integer numeric values carefully as often the values will have very slight differences.
The property qualifier pyDecimalPrecision can set a decimal precision when working with numeric properties.
Many articles and posts are available on the web regarding this issue and can be found with a web search for "binary rounding errors".
Published December 7, 2016 - Updated December 9, 2016