Coordinate values for data stored in the high precision geodatabase do not match values arrived at through arithmetic calculations.
Coordinates in the high-precision geodatabase are stored as floating point numbers. Floating point numbers are approximations that cannot precisely mimic real numbers, and floating-point operations cannot exactly mimic true arithmetic operations.
The Institute of Electrical and Electronics Engineers (IEEE) adopted a standard referred to as IEEE 754 in 1985. This standard is a method of storing floating-point numbers in a compact way that is easy to manipulate. This standard is used by Intel coprocessors and most PC-based programs that implement floating-point math.
IEEE 754 specifies that numbers be stored in binary format to reduce storage requirements and allow the built-in binary arithmetic instructions that are available on all microprocessors to process the data in a relatively rapid fashion.
However, some numbers that are simple, nonrepeating decimal numbers are converted into repeating binary numbers that cannot be stored with perfect accuracy.
For example, the number 1/10 can be represented in a decimal number system with a simple decimal: 0.1
However, the same number in binary format becomes the repeating binary decimal: .0001100011000111000111...
Since this number cannot be represented in a finite amount of space, this number is rounded down by approximately -2.78E-17 when it is stored.
If several arithmetic operations are performed on floating point numbers to obtain a given result, these rounding errors may be cumulative.
Portions of the IEEE documentation on standard 754-1985 for Binary Floating-Point Arithmetic can be accessed in the Related Information section below. Other technical information related to this topic is also available from the links below.