Frequently asked question

Why are the numbers returned by the DB2 spatial extender Z and M functions slightly different than the actual stored values?

Last Published: April 25, 2020

Answer



** Internal Publish Only! This article may contain information that is not intended for external circulation. **



The DB2 spatial extender function is returning values that are different than what has been stored. For instance, storing a measure of 0.20000000 may be
returned by the spatial extender M function as 0.19999999.

The Z and M functions are incorrectly converting the integer stored value to double-precision real-world values.

Article ID:000003654

Software:
  • Legacy Products

Get help from ArcGIS experts

Contact technical support

Download the Esri Support App

Go to download options

Discover more on this topic