PROBLEM
When a FLOAT32 data type is used in an SDE dynamic WHERE clause constraint, the query could return unexpected results.In the following example, the variable float_val is bound dynamically to the WHERE clause in the query and the constraint is "SE_FLOAT_TYPE > ?". When the query is executed, 0.03 is also returned even though the constraint asked for values greater than the variable.(1) Check the contents of the table to see what values are present.
Code:(2) Execute the query using SDE dynamic binding.
SQL> select * from qa_cq00238016;SE_FLOAT_TYPE
-------------
.01
.02
.03
.04
.05
.06
.07
.08
.099 rows selected.
Code:The query results in the following fetched rows: 0.030000
float_val = 0.03;
where = "SE_FLOAT_TYPE > ?"
This unexpected result is not a defect of SDE, but is caused by the nature of floating point data types: Floating point math is not an exact arithmetic; different compilers and CPU architectures store temporary results at different precisions, which could lead to different floating point comparison results.
For instance, in the following simple IF statement:
Code:
if (result = = expectedResult)
Article ID:000009002
Get help from ArcGIS experts
Download the Esri Support App