English

How To: Customize numeric data precision and scale in ArcMap

Procedure

When adding a numeric field to a file or personal geodatabase attribute table, Precision refers to the number of significant digits that can be stored in a field. The Scale parameter represents the number of decimal places the field will store.

Setting the precision and scale when creating a new numeric field

The Add Field (Data Management) tool is used create a new field in the attribute table. The Add Field dialog box allows the user to specify precision and scale values. However, these values are ignored if the selected field type does not match the precision and scale values the user has entered.

Use the following guidelines to select the field type for a given precision and scale:

  • When creating a float, double, or integer field and accepting the default of 0 for Precision and Scale, the Add Field tool creates a binary type field if the underlying database supports binary data type. As personal and file geodatabases support only binary type fields, precision and scale are ignored.
  • When creating float and double fields and specifying a precision greater than six, use a double, otherwise, use a float. By default, when creating a double field and specifying a precision of six or less, a float field is created. Likewise, creating a float field and specifying a precision greater than six, creates a double field.
  • When specifying a scale of 0 and a precision of 10 or less, create integer fields. Setting a precision of more than 10 when creating an integer field, creates a double field.
Setting the precision and scale of numeric field in tables

In some instances, numeric values are truncated in the field. This is because the default number of decimal places that can be displayed in attribute table is six. The number of decimal points and significant digits can be reviewed in the Field Properties window. To do this in ArcMap, refer to the following documentation, ArcGIS Help: Formatting numeric fields in tables.

Setting the scale during an editing session

When editing, ArcMap displays numerical values in three decimal places. These values include the values reported when new features are created, the distance, coordinates, and the values on the Edit Sketch Properties window. The number of decimal places displayed for measurements can be changed as desired in the Editing Options window. For more information, refer to the following documentation, ArcGIS Help: Setting the number of decimal places displayed for measurements.

Related Information