Problem: Exported shapefiles have truncated character strings in ArcGIS 10.2.x
With certain language locale settings, a problem occurs when creating shapefiles in ArcGIS for Desktop 10.2.1 and 10.2.2.
This issue may cause attribute table field names and values to be truncated, and throw errors such as the following:
"Failed to add the field to the table /Feature class. The field type is invalid or unsupported for the operation."
This issue is related to the following NIMs:
• NIM100355 - Adding Japanese characters as field names for a shapefile is generating the error: "Failed to add the field to the table /Feature class. The field type is invalid or unsupported for the operation".
• NIM100357 - Setting the code page in the registry does not properly change the code page used by a shapefile on creation.
• NIM098820 - A shapefile created at 10.2, and then consumed and exported in 10.2.1, loses the attribute values in the last field.
Before ArcGIS 10.2.1, shapefiles were encoded in the language locale of the system on which the shapefile was created. As of version 10.2.1, shapefiles are always encoded in UTF-8, which presents a problem on systems that use languages with multibyte character encodings, such as Chinese, Japanese or Russian.
This problem occurs because the string length for shapefiles are measured in bytes, not characters. So the ten-character limit imposed by UTF-8 for the length of a shapefile field name is actually ten bytes.
For example, since Japanese characters require three bytes per character, this reduces the limit to three characters for a shapefile field name. The other issue is text fields. A text field defined as three bytes wide can only hold one character. This means that data exported from Japanese (SJIS) to UTF-8 may see truncation in its text field values. The truncation is reported and each string has an asterisk (*) added to it.
Solution or Workaround
- Download and install the appropriate code page patch for the local version of ArcGIS for Desktop:
• ArcGIS 10.2.1 for (Desktop, Engine, Server) Set Shapefile Default Code Page Patch
• ArcGIS 10.2.2 for (Desktop, Engine, Server) Set Shapefile Default Code Page Patch
- Follow the instructions in Knowledge Base article 21106, How To: Read and write shapefile and dBASE files encoded in various code pages.
- How To: Read and write shapefile and dBASE files encoded in various code pages