ArcGIS Pro: Field larger than 131,072 field limit

Error Message

In ArcGIS Pro, creating polygons from a CSV containing well-known text (WKT) using ArcPy returns one of the following error messages:

Error:   
_csv.Error: field larger than field limit (131072)

Error: 
field larger than field limit (131072)

Cause

The WKT geometry is stored as strings in the CSV file, and Python’s csv module commonly enforces a default maximum field size of 131,072 characters. Large WKT geometries can exceed this limit, resulting in the error during conversion.

Solution or Workaround

Increase the field limit to the maximum allowed by the system

1. Open the project in ArcGIS Pro.
2. Click the Insert tab > New Notebook.
3. Specify the script below:
1. Import the necessary modules.

import arcpy
import csv
import os
import sys

2. Insert the function to increase the limit to the maximum allowed by the system.

csv.field_size_limit(sys.maxsize)

3. Define the CSV input file path, WKT column name, path to the new output geodatabase, output name, and spatial reference.

In_csv_path = r"<CSVFileFolderPath>\<fileName>.csv"
In_wkt_column_name = "<columnName>"
out_gdb = r"<OutputFileFolderPath>.gdb"
out_name = "<newShapefileName>"
spatial_ref = arcpy.SpatialReference(<projectedCoordinateSystemWKID>)

4. Enable overwriting of existing datasets and define the output feature class path.

arcpy.env.overwriteOutput = True
out_fc = os.path.join(out_gdb, out_name)

5. Open the CSV file, read the headers, and map the original field names to the validated field names using arcpy.ValidateFieldName().

with open(In_csv_path, 'r', encoding='utf-8-sig') as f:
    reader = csv.DictReader(f)
    original_csv_fields = reader.fieldnames
    
    field_mapping = {}
    for f_name in original_csv_fields:
        if f_name != In_wkt_column_name:
            valid_name = arcpy.ValidateFieldName(f_name, out_gdb)
            field_mapping[f_name] = valid_name

6. Create the feature class in the geodatabase and specify the geometry type, output path, output feature name, and spatial reference. In this example, the geometry type is specified as POLYGON.

arcpy.management.CreateFeatureclass(out_gdb, out_name, "POLYGON", spatial_reference=spatial_ref)

7. Add the validated fields to the feature class table.

valid_field_list = list(field_mapping.values())
for v_name in valid_field_list:
    arcpy.management.AddField(out_fc, v_name, "TEXT", field_length=255)

8. Open the CSV file, convert the WKT strings to polygons, and insert the row data into the feature class using the validated field names.

cursor_fields = ["SHAPE@"] + valid_field_list

success_count = 0
with open(In_csv_path, 'r', encoding='utf-8-sig') as f:
    reader = csv.DictReader(f)
    
    with arcpy.da.InsertCursor(out_fc, cursor_fields) as cursor:
        for row in reader:
            try:
                wkt_text = row.get(In_wkt_column_name)
                if wkt_text and wkt_text.strip():
                    geom = arcpy.FromWKT(wkt_text, spatial_ref)
                    
                    if geom:
                        row_data = [geom] + [row[orig_name] for orig_name in field_mapping.keys()]
                        cursor.insertRow(row_data)
                        success_count += 1
            except Exception:
                continue

9. Apply the print() function to return a message indicating the number of polygons successfully imported.

print(f"Finished! Successfully imported {success_count} features.")

The code below is an example of the full working script.

import arcpy
import csv
import os
import sys

csv.field_size_limit(sys.maxsize)

In_csv_path = r"C:\Users\hmahmud\Documents\test\wkt_test.csv"
In_wkt_column_name = "GEOM_TRACT_FOR_MAP" 
out_gdb = r"C:\Users\hmahmud\Documents\ArcGIS\Projects\Trial\Trial.gdb"
out_name = "Imported_Polygons"
spatial_ref = arcpy.SpatialReference(4326)

arcpy.env.overwriteOutput = True
out_fc = os.path.join(out_gdb, out_name)

with open(In_csv_path, 'r', encoding='utf-8-sig') as f:
    reader = csv.DictReader(f)
    original_csv_fields = reader.fieldnames
    
    field_mapping = {}
    for f_name in original_csv_fields:
        if f_name != In_wkt_column_name:
            # ValidateFieldName ensures it doesn't start with a number and removes spaces
            valid_name = arcpy.ValidateFieldName(f_name, out_gdb)
            field_mapping[f_name] = valid_name

arcpy.management.CreateFeatureclass(out_gdb, out_name, "POLYGON", spatial_reference=spatial_ref)

valid_field_list = list(field_mapping.values())
for v_name in valid_field_list:
    arcpy.management.AddField(out_fc, v_name, "TEXT", field_length=255)

cursor_fields = ["SHAPE@"] + valid_field_list

success_count = 0
with open(In_csv_path, 'r', encoding='utf-8-sig') as f:
    reader = csv.DictReader(f)
    
    with arcpy.da.InsertCursor(out_fc, cursor_fields) as cursor:
        for row in reader:
            try:
                wkt_text = row.get(In_wkt_column_name)
                if wkt_text and wkt_text.strip():
                    geom = arcpy.FromWKT(wkt_text, spatial_ref)
                    
                    if geom:
                        row_data = [geom] + [row[orig_name] for orig_name in field_mapping.keys()]
                        cursor.insertRow(row_data)
                        success_count += 1
            except Exception:
                continue

print(f"Finished! Successfully imported {success_count} features.")

The message from ArcPy below indicates the number of features successfully imported after increasing the field limit.