Knowledge Base - Technical Articles


Technical Article   Problem:  W3WP and SOM processes consume too much memory when GIS services are under heavy load

Article ID: 34872
Bug Id: N/A
Software:  ArcGIS Server 9.2, 9.3, 9.3.1
Platforms:  Windows XP, 2003Server

Description

The IIS W3WP worker process and SOM process may begin to adversely affect other applications by consuming large amounts of memory when GIS Services are under very heavy load.

Cause

The .NET garbage collection has a concept of generations (Garbage Collector Basics and Performance Hints). Local variables and short term objects are classified as generation one and are cleaned up frequently when garbage collection is called. Objects that have longer lifetimes are classified as generation two or higher and are less frequently cleaned up when garbage collection is called.

ArcGIS Server DCOM runtime callable wrapper (RCW) proxies and their references to DCOM objects in the SOM process are particularly susceptible to being pushed into higher generations and accumulate on the Web server and SOM until the less frequent second or third generation garbage collection algorithm occurs.

Under load, when second and third generation garbage collection may be delayed due to its high CPU utilization cost, the W3WP and SOM process memory utilization can grow rapidly.

Solution or Workaround

If memory is a critically limiting resource on the Web services machine, it may be desirable to force recovery of resources by decreasing the default .NET Garbage Collection (GC) interval. Decreasing the GC interval encourages the GC algorithms to clear out generation two and three objects no longer being used.

To address this issue, the ArcGIS Server Web services use a web.config setting to force garbage collection after a specified number of requests.

The default GC Interval (every 10,000 requests) has been chosen to address the memory consumption problem described above based on testing representative services under load.

If a problem with memory growth in the W3WP worker process or SOM process persists, the GC interval can be lowered from 10,000 to a smaller number (for example: 1000). This results in more frequent garbage collection. However, garbage collection is a CPU intensive operation and decreasing the GC interval results in the reduction of throughput. If a more frequent GC interval than 10,000 is chosen, select a number cautiously so that the throughput is not significantly affected.

The steps required to change the default GC interval are outlined below.

  1. Navigate to the ArcGIS Server Web service instance directory. This is typically:

    C:\Inetpub\wwwroot\ArcGIS\Services
  2. Open the web.config file in a text editor, such as Notepad.
  3. Edit the GCInterval key in the <appSettings> subElement of the <configuration> element by changing the value from the default of 10000 to something smaller.

    <configuration>
    
    ...
    <appSettings>
    ...
    <add key="GCInterval" value="1000" />
    ...
    </configuration>

  4. Restart IIS by issuing 'iisreset' in a command window or by restarting the IISAdmin windows service. -show me-

    1. Open the Services control panel from Start > Control Panel > Administrative Tools > Services.
    2. In the local services pane, select 'IIS Admin Service' and click the restart link or button.

Related Information


Created: 5/21/2008
Last Modified: 6/17/2010

If you would like to post a comment, please login