Applying the ArcGIS Server Log4j Patch on an AWS (Amazon Web Services) deployment may cause a machine to be removed from the ArcGIS Server site.

Description

Applying the ArcGIS Server Log4j Patch on an AWS (Amazon Web Services) deployment may cause a machine to be removed from the ArcGIS Server site. The attached script can be used to prevent and correct this issue.
 

Important Note (April 11, 2022): New patches have been released to prevent this defect (BUG-000148146) on AWS (Amazon Web Services) deployments. The B patches for the affected versions (10.9.1, 10.9, 10.8.1) will install over the top of the original patches if you have already installed the originals. If you did install the original patches, please follow the steps of this tech article to ensure your system will not encounter a possible problem where ArcGIS Server machines are removed from a site after a machine restart. If you did not previously install the original patches and are only applying the "B" version of the patch, the issue should not exhibit itself.

The script is provided to address issues introduced on an AWS deployment after installing the ArcGIS Server Log4j patch on the 10.8.1, 10.9, and 10.9.1 releases.
 

• The problem only happens when version 1 of the AWS metadata service is disabled.  This affects all ArcGIS Server sites created by the Cloud Builder or by Esri’s CloudFormation templates.  For those installing manually, Amazon's default is to enable version 1 of the AWS metadata service.
• The problem happens after ArcGIS Server is restarted after the Log4j patch is installed.  ArcGIS Server may function for a while after the patch is installed.
• Regardless of which versions of the AWS metadata service are enabled, it is recommended to run the script to ensure the issue will not be encountered later.
• The attached script is written to work with AWS deployments based on Esri's AWS AMIs, or deployments created using Cloud Builder or by Esri’s CloudFormation templates.  If your deployment was configured manually, you may not be impacted by this bug.  Please consult Esri technical support for additional assistance.

Workaround

The attached Python script should be downloaded and executed on any AWS EC2 instances where the patch has been run. The steps vary slightly depending on whether the problem has occurred or if it is latent (pending the next restart of the ArcGIS Server service).

Download either the zip file (for Windows) or tar.gz file (for Linux). Ensure that the file hasn’t been tampered with by computing a sha256 checksum, as follows:

Computing the Checksum on Windows

1. Start Powershell.
2. Change directory to the directory with the zip file.
3. Type this command:

Get-FileHash log4jserverpatchfix.zip

4. Ensure that the hash is:

2C73CC8233770F65C5760D1872630DA41A58574D39DD4A1D353D1CDB976D1645

Computing the Checksum on Linux

1. Start a shell.
2. Change directory to the directory with the tar.gz file.
3. Type this command:

sha256sum log4jserverpatchfix.tar.gz

4. Ensure that the hash is:

48861fa6ec6646d425d5952c33d9eff2eaae58fb90b458b48800d1ced51969be

Extract
Download and extract the Python file from the zip or tar.gz file.

Script Execution
The general syntax for running the Python script is the following:

python3 log4jserverpatchfix.py <installation location> [properties file]

The installation location is always required. The properties file is only required if the problem (the machine has been removed from the site) has already occurred.

Windows:

1. Start a command prompt.
2. Change directory to the directory where you placed the script.
3. Run the Python command: 

"C:\Program Files\ArcGIS\Server\framework\runtime\ArcGIS\bin\Python\envs\arcgispro-py3\python.exe" log4jserverpatchfix.py "C:\Program Files\ArcGIS\Server"

Linux:

1. Start a shell.
2. Change directory to the directory where you placed the script.
3. Run the Python command:

python3 log4jserverpatchfix.py /arcgis/server

Properties file:
The properties file, log4jserverpatchfix.txt, is included in the download, and is needed to re-establish the connection to the ArcGIS Server config-store if the machine has been removed from the site. An example of this text file is included with the script. For each instance (or machine) that must be updated, the following properties are required and need to be listed in comma-separated format:

• private IP address
• config-store connection string
• local repository path

Private IP
On Windows, the private IP is commonly displayed on the desktop in the upper-right corner, as shown in the image below.

On Linux, the private IP can be obtained from the terminal by running: $ ip addr, as seen in the next image:

The first IP is the loopback IP and the second one is the private IP address.

Configuration Store Connection String
For the config-store connection string, if the ArcGIS Server config-store is stored at the filesystem level in AWS, the path is very likely “\\FILESERVER\config-store” on Windows or “/net/FILESERVER/gisdata/arcgisserver/config-store” on Linux.

If using a highly available ArcGIS Server, the settings (the config-store) are stored in AWS DynamoDB. To tell ArcGIS Server how to connect to DynamoDB you must obtain the region id (such as ‘us-west-2') and information about the DynamoDB table.

The region can be obtained in the upper right corner of the AWS console as shown in the image below by clicking the region button.  When the region is clicked, it provides both the name - “US West (Oregon)” for instance. and the region id “us-west-2".  The region id is needed for the properties file, as seen in the following image:

The DynamoDB information can be obtained by navigation to DynamoDB in the AWS Console and clicking Tables in the left menu.  In the main panel you should see something like the image below:

In the main panel tables are listed such as “ArcGISConfigStore.SOMETHING”.  The “SOMETHING” is the namespace that you must use to restore the connection.

Local Repository Path
The local repository path is the last piece of information needed in the properties file. The local repository path is a folder on the machine where a local copy of the settings are stored.  The default locations for these folders are as follows:

• Windows: C:\arcgisserver\local
• Linux: /arcgis/server/usr/local

After the script is executed, the ArcGIS Server service must be restarted for the changes to take effect.

Workaround

Updated as of April 11, 2022: New patches have been released to prevent BUG-000148146 on some AWS (Amazon Web Services) deployments. The B patches will install over the top of the original patches if you have already installed the originals. If you did install the original patches, please follow the instructions in the tech article https://support.esri.com/en/Technical-Article/000027487 to ensure your system will not encounter a possible problem where ArcGIS Server machines are removed from a site after a machine restart. If you did not previously install the original patches and are only applying the "B" version, the issue should not exhibit itself. The new patches are available at:

ArcGIS Server 10.9.1 Log4j Patch  https://support.esri.com/en/download/7963

ArcGIS Server 10.9 Log4j Patch  https://support.esri.com/en/download/7976

ArcGIS Server 10.8.1 Log4j Patch  https://support.esri.com/en/download/7965

Steps to Reproduce