Unable to add a portal connection in ArcGIS Pro through a firewall when the URL is accessible via a web browser

Description

Attempting to add a Portal connection in ArcGIS Pro results in an error similar to the following:

"Unable to add portal. Please make sure that the specified portal is up, running and the URL is correct"

However the Portal URL in question can be accessed without issue in a web browser on the same machine where ArcGIS Pro is installed. Machines on the network, which don't have to go through the firewall to access the Portal URL, can successfully add it in ArcGIS Pro.

Cause

When attempting to add a Portal connection in ArcGIS Pro, it makes a test http header request to see if the Portal is accessible (as of Pro 3.4.x where this issue was encountered). This uses different request headers than when accessing the Portal in a web browser. If this fails due to rules, filtering, etc., on the web application firewall or load balancer side, an error is received when adding the URL as a Portal connection.

You can use a tool like Fiddler to capture network traffic and see if there are any requests failing with something like a 504 gateway timeout error. Note that in this case - using Fiddler actually avoided the issue since it meant there was a body in the request response but you could still see an error was being returned.

Based on what you observe in Fiddler, you can construct a similar request in PowerShell from the ArcGIS Pro machine using something like the following example:

Invoke-WebRequest 'https://<portal url>/portal/sharing/rest/portals/self?f=json' -Method Head -Headers @{'Cache-Control'='no-cache';'User-Agent'='ArcGIS Pro 3.4.0 (00000000000) - Testing ArcGIS for Portal authentication';'Accept'='*/*';'Accept-Encoding'='gzip, deflate'}

If this fails, this test can be shared with firewall/load balancer admins to track down the actual source of the problem on their side. You can run the same request against a Portal that can be added successfully for comparison. 

Solution or Workaround

In this particular situation, using the above test, IT was able to narrow down the solution which involved enabling a setting called "Disable compression support" on the Sophos Firewall.

Without this enabled, the 'Accept-Encoding'='gzip, deflate' headers cause the request to fail.