In ArcGIS Server, publishing a service returns the following error message:
Error: System.PublishingTools.GPServer: java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is: java.net.SocketException: Connection reset
The ArcGIS Server account does not have the appropriate permissions to access the required folders. This commonly occurs in environments that apply group policies through an Active Directory, which can automatically revoke account privileges based on the policy configured.
Warning: It is recommended that users backup their data before proceeding. If necessary, consult with the organization's IT administrator before making the following changes.
The following Microsoft TechNet document describes how to do this: How to Add a Computer Account to a Group.
For more information on the required permissions, refer to the following web help page: ArcGIS for Server: What permissions do I need to grant to the ArcGIS Server account?
Note: Another method to repair the ArcGIS Server installation is to re-run the ArcGIS Server installation file.