For an ArcMap Service, a TARGETLAYER is a layer in which features are selecting based on a buffer of features belonging to a different layer.
When using the BUFFER project attribute, the attributes FEATURECOORDSYS and FILTERCOORDSYS can be used to define a new coordinate system.
When the coordinate system is changed, the coordinates of the TARGETLAYER result do not match the new coordinate system. the results remain in the original coordinate system.
Coordinates, or geometry, returned by ArcMap will always reflect the actual coordinate system of the features. This data does not change when the service is projected on the fly.
If this is so, why does an ArcMap Service that is projected on the fly display the correct image? Images reflect a change in the coordinate system because ArcMap Server performs the necessary coordinate transformation for image export in memory. No changes are made to the actual data.