ArcSDE geodatabase upgrade for Oracle fails if the Geoprocessing upgrade script is executed using ArcGIS Server or Engine. Please see technical article Technical Article 37974 for more details.
You can use ArcCatalog to upgrade the geodatabase to version 10 or you can run the Geoprocessing script from a Windows platform that has either ArcGIS 10 Desktop, Server or Engine RT installed.For more information on upgrading please see the {LINK}TEXT=Upgrading an ArcSDE geodatabase in Oracle;URL=<a href="http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//002n000000p8000000.htm" target="_blank">http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//002n000000p8000000.htm</a>{/LINK} topic in the ArcGIS 10 Webhelp.