Cloning the arcgispro-py3 with deep learning libraries is successful in ArcGIS Pro 2.9.
The cause of the trouble is when the deep learning libraries are installed before ArcGIS Pro is upgraded. Uninstall the deep learning libraries before upgrading ArcGIS Pro. Refer to the 'Upgrading From a Previous Version' section at https://github.com/Esri/deep-learning-frameworks#installation.
If this issue persists at ArcGIS Pro 3.0, log a new case or re-open this issue with the details of the error produced. At ArcGIS Pro 3.0, the clone process in the application produces detailed log messages which can use to troubleshoot.
Another workaround is to not install the deep learning libraries into arcgispro-py3. Clone arcgispro-py3, activate the clone, then `conda install deep-learning-essentials` into the clone instead.