Introduction
Artifactory supports PyTorch PyPI repository with the public remote registry URL: https://download.pytorch.org. We can resolve most packages per this article: How to configure PyTorch remote repo and resolve packages from Artifactory. But for some special PyTorch packages, we need to change the configuration. In this article, we will summarize some cases where PyTorch packages are not getting cached and their solutions.
Note:
This article assumes you already have a PyTorch remote repository configured in Artifactory. If you haven't set one up yet, start with How to Configure a PyTorch Remote Repository and Resolve Packages from Artifactory (add link) and then return to this guide. This article focuses on edge cases that the standard configuration does not address, including packages that redirect to files.pythonhosted.org and CUDA-specific package variants such as +cu118.
Causes
case 1:
Download URL is different from Registry URL, such as the sympy package. We can check the download URL in the PIP client output or the UI:
pip install sympy==1.13.1 --index-url https://download.pytorch.org/whl -vvv
...
Looking in indexes: https://download.pytorch.org/whl
1 location(s) to search for versions of sympy:
* https://download.pytorch.org/whl/sympy/
Fetching project page and analyzing links: https://download.pytorch.org/whl/sympy/
Getting page https://download.pytorch.org/whl/sympy/
Found index url https://download.pytorch.org/whl
Looking up "https://download.pytorch.org/whl/sympy/" in the cache
Request header has "max_age" as 0, cache bypassed
Starting new HTTPS connection (1): download.pytorch.org:443
https://download.pytorch.org:443 "GET /whl/sympy/ HTTP/1.1" 304 0
Found link https://files.pythonhosted.org/packages/3f/84/f26ac19caa7594383a6bba49f55e5d1d55e28e1544f34a0bba8f53fbd324/sympy-0.5.14.tar.gz#sha256=8672fb7b61677a7a6006d25ac5aaff97233bf1f6310415cdf17af7afecd0c1df (from https://download.pytorch.org/whl/sympy/), version: 0.5.14
We can see the Index URL is https://download.pytorch.org, and the download URL is https://files.pythonhosted.org
case 2:
The Index URL does not include the package version, such as torch==2.1.0+cu118. We can check the Registry URL in the PIP client or the UI:
root@18b1794b058b:~# pip3 install torch==2.1.0+cu118 --index-url https://download.pytorch.org/whl
Looking in indexes: https://download.pytorch.org/whl
ERROR: Could not find a version that satisfies the requirement torch==2.1.0+cu118 (from versions: 1.4.0+cpu, 1.5.0+cpu, 1.5.1+cpu, 1.6.0+cpu, 1.7.0+cpu, 1.7.1+cpu, 1.8.0+cpu, 1.8.1+cpu, 1.9.0+cpu, 1.9.1+cpu, 1.10.0+cpu, 1.10.1+cpu, 1.10.2+cpu, 1.11.0+cpu, 1.12.0+cpu, 1.12.1+cpu, 1.13.0+cpu, 1.13.1+cpu, 2.0.0+cpu, 2.0.1+cpu, 2.1.0+cpu, 2.1.1+cpu, 2.1.2+cpu, 2.2.0+cpu, 2.2.1+cpu, 2.2.2+cpu, 2.3.0+cpu, 2.3.1+cpu, 2.4.0+cpu, 2.4.1+cpu)
ERROR: No matching distribution found for torch==2.1.0+cu118
root@18b1794b058b:~# pip3 install torch==2.1.0+cu118 --index-url https://download.pytorch.org/whl/cu118/
Looking in indexes: https://download.pytorch.org/whl/cu118/
Collecting torch==2.1.0+cu118
Downloading https://download.pytorch.org/whl/cu118/torch-2.1.0%2Bcu118-cp38-cp38-linux_x86_64.whl (2325.9 MB)
|██ | 141.3 MB 41.2 MB/s eta 0:00:53^C
the 2.1.0+cu118 Registry URL is https://download.pytorch.org/whl/cu118/, not the https://download.pytorch.org/whl
Resolution
For case 1:
We need to change the Download URL to https://files.pythonhosted.org:
For case 2:
We need to change the Registry URL to https://download.pytorch.org/whl/cu118/ and set the Registry Suffix to empty:
Or we can set the Registry URL to https://download.pytorch.org/ and set the Registry Suffix: whl/cu118
