ARTIFACTORY: How to download private/gated models from Artifactory using HuggingFace Repositories

Introduction 

The integration of Artifactory with Hugging Face provides a streamlined approach for managing machine learning models, aligning ML/AI development processes with your existing software supply chain (SSC). By utilising Artifactory, you can seamlessly incorporate machine learning into your technology stack.

This Knowledge Base article offers a step-by-step guide to configuring Hugging Face Remote Repositories in Artifactory leveraging the Hugging Face SDK, and efficiently downloading the private/gated models.

What is an access-gated model as a user?

As a user, if you want to use a gated model, you will need to request access to it. This means that you must be logged in to a Hugging Face user account.

Requesting access can only be done from your browser. Go to the model on the Hub and you will be prompted to share your information.

Once the access request is sent, there are two possibilities. If the approval mechanism is automatic, you immediately get access to the model files. 
Otherwise, the requests have to be approved manually by the authors, which can take more time.

To download files from a gated model you’ll need to be authenticated. In the browser, this is automatic as long as you are logged in with your account. 
If you are using a script, you will need to provide a user token. In the Hugging Face Python ecosystem (transformers, diffusers, datasets, etc.).

For more information kindly refer to this Official documentation from HuggingFace.

Pre-requisites before downloading the model

1. Resolving models via private repositories is only possible from Artifactory 7.77.x versions.
2. Make sure the huggingface cli is downloaded. If not, we can run pip3 install transformers command and it will download all the necessary dependencies.
3. The user should have an account on https://huggingface.co.
4. Have a HuggingFace Remote Repository configured and it should point to https://huggingface.co

 

Generating Access Token on HuggingFace

1. Open the https://huggingface.co in the browser and log in as a valid user.
2. Now, navigate to the Profile section(Top Right) and select the Settings option, or can directly select the Access Tokens option
        
3. In Settings, select Access Tokens and click on Create new token.
   
    
4. Now, click on the Read Only option give a name to the token, and click Create.
        
    
5. This will generate the token. Keep it safe as we need to use it in the Artifactory Remote Repository.

 

Downloading of a model

In order to download a private model from Artifactory. Below are the steps in detail.

1. Select the remote repository by navigating to Administration -> Repositories -> Remote Repository -> Select Repository.
2. Enter the token in the Password / Access Token field and Save it.
3. Now, To configure the Hugging Face SDK to work with Artifactory:
   > Go to the JFrog Platform Artifacts page, select your Hugging Face repository, and click Set Me Up.
   > To create a Hugging Face token, enter your JFrog Platform password, and click Generate Token & Create Instructions.
   > Connect your HuggingFaceML repository to the Hugging Face SDK in Artifactory using the following command:

   export HF_HUB_ETAG_TIMEOUT=86400
   export HF_HUB_DOWNLOAD_TIMEOUT=86400 
   export HF_ENDPOINT=https://<YOUR_JFROG_DOMAIN>/artifactory/api/huggingfaceml/<REPOSITORY_NAME>
   For ex:-
   export HF_HUB_ETAG_TIMEOUT=86400
   export HF_HUB_DOWNLOAD_TIMEOUT=86400
   export HF_ENDPOINT=https://myArtifactory/artifactory/api/huggingfaceml/testing-vj-hf-remote

4. Now, create a new .py file (download.py) and add the download command generated from Artifactory(Resolve section)

   from huggingface_hub import snapshot_download
   snapshot_download(
       repo_id="<model_name>", revision="<model_revision>", etag_timeout=86400
   )

   
   Here, replace <model_name> and <model_revision> with the private model and the revision number that needs to be downloaded. Below is the sample py file

   snapshot_download(    repo_id="bigcode/starcoder",revision="6bd835911561b927265e9e2c70ecf32521518c0f", etag_timeout=86400
   )

   
    
5. Now execute the .py file with the following command(python3 download.py). The model download will fail. Below is the sample output from the client   

    requests.exceptions.HTTPError: 404 Client Error:  for url: https://myArtifactory.jfrog.io/artifactory/api/huggingfaceml/testing-vj-hf-remote/mistralai/Mixtral-8x7B-Instruct-v0.1/resolve/2f41fd905c1cb30e991432ead7a4cd324f8a99a0/.gitattributes 

   
    
6. As it failed with a 404 error, the model was not found. The reason behind this is the user does not have access to the model.
    
7. Now, we need to navigate to huggingface upstream and search for the package. Below is the UI we will see for this model
   
    
8. Hence, we need to agree the Terms and Condition in order to access this model. Once this is done, we will run the py file again and this time the model will be downloaded.
   Below is the sample output from the client.

   > python3 priv-model.py
   (…)b30e991432ead7a4cd324f8a99a0/config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 720/720 [00:00<00:00, 1.92MB/s]
   (…)e991432ead7a4cd324f8a99a0/.gitattributes: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.52k/1.52k [00:00<00:00, 10.9MB/s]
   Fetching 36 files:   3%|████▊                                                                                                                                                                       | 1/36 [00:02<01:16,  2.19s/it]

Therefore, now the private models can be downloaded from the upstream.

For more details on HuggingFace Repositries, kindly refer to the below documentation.

• How to Create HuggingFace Repostories in Artifactory
• How to Configure the HuggingFace SDK to work with Artifactory