Upload and download huggingface warehouse files (models, data, etc.)

download

For example, want to download the llama-13b model file from huggingface hub to local:

You can use the following command, local_dirwhich is the local folder you want to download to:

from huggingface_hub import snapshot_download
snapshot_download(repo_id="decapoda-research/llama-13b-hf",cache_dir="./cache", local_dir="./model_weights/llama-13b-hf")

The above command is equivalent to git clonemore parameters, such as filtering, specifying files, establishing symbolic links, etc., see the official website tutorial for details:

• Download files from the Hub

upload

If you want to upload files, such as model weights, you must first find your own huggingface hub access code:

Then set your own access code to your own machine:

pip install huggingface_hub  ## 如果没有安装huggingface_hub库
python -c "from huggingface_hub.hf_api import HfFolder; HfFolder.save_token('MY_HUGGINGFACE_TOKEN_HERE')"

Finally, use the following command to upload the relevant files (model, tokenizer) (equivalent to git push).

pt_model.push_to_hub("my-awesome-model")
tokenizer.push_to_hub("my-awesome-model")