[linux] How to download huggingface transformers model to local & git lfs install error

bert-base-uncased at main

1. The official order:

# Make sure you have git-lfs installed (https://git-lfs.com)
git lfs install
git clone https://huggingface.co/bert-base-uncased



# if you want to clone without large files – just their pointers
# prepend your git clone with the following env var:
GIT_LFS_SKIP_SMUDGE=1

But an error was reported when git lfs install.

So I checked it out. . . . Install lfs like this: 

Two, install lfs

You can't directly use

git lfs install

Instead of that, you can use these commands to download and install (you have to download it before installing).

sudo apt-get install git-lfs
git-lfs install

Third, use the official command to download.

If it fails, then . . . It may be that the network is too bad. . . .

In the end, I still set a loop for from_pretrained to solve it. . . . . . .

If there is no loop, an error will be reported " requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(104, 'Connection reset by peer'))" . . . . . . . . . . . . . . . . . . . . .

class Trainer(object):
    def __init__(self, args, train_dataset=None, dev_dataset=None, test_dataset=None):
        self.args = args
        self.train_dataset = train_dataset
        self.dev_dataset = dev_dataset
        self.test_dataset = test_dataset

        self.intent_label_lst = get_intent_labels(args)
        self.slot_label_lst = get_slot_labels(args)
        # Use cross entropy ignore index as padding label id so that only real label ids contribute to the loss later
        self.pad_token_label_id = args.ignore_index

        self.config_class, self.model_class, _ = MODEL_CLASSES[args.model_type]
        #self.config = self.config_class.from_pretrained(args.model_name_or_path, finetuning_task=args.task, output_hidden_states=args.output_hidden_states)
        self.config = self.config_class.from_pretrained(args.model_name_or_path, finetuning_task=args.task)
        ################## [O.O]这是一个循环，解决下不下来模型的问题 #################
        nb_tries = 20
        while nb_tries>0:
            nb_tries -= 1
            try:
                self.model = self.model_class.from_pretrained(args.model_name_or_path,
                                                      config=self.config,
                                                      args=args,
                                                      intent_label_lst=self.intent_label_lst,
                                                      slot_label_lst=self.slot_label_lst)
                break
            except:
                time.sleep(0.1)
        #########################################################################
        # GPU or CPU
        self.device = "cuda" if torch.cuda.is_available() else "cpu"
        # self.device = "cuda" if torch.cuda.is_available() and not args.no_cuda else "cpu"
        self.model.to(self.device)