TigerBot and ChatGLM-6B large language model

Table of contents

1 TigerBot large language model

 1.1 Environment installation

1.2 Model download

1.2.1 hugging face website download

1.2.2 Baidu network disk download

1.3 Reasoning

2 ChatGLM large language model

2.1 Environment Construction

2.2 Model download

2.3 Reasoning

---

1 TigerBot large language model

"Hubo Technology" released the self-developed multi-modal large model TigerBot, open source model, code and data. Today, let's build an environment to experience his large model. Find it on github: GitHub - TigerResearch/TigerBot: TigerBot: A multi-language multi -task LLM

 1.1 Environment installation

conda create --name tigerbot python=3.8
conda activate tigerbot
conda install pytorch torchvision torchaudio pytorch-cuda=11.7 -c pytorch -c nvidia 

The cuda of my server is version 11.0. If I change it to pytorch-cuda=11.0, I find that this package cannot be found. So go to CUDA Toolkit 11.7 Downloads | NVIDIA Developer

Downloaded 11.7 cuda, and then installed the new version of cuda.

Then

git clone https://github.com/TigerResearch/TigerBot
cd TigerBot
pip install -r requirements.txt

1.2 Model download

1.2.1 hugging face website download

 Click the blue link on the left to go to the huggingface website.

 Then at this time you cannot directly

git clone https://huggingface.co/TigerResearch/tigerbot-7b-sft

This download will not contain large files. You must install lfs first and then download it.

curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
sudo apt-get install git-lfs
git lfs install
git clone https://huggingface.co/TigerResearch/tigerbot-7b-sft

1.2.2 Baidu network disk download

there is nothing more to say,

1.3 Reasoning

Use the following command for single-card reasoning

CUDA_VISIBLE_DEVICES=0 python infer.py --model_path ${MODEL_DIR}

Note that MODEL_DIR at the end is the directory, not the name of the model file. Just put the model folder you downloaded here.

CUDA_VISIBLE_DEVICES=0 python infer.py --model_path ./tigerbot-7b-sft

get the following result

I'll try the web interface again.

CUDA_VISIBLE_DEVICES=0 python ./apps/web_demo.py
loading model: tigerbot-7b-sft...
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:09<00:00,  4.73s/it]
Using the following device map for the model: {'': 0}
/root/anaconda3/envs/tigerbot_chw/lib/python3.8/site-packages/gradio/components/textbox.py:259: UserWarning: The `style` method is deprecated. Please set these arguments in the constructor instead.
  warnings.warn(
Running on local URL:  http://127.0.0.1:7860

Directly run CUDA_VISIBLE_DEVICES=0 python ./apps/web_demo.py and then use the company's intranet address to access it. You need to modify the code and change the last line of web_demo.py

demo.queue().launch( share=False,  inbrowser=True)

修改为

demo.queue().launch(server_name="0.0.0.0", share=False,  inbrowser=True)

即可。

2 ChatGLM large language model

2.1 Environment Construction

This is from Tsinghua University, go directly to github.

git clone https://github.com/THUDM/ChatGLM-6B
cd ChatGLM-6B
conda create -n chatglm_chw python=3.8
conda activate chatglm_chw
pip install -r requirements.txt

On the official website, just use pip install. I still create a conda environment first, and then install the dependencies.

2.2 Model download

git lfs install
git clone https://huggingface.co/THUDM/chatglm-6b

After a long wait, the download will be completed.

2.3 Reasoning

Because we downloaded the code ourselves, modify these two lines in cli_demo.py as follows

tokenizer = AutoTokenizer.from_pretrained("./chatglm-6b", trust_remote_code=True)
model = AutoModel.from_pretrained("./chatglm-6b", trust_remote_code=True).half().cuda()

Then python cli_demo.py can start the program and then the question can be entered 

references:

GitHub - TigerResearch/TigerBot: TigerBot: A multi-language multi-task LLM

Hugging Face——Large-scale pre-training model download_GeekZW’s blog-CSDN blog