Summary
Minigpt4, which claims to be able to reach 90% of chatgpt's capabilities, is open source! This article shows you how to deploy minigpt4, using 13B as an example!
download code
GitHub link:
https://github.com/Vision-CAIR/MiniGPT-4
Download it and unzip it. Then build the environment
Installation Environment
You can run it directly environment.ymlto create a virtual environment. Excuting an order:
conda env create name environment_name -f environment.yml
environment_name refers to the name of the virtual environment.
If you are worried that the pytorch version is not compatible with the local cuda version, you can manually create it yourself. This is the way I use:
conda create --name gpt python=3.9
Then, find the appropriate pytorch version, install it into the virtual environment created above, and execute the command:
conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.7 -c pytorch -c nvidia
Then, take out the pip part of environment.yml, create a new requirements.txt, and put it in. The complete content:
accelerate==0.16.0
aiohttp==3.8.4
aiosignal==1.3.1
async-timeout==4.0.2
attrs==22.2.0
bitsandbytes==0.37.0
cchardet==2.1.7
chardet==5.1.0
contourpy==1.0.7
cycler==0.11.0
filelock==3.9.0
fonttools==4.38.0
frozenlist==1.3.3
huggingface-hub==0.13.4
importlib-resources==5.12.0
kiwisolver==1.4.4
matplotlib==3.7.0
multidict==6.0.4
openai==0.27.0
packaging==23.0
psutil==5.9.4
pycocotools==2.0.6
pyparsing==3.0.9
python-dateutil==2.8.2
pyyaml==6.0
regex==2022.10.31
tokenizers==0.13.2
tqdm==4.64.1
transformers==4.28.0
timm==0.6.13
spacy==3.5.1
webdataset==0.2.48
scikit-learn==1.2.2
scipy==1.10.1
yarl==1.8.2
zipp==3.14.0
omegaconf==2.3.0
opencv-python==4.7.0.72
iopath==0.1.10
decord==0.6.0
tenacity==8.2.2
peft
pycocoevalcap
sentence-transformers
umap-learn
notebook
gradio==3.24.1
gradio-client==0.0.8
wandb
Try to build the environment according to the above version, otherwise it is prone to various strange problems.
download model
vicuna model https://huggingface.co/lmsys
vicuna model, it is recommended to download the v1.1 version. The dimensions of the old and new versions are inconsistent!
LLaMA：https://huggingface.co/decapoda-research
The LLaMA and vicuna models remain the same. If 13B is selected, both models use 13B.
model conversion
For model conversion, the required library FastChat, github link: https://github.com/lm-sys/FastChat
installation method:
git clone https://github.com/lm-sys/FastChat.git
cd FastChat
pip install --upgrade pip # enable PEP 660 support
pip install -e .
Create a new LLAMA folder in the root directory of FastChat, and put the downloaded llama model and icunb model under the LLAMA folder.
Execute the command after the installation is complete:
python -m fastchat.model.apply_delta --base LLAMA/llama-13b-hf --target LLAMA/vicuna-13b --delta LLAMA/vicuna-13b-delta-v1.1
The conversion is complete. Note: The conversion consumes a lot of memory, and the configuration requirements of the machine are relatively high.
After the conversion is complete, create a new weights folder in the root directory of MiniGPT4, and put the converted model folder into it
Download the minigpt4 weights file
There are two, one is 13B and the other is 7B.
| Checkpoint Aligned with Vicuna 13B | Checkpoint Aligned with Vicuna 7B |
|---|---|
| Download | Download |
| They are all on the Gongle network disk, there is helplessness, and someone has shared the weight file, the link: https://huggingface.co/wangrongsheng |
Specific link:
# 13B
https://huggingface.co/wangrongsheng/MiniGPT4/tree/main
# 7B
https://huggingface.co/wangrongsheng/MiniGPT4-7B/tree/main
Of course, there are also converted models, link:
# 13B
https://huggingface.co/wangrongsheng/MiniGPT-4-LLaMA/tree/main
#7B
https://huggingface.co/wangrongsheng/MiniGPT-4-LLaMA-7B/tree/main
Boss Rongsheng is so considerate! ! !
configuration
The path to the modified eval_configs/minigpt4_eval.yamlminigpt4.
ckpt: './weights/pretrained_minigpt4.pth'
Modify minigpt4/configs/models/minigpt4.yamlthe file , modify llama_model to the converted model directory.
run
python demo.py --cfg-path eval_configs/minigpt4_eval.yaml --gpu-id 0
question
RuntimeError: The size of tensor a (32000) must match the size of tensor b
(32001) at non-singleton dimension 0
Cause:
The fastchat version is not compatible with the model version
Solution:
replace vicuna-7b-delta-v0 with vicuna-7b-delta-v1.1 version,
It can be converted after changing to version 1.1.
Excuting an order:
python -m fastchat.model.apply_delta --base LLAMA/llama-13b-hf --target LLAMA/vicuna-13b --delta LLAMA/vicuna-13b-delta-v1.1
