Background of the project

On April 19, 2023, the open source project MiniGPT-4 was released, which was developed by KAUST (King Abdullah University of Science and Technology, Saudi Arabia) and several Ph.D.
Project address: https://github.com/Vision-CAIR/MiniGPT-4
MiniGPT-4 can provide image understanding and dialogue capabilities similar to GPT-4.
Online experience: https://minigpt-4.github.io/The
insert image description here
team integrated the image encoder with the open source language model Vicuna (little alpaca), and frozen most of the parameters of the two, only a small part of training is required.

deployment process

Environment configuration and file preparation

1. Download the project to the local, configure the environment

git clone https://github.com/Vision-CAIR/MiniGPT-4.git
cd MiniGPT-4
conda env create -f environment.yml
conda activate minigpt4

2. Prepare pretrained Vicuna weights
Here we use the git command to download

First download and install git-lfs (git-lfs supports downloading large files and binary files): installation reference https://www.cnblogs.com/allmignt/p/12353756.html
Then use the git command to download the weight file, download the weight of 7b here.

# 先进入指定的路径下 
cd /data/sim_chatgpt/MiniGPT-4
# 进行下载
git clone https://huggingface.co/lmsys/vicuna-7b-delta-v0

3. Modify the model path of the configuration file
In minigpt4/configs/models/minigpt4.yaml, line 16 modifies the path of the weight just downloaded, I put it here in
/data/sim_chatgpt/MiniGPT-4/vicuna_weights/prerained_minigpt4_7b.pth

4. Prepare pretrained MiniGPT-4 checkpoint

Download address: https://drive.google.com/file/d/1RY9jV0dyqLX-o38LrumkKRh6Jtaop58R/view?usp=sharing
In the eval_configs/minigpt4_eval.yaml file, line 11 modifies the path of the weight just downloaded, and I put it here in
/students /julyedu_522454/MiniGPT-4/pretrained/ckpt/

deployment reasoning

I have a single GPU here, so --gpu-id is set to 0

python demo.py --cfg-path eval_configs/minigpt4_eval.yaml  --gpu-id 0

error 1

name ‘cuda_setup’ is not defined

Solution
pip install -U bitsandbytes
Reference: https://github.com/Vision-CAIR/MiniGPT-4/issues/117

error 2

insert image description here

Could not create share link. Please check your internet connection or our status page: https://status.gradio.app

Solution (unresolved, refer to: https://github.com/Vision-CAIR/MiniGPT-4/issues/99 )
Attempt:
modify the last line of code and add the server IP, as follows: demo.launch(server_name ="xx.xxx.xxx.xxx", share=True, enable_queue=True)
unresolved, to be updated...

pip install gradio==3.13.0

Test whether gradio is available:

import gradio as gr
def greet(name):
    return "Hello " + name + "!"
demo = gr.Interface(fn=greet, inputs="text", outputs="text")
demo.launch(share=True)

MiniGPT-4 deployment process

Article directory