Author: Dr. Wu Zhuo Intel AI Evangelist
Liu Li Intel IoT Industry Innovation Ambassador
What is a stable diffusion model?
Stable Diffusion is stability.ai 's open source AI image generation model, which can input text and generate images. Stable Diffusion takes AI image generation to a whole new level and will revolutionize media creation.
Quoted from: https://github.com/Stability-AI/StableDiffusion
What is Rubik's Cube?
Rubik's Cube is a mini host that can be DIYed. It adopts a drawer design. Subsequent assembly, upgrade, and maintenance only need to plug in and out the module. By selecting the version of the computing module and matching different IO modules, a rich configuration can be formed to adapt to different scenarios. When the performance is not enough, you can upgrade the computing module to increase the computing power. When the IO interface does not match, you can replace the IO module to adjust the function without rebuilding the entire system
Based on OpenVINO, the inference acceleration of the Stable Diffusion model was realized on the Rubik's Cube, and the performance of drawing in 14.9 seconds (excluding model initialization) was obtained .
During image generation: CPU occupies 16%, memory occupies 7.5GB, GPU occupies 100%, and video memory utilization is 98.5%.
The test code is as follows, which is convenient for readers to reproduce on the Rubik’s Cube: openvino_notebooks/notebooks/225-stable-diffusion-text-to-image at main openvinotoolkit/openvino_notebooks GitHub .
Then, let's draw the key points for everyone, and let's take a look at the important steps and the effect of the generated paintings. In this code example, we take the Stable Diffusion model and convert the model to the OpenVINO Intermediate Representation (IR) format so that it can run efficiently on Intel® GPUs . Also, by compressing the FP32 model to FP16 , we cut the model size in half (nearly in half), and it now requires a lot less RAM/VRAM to run. Most importantly, due to the addition of Intel® Xe Matrix Extensions ( XMX ), GPU processing speed has also been significantly improved.
Here are some interesting results I got from running this notebook . With the Intel® Sharp™ A380 discrete graphics, I can achieve about 6.0 iterations per second (without using debug mode). This means it typically takes less than 10 seconds to generate one of the high-quality images below.
Figure 1 is based on OpenVINO Notebooks and Rubik's Cube (Intel Sharp™ A380), the results generated by the text-to-image function of the "Stable Diffusion" model.
After reading the results of the above AI painting, are you tempted and want to try it yourself? Next, let's take a look at the main steps you need to run our Stable Diffusion notebook code example on your own machine.
First, this is the repository for OpenVINO Notebooks. It has everything you need to make your presentation today.
openvino_notebooks/notebooks at main · openvinotoolkit/openvino_notebooks
Stable Diffusion is located under the 225-stable-diffusion-text-to-image folder
In the notebook code example, we not only introduced the famous text-to-image pipeline, but also added the image-to-image generation pipeline. But what does it really mean? How do we run it?
pipeline
how to install
To install OpenVINO Notebooks , you can follow the instructions here (for Windows ): Windows · openvinotoolkit/openvino_notebooks Wiki · GitHub
If you are a Linux user, you can click this link: Ubuntu · openvinotoolkit/openvino_notebooks Wiki · GitHub
In general, you can implement the following steps:
Install Python 3.10.x. ( or below) and create a virtual environment
python3 -m venv openvino_env
source openvino_env/bin/activate #for linuxImplement a Git clone for a directory
git clone --depth=1 https://github.com/openvinotoolkit/openvino_notebooks.git
cd openvino_notebooksInstall all libraries and dependencies
pip install -r requirements.txtRun Jupyter Notebook
jupyter lab notebooks
run all cells and wait =)
Now, look at the code. We realistically optimize the PyTorch pipeline and execute the code using OpenVINO .
The first download and conversion may take a while. Once done, you will have a set of IR files. For convenience, I have updated these pre-trained optimization models to huggingface ( https://huggingface.co/bes-dev/stable-diffusion-v1-4-openvino ), you can download and use directly
Now, if you are lucky enough to use an Intel RAW discrete graphics card, you can change the code of the device name in the image below to " GPU ". By default it uses " AUTO " and will automatically switch to the detected GPU .
let it run on the GPU
Automatic plugin. It uses the CPU first, then automatically switches to the GPU.
in conclusion
The Rubik's Cube is a small but powerful computing device, they are small in size, low in power consumption, rich in interfaces, and highly expandable, and are widely used in industries such as education, healthcare, robotics, automation, and the Internet of Things.
By combining Rubik's Cube with OpenVINO, various complex AI applications can be realized, including accelerated AI model inference calculations, real-time image and video processing, and low-latency communication between devices.
To evaluate the AI acceleration effect of OpenVINO on Rubik's Cube , OpenVINO Notebooks is undoubtedly the first choice. Welcome to download and install OpenVINO Notebooks on Rubik's Cube! Happy coding. https://github.com/openvinotoolkit/openvino_notebooks