I. Introduction
On July 26, Stability AI released SDXL 1.0, which is claimed to be the most powerful open image generation large model so far.
Is it as explosive as the Internet says? Have you really achieved the function of making pictures by mouth like midjourney ? Compared with previous versions, what new features have been added?
I tried it today, let’s take a look!
2. New features
1. Clear text generation
As we all know, the current painting tools are not particularly mature in supporting text generation. Although there are some tools that can generate text, the results are always unsatisfactory. SDXL 1.0 takes another step forward in text generation.
For example, with just one prompt word: a cute cat holds a paper with text "cool",professional photography, a kitten holding a piece of paper with the word "cool" written on it can be generated.
2. Better understanding of human body structure
The previous Stable Diffusino model had obvious problems in generating correct human body structures. For example, they often develop redundant or incomplete limbs, extremely deformed faces, etc. SDXL 1.0 solves this problem to some extent.
We all know that AI is not good at drawing hands. For example, in order to generate a normal hand in the past, a lot of negative prompts, Embedding, or plug-ins such as OpenPose were used. However, in SDXL 1.0, all of these are no longer needed, or more To be exact, even without these, the generated characters are much better than the previous version.
3. The degree of freedom is greatly improved
Before SDXL 1.0, if you wanted to generate images of different styles, you had to change the large model or download the corresponding LoRA model. However, in SDXL 1.0, you can seamlessly switch between more than ten styles through prompt words, including animation. , photography, digital illustration and more.
4. Shorter prompt words
In terms of prompt words, the spells have become shorter and simpler, and the understanding of natural language has been enhanced, greatly lowering the threshold for us to write prompt words.
- The previous prompt words were lexical descriptions composed of words, words, commas and other symbols, but in SDXL 1.0, they can be described directly using natural language (such as a whole sentence).
- When describing the previous prompt words, in addition to writing prompt words such as the generated subject, scene, and ambient light, you also need to add image quality prompt words such as masterpiece, best quality, highres, and a large number of negative prompt words. In SDXL 1.0, these quality prompt words and negative prompt words no longer need to be written.
- The understanding of some concepts and the restoration of the environmental atmosphere are better. For the understanding of concepts, the official website gives an example: For example, for the two concepts "The Red Square" (a famous attraction) and "red square" (a shape), SDXL 1.0 can already distinguish them.
5. Support photos with larger size and precision
Friends who have experience in Stable Diffusion picture production know that if you directly generate a large picture with a resolution of 1024 x 1024 or higher, there may be multiple people with multiple heads, limb misalignment, etc., and you need to use high-definition restoration or Tiled Diffusion and other other methods. method can be achieved.
Now, you can directly produce large images with a resolution of 1024 x 1024 or higher without any problems.
6. Color vividness and accuracy
SDXL1.0 has made great improvements in the vividness and accuracy of colors. Compared with the previous version, the contrast, light and shadow are more realistic.
3. How to experience SDXL 1.0
1、Remote AI
If it’s just for experience, I recommend the simplest and fastest online method: Liblib AI.
Liblib AI produces pictures online and can produce 300 pictures a day for free, which basically meets the needs of most students.
ps: The disadvantage of Liblib AI online experience is that there are few plug-ins, and the drawing may be stuck during peak periods. A simple experience is still possible. If you want an in-depth experience, you still need to deploy Stable Diffusion in the cloud or deploy Stable Diffusion locally.
The steps to experience Liblib AI are very simple, just follow the instructions and you’ll be done in 5 minutes!
- In Liblib AI, click "Online Stable Diffusion".
- Model Select the model with "SDXL" and enter a simple prompt word.
- parameter settings.
If you don’t know which parameter is best for drawing, you can copy it directly from the picture.
Sampling method : DPM++ 2M Karras (or other)
Number of sampling steps : 30
Many times before, we usually set the number of sampling steps to 20, but in SDXL, if the number of sampling steps is set to 20, it will make people feel that the picture is not fine enough. Therefore, the number of sampling steps can be appropriately increased.
Resolution : 1024 x 1024 or other resolutions. Too low may affect the image quality.
Other parameters : You can leave them as default.
- Click "Generate Image".
A picture generated using SDXL 1.0 comes out. Isn’t it very simple?
2. Local deployment
Warm reminder: You need to update the local Stable Diffusion WebUI to 1.5.1 first.
If Stable Diffusion WebUI has not been deployed locally before,
Mac computers can refer to this article: Mac local deployment of Stable Diffusion (super detailed, including pitfalls)
Windows computers: Just use Akiye Boss’s one-click deployment and installation.
If Stable Diffusion WebUI has been installed and deployed locally, directly download the following two SDXL 1.0 models and place them in the /models/Stable-diffusion directory of the root directory of SDW.
SDXL 1.0 refiner model download
Here you can find that SDXL 1.0 has two models, a base model and a refiner model. When using it, first generate the graph through the base model, then select "Send to graph to generate graph", and use the refiner model for optimization.
The second step is to use the refiner model for optimization, which is actually equivalent to low-level redrawing of the image to improve the image quality (the redrawing rate here should not be set too high, such as 0.2 or 0.3 is ok, or it can be According to your own needs).
Of course, you can also skip the second step and just use the base model to generate images.
For the drawing method and specific parameter settings, refer to the method of Liblib AI above, so I won’t go into details here.
3. Officially provided methods
Stability AI officially also provides several experience methods :
4. Current issues
After talking for a long time, SDXL 1.0 is so powerful, is it really invulnerable? of course not!
1. Some old models, LoRA models and ControlNet are not currently supported.
For example, most of the old models, LoRA models and ControlNet will be invalid when used on SDXL 1.0 because they are not currently supported and need to be updated to adapt to SDXL 1.0.
In addition, SDXL 1.0 is just a basic large model, just like the previous SD 1.4 and SD 1.5. When we draw daily, we often do not use these officially provided basic models, but use these basic models for training, fine-tuning, Specific model after fusion.
For example, if we want to draw two-dimensional style pictures, we will choose two-dimensional style large models such as Cetus-Mix, Counterfeit, AbyssOrangeMix, etc., instead of using the official basic large model.
If we want to draw realistic-style pictures, we will choose large models such as Deliberate, LOFI, Realistic Vision, etc., and we will not use the official basic large models.
After the advent of SDXL, many large models based on SDXL training, fine-tuning, and fusion will also emerge. Currently, there are models based on SDXL training on platforms such as Liblib, and you can try them out. For our future drawings, there is a high probability that we will use these models based on basic large model training, fine-tuning, and fusion, instead of the currently officially provided SDXL model.
In other words, SDXL 1.0 is currently only a transitional product. Those who are interested can try it out and learn some knowledge in advance, but it may take some time to use it in large-scale production.
2. Too much video memory consumption
Compared with the previous Stable Diffusion version, it obviously consumes more video memory. It is officially recommended to run on a graphics card with more than 8G memory or on a cloud platform .
5. Summary
The biggest benefit that SDXL 1.0 brings to us is that we can basically produce pictures by mouth , making the drawing method simpler .
Newbies can use natural language to easily draw pictures without having any knowledge of complex prompt word structures, LoRA, Embedding, extension plug-ins, etc., which greatly lowers the threshold for use .
There will definitely be a batch of large models trained based on SDXL 1.0 that will emerge in the future, and they will definitely be even more amazing. Let’s look forward to it!