When initially getting involved in Stable Diffusion, you may be overwhelmed by various new concepts and models. For example, as new mages, we need to understand how to use all kinds of mage equipment—that is, these models—in order to better deal with problems and maximize the effect.
To understand a safetensorsconcept called . This is the name of many model suffixes, even though the model suffixes are varied, but safetensorsalways occupies a place. This is because of safetensorsthe support for various AI models, each of which had its own unique suffix before it appeared. So each model can be used either safetensors, or with its own suffix, which can be confusing for newcomers when getting started.
In fact, safetensorsit is an open source model format developed by huggingface. It has many advantages: it can effectively prevent DOS attacks, fast loading speed, support lazy loading and high versatility. Therefore, most open source models are now available in 'safetensors' format.
Article Directory
Model introduction
- Checkpoint｜Stable Diffusion : Checkpoint is the most important model in Stable Diffusion and is also regarded as the main model. Almost all operations need to rely on it. All main models are trained based on the Stable Diffusion model and are therefore sometimes referred to as Stable Diffusion models.
- LoRA and LyCORIS : These two models are fine-tuned models, which are usually used to control the style, generated characters, and character poses.
- Textual Inversion : This is a text encoder model for changing textual vectors, which can be understood as a set of prompts.
- Hypernetworks : The Hypernetworks model is used to adjust the model neural network weights and fine-tune the style.
- ControlNet : This is a very powerful control model that can perform picture control, motion control, color depth control, color control, etc.
- VAE : The VAE model is usually used to correct the brightness and saturation of the picture, perform picture correction and fill light, etc.
- CodeFormer : This is an inpainting model, usually used to increase image resolution, modify black and white photos to color photos, face restoration, etc.
list table
Control the main style of the picture through Checkpoint in Stable Diffusion; adjust the fill light and brightness of the picture through VAE; control the style and character of the model through LoRA and LyCORIS; simplify Prompt through Textual Inversion; control pose, color and repair through ControlNet .
| model name | effect | suffix | size | Folders in WebUI |
|---|---|---|---|---|
| Checkpoint | main model | .ckpt or .safetensors | 2G - 7G | models/Stable-diffusion |
| LoRA 和 Lycoris | Fine-tuning the model, generally used to control the style of painting, control the generated character, control the pose of the character, etc. | .safetensors | 2G - 7G | models/Stable-diffusion |
| Textual Inversion | Text Encoder Model | .pt or .safetensors | KB level | embeddings |
| Hypernetworks | Adjust the model neural network weights to fine-tune the style | .pt or .safetensors | 20M - 200M | models/hypernetworks |
| ControlNet | Powerful control model, which can be used for image control, motion control, color depth control, color control, etc. | .safetensors | KB level | models/ControlNet |
| FEET | Image brightness and saturation correction, image correction and fill light, etc. | .pt or .safetensors | 335M or 823M | models/VAE |
| CodeFormer | Repair models, repair faces, improve resolution, etc. | - | - | models/codeformer |
model use
Just put the corresponding model in the corresponding folder.
Click on the interface of Vincent graph and graph graph Show/hide extra networks.