I. Introduction
This chapter is the fourth part of the series "Stable Diffusion from Entry-level to Enterprise-level Practical Practice", the fourth part of the advanced capability chapter "Stable Diffusion ControlNet v1.1 Precise Image Control", Section 03, using the Stable Diffusion ControlNet Canny model to accurately control image generation. The content of this part is located in the entire Stable Diffusion ecosystem as shown in the yellow part of the figure below:
2. Definition
ControlNet v1.1 provides a total of 14 functional models, and each model corresponds to an applicable business scenario. The specific model information is shown in the figure below:
This article introduces the ControlNet Canny model. The Canny edge detection algorithm is an algorithm for extracting edge information in images. It was developed by John F. Canny in 1986 and is one of the most widely used edge detection algorithms. Stable diffusion combines canny edge detection, an innovative attempt to combine the classic edge detection algorithm with AI generation, which can achieve better generation results and effectively expand the
application scope of stable diffusion.
3. Work flow
The workflow using ControlNet Openpose is shown in the figure below:
The complete workflow description is as follows:
- Canny edge detection is performed on the input image to obtain the edge image.
- The edge line part in the edge image is used as a mask, and combined with the non-edge part in the original image to obtain the edge mask image of the input image.
- The combined edge mask graph is used as one of the conditions and input into the stable diffusion model.
- Stable diffusion generates new images containing corresponding edge features based on edge information and other text descriptions.
- You can create multiple masks with different edge styles, input them into stable diffusion, and generate images with these edge styles.
4. Creative achievements
Utilizing ControlNet Canny technology, through attitude detection, the effect of precise image control is shown in the following figure:
The target building and the source building,have the same edge characteristics. First, identify the edge features of the source image, and then generate the target image based on the edge features.
5. Creative process
5.1 Working steps
The entire creative process can be divided into 4 steps, as shown in the figure below:
Environment deployment: Start ControlNet Canny WebUI service;
Model download: Download the ControlNet Canny model;
Practical operation: selecting inputs, configuring parameters and debugging;
Run the demo: show the effect of image generation;
5.2 Environment deployment
In order to reduce the impact of integrated encapsulation on our understanding of the underlying implementation, we use ControlNet v1.1 native framework deployment instead of integrated visual interface environment. The specific ControlNet Openpose service program is shown in the figure below. We only need to start the program:
5.3 Model download
There are two main ControlNet v1.1 canny pre-training models, as shown in the figure below:
5.4 Practical operation
Because it is a visual operation interface, you can understand the operation methods and processes at a glance. We will not go into details again and go directly to the configuration interface, as shown in the figure below:
5.5 Running the demo
6. Summary
This chapter is the fourth part of the series "Stable Diffusion from Entry-level to Enterprise-level Practical Practice", the fourth part of the advanced capability chapter "Stable Diffusion ControlNet v1.1 Precise Image Control", Section 03, using the Stable Diffusion ControlNet Canny model to accurately control image generation. In the next section, we will show you how to accurately control image generation using Stable Diffusion ControlNet Depth depth information.