Personal website: https://tianfeng.space
Preface
I don’t know what to post recently, so I’ve summarized the usage of controlnet. If you know all the usage of controlnet, you may understand it but it’s a bit vague. I hope it will be useful to you.
1. SD controlnet
I unify other parameters and only switch the controlnet model to demonstrate the effect for you.
Forward prompt words: (just some basic character prompt words for convenience and trouble-free, please reverse the prompt words according to the picture in the actual scene)
best quality,masterpiece,4k,1girl,solo,long hair,dress,jewelry,choker,brown hair,realistic,necklace,brown eyes,black dress,lips,<lora:tianfeng_cutegirls4:0.8>,
Negative prompt words:
ng_deepnegative_v1_75t,(badhandv4:1.2),(worst quality:2),(low quality:2),(normal quality:2),lowres,bad anatomy,bad hands,((monochrome)),((grayscale)) watermark,moles,large breast,big breast,sunshine,bright,
intermediate parameters
1.Canny
It can be preprocessed into line drawings based on the pictures you provide, and then the image diffusion can be controlled through the canny controlnet model. The weight and guidance timing can be adjusted to control the proportion of the line drawings. It is suitable for scenes where the line drawings of people, cars, animals, etc. need to be extracted for reconstruction. painted
2.depth
Depth controlnet is mainly used to highlight the foreground, background and spatial relationship of the image. The foreground has a large proportion and the background is shallow (prompt words plus space, etc.). It is suitable for use scenarios where there is a need for spatial depth.
There are several preprocessors for depth. You can try them all. The difference between them can be summarized as different proportions of the foreground and background. The main character is a little highlighted, while the background has a smaller proportion. Some of them give the main and background a proper balance.
3.Normalmap
Normal maps are generally used in 3D sculptures to bring information about 3D depth sculptures. The usage scenarios are generally 3D sculptures. My example pictures may not be appropriate. You probably know what to do.
4.openpose
Extract pose information based on the provided image, and control the pose image you want to generate in a style. dw_openpose is recommended.
5.MLSD
It has always been able to only recognize straight lines, which can be used to extract line drafts for interior design, and then redraw them for one-click decoration.
Prompt words:
qingshe,living room,HDR,UHD,8K,best quality,realistic,masterpiece,Highly detailed,light,lamp,Fuji_Mountain out of window,
6.Lineart
Lineart preprocessors are respectively targeted at realism and animation. You can choose a preprocessor according to your own needs. The effect is better than canny.
7.softedge
Soft edge detection is different from canny and lineart. It uses a softer line information that is consistent with the real scene and is more consistent with the original image.
8.scribble
Are you still worried about the ugly drawings? Upload a simple hand-drawn graffiti to generate a cool picture, for example, upload a super ugly sports car graffiti.
9.segmentation
Segment the objects that exist in the picture to ensure that the redrawn objects are highly reducible. At the same time, you can add color blocks to the pre-processed pictures. Segmentation has a correspondence table between objects and color blocks. For example, draw a table, paint The corresponding color on the top can be drawn with controlnet to generate a table more accurately.
10.shuffle
For style fusion, you can upload a picture with a style such as ink, oil painting, or science and technology. After the preprocessor passes through controlnet, it will be fused with the original model style to generate a picture with a fusion style.
11.Tile
There are many ways to use tiles. As far as usage is concerned, it is currently the controlnet model that I feel the strongest about, especially in terms of restoring the picture style. Because it is resampled in blocks, it is basically a perfect reproduction. At the same time, the details of the high-definition repair effect are far beyond the Ordinary amplification algorithm
12.inpaint (partial redraw)
Similar to the partial redrawing of a picture, use a brush to paint the parts that need to be redrawn on the original picture, and then add the corresponding prompt words, such as changing clothes.
13.instructP2P
pix2pix is mainly used for style transfer, such as weather conversion
14.reference only
You can add prompt words to regenerate a picture of the same style based on the picture you uploaded, such as a girl turning sideways, new prompt words are added, and the body is facing the audience.
15. Recolor
Mainly used for coloring some old photos, or black and white pictures, both can be used
16.Revision
For some pictures with too many complex elements, the style transfer cannot be carried out very well. In the past, prompt word inference may have been used, but it still could not perfectly contain the details of the original picture. You can use Revision to cooperate with the original picture for style transfer.
17.IP_adapter
It is perfect for style transfer. It is recommended to use it first, not only the style, but also the original image, which basically includes all the content.
That’s almost it! ! ! Let me tell you what I personally use often:
Basically Lineart, softedge, tile, ip_adapter, dw_openpose