Stable Diffusion Practical Example

1. Negative prompts

Solve the problem: The generated image has flaws, such as multiple eyes, multiple fingers, etc. Some commonly used bad results can be avoided through embeddings.

Method:  Download the embeddings model of negative prompt words from https://civitai.com/?utm_source=nettsz.com ,

EasyNegativeV2, easynegative, badhandv4 are all negative models for the second element,

NG_DeepNegative_V1_75T is a negative model for real people,

After downloading these embeddings models, put them in the sd root directory/embeddings, as shown in the following figure:

 The way used on webui:

2. Hi-Res Fix

Solve the problem: The pictures produced by the real model are partially blurred, without real-life texture, for example, the hands in the picture below are blurred.

Reason: The resolution is set too low, and the AI ​​does not have enough operating space to reflect the details.

HD repair operation:

Select the redraw range not greater than 0.5 to avoid changes in the screen structure.

Parameter selection reference:

Contrast effect: (left is before repair, right is after repair)

  

 You can see that the back of the right hand is more delicate.

Final version parameters:

prompt:1girl, detailed background filled with (many:1.1) (colorful:1.1) (flowers), (best best quality), (photorealistic:1.1), (resolution:1.1), (sharpness:1.1), (cinematic lighting), depth of field, Canan EOS R6, 135mm, 1/1250s, f/2.8, ISO 400, white cloth (lace trim:1.3), close-up, portrait, SFW, hands up, 

negative prompt:EasyNegativeV2, easynegative, badhandv4, NG_DeepNegative_V1_75T, child, lowres, worst quality, low quality, blurry,  fake, 3d, anime, bad anatomy,  disabled body, disgusting, ugly, text, watermark, four fingers, wrong fingers,  lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, artifacts, signature, watermark, username, blurry, missing arms, long neck, humpbacked, bad feet, nsfw, malformed limbs, poorly drawn hands, mutilated, more than 2 thighs, poorly drawn, more than 2 nipples, unclear eyes, missing legs, deformed, fused fingers, mutated hand and finger, bad anatomy disfigured malformed mutated, bad proportions, malformed mutated, three legs, poorly drawn face, bad hands, bad feet, cloned face, disfigured, poorly drawn, worstquality, mutated hands, bad anatomy, morbid, extra limbs, too many fingers, mutation, malformed, multiple breasts, bad face, extra legs, mutated, ugly

Random number seed: 3742559524

Model: majicmixRealistic_v4

3. Partial redrawing inPaint

Solve the problem: 99% of a picture is satisfactory, and the unsatisfactory 1% cannot be repaired only by prompt words. For example, if you have 6 fingers, no matter how you modify the prompt words, the problem may not be fixed.

1. The special effect of "close your eyes"

First, generate a picture through the prompt word, then click "Send to Redraw"

Then, the eye position is redrawn, and the prompt is added with "(closed eyes)". After clicking Generate, only the eyes are closed in the original image, and the parts outside the redrawing area have not changed.

2. The special effect of "replace flowers with butterflies"

First, add a mask to a certain flower, which is the redrawing area in the previous example.

Then, select " Redraw mask content ", the prompt only has "butterfly", and the corresponding effect will be achieved, as shown in the figure below:

ps: If the butterfly is added after the original prompt, it will be difficult to get the redrawing pattern of the butterfly.

3. Change the background

First, add a mask mask to the whole person

Then, select " Redraw non-masked content ", the prompt only has "seaside, sunlight," and the corresponding effect will be achieved, as shown in the following figure:

 4. Graffiti repainting

1. The difference from partial redrawing

Partial redrawing is to increase the mask mask, and then redraw the masked area (or non-masked area) through prompt words.

For graffiti redrawing, you can add the content of the picture through the color board, and then SD redraws through colors and prompt words.

The picture above shows the color palette only available for graffiti repainting.

2. Repair fingers

As shown in the picture above, there is a big problem with the finger part. The graffiti redrawing operation is as follows:

 First, paint the wall color through the eyedropper, then wipe it off with your hand

Then, draw 5 more fingers and enter prompt: (5fingers, detailed hand, high five:1.2).

There are still blemishes, but 5 fingers can now be seen.

5. ControlNet

1. OpenPose model transformation pose

Function: directly grasp the "lifeline" of the body posture presentation

step:

(1) Vincent graph or graph graph, get an initial image, and then fix the random value

(2) Enable ControlNet, select "OpenPose" as the control type, click the explosion button next to the preprocessor to see the preview result of the pose, as shown in the figure below:

 After clicking generate, you will get another picture of posture:

2. Outline drawing

Function: Obtain the approximate outline of a picture through algorithms such as canny, and then generate a picture based on this outline.

step:

(1) Select a picture in ControlNet, select "SoftEdge" as the control type, and you can get the preview results as follows:

We can download the previewed outline map for secondary editing to get a more satisfactory picture. The new line drawing needs to be processed in the drawing tool, and the position is placed from the middle of a picture to the lower 1/2. Prepare for the subsequent layout redrawing function.

 

(2) Through the picture generation function, you can put the car in a repair shop.

prompt:(highly detailed:1.2), ultra-detailed, best quality, (photorealistic:1.1), (resolution:1.1), (sharpness:1.1), (cinematic lighting), depth of field, Canan EOS R6, 135mm, 1/1250s, f/2.8, ISO 400, realistic, indoors,
tesla model y,

negative prompt:NG_DeepNegative_V1_75T, blurry,mutation poorly drawn, wort quality , poorly drawn, bad proportions, worstquality, poorly drawn,

六、SD upscale

Solution: Repair some defects in high definition, such as the generated car front and rear as shown in the figure below, there are problems, and the lines are not smooth

step:

(1) In the image generation function, load the image to be processed, the prompt retains the previous information, and the random number seed remains unchanged

prompt:(highly detailed:1.2), ultra-detailed, best quality, (photorealistic:1.1), (resolution:1.1), (sharpness:1.1), (cinematic lighting), depth of field, Canan EOS R6, 135mm, 1/1250s, f/2.8, ISO 400, realistic, indoors, tesla model y, 

negative prompt:NG_DeepNegative_V1_75T, blurry,mutation poorly drawn, wort quality , poorly drawn, bad proportions, worstquality, poorly drawn,

Adjust the redrawing range to less than 0.3 (such as 0.25), because it is necessary to keep the existing information as much as possible, and only repair a small part of the problem.

(2) ControlNet loads the above picture again, click the "send dimension to stable diffusion" function, and the width and height data in the redrawing dimension will be reset after clicking.

 After resetting, delete the pictures in ControlNet, and select "Tile" for the control type. We only need its resolution, and we don't need to use the corresponding picture. If the picture is not deleted, the following effects will result:

The configuration after deletion is shown in the figure below:

(3) Use SD upscale function

The parameter selection is as follows: select the R-ESRGAN 4x+ algorithm, and the model will be automatically downloaded for the first use

 If the block overlapping pixel width is selected to be 64, 64 needs to be added to the corresponding width and height in the image generation function, which becomes 832=768+64:

 Finally, the resulting image is as follows: 

The picture has changed from the original 721K to 2.3M, and some defects in the front and rear of the car have been improved a lot.

7. SadTalker voice + picture to generate virtual digital human

1. Installation

(1) Download SadTalker

github地址:GitHub - OpenTalker/SadTalker: [CVPR 2023] SadTalker:Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation

The code is placed under: sd root directory /extensions/

(2) Download the model

There is a script: scripts/download_models.sh, you can directly bash scripts/download_models.sh

Each file can be downloaded individually if it is too slow to perform on the server.

SadTalker 4 files are downloaded and placed in the sd root directory /extensions/SadTalker/checkpoints/

gfpgan4 files are downloaded and placed in the sd root directory //models/GFPGAN/

2. Use of SadTalker

You can refer to: stable diffusion An Ai plug-in makes the goddess become a chatterbox, and the beautiful photo becomes a virtual anchor in seconds! One step is very simple_哔哩哔哩_bilibili

(1) Generate wav voice

Use Baidu's text-to-speech technology: Baidu Smart Cloud - log in to https://console.bce.baidu.com/ai/?fromai=1#/ai/speech/overview/index

(2) SadTalker generates video

 The log information is as follows:

 

Guess you like

Origin blog.csdn.net/benben044/article/details/131198577