Stable Diffusion Accurately draw character movements and details of hands and feet (ControlNet extension required)

At present, AI drawing is still more like opening a blind box. Although it can provide stunning pictures based on semantics, it still has a certain distance from the goal conceived by the designer.
So programmers like us who have no drawing skills want to create the poses of characters (especially hands and feet) in their hearts, how to reduce the number of random box openings?
This article provides several methods to precisely control the movement of characters.

• Installation and problem solving reference: " Windows installation Stable Diffusion WebUI and problem solving records ";
• Problems when running and using " Sorting out various problems encountered when Windows uses Stable Diffusion ";
• Model application and parameters " Stable Diffusion Personally Recommended Various Models, Setting Parameters, Extended Applications, etc. Collection ";
• Prompt Words Create Spells " AI Drawing Prompt Words/Spells/Affixes/Keywords Usage Guide (Stable Diffusion Prompt Designer Operation Manual) ";
• Different types of models Models description " Analysis of different types of StableDiffusion model Models ";
• Draw character movements and details of hands and feet " Stable Diffusion Accurately draw character movements and details of hands and feet (ControlNet extension required) ";
• Comparison of various styles and practical application " AI drawing style comparison table / detailed research records and experience summary of drawing style samples ";

ControlNetNote: Please confirm that extensions and plug-ins have been installed in your Stable Diffusion Webui before official use OpenPose Editor, and ControlNetthe models required for extensions candy, depth, openpose(recommended to be installed) are also deployed in place.

Reference for students who will not install: " Installation Tutorial of ControlNet and OpenPose in Stable Diffusion "

Process arrangement: no matter which method, the operation ideas are similar

1. Reference picture: Pose out the reference picture and try to express it in place;
2. Analysis and debugging: provide the reference image to controlnet for use and fine-tune;
3. Drawing: Stable Diffusion opens the blind box to output the drawing (some parameter adjustments are required during the period).

1. Online puppet pose method

Prepare

Most of them are similar, I won't write about the specific pose method, and you can master it with more adjustments.
URL:
https://app.posemy.art/
https://webapp.magicposer.com/
https://www.anyposes.com/

reference image

Set up the camera position, clear the background, and generate a pose reference image.

Analysis and debugging

Put the picture into the controlnet, analyze the openpose, and get the structure diagram (important), the depth map and the edge map according to the needs, and pay attention to the weight, timing of intervention and exit.

Whiter and brighter means closer to the viewer. In this case, the character on the right is brighter and whiter, which shows that the analysis of the depth map by controlnet is wrong.

raw picture

The details of the characters in the reference picture in this section overlap, so the confusing parts are constrained by text descriptions.

Generation Data：

modelshoot style, 2warrior, fall backward with arms outstretched, push down, Roman Colosseum, helmet, cloudy sky, 
Negative prompt:  easynegative
Seed: -1, Steps: 32, Sampler: DPM++ SDE Karras, CFG scale: 10

Two, OPENPOSE self-portrait method

Prepare

• Installation OpenPose Editortutorial:
  https://github.com/fkunn1326/openpose-editor/blob/master/README.zh-cn.md
• Restart Stable Diffusion WebUI

reference image

OpenPose EditorEdit the character form by .

Analysis and debugging

sent to >>>文生图, and configuration parameters

raw picture

Write whatever suits you.

Generation Data：

modelshoot style, (extremely detailed CG unity 8k wallpaper), full shot body photo of the most beautiful artwork in the world, beautiful women, (1dancer:1.5),(2hands:1.5)
Negative prompt:  easynegative, canvas frame, cartoon, 3d, ((disfigured)), ((bad art)), ((deformed)),((extra limbs)),((close up)),((b&w)), wierd colors, blurry, (((duplicate))), ((morbid)), ((mutilated)), [out of frame], extra fingers, mutated hands, ((poorly drawn hands)), ((poorly drawn face)), (((mutation))), (((deformed))), ((ugly)), blurry, ((bad anatomy)), (((bad proportions))), ((extra limbs)), cloned face, (((disfigured))), out of frame, ugly, extra limbs, (bad anatomy), gross proportions, (malformed limbs), ((missing arms)), ((missing legs)), (((extra arms))), (((extra legs))), mutated hands, (fused fingers), (too many fingers), (((long neck))), Photoshop, video game, ugly, tiling, poorly drawn hands, poorly drawn feet, poorly drawn face, out of frame, mutation, mutated, extra limbs, extra legs, extra arms, disfigured, deformed, cross-eye, body out of frame, blurry, bad art, bad anatomy, 3d render
Seed: -1, Steps: 32, Sampler: DPM++ SDE Karras, CFG scale: 10

Using this method alone, it is currently impossible to generate depth maps and edge maps.

3D evolution

For the 3D version of openpose, refer to this article for operation:
AI drawing practice (11): Modify paper children's drawings into electronic photos/3D Openpose plug-in | Stable Diffusion becomes a productivity tool for designers

3. Blender 3D modeling method

This method can provide depth map and structure map~~~

Prepare

• Download blender and install
  https://www.blender.org/
  
• Download the 3D model (currently the latest version is 5.0)
  https://toyxyz.gumroad.com/l/ciojz
  
• If you cannot open this page, you can download it:
  https://download.csdn.net/download/snans/87581401

reference image

Open the downloaded model with blender and adjust the pose of the character, as shown in the figure:

Adjust resolution

Tick depth​​, canny, poseand then render the graphics to get the pictures required by controlnet

The drawing method is as follows:

• Depth map
  Select depth as shown above, render the depth map and save it as a file
  

• Edge map
  After selecting canny, configure according to the following figure compositing, and then render and save the canny map
  
  

• Skeleton diagram
  Cancel the reverse color set in the canny diagram, otherwise the diagram will be incorrect
  

Render graphics after openpose is selected

Analysis and debugging

Import the three pictures into the controlnet respectively, refer to the scheme one.

raw picture

Write a keyword-generated image.

It seems that stable diffusion does not analyze the body shape well at this camera angle, such thick legs. -_-!

Four, commonly used image recognition method

Prepare

Real person poses for a photo

reference image

This article uses photos of Anna Shcherbakova~~

Analysis and debugging

Controlnet analyzes and draws edge maps, bone maps, and depth maps.

This clean and single picture, the results analyzed by controlnet are relatively accurate, so if it is really used in production, then a green screen or a clean background is still necessary.

raw picture

figure skating, Style-Empire, 8k portrait of beautiful young woman, sports
Negative prompt: [lowres, cropped, watermark, jpeg artifacts, low quality, normal quality, bad anatomy, text error, worst quality]
Size: 1024x691, Seed: -1, Steps: 32, Sampler: DPM++ SDE Karras, CFG scale: 5,

If the constraint parameters of controlnet are lowered, it is very likely that the picture will produce pair skating~~

It is easy to use, and you can do the action yourself if you want to draw a picture in the future.

Five, PS repair method

This is a method for modifying the details after the drawing is finished.

Prepare

• Prepare photo editing software such as PHOTOSHOP
• Prepare a picture to be edited

• looking for a hand chart
  

reference image

• In photoshop, putting your hands in the right position and making the colors as consistent as possible involves part of the knowledge of photoshop.

Analysis and debugging

Generate a reference map and import it into Stable Diffusion 图生图>>>局部重绘

After the brush paints the object to be modified (here is the hand), configure the parameters (requires multiple adjustments) to generate a picture until you are satisfied.
In addition to modifying the main object, the parts around the object may also be modified by smearing.

raw picture

The light and shadow color of the hand is coordinated (in fact, wrong), but the details are lost.

For minor revisions like this, it may be faster, more delicate and more accurate for painting professionals to make changes in the traditional way.