Mr. Shu introduced a project DragGAN to you before, and now it is officially open source!
Super cool project is coming! DragGAN is hot in the open source community!
So what are you waiting for, grab it!
What is DragGAN?
Unlike the diffusion models used by Stable Diffusion and Midjourney, DragGAN is a photo editing project based on generative confrontation network (GANs) technology. Tracing back to the source, it is actually a derivative project of NVIDIA's StyleGAN.
DragGAN consists of two main parts:
The first part is feature-based motion supervision, which realizes image deformation by controlling the movement of the starting point to the target point;
The second part is a new point tracking method, nearest neighbor retrieval, which improves efficiency and avoids cumulative error loss by performing point tracking on the same feature space.
Simply put, it allows you to easily control the characters and animals in the picture, and adjust their position, shape, posture, etc. by dragging control points and target points. At present, this project has gained 20K stars on Github, ranking first in the trend list, which shows its popularity!
deployment requirements
-
Both Linux and Windows are supported, Linux is recommended
-
RAM 12 GB or more
-
1–8 NVIDIA high-end GPUs
-
Python >= 3.8，PyTorch >= 1.9.0
-
CUDA toolkit >= 11.1
Installation and deployment
During the installation process, there was a lot of snot and tears, and there were pits everywhere. First, I stumbled on the Ali PAI platform. After a long time, I found out that it was a basic environmental problem. Later, I overturned the installation method. The official installation guide also reported various errors. .
After groping hard all the way, now I will bring you a set of deployment methods that are actually tested and feasible.
Here we choose the AutoDL cloud platform and use the image of Python 3.8 and CUDA 11.8. The environment of this image already meets the requirements of the project.
download source code
git clone https://github.com/XingangPan/DragGAN.git
install dependencies
cd DragGAN
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
If the following error occurs, it means that there is no corresponding version in the pip source.
Modify requirements.txtthe file, remove the specific version dependency, and execute it again.
torch>=2.0.0
scipy
Ninja==1.10.2
gradio>=3.35.2
imageio-ffmpeg>=0.4.3
huggingface_hub
hf_transfer
pyopengl
imgui
glfw==2.6.1
pillow>=9.4.0
torchvision>=0.15.2
imageio>=2.9.0
download model
sh scripts/download_model.sh
run
Choose to run the WebUI here, and if you see the URL, it means the startup is successful.
# 运行 WebUI
python visualizer_drag_gradio.py
Show results
It is very simple to use, just click the mouse in the picture, the red point is the starting point, the blue point is the target point, and the dragging effect of the image is realized.
You can also control the regional image to avoid the overall image from changing.
Many people think that this project can directly select a photo, and you can make a P picture like PS. In fact, it is not, at least it is not supported yet. The pictures you see above are models generated after training, so Essentially, you are manipulating a model, not an image!
If you want to manipulate your own pictures, then according to the official website, you need to use the PTI project to train your custom pictures into a StyleGAN model.
This part of the content will be introduced in the next issue. It took a lot of effort to deploy DragGAN, and I am a little tired. Remember three times~