Reference link: https://www.unrealengine.com/zh-CN/tech-blog/setting-up-a-chroma-key-material-in-ue4?sessionInvalidated=true&lang=zh-CN
The above link has made it very clear, the following is a record of my experience in practice.
process:
- Cutout Alpha Obtained
- remove spill
- Add reflectors
Material screenshot:
Example cutout:
It may be that the original image was not captured clearly enough, or the parameters I adjusted did not reach the end, and the edge could not be adjusted well. 
Remove brightness:
Saturation-based luminance: The image needs to be desaturated first, and a simple exponential function e^-x is used to generate the luminance curve.
float3 ExtractColor(float3 Color, float LumaMask)
{
float Luma = dot(Color, 1);
float ColorMask = exp(-Luma * 2 * PI / LumaMask);
Color = lerp( Color, Luma, ColorMask);
return Color / (dot(Color, 2));
}
These parameters in the material affect alpha generation.
LumaMask: It can be understood as the range of brightness. The larger the value, the larger the color range processed. You can only use one color at a time for the cutout color. If you want to deduct the color that is close to this color, increase the value.
Alpha Min: Define all images smaller than this value to be cut out, and Alpha Max larger than this value to be cut out. In actual use, these two values are difficult to adjust, and many attempts are required.
Alpha Exponent, the internal brightness adjustment, how much the Alpha itself is transparent.
remove spill
There will be an edge, which is solved by adding a reflector at the back, but the edge still exists and cannot be eliminated. For backgrounds a bit more complex, it's easy to see the edges.
Reflector:
Use background color for edges
