The performance of vertex index in the program is to put all the vertices into an array (vertex array), and then use another integer array as an index to express the composition of the triangle (the integer represents the index subscript in the vertex array).

There are several index data, there are several UV coordinates, they are composed of two floating-point numbers, the range of these two floating-point numbers is 0 to 1, 0 indicates the starting position of the upper left corner of the texture, and 1 indicates the maximum offset of the texture The position, that is, the lower right corner, we should know the coordinates on the picture when we hear the two letters UV.

When drawing a 3D model, in addition to the vertex and index arrays, there is also an array called UV array, which exists for storing UV coordinates. Since there are already indexes to express the three vertices of the triangle, the UV array does not need to use indexes to express it anymore. It is only necessary to customize the order of the UVs according to the triangle formed by the vertex indexes.

4 vertices: [(0,0,0),(0,1,0),(1,1,0),(1,0,0)]
Vertex indices, representing 2 triangles: [0,1,2,2,3,0]
UV array, indicating the drawing range of the texture on the two triangles: [(0,0),(0,1),(1,1),(1,1),(1,0),(0,0)]

二、Mesh、MeshFilter、MeshRenderer、SkinnedMeshRenderer

1. Mesh

Grid (Mesh) is a data resource, it can have its own resource file, such as XXX.FBX. The mesh stores the data necessary for rendering such as vertices, UVs, vertex colors, triangles, tangents, normals, bones, and bone weights.

2. MeshFilter

MeshFilter is a class that carries grid data, and the grid is instantiated and stored in the MeshFilter class. MeshFilter contains two types, instance and shared variables (mesh and sharedMesh). Operating on mesh will generate a new mesh instance, while operating on sharedMesh will change the specified grid data shared with other models instance.

3.MeshRenderer

MeshRenderer is a class for drawing grids, with rendering functions, it will extract the grid data in MeshFilter, combined with its own materials or sharedMaterials for rendering.

4. MeshRenderer and SkinnedMeshRenderer (skinned mesh)

These two components, MeshRenderer and SkinnedMeshRenderer, are used to render 3D models and 3D model animations respectively. Their model data are stored in MeshFilter, so they all depend on the MeshFilter component. Among them, MeshRenderer is only responsible for rendering the model, we can also call it a common grid rendering component, which extracts the grid vertex data from the MeshFilter. Although the SkinnedMeshRenderer (skinned mesh) also renders the model, it also extracts the model mesh vertex data from the MeshFilter, but the skinned mesh is mainly used for rendering animation services, so the skinned mesh has bones in addition to the 3D model data data and vertex weight data.

If there is no skeletal data stored on the skinned mesh, then it has no difference from the normal mesh MeshRender, and what is rendered is a 3D model without animation.

In short, MeshRenderer is suitable for static mesh rendering without deformation, while SkinnedMeshRenderer is suitable for mesh rendering that requires skeletal animation and deformation. In games, common character models usually use SkinnedMeshRenderer to implement skeletal animation, while environment models and other static objects usually use MeshRenderer.

3. Related components in Unity

1. mesh and material

Mesh (see above) defines the shape and vertex data of the model, while material defines the appearance properties of the model, such as color, texture and lighting effects.

Both mesh and material are instance-type variables. To perform any operation on mesh and material, an additional copy is made and then reassigned. Even if it is just a get operation, the copy operation will also be performed. That is to say, after the mesh and material are operated, it will become another instance. Although it looks the same, it is actually a different instance.

2. sharedMesh和sharedMaterial

sharedMesh and sharedMaterial are different from the previous two variables, they are shared. Multiple 3D models can share the same specified sharedMesh and sharedMaterial. When modifying the parameters in sharedMesh or sharedMaterial, multiple models pointing to the same sharedMesh and sharedMaterial will change the effect at the same time. That is to say, after sharedMesh and sharedMaterial are changed, all 3D models using sharedMesh and sharedMaterial resources will show the same effect.

3. materials和sharedMaterials

Like material and sharedMaterial, materials are instance-type, and sharedMaterials are shared-type, but now they become arrays.

No matter what operation is performed on materials, an identical copy will be copied to replace it. After sharedMaterials is operated, all models pointing to this material ball will change the effect. The difference between materials&sharedMaterials and material&sharedMaterial is that materials and sharedMaterials can target different sub-grids, and material and sharedMaterial are only for the main grid. That is, material and sharedMaterial are equal to materials[0] and sharedMaterials[0].

4. Skinned skeleton animation

The 3D model needs to move, first of all, the points, lines, and surfaces on the model grid must be moved. Only when the points, lines, and surfaces are moved, can different grid shapes be rendered when each frame is rendered, so that it can be seen. A picture that moves when you get up. So how to make points, lines, and planes move?

There are two main methods: one is to use an algorithm to change the vertex position, which we usually call vertex animation; the other is to use bones to affect mesh vertices, which we call skeletal animation. These two animation methods are to deform the model by offsetting each vertex on the model mesh in each frame, thereby forming the effect of animation. The shape of the model grid is different in each frame, and animation is formed during playback. Although the two methods are different, they all follow the same principle.

        Rigid Hierarchical Animation

        At first, 3D model animation only had rigid hierarchical animation (rigid hierarchical animation), which split the entire model into multiple parts, and then installed them in the form of hierarchical nodes.

        In this way, the model is arranged on nodes in a hierarchical manner, and when the parent node moves, rotates, and scales, the child nodes also move accordingly. There are many problems with rigid hierarchical animation, the most serious of which is that the joint connection position often produces "cracks", because they are not connected by one model, but pieced together by multiple models.

Morph Target Animation

The method of morph target animation (morph target animation) is often used in facial animation. It makes the animation into several models with fixed extreme poses, and then performs linear interpolation between each vertex of the two models, and the facial movement Requiring about 50 sets of muscle actuation, animations of this level of complexity would be better represented by linear interpolation between the vertices of the two meshes.

1. Bones

Skeletal animation consists of skeletal points, which can be considered as data entities with relative space coordinate points. There can be many skeletal points in skeletal animation, but there is only one root node. (In modern mobile games, the number of bone animations for each character is generally about 30, and it can reach about 75 in PC stand-alone games. The more bones, the more dynamic the animation, but at the same time it will consume more Computation.)

The bone point is a tree structure. A bone can have many child bones. The child bone exists in the relative space of the parent bone. The child bone has the same function as the parent bone. Since the child bone is in the space of the parent bone, when the parent bone When the bone moves, rotates, and scales, the child bones also move, rotate, and scale with the parent bone, and their relative positions, angles, and proportions remain unchanged.

In the skinned grid component of Unity3D, the bones variable is used to store all bone points. The bone points are stored in the form of Transform array in the skinned grid. This can be known from the fact that the bones variable is the Transform[] array type. .

A bone point can affect vertices within a certain range around it, and a single vertex can also be affected by multiple bones. In addition to the bone data, each vertex in the model has the weight values of the 4 bones that have the most influence on the vertex itself. Unity3D stores the weight values of these 4 bones and stores them in the BoneWeight Struct structure. Each SkinMeshRender class has a boneWeights array variable, which is used to record the bone weight values of all vertices. Those meshes without bone animation do not have these data.

From the Quality setting of Unity3D, we can see that the Blend Weights parameter can be used to set how many bones a vertex can be affected by. There are parameters such as 1 Bone, 2 Bones, and 4 Bones, which mean that a vertex can be affected by 1 bone, or by 2 bones, or by 4 bones. The more bones are affected, the longer the CPU spends on bone calculation skinning, and the greater the consumption.

Skeletal animation uses the bone weight data of the vertex to determine which bone points the vertex is affected by, and each vertex can be affected by the bone points. In Unity3D, each vertex is affected by up to 4 bone points, and these data are stored in the BoneWeight instance, which is used to describe which 4 bone points the current vertex is affected by and how much weight they occupy respectively.

When the bone point moves, the engine uses these vertex weight values to calculate the rotation, offset and scale of the vertex. To put it simply, use the bone weight data on the vertex to determine which bone points the point is affected by and how much it is affected.

2. Steps for making skin animation

We usually divide skin animation into three steps:

The first step is to use 3D modeling software such as 3DMax and Maya to construct a series of bone points (bones) on the geometric model, and calculate the weight value (BoneWeight) of each vertex of the geometric model affected by these bone points.

The second step is for the animator to make a series of animations through 3D model software. These animations are all done through the offset, rotation, and scaling of bone points. Each frame may change, and there will be a change between keyframes and keyframes. Tween some non-keyframed animation. After production, export the engine's proprietary animation file format. In Unity3D, we use .fbx as a proprietary format file.

The third step is to import and play the animation in Unity3D. When the animation is played, the data of the bone point changes made by the animator has been stored every frame. The animation sequence frame will continuously change a series of bones according to the animation data of each frame. point, the change of the bone point will lead to the corresponding change of the vertices on the geometric model grid.

3. Keyframes

Usually we use key frame animation, which is the Animation file in Unity3D. Keyframe the bones that need to be changed at a certain point in time, instead of performing keyframe operations on every frame. The advantage of using keyframes as the rotation displacement points of bones is that there is no need to set the position changes of bone points for each frame. Between keyframes and keyframes, the bone position can be smoothly interpolated by the Animation component, which can greatly reduce data. The amount is equivalent to a tween animation between key frames. The purpose of tween animation is to perform smooth displacement, rotation, and scaling interpolation calculations on the bones that need to be changed, so as to obtain corresponding results in real time and reduce data usage.

Since the 4×4 matrix can fully express the offset, scaling and rotation of the point, and can also calculate the specific orientation from the root node to the parent node to the child node through continuous right multiplication, the 4×4 matrix is The necessary data of the bone point, which expresses the offset of the relative space, that is, the bone node change matrix = root node matrix × parent parent node matrix 1 × parent parent node matrix × parent node matrix × bone node matrix.

5. The process of grid data from production to rendering

First, artists make 3D models and export them into a format that Unity3D can recognize, that is, .fbx files, which already contain vertex and index data.

Then instantiate .fbx into Unity3D GameObjects in the program, and the MeshFilter components attached to them store the vertex data and index data of the grid (we can also create vertex arrays and index arrays by ourselves, and input vertex data and indexes manually data).

MeshFilter can be used to store vertex and index data, and MeshRender or SkinMeshRender can be used to render models. These vertex data are usually combined with shaders and sent to the graphics card when rendering. Unlike what we expected, when sending Instead of index data, triangle vertices consisting of groups of three vertices are sent to the graphics card. Then the graphics card is responsible for processing the data we send in, and then rendering the frame buffer and outputting it to the screen.

6. Dynamically realize the whole process from modeling to skin animation

1. Code

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class TestMesh : MonoBehaviour
{
    public SkinnedMeshRenderer rend;
    public Animation anim;
    public AnimationCurve curve;
    public AnimationClip clip;
    public string clipName = "test";
    // Start is called before the first frame update
    void Start()
    {
        // 新建一个动画组件和蒙皮组件
        gameObject.AddComponent<Animation>();
        gameObject.AddComponent<SkinnedMeshRenderer>();
        rend = GetComponent<SkinnedMeshRenderer>();
        anim = GetComponent<Animation>();

        // 新建一个网格组件，并编入4个顶点形成一个矩形形状的网格
        Mesh mesh = new Mesh();
        mesh.vertices = new Vector3[] { new Vector3(-1, 0, 0), new Vector3(1, 0, 0), new Vector3(-1, 5, 0), new Vector3(1, 5, 0) };
        mesh.uv = new Vector2[] { new Vector2(0, 0), new Vector2(1, 0), new Vector2(0, 1), new Vector2(1, 1) };
        mesh.triangles = new int[] { 0, 1, 2, 1, 3, 2 };
        mesh.RecalculateNormals();

        // 新建一个漫反射的材质球
        rend.material = new Material(Shader.Find("Diffuse"));

        // 为每个顶点定制相应的骨骼权重
        BoneWeight[] weights = new BoneWeight[4];
        weights[0].boneIndex0 = 0;
        weights[0].weight0 = 1;
        weights[1].boneIndex0 = 0;
        weights[1].weight0 = 1;
        weights[2].boneIndex0 = 1;
        weights[2].weight0 = 1;
        weights[3].boneIndex0 = 1;
        weights[3].weight0 = 1;

        // 将骨骼权重赋值给网格组件
        mesh.boneWeights = weights;

        // 创建新的骨骼点，设置骨骼点的位置、父骨骼点和位移旋转矩阵
        Transform[] bones = new Transform[2];
        Matrix4x4[] bindPoses = new Matrix4x4[2];

        bones[0] = new GameObject("Lower").transform;
        bones[0].parent = transform;
        bones[0].localRotation = Quaternion.identity;
        bones[0].localPosition = Vector3.zero;
        bindPoses[0] = bones[0].worldToLocalMatrix * transform.localToWorldMatrix;

        bones[1] = new GameObject("Upper").transform;
        bones[1].parent = transform;
        bones[1].localRotation = Quaternion.identity;
        bones[1].localPosition = new Vector3(0, 5, 0);
        bindPoses[1] = bones[1].worldToLocalMatrix * transform.localToWorldMatrix;

        mesh.bindposes = bindPoses;

        // 将骨骼点和网格赋值给蒙皮组件
        rend.bones = bones;
        rend.sharedMesh = mesh;

        // 定制几个关键帧
        curve = new AnimationCurve();
        curve.keys = new Keyframe[] { new Keyframe(0, 3, 0, 0), new Keyframe(2, -3, 0, 0), new Keyframe(4, 3, 0, 0) };

        // 创建帧动画
        clip = new AnimationClip();
        clip.legacy = true;     //正确
        clip.SetCurve("Lower", typeof(Transform), "m_LocalPosition.z", curve);
        //clip.legacy = true;   //错误，应先设置legacy为true，再SetCurve,否则发布exe后报错
        clip.wrapMode = WrapMode.Loop;

        // 将帧动画赋值给动画组件，并播放动画
        anim.AddClip(clip, clipName);
        anim.playAutomatically = true;
        anim.clip = clip;
        anim.Play(clipName);
        Debug.Log("play");
    }

    // Update is called once per frame
    void Update()
    {

    }
}

2. Related data

(1) Curve attribute effect

(2) mesh data

3. Overall effect

4. Attention to detail

The clip.legacy should be set before SetCurve, otherwise an error will be reported after the exe is released:

Unity | The whole process from modeling to skin animation

1. Vertex array, index array and UV array