Knowledge Morning Bus:
1. When n is greater than 1, the nth power of 2 must be divisible by 4; prove: 2^n = 2^2*2^(n-1) = 4*2^(n-1)
Multiples of 2 and 4 are not necessarily all powers of 2; prove: 4*3 = 12; 12 is not a power of 2
3. Pixel is the smallest unit that makes up a picture; Byte is the smallest unit of computer file size; bit (bit/bit) is the smallest unit of computer processing operations.
4. Common RGB 16-bit, ARGB/RGBA 16-bit, RGB 24-bit, ARGB/RGBA 32-bit, etc. Their usual channel allocations are:
RGB 16-bit = RGB565 = RGB555 (the highest bit is not used), ARGB/RGBA 16-bit = ARGB4444 = RGBA4444 = ARGB1555 = RGBA5551
RGB 24-bit = RGB888, ARGB/RGBA 24-bit = ARGB/RGBA6666, RGB 32-bit = RGB888 (the remaining 8 bits are reserved), ARGB/RGBA 32-bit = ARGB/RGBA8888
Knowledge starting point
When we optimize resources, we must inevitably deal with texture formats. Any image file format will be converted to Textur2D format after being imported into Unity. Then we can set the compression format of the texture according to different platforms in the import setting options of Texture2D.
So why do we need texture compression formats?
For example, uncompressed image formats such as RGB565, ARGB4444, ARGB1555, RGB888, and ARGB8888 are native texture formats that can be directly read by the GPU. But under low-end hardware devices or mobile platforms, there are two problems that need to be solved: memory + bandwidth;
1. For example, one pixel in the ARGB8888 format occupies 4Byte. If the resolution is 512x512, the memory usage is: 512x512x4B=1048576B=1M. This kind of memory consumption is simply unacceptable on low-end devices.
2. Another important thing is the bandwidth during data transmission. Bandwidth is the culprit of heat generation. When rendering 3D scenes, a large number of textures will be transferred to the GPU. If not restricted, the bus bandwidth will soon become a bottleneck, and the mobile phone will quickly become a bottleneck. Warming the handheld device will seriously affect the rendering performance.
Therefore, we need a format that has a small memory footprint and can be read by the GPU - a compressed texture format. Texture compression corresponds to an algorithm that encodes fixed-size blocks of pixels into fixed-size blocks of bytes using some form of fixed-rate lossy vector quantization.
Lossy: Indicates that lossy compression is acceptable for rendering. Generally, when choosing a compression format, you need to find a balance between texture quality and file size.
Fixed rate compression: Because the GPU needs to be able to efficiently access a pixel randomly, this means that for any pixel, the decoding speed should not change much. Therefore, our common texture compression algorithms are lossy compression. In contrast, zip, for example, is a variable rate compression.
Vector Quantization: A quantization technique that divides a large set of points (vectors) into groups with approximately the same number of points closest to them. Each group is represented by its centroid point and therefore subject to data errors, suitable for lossy compression. To understand it in texture compression, for example, the color of a 4x4 block of pixels is represented by 2 base colors.
Encoding and decoding speed: Generally speaking, it does not matter if the encoding speed is slow, because usually texture compression only needs to be performed once when the game is packaged, and has no impact on the user's runtime experience. But the decoding speed must be fast enough without basically affecting rendering performance.
Compression ratio: Usually expressed in terms of bit rate or the average number of bits per pixel (bits per pixel, bpp), commonly 1 to 8bpp. Generally, the pixels of RGB native textures refer to 24 bits, that is, each pixel occupies 24 bits (bit). The existing 4bpp means that each pixel occupies 4 bits (bit), so it can be considered that 4bpp means that the compression ratio is 6:1.
In addition, Unity will convert the imported resource images, whether in PNG format or JPG format, into the Texture2D format processed by the Unity engine.
Why don't we use common compression formats such as png and jpg?
Although jpg and png have high compression rates, they are not suitable for textures. The main problem is that they do not support random access to pixels, which is quite unfriendly to the GPU. The GPU only uses the required texture part when rendering. We cannot always access a certain Pixels to decode the entire texture? Without knowing the order, and not sure whether adjacent triangles sampled on the texture are also adjacent, it is difficult to optimize. This type of format is more suitable for downloading and transferring and reducing disk space.
Common texture compression formats
1、ETC
ETC is mainly the standard compression scheme for Android. Its principle is simply to encode a 4x4 pixel block into two blocks of 2x4 or 4x2 pixels. Each block specifies a base color, and the color of each pixel is encoded as a relative The grayscale values of these primary color shifts are determined.
Commonly used ETC compression formats in Unity are:
RGB ETC1 4bit: 4bit/pixel, the RGB compression ratio is 6:1, does not support alpha, most android devices support it
RGB ETC2 4 bit: 4 bits/pixel, RGB compression ratio 6:1. Alpha is not supported. ETC2 is compatible with ETC1. The compression quality may be higher, but the error will also be greater for blocks with large chroma changes. OpenGL ES 3.0 and OpenGL 4.3 or above are required.
RGBA ETC2 8bit: 8 bits/pixel, RGBA compression ratio 4:1. Supports complete transparent channels, the version requirements are the same as above.
RGB +1bit Alpha ETC2 4bit: 4 bits/pixel. Supports 1-bit Alpha channel, that is, only supports hollow images. The image only has transparent and opaque parts, and no intermediate transparency.
2、DXT 原名S3TC（S3 Texture Compression ）
The principle is simply to describe a 4x4 RGB pixel block by a pair of low-precision "primary colors" and allow each pixel to specify an interpolation between these primary colors. There are several variations of S3TC, each designed for a specific type of image data, but they all convert 4x4 blocks of pixels into 64-bit or 128-bit data.
3、ASTC
ASTC is a high-quality compression method under the Android and iOS platforms, supporting Android5.0 and iPhone6 and above models. It is also a block-based lossy compression algorithm. Its blocks are of fixed size of 128 bits. Although it is a fixed size, unlike the fixed 4x4 blocks of the previous format, each texture can have different sizes in these 128 bits. Blocks can be square or non-square. In 2D texture coding, they range from 4x4 to 12x12 pixels, and the corresponding compression ratio is 1 to 8bpp. It has many features:
High flexibility: supports 1-4 component maps
Flexible compression rate/quality: different compression rate algorithms will be selected according to different pictures.
Support 2D/3D textures
Cross-platform: iOS, Android, PC
Supports both LDR and HDR: BC6H supports HDR but does not support Alpha channel
The larger the blocks, the worse the compression quality, but the smaller the image. The bit rate of ASTC can be changed at the decimal level. This technology is called BISE (Bounded Integer Sequence Encoding).
But be aware that although textures can be encoded as 1-4 channel images, the decoded values are always output in RGBA format. Color values are returned as 8-bit integers in LDR sRGB mode, or as 16-bit floats in HDR.
When setting the ASTC format, you need to pay attention to the following points:
Normal map: Try to choose 4x4 to avoid losing too much data.
Detailed textures: choose 4x4 or 6x6, otherwise details will be lost.
General maps: choose 6x6 or 8x8. 4. Pictures that are insignificant but extremely large in size: consider 8x8 or 10x10 or 12x12, otherwise they will be too large when packaged.
Through the above three compression formats, we found that the three compression algorithms all use 4x4 pixel blocks for operation, which makes us think that when we do image optimization, we often ask art students to keep the long distance when outputting the image size. The reason why the width is a multiple of 4 is so that we can achieve better results when setting the compression format for images imported into Unity. However, some students may still ask, why do you sometimes hear the image size being set to a power of 2? This is because the original ETC compression algorithm does not support transparent channels, and the image width and height must be a power of 2. In addition, the PVRTC compression format under the iOS platform only supports a power of 2. In addition, when n>1, 2^n must be a multiple of 4, so there is a requirement that the image output size be a power of 2; fortunately, most andorid platforms now support ETC1/ETC2. ETC2 is an extension of ETC, supports transparent channels, and the image width and height As long as it is a multiple of 4; PC platforms also support DXT1/DXT5/DXT5. The principles of DXT3 and DXT5 are the same as DXT1, except that DXT3 and DXT5 support transparent channels, and the image width and height are also required to be a multiple of 4; ASTC The format is also supported by more and more platforms, so when we set the image size, we only need to set it to a multiple of 4. Also, if Generate Mip Maps is turned on, only the power of 2 can be compressed, because the generated mipmap must be a power of 2 pixel.
Knowledge tips:
1. For pictures, sprite atlas is now used to manage pictures. For pictures that are to be entered into the atlas, the size does not need to be set in multiples of 4, because these pictures will eventually be entered into the atlas. We can modify the atlas. Setting multiples of 4, but one disadvantage of this is that in order to maintain the size of the set multiples of 4, the album has to waste some space, so the best way is to set all pictures to multiples of 4; do not enter the picture The size of the set pictures must be strictly controlled, preferably a multiple of 4, and the maximum size should not exceed 1024x1024.
2. Turn off the Read/Write option, because turning on this option will allow texture data to be accessed from scripts, which will produce a copy of the texture data. The copy will occupy memory, which is equivalent to double the memory consumption of one texture data.
3. Set the Filter Mode to Bilinear texture: Trilinear trilinear filtering (trilinear interpolation), the texture will be blurred between different mip levels, thereby increasing GPU overhead.
4. Turn off Mipmap for Sprite texture: After Mipmap is turned on, the memory will be 1.33 times that of Mipmap that is not turned on, because Mipmap will generate a set of texture sequences whose length and width are reduced by one time, until 1*1. Mipmap improves GPU efficiency; generally used in 3D scenes or characters, it is not recommended to turn it on in the UI.
5. Set Wrap mode to Clamp: Wrapmode uses Repeat mode, which can easily cause noise to appear on the edge of the map.
Take the PC platform as an example:
If the length and width of the imported image are a multiple of 4 and a power of 2, it will be compressed into RGB Compressed DXT1 or RGBA Compressed DXT5;
If the length and width of the imported image are multiples of 4 but not powers of 2, they will be compressed into (NPOT)RGB Compressed DXT1 or (NPOT)RGBA Compressed DXT5;
If the length and width of the imported image are not a multiple of 4 or a power of 2, it will be compressed into (NPOT) RGB24/(NPOT) RGBA32. These are uncompressed formats and take up a lot of memory.