Operating systems, compiling principles, and computer graphics are said to be the three romances of programmers. Each direction is easy to understand but difficult to understand. However, as a programmer, you must have a basic understanding and judgment for each direction. After all, it runs through virtual and reality. The bridge of life is built on abstraction, and these three romances are the representative of abstraction. This time we get to know the 3D rendering part of computer graphics, the purpose is to accumulate some 3D knowledge that we have come into contact with recently, and at the same time help everyone have a simple understanding, which is also a pleasant surprise.
3D rendering engine technology
When rendering and drawing 3D models, we are very concerned about what kind of technology is used at the bottom layer. Webgl technology is also widely used in web front-end work, so what is the relationship between webgl, opengl, opengl es and other technologies Woolen cloth? Which engine do we use when drawing 3D graphics on the web and on the client? Let's briefly discuss these two issues first, and first talk about the concepts and uses of each technology.
▐ OpenGL (Open Graphics Library)
OpenGL is the earliest popular cross-platform graphics drawing interface, and it has been widely used on various platforms since its release in 1997. Currently the main graphics engine, the OpenGL specification describes an abstract API for drawing 2D and 3D graphics. Although these APIs can be implemented entirely in software, it is designed for most or all use of hardware acceleration. OpenGL is not only language independent, but also platform independent. The specification says nothing about obtaining and managing OpenGL contexts, leaving these as details to the underlying windowing system. For the same reason, OpenGL focuses purely on rendering and does not provide input, audio, or window-related APIs.
▐ OpenGL ES (OpenGL for Embedded Systems)
OpenGL ES is tailor-made from OpenGL. It is mainly designed for embedded devices such as mobile phones, PDAs and game consoles. It removes many non-absolute primitives such as glBegin/glEnd, quadrilaterals (GL_QUADS), polygons (GL_POLYGONS), etc. necessary features. After years of development, there are now two main versions, OpenGL ES 1.x for fixed pipeline hardware, and OpenGL ES 2.x for programmable pipeline hardware. OpenGL ES 1.0 is based on the OpenGL 1.3 specification, and OpenGL ES 1.1 is based on the OpenGL 1.5 specification, and they support two profiles, common and common lite, respectively. The lite profile only supports fixed-point real numbers, while the common profile supports both fixed-point and floating-point numbers. OpenGL ES 2.0 is defined with reference to the OpenGL 2.0 specification, and the common profile was released in August 2005, introducing support for programmable pipelines. OpenGL ES 3.0 was released in 2012, adding a lot of new features.
OpenGL ES 1.0 is based on OpenGL 1.3, released in 2003;
OpenGL ES 1.1 is based on OpenGL 1.5, released in 2004;
OpenGL ES 2.0 is based on OpenGL 2.0, released in 2007;
OpenGL 2.0 is backward compatible with OpenGL 1.5, while OpenGL ES 2.0 and OpenGL ES 1.x are not compatible.
OpenGL 3.x Based on OpenGL 3.x, released in 2012
▐ WebGL
WebGL (Web Graphics Library) is a JavaScript API that renders high-performance interactive 3D and 2D graphics in any compatible web browser without the use of plug-ins. WebGL does this by introducing an API that is very consistent with OpenGL ES 2.0 and can be used within the HTML5 <canvas> element. This consistency allows the API to take advantage of hardware graphics acceleration provided by the user's device.
The relationship between OpenGL, OpenGL ES, and WebGL can be well represented by the following figure. In the actual application process, the front-end WebGL operates part of the local OpenGL interface through JS statements. For mobile devices, this is the interface for operating OpenGL ES. To realize the rendering of graphics on the page, WebGL is only a layer that binds the external interface, and some internal core content, such as shaders, materials, lights, etc., all need to be operated with the help of GLSL ES syntax.
The relationship between the three is shown in the figure below
WebGL and OpenGL ES encapsulate the call relationship itself, which only has a small js runtime instant, but the current mid-end machine experience has no sense. The code implementation related to webgl is interested in code implementation. You can take a look at it yourself. It is just the call encapsulation of opengl. .
There is another article on the relationship and difference between OpenGL, OpenGL ES, and WebGL that can be explained better. Please refer to: https://blog.csdn.net/qq_23034515/article/details/108283747
It is also important to understand that these techniques all take advantage of the parallel computing of the GPU, which is very different from the parallelism of the CPU. In a multi-core CPU, each CPU can run another program in parallel. In the GPU, each computing unit of the GPU executes exactly the same program in parallel on different data . GPU's arithmetic units are designed to perform simple processing at high speed and are poor at conditional branches. Unlike a CPU, it has considerable limitations on the programs it can execute. The actual GPU mechanism is much more complicated, but for the time being, this level of understanding is enough for the introduction of OpenGL.
▐ Shader drawing language GLSL (OpenGL Shading Language)
GLSL is a subset of C language that does not support implicit type conversion and static compilation, and is a program that runs on the graphics card. The main function is to replace the vertex shader and chip source shader in the previously fixed rendering pipeline to make it flexible and programmable to complete complex rendering customization tasks. The roughly replaced part is shown in the figure below
Familiar with GLSL is recommended to practice through the following two URLs.
https://thebookofshaders.com/edit.php
https://glslsandbox.com/
rendering
In simple terms, the rendering of a 3D model is actually based on the basic coordinate parameters of the model itself and the relationship between the response material and the ambient light. However, compared with two-dimensional photo rendering, 3D rendering has the requirements for calculation of ambient light and material reflection results and real-time calculation, so relatively speaking, the technical complexity is higher and the amount of calculation is greater. I believe that everyone has a brief understanding of 3D-related technologies in daily life. During the learning process, you will hear a lot of professional terms such as rendering pipeline, coloring, material, texture, scene, light source, opengl, webgl, etc. If you are not engaged in related These nouns alone are enough to learn for a while, so this article will briefly explain the more important rendering methods and nouns, with the focus on application, and more fundamental things will be mentioned in later articles. Currently, there are two main rendering methods for 3D rendering: rasterization and ray tracing.
The rendering pipeline is actually a projection calculation process facing the camera. The general process is shown in the figure below. What each part does specifically, you can learn about it yourself if you are interested. This article only talks about the most understandable part.
Rasterization is the process of converting vector graphics into raster images (also known as bitmaps). The process involves dividing the image into small, individual pixels and assigning each pixel a color based on the original vector graphic's color at that location. The result is a grid-like representation of the original image that can be displayed on a digital screen. In fact, it is the process of projecting continuous geometric figures into the mathematical representation of discrete pixel matrices. It uses perspective, projection matrix and other related knowledge. If you are interested, you can read it from https://github.com/QianMo/Real-Time-Rendering-3rd-Summary-Ebook/raw/master/%E3%80%8AReal-Time %20Rendering%203rd%E3%80%8B%20%E6%8F%90%E7%82%BC%E6%80%BB%E7%BB%93.pdf This book is a simple understanding.
Ray tracing is a technology that has recently become popular with 30 series graphics cards. It is a relatively old technology. It was proposed in the 1980s, but due to the high computing cost, it has only recently been widely used in offline rendering and high-reduction scenes of games. The basic principle is that the reverse calculation of the light from the camera to the light source reduces the engineering complexity and the reality is more realistic.
▐Model format
All data packets have a corresponding specification format. Knowing the corresponding consumption data when understanding a technology can help us quickly see through the whole process. Currently, the more popular 3D formats include obj, fbx, gltf, stl, etc. The format, where fbx is equivalent to jpeg in the model world is relatively widely used, and it is also the format used in this model rendering example. The fbx contains the information required for drawing such as materials, textures, mesh, submesh, etc. required by the model.
Detailed format data view -> https://docs.fileformat.com/3d/fbx/https://mossec.eurostep.com/documentation/gui-user-guide/generic/bin-file-format.html
▐ Front-end rendering model method
threejs car model
It is more complicated to call webgl rendering 3D library on the front end. There are already many libraries packaged such as Threejs, Babylonjs, PlayCanvas, etc. The most widely used one is threejs. The usage and examples of threejs can be learned on its official website. I won’t go into details here, https://threejs.org/
other engines
Those who are interested in other engines can do their own research, and some references are given below.
web engine: babylon.js, oasis-engine, etc.
Professional game engine unity, UE, etc.
modeling
Now most of the model modeling solutions are manual accurate modeling. In recent years, due to the development of civilian lidar and deep learning, 3D model-assisted modeling methods based on vision + spatial distance data or pure vision have gradually increased.
Let’s not mention the traditional modeling methods. I believe that with the continuous development of deep learning, there will be more and more methods based on text modeling, sketch modeling, and NERF modeling, and there will be more and more tools combined with chatgpt Thereby greatly improving the modeling efficiency, which may be tested in various software in less than 5 years. At present, NVIDIA modeling software has been well implemented, and the ways of seeing and getting in the future will be more diverse. Traditional methods Ultimately, it will be a supplement to the means of intervention.
In recent years, under the rapid development of Nvidia toothpaste and deep learning, a new auxiliary modeling method - NERF has brought hope to rapid modeling. Compared with the traditional hand-pinch modeling method, NERF directly shoots videos Modeling in the way of modeling, like taking pictures in the two-dimensional era, can quickly generate a model, and compared with the hand-made model, it has more "temperature", which is an objective response to the real situation, and can bring More fancy ways of playing, such as in the field of automobiles, quickly model customers, bind skeletal animations with car models for online test drive experience, realize online test drives, solve the matching problem of vehicle size and people, and make customers more confident Shopping experience. Thus gradually breaking the boundary between the physical world and the virtual world, complementing the deficiencies of the two worlds.
You can refer to this article https://blog.csdn.net/BIT_HXZ/article/details/128055763 to learn the principle of NERF. This article translates and simplifies the paper, which can help you understand it quickly. If you want to try it out, there is also a lighter way, https://github.com/NVlabs/instant-ngp.
For an easier way, it is recommended to search for Luma in the App Store and use it. Luma also uses nerf technology behind it, so you can experience it relatively easily.
Model building, viewing tools
▐ blender
Blender is open source software, free and powerful, recommended for personal use. https://www.blender.org/
▐ Unity 3D
The advantages and scope of application brought by 3D rendering
3D rendering is still an unused land compared to our current two-dimensional web Internet world. The advantages of 3D over 2D are as follows:
Improved realism and depth perception: 3D graphics provide a more realistic representation of objects and environments, enhancing immersion and enabling a better understanding of spatial relationships.
Improved Visual Appeal: 3D graphics are more visually appealing and interactive than 2D graphics, making them ideal for use in high-value merchandise reproductions, advertising, and gaming.
More flexible representation: 3D graphics can be rotated and viewed from any angle, providing a more flexible representation of objects and environments.
Better simulation capabilities: 3D graphics are ideal for simulations, allowing the creation of highly realistic virtual environments and scenarios for testing, training and research purposes.
It has better applicability in the home improvement and automobile industries. It has been widely used in the home improvement industry and is more intuitive. In the automotive industry, 3D has been introduced in Apps such as Dianchedi and Autohome outside the Ali department for a long time, as an important means for users to compare cars and increase the length of time users stay.
Some car directions are used:
https://ezshine.gitee.io/www/showcase/smart3dh5/loader.html
https://car.taobaocdn.com/static/realibox/breeze/index.html?wh_weex=true&spm=acar.rb-gb.list.breeze
team introduction
Our big Taobao technical team, the team is mainly responsible for the industry business of the brand business development center, based on the depth of the industry to make breakthroughs in business models and accumulation of capabilities, to better support business development. Supported businesses include but are not limited to Tmall Youpin, Tmall Automobile, home improvement, consumer electronics, sports and other industries.
Our team is trying new technologies and methods in multiple directions to improve user experience and help the business achieve sustainable growth. Welcome everyone to join the Ali family and use technology to help business grow together.
¤ Extended reading ¤
3DXR Technology | Terminal Technology | Audio and Video Technology