Rendering is the core of computer graphics, which can convert 2D images into 3D models. It is also to establish a bridge between the conventional means of 3D scene attribute (scene properties) and 2D image pixels. But the traditional rendering engine can not be differentiated, so they can not be merged to a depth of study and work in the pipeline. PyTorch3D modular built-differentiable renderer, can be used to process the 3D data differentiable.
Facebook recently open source library for the PyTorch 3D depth study of PyTorch3D , which is a modular and highly optimized library, with unique features designed to simplify the 3D depth study by PyTorch. PyTorch3D 3D data provides a common set of 3D operators and fast and differentiable function loss (loss function), and modular differentiable rendering API. Through the above functions, researchers can use these functions to import the most advanced deep learning system immediately.
Researchers and engineers can use PyTorch3D various 3D depth study and research (both 3D reconstruction, bundle adjustment, and even 3D reasoning), and improved recognition task in two-dimensional space.
Cognitive three-dimensional space, plays an important role in the interaction with the real world of artificial intelligence. For example robot navigation in physical space, virtual reality experience improved, 2D content and identifying the object or the like is blocked. But even Facebook has accumulated rich depth of learning technology, it will still be troubled in the face of 3D depth learning problems. Facebook said that the reason for the less deep 3D scene learning techniques, because of lack of adequate tools and resources to support the complexity of the neural network with the combination of 3D data, this scenario requires more memory and a higher operator force , unlike the 2D image may be represented using tensors, many of the traditional graphics and non-differentiable operator, so 3D depth study learning technology is limited.
To this end, Facebook PyTorch3D library was constructed in order to promote the 3D depth study and research, to provide highly optimized library Like PyTorch as 2D recognition task, PyTorch3D to optimize training and reasoning by providing batch processing, and support for 3D operators and loss of function . In order to simplify the complexity of the 3D model of the batch, Facebook created Meshes format , which is a designed depth learning applications, for a heterogeneous batch mesh data structure.
This data structure allows researchers to quickly and easily convert the data into the base mesh model different views, so that the operator with the most efficient representation of data match. More importantly, PyTorch3D embodiment provides the flexibility to efficiently switch between the different views represented for researchers and engineers, and access to different mesh properties.
Rendering is the core of computer graphics, which can convert 2D images into 3D models. It is also to establish a bridge between the conventional means of 3D scene attribute (scene properties) and 2D image pixels. But the traditional rendering engine can not be differentiated, so they can not be merged to a depth of study and work in the pipeline. Therefore, Facebook highly modular built-in PyTorch3D differentiable renderer, can be used to process the 3D data differentiable. This function may be implemented by a combination of units, allowing users to easily extended to support custom renderer lighting or shading effects.
Facebook these features packaged into kits, and provides operators, heterogeneous batch functions and modular differentiable rendering API, to help researchers study complex 3D applications of neural networks.
View PyTorch3d document: https://pytorch3d.org/docs/why_pytorch3d.htm