If you've ever tried to upload a 3D model exported from a CAD program to a WebGL or AR service, you've probably run into issues with maximum file sizes, never-ending progress bars, and poor frame rates.
Optimizing the size and performance of 3D data is critical to authoring a good online interactive experience. It's also good for your bottom line, since smaller files require less cloud storage and less data to push through a CDN.
This article describes how to design an automated pipeline to generate well-visualized 3D models. This allows you to author fully detailed models ready to use for web and AR with minimal manual effort.
When 3D models are authored for fabrication or visualization in offline renderers, they are often not suitable for display in handheld devices, web browsers, AR apps, and other low-spec devices. This means content production teams often end up spending a lot of time optimizing or converting source assets to low-end devices to ensure smooth rendering and fast downloads.
In this article, we'll cover the optimization of 3D assets in general, and in particular how this process interacts with Substance materials and libraries.
Recommendation: Use NSDT editor to quickly build programmable 3D scenes
Not only is manual optimization of 3D models tedious and time-consuming, it can easily become a bottleneck in the production process. The problem is that optimization is inherently downstream of the source asset, meaning that any changes to the source asset (3D model, materials, etc.) need to be reflected in the optimized asset. Therefore, there is a conflict between being able to preview optimized content early on and the time it takes to optimize.
If the source model is expected to change several times during production, it is more efficient to optimize it only after finalization.
This makes optimization a prime goal for automation. This is not a place for artistic expression. An automated pipeline will detect when any aspect of the asset library changes and re-optimize the affected assets.
1. Overview of 3D optimization pipeline
Let's look at an e-commerce-like setup.
Source 3D models can come from many different places, such as CAD packages, DCC packages, etc. In addition to vertices and polygons, we assume they also have texture UV and normal information.
A material library is a set of source materials for a 3D model. The image below is from Substance Source, which is a good starting point for materials - but any such material could also be built in-house, perhaps created from a real-world material sample.
Substance's materials are procedural, allowing users to set parameters and define presets. Materials, together with application-specific settings, are called Material Instances. As an example, the same leather material could be instantiated as a red and blue leather material.
After choosing a material, we assign it to a specific part of the object. For a sofa, you can specify fabric material on the cushions and metal on the legs:
Model without material (left) and with material assigned (right)
The same model can also have multiple distribution configurations: in this example, the same sofa can have different fabric and leather cushions.
Two Material Arrangements of the Same Model
1.1 Output target
Different devices have different capabilities, and what worked well in a browser on a previous-generation phone might work very differently from a high-end PC, so we wanted to be able to produce different models for different purposes.
In its simplest form, the pipeline looks like this:
Note that this process shows one model being processed, but the idea is that multiple objects with multiple outputs from multiple material configurations will go through this process.
There are two stages in the concept pipeline: optimizing the model, then applying the material. Note that optimizing the model is the first of these steps, applying materials happens downstream. This means that any change to the 3D model will trigger optimization and reapplication of the material, but changes to the material can be made without re-optimization.
The pipeline is simplified and there could be more stages and dependencies in a real life pipeline.
1.2 Considerations for Efficient Automation
With a structured pipeline like the one above, we can automate the process of generating optimized models - meaning we can have up-to-date visualization models throughout the life of the product.
To run asset pipelines effectively, we need to understand the relationships between operations, and which data affects which outputs. This means that when source data changes, we can quickly figure out what needs to be built.
Some examples:
- When a material in the material library is changed, any output using that material will need to be rebuilt;
- When adding a new output target, all models need to be processed for that target;
- When a 3D model changes, all configurations of all targets for that model need to be rebuilt.
- When and how to trigger the automated process requires a balance between how quickly results are required and how much computing power is used. Triggering a run every time there is a change will give you fast results, but it can also mean that you spend a lot of time working on a model that will change again soon.
Another approach is to run the process nightly when processing power is more available or cheaper, meaning the latest model should be ready every morning.
It is also possible to let users decide when to run the process of the model, so that they can get the latest model when they need it.
A common problem with the automation part of 3D workflows is loss of artistic control. This shouldn't be the case, as all artistic decisions are made before automation happens. The optimization phase should be viewed in the same way that an image stored as a .tiff file might be compressed to a .jpeg format before being placed on a website. Automating the optimization phase should free up more time for creative work, since you don't need to spend time optimizing the deployment model.
The main considerations when using an automated solution are how and when manual remediation takes place. If the output of the process is not good enough for a certain model, the instinct is often to fix the generated output assets. The problem with this approach is that the fix needs to be reapplied every time the asset is changed (since the change is downstream of the automation pipeline). In general, all output adjustments should be done as settings in the automation pipeline so that they can be applied the next time the pipeline is run. Manual fixes are discouraged unless you are sure the model is its final state.
A great way to tweak is to layer settings overrides. The pipeline's default values can be overridden on a per-asset level, so you can set higher quality values for underperforming assets without affecting any other assets.
2. Why optimize the 3D model?
Optimization of input data is key to producing good 3D models for web and handheld devices. CAD models or models made for high-end rendering are too detailed and too large to be generally suitable for this purpose.
The main things you want to improve through optimization are:
- rendering performance
- Battery Life
- download size
- memory usage
These different goals are often aligned: smaller models typically render faster and use less battery power on the device.
2.1 Evaluate the optimized model
Before diving into what we optimize, it's important to make sure we know how to evaluate the visual quality of the results. The key is always to evaluate the model in terms of its usefulness. If you're making mockups designed to look good in a 400x400 pixel web viewer or handheld device, taking the time to tweak settings to preserve detail or remove artifacts that aren't visible without zooming in will mean "We're going to provide a model that's too detailed and too heavy to download.
When it comes to rendering performance, it's not always easy to predict what will produce the best results without benchmarking, but there are a number of heuristics that combine to tend to produce good results as described below. Also keep in mind that even if you hit your target framerate and download size, on a handheld it's still important to be smaller and faster because less busy GPUs consume less battery. The article GPU performance for game artists covers this topic in more detail and is a good resource for understanding what makes for fast rendering of 3D models.
2.2 Number of polygons
Polygon count and polygon density are an important part of making models render fast and download fast. More polygons and vertices means more calculations the GPU has to do to generate the image.
GPUs are generally better suited for rendering larger and more uniformly sized polygons. GPUs are highly parallel and optimized for larger polygons. The smaller and thinner the polygons, the more parallelism is wasted on shaded areas. This problem is known as overcoloring and is covered in the article linked above.
In general, it is best to look at the wireframe of the resulting model and make sure that the wireframe is not very dense when visualizing the model at the size you intend to view it.
An effective way to reduce polygon count without sacrificing detail is to use normal maps. The idea is that we are more sensitive to how light interacts with objects than to how their silhouettes interact. This means we can move detail from triangle data to normal maps (i.e. from model to texture) and use larger polygons and still get detail from raw data in lighting.
Here is the same model optimized without (left) and (right) a normal map containing details of the original model:
Optimizing the model to a level where most of the small details are in the normal map also allows the mesh to have a better UV map. Small details and sharp edges are often a problem when creating UV maps. Simpler models tend to have fewer large diagrams, less wasted fill space, and fewer visible seams - which can be an issue for auto UV mapped models.
2.3 GPU draw calls
Draw calls represent how many times the renderer needs to communicate with the GPU to render an object. In general, the GPU needs to be notified whenever you switch from one object to another or want to use a different texture. This means that if the same model consists of only one mesh and one set of textures, objects that are split into multiple parts or use many different materials will be more expensive to render.
2.4 Texture resolution
GPUs are good at choosing textures with a resolution appropriate to the viewing dimensions of the model, and avoiding the performance and quality issues associated with using too high a texture resolution. Given the viewing limitations of the transferred model, it is easy to include unnecessarily high-resolution textures. They won't actually be displayed at full resolution, which causes users unnecessary download times.
2.5 Overdraw
Overdraw is what happens when rendering polygons that end up behind other polygons. Some overdraw is unavoidable for most objects. However, for simple viewing scenarios, there may also be polygons that will never be seen no matter how the user interacts with the model. For example, imagine a sofa whose seat cushion is placed as a separate object on top of the sofa frame. In cases where the user is unable to remove the cushion, the bottom of the cushion and the portion of the frame where the cushion rests will never be seen.
A good optimization solution can identify these unnecessary areas and eliminate them so that polygons in invisible areas don't have to be downloaded or rendered. What's more, in many textured scenes, texture image space is allocated to these invisible polygons, which means you'll be wasting texture data, negatively impacting download size, and reducing texture resolution in areas that are actually visible.
2.6 The role of model optimization in protecting intellectual property rights
Finally, when working with data originating from CAD programs, 3D models often contain details about how a product is made. An optimized solution removes internal objects and converts small details into normal maps and texture information. This makes it difficult to reverse engineer a product from a model that is only used for visualization.
3. The basis of good pipeline
Implementing an automated pipeline that encompasses all of the above can be a challenging task. But getting the basics right can save you a lot of trouble later on.
3.1 Data layout
A prerequisite for any successful automation effort is a structured approach to data. It is crucial to have a clear idea of the material libraries used and which materials are assigned to which objects. You also want to make this material visible to the pipeline, this allows the pipeline to track changes so you don't waste time reprocessing things that haven't changed.
All data should be stored in some central repository and not allow references to any data outside of that repository to ensure that the entire source asset can be found without mounting an additional shared network drive (or other potential Location). When choosing a data format, they should be self-contained, or any references to other files should be easily accessible, to aid in tracking.
3.2 Dependency Tracking
Tracking relationships between assets allows you to rebuild only what has changed. The more granular your dependency tracking and job execution is, the smaller your incremental asset builds will be.
For example, suppose your dependency tracking treats a material library as a single opaque entity. If you make a change to a material, it will force a rebuild of every model to keep all material data up to date. If you track individual material changes, only those objects that use the modified material will be allowed to be rebuilt.
3.3 Execution
After resolving dependencies, you will eventually need to perform multiple build tasks to produce the desired output. These tasks are usually largely independent and can run in parallel. This means that the build process can scale across multiple CPUs or machines. The execution part involves scheduling these tasks and making sure the appropriate tools are invoked to produce the desired output.
Examples of tasks performed in the optimization pipeline include:
- grid optimization
- texture rendering
- texture compression
- scene assembly
3.4 Caching
Intermediate results during the build process can be cached to speed up incremental builds. An example is optimized low poly output. This probably won't work directly as an asset, as it requires textures to be applied before it's ready to deploy. However, this low poly asset can be cached as an intermediate step. This way, when a material used in an asset is updated, it doesn't have to be rebuilt, and the process can be simplified to avoid re-optimization.
Caching is a convenient solution to this problem, since the cache can be cleared without losing any data that cannot be recreated. This ensures that the size can be continually trimmed to get a good balance between incremental build performance and temporary data storage.
4. Example of 3D asset optimization pipeline
To provide some concreteness to this somewhat abstract article, I decided to build a simple implementation of an asset-optimized pipeline.
My aim here is to show different aspects of the pipeline, with a particular emphasis on showing how Substance's materials participate in the asset optimization workflow.
The goal of this pipeline is to take high-resolution 3D furniture models and automatically generate models that are small and fast to render. The pipeline is primarily implemented in Python and is designed to run natively on a single Windows machine.
The original pipeline described looks like this:
Adding details to the processing box, we get the following result:
Here is an overview of a single optimized model through the example pipeline. This pipeline can be applied to multiple models and multiple masses of the same model. Derived assets represent intermediate output that can be cached by the build system to speed up incremental builds.
4.1 Data
The data in this pipeline is a set of files on disk to keep things as simple as possible. They are divided into:
- Mesh: .obj file
- Source material: Substance .sbsar file
- Material Library: A .json file that references a .sbsar file, plus additional settings for selecting presets, parameters, and material scaling.
- Material assignments: .json files that associate parts of the model with material instances and scales for each model.
- Pipeline: A .json document describing the optimized target profile (eg mobile, VR, etc.).
- Job: A .json document describing which models to optimize, which material arrangements to use, and which pipelines to generate the output through.
4.2 OBJ as mesh file format
The reason I use .obj files for geometry is simplicity. They are easy to create and share. Since they are text-based, they can also be easily edited if the group name is missing or wrong for some reason.
The main limitation of meshes in the pipeline is that the UV chart must fit on the UV page. They cannot be used to tile materials on shapes, but they can overlap or be on different pages if preferred. Ideally, all UV charts should be sized relative to their size in world space on the model, so that textures are applied at the same scale on all parts.
Below you can see two different UV layouts of the same model. The layout on the left will cause problems because its graph crosses the UV tile boundary; in contrast, the layout on the right will function normally.
4.3 Material Library
The material library format used is a custom .json file, not the MTL format of .obj. The MTL format does not support binding Substance materials or setting procedural parameters, so I decided to introduce a simple custom format with these capabilities.
Here is an example of a material instance from the material library:
{
// ...
// Leather is the name of the instance
"Leather": {
// sbsar file referenced
"sbsar": "${assets}/material_library/sbsar/Sofa_Leather.sbsar",
"parameters": {
// These are parameters on the sbsar
"normal_format": 1,
"Albedo_Color": [
0.160,
0.160,
0.160,
1
],
"Roughness_Base": 0.353
},
"scale": [
// A material relative scale for the UV
20.0,
20.0
]
}
// Additional material instances
// ...
}
In the example pipeline, all material instances are stored in a single material library file.
These materials are based on PBR metal roughness, using the following maps:
- Base color: basic color map
- Normal: normal map
- Roughness: roughness map
- Metallic: Metallic map
4.4 Material assignment
A material assignment is a separate file that associates a part in the model with a specific material instance. Material assignments also include model-specific scale factors to compensate for differences between texture graph scales between different models.
Separating material assignment from geometry allows users to assign different material configurations to the same model, or to share material configurations between models that share the same group name.
Here's an example of a material assignment configuration:
{
// Legs is the name of the part in OBJ file
// If similar scenes share part names the same
// material assignment file can be used for all of them
"Legs": {
// Material refers to a material instance
// in the library
"material": "Metal",
// Scale is the scale of the material associated with this
// part. It will be multiplied by the scale from the
// material instance
"scale": [
1.0,
1.0
]
},
// Additional assignments to other parts
"Cushions": {
"material": "Leather",
"scale": [
1.0,
1.0
]
},
"Frame": {
"material": "Wood",
"scale": [
1.0,
1.0
]
}
}
4.5 GLB as output format
A .glb file is a version of .gltf where mesh data, scene data, and textures are packed into a single file. I use .glb as the output format for this process, as it is a compact file representation of meshes, materials, and all generated textures, and has broad industry support for web and mobile 3D viewers.
4.6 Pipeline definition
The pipeline describes different aspects of optimization to be performed for a specific target hardware.
An example pipeline looks like this:
{
"import": {
// Resolution for reference source textures
// Insufficient resolution in the source textures
// will come out as blurry areas in the model
// Must be an integer power of 2
"material_resolution": 2048
},
"reference": {
// Enable or disable reference model
// creation
"enable": true
},
"optimize": {
// Target size in pixels for which the model
// quality should be optimized. Anything above
// 2000 will be very time consuming to produce
"screen_size": 600,
// Resolution for the utility maps for the model
"texture_resolution": 1024,
// Bake tangent space using Substance Automation
// Toolkit if true, use Simplygon if false
"bake_tangent_space_SAT": true,
"remeshing_settings": {
// Angle in degrees between surfaces in
// a vertex to split it with discrete
// normals
"hard_edge_angle": 75
},
"parameterizer_settings": {
// How much stretching is allowed inside
// a chart in the generated UV layout for
// the model
"max_stretch": 0.33,
// How prioritized large charts are for the
// UV layout
"large_charts_importance": 0.2
}
},
"render": {
// Texture resolution for the atlas
// for the optimized model. Should typically
// be the same as the texture_resolution in
// optimize
"texture_resolution": 1024,
// Offset for mip map selection. 0 is default,
// Negative values gives sharper and noisier results
// Positive values give blurrier results
"mip_bias": 0,
// Enable FXAA post processing on the map to give
// smoother edges between different materials
// (doesn't apply to normal maps)
"enable_fxaa": true,
// Blurring of the material id mask before compositing
// to give smoother borders between materials
// (doesn't apply to the normal map)
"mask_blur": 0.25,
// Enable FXAA post processing on the normal map to
// give smoother edges between different materials
"enable_fxaa_normal": true,
// Blurring of the material id mask before compositing
// the normal map to give smoother borders between
// materials
"mask_blur_normal": 0.25,
// Clean up edges around charts on normal maps
"edge_clean_normal_maps": false,
// Normal map output format and filtering
// For most cases 8 bpp is enough but
// for low roughness and 16bpp is needed to avoid
// artifacts
"output_normal_map_bpp": 8,
// Enable dithering for the normal map. Typically only
// relevant for 8 bpp maps
"enable_normal_map_dithering": true,
// Dithering intensity. Represents 1/x. Use 256 to
// get one bit of noise for an 8bpp map
"normal_map_dithering_range": 256,
// Paths to tools for compositing materials and
// transforming normal maps
"tools": {
"transfer_texture": "${tools}/MultiMapBlend.sbsar",
"transform_normals": "${tools}/transform_tangents.sbsar"
}
}
}
4.7 Homework
A job is the entry point for specifying all model, material assignment, and pipeline processes. Here is an example of a job:
{
// Scenes to optimize
"scenes": {
"sofa-a1": {
// OBJ file with geometry in
"mesh": "${assets}/meshes/sofa-a1.obj",
// Different material variations to produce for this model
"material_variations": {
// These are references to material assignment files
"sofa-a1-leather": "${assets}/material_bindings/leather.json",
"sofa-a1-fabric": "${assets}/material_bindings/fabric.json"
}
},
// Additional scenes goes here
// ...
},
// The material library with material instances in to use
"material_library": "${assets}/material_library/material_library.json",
"pipelines": {
// A pipeline to run for the scenes
"lq": {
// Reference to the definition file
"definition": "${assets}/pipelines/lq.json",
// Paths for reference models and optimized models for
// this pipeline
"output_reference": "${outputs}/lq/reference",
"output_optimized": "${outputs}/lq/optimized"
},
// Additional pipelines to run
// ...
}
}
4.8 Python as the core language of the pipeline
Python is used to implement the pipeline. It's a widely supported language and has features that solve many of our problems out of the box. I'd like to use bindings for multiple tools in the process, allowing me to easily focus on building the pipeline rather than creating bridges to other applications.
4.9 SCons dependency tracking and execution
The dependency tracking and execution system used for the pipeline is the SCons build system. It's a Python-based build system that keeps track of dependencies and tries to minimize the cost of incremental builds by rebuilding only data that has changed since the last build. This means it works as an executor and also performs caching of intermediate results for us.
Using a Python-based build system is convenient because it makes interacting with build operations trivial. It's also available in the pip module system, and anyone with a Python environment can easily install it.
The pipeline also supports running directly as a Python script, making it easier to debug. When running a pipeline directly from a script, each run is rebuilt from scratch and there is no parallel processing of independent tasks.
4.10 Polygon Optimization
For optimization, I use Simplygon's Remesher 2.0. Simplygon is considered the gold standard for polygon optimization in the gaming industry and can generate very compact meshes with a great deal of control. It has several different optimization strategies that can be applied in different scenarios, but for simplicity I chose one for the pipeline.
The remester has many properties needed to optimize the mesh:
- It aggressively optimizes the model and, if used correctly, can often produce good results using orders of magnitude fewer polygons.
- It cleans up all internal geometry of the model, reduces overdraw, and texture assignments to unseen surfaces, as well as removes irrelevant or proprietary information from the model.
- It creates a new texture atlas for the entire model, which means it can be rendered with the same amount of texture data as a single draw call, no matter how the model was originally set up.
- It generates a mapping from the source mesh to the target mesh so that textures, normals, etc. on the source mesh can be transferred to the optimized model correctly and with high quality.
- It can be optimized to a specific viewing size. If you want to display a model with predictable quality in a viewer of a specific size or resolution, you can provide this information as a target resolution; your result will be a model that fits that size. The remesterizer can also suggest texture sizes for this particular quality, although this functionality is not used in this pipeline.
- I've worked at Simplygon in the past; so one of the reasons I'm using Remesher 2.0 in this case is because I'm very familiar with it - the types of results it produces, and how to configure the tool to work better with it.
Simplygon also has the following properties that make it suitable for this pipeline:
- It has a Python API that drives all optimizations and scene creation, making it easy to integrate the tool with the rest of the pipeline, which is also created in Python.
- Can read and write .obj and .glb files. This provides a lot of control over how materials and textures are managed; thus, the tool can be used to read source data with custom materials and write out .glb files with textures generated using Substance.
Remeshing is an aggressive optimization technique that is very effective for models viewed from a distance. However, it is not suitable for models that need to be examined too closely, as it can greatly affect object outlines and mesh topology. Therefore, it might not be the right solution for every visualization scenario. Specifically, the implementation used here does not handle transparency well, which should be avoided in this pipeline. By breaking the object's mesh topology, parts that were separated for animation purposes or similar may merge together, so this process requires special attention in this case.
There are many other 3D optimization solutions out there, but Simplygon was my natural choice due to its large number of features and my previous experience with it.
4.11 Texturing the Optimized Model
I'm using Substance Automation Toolkit to apply textures from a source mesh to an optimized mesh.
The Substance Automation Toolkit allowed me to build all the operations used in the texture transfer process using Substance Designer and call them from Python scripts in the pipeline. It also allows me to separate optimization from texture generation into two separate stages, which means that SCons can track the intermediate files independently; this way, as long as the model remains the same, changes in material data will not trigger geometry optimization, since the optimization itself is separate from the Material is irrelevant.
4.12 Format Conversion of 3D Assets
Sometimes you may need to convert .obj or .gltf to 3D assets in other formats, such as DAE, GLB, PLY, etc. You may consider adding NSDT 3DConvert, a powerful online 3D format conversion tool, to your optimization pipeline.
5. Pipeline
The pipeline consists of the following stages:
5.1 Texture rendering
At this stage we look at the material rigs of all processed models and render Substance PBR images of all used materials.
5.2 Reference model creation
The reference stage combines the original model with render textures to generate a reference .glb file.
This reference .glb file will not be used downstream, but it is a good resource for "before" and "after" footage. It also helps to debug whether problems in the output were introduced during optimization or in the source data.
Due to the limitations of the input UV coordinates described in the .obj section above, the UV chart is scaled according to the material scale and allocation scale. This ensures that the texture has the same scale as the optimized model.
5.3 Optimization
The optimization phase loads the source model and runs the remeshing process using Simplygon. During this process, the model will get a new UV set unrelated to the original UV set.
In addition to doing remeshing, it also uses Simplygon Geometry Casters to generate a set of textures, transferring texture data from the source model to the optimized model. These maps are represented in the new UV space of the optimized model:
-
Material ID. This map encodes the material index for each texel. From this map we can determine which material to assign to which point on the source model.
-
UVs. This map encodes the UV coordinates of the source model in the optimized model's texture space. Using this map, we can determine where to look in the textures on the source model to transfer data to the optimized model.
Note that this map is a 16bpp map between 0-1, that's why we can't use UVs to tile the material. The assigned material tiles are applied to the texture rendering stage.
UV remapped textures allow us to find the UV coordinates on the high poly model so that we can texture the optimized model with the original UVs:
- Ambient occlusion of the source model, expressed in the UV space of the optimized model. Since ambient occlusion is created using details from the source model, it will provide visual cues for lost geometry by occluding areas that might be lost in optimization.
- The world space normals and tangents of the source model, in the UV space of the optimized model. Using these two maps, we can capture the missing normals of the original model and transfer the tangent space normal map applied to the source model to world space for further processing.
- Optimize the model's world space normals and tangents. Using these maps, we can transfer the normals from the source model to the optimized model's tangent space normal map, capturing both the source mesh normal and the tangent space normal map applied to it.
5.4 Optimize the texture rendering of the model
This stage is a multi-stage process for transferring all source material maps from the source model using Substance Automation Toolkit. Since the substance graph cannot involve geometry in processing, it uses the graph from the optimization stage for the substance transfer. The basic process uses the UV transfer map to select sample locations, and picks textures based on the material ID map.
In addition to utility and material maps, this stage takes as input the scale and parameters of each material related to mipmaps to control tiling and filtering.
For normal maps, the process is more complicated because it not only needs to sample the source texture space normal map, but also involves the normal and tangent of the source and destination meshes in order to generate a tangent space normal map for the optimized model .
Note that the entire process may be delayed until after deployment. If the model is going to be used in a material profiling scene, this process can be run using the Substance engine or the Substance Automation Toolkit on the server to generate images on demand based on user input; this is much faster than running a full optimization pipeline.
5.5 Final Scene Assembly
The scene component uses Simplygon to load optimized geometry and assign it a new rendermap for saving .glb files with materials.
5.6 Implementation of job processing
Job processing is implemented as a Python script that understands and keeps track of all dependencies for jobs, pipelines, material libraries, etc.
The different build stages are separate Python files, and the pipeline applies the relevant parameters and input files to each stage. The contents of the file and parameters act as cache keys, meaning that if none of them have changed since the last run, you can reuse the old result instead of rebuilding it.
This means we try to apply the smallest set of parameters to each operation to avoid unnecessary reprocessing. As an example of how to prune data, only material instances referenced in the object's used material assignment file are applied as parameters, ensuring that only changes to material instances referenced by the model being processed will trigger a new build.
Processing scripts can be run in two ways:
- build mode. In this mode, it will recognize all build operations and execute them directly from Python. Note that this doesn't take any caching into account, and the entire pipeline will be re-evaluated on every run.
- Dry run mode. In this mode, all dependency resolution will be performed, but instead of running the pipeline, a list of build actions with corresponding parameters and all input and output files specified will be generated.
The output of dry run mode can be consumed by any build system, which can then build the results in parallel in dependency order.
5.7 SCons script
The SCons script will execute the pipeline in dry-run mode and use the output to specify all build tasks.
It uses the same build actions that python scripts use in build mode to perform tasks. SCons will then identify which tasks are independent and try to run as many tasks as possible in parallel to ensure the build runs quickly. It will also identify which targets are already up-to-date and keep them unchanged so that only the ones that have changed are built.
5.8 Code package
The code for the pipeline can be found on Substance Share. The package contains installation and running instructions so you can explore the pipeline yourself.
6. Results
6.1 Output model
The comparison here is to compare the created reference model with the optimized model.
This isn't necessarily a like-for-like comparison, since the texture density is significantly higher on the reference model, but it should give an estimate of the difference between the models. Also note that the numbers may not be the same between your own run and ours, since neither the optimization nor the texturing process is deterministic.
6.2 Sample lines
There are two lines in this process, LQ and HQ, for low quality and high quality respectively. LQ is aggressively optimized for use with thumbnail-sized models. HQ is for a 600x600 pixel viewer.
Check the model:
6.3 Number of polygons
| Model | polygon count | HQ Polygon Count | LQ polygon count |
|---|---|---|---|
| Sofa A1 | 96948 | 2226 | 256 |
| Sofa A2 | 133998 | 1496 | 168 |
| Sofa B1 | 740748 | 2628 | 238 |
| Sofa B2 | 1009690 | 1556 | 118 |
As you can see, the polygon count in the optimized model is orders of magnitude lower. Note also that denser source models do not generate significantly higher polygon counts than lower source models. This is a function that optimizes for a percentage of the target screen size rather than the original polygon count. This is desirable because different CAD packages, workflows, and designers can produce very different source data, but when visualizing data we want a consistent package size for a particular viewing scenario.
6.4 File size
The file size is determined by the combination of polygons and textures that generate the image.
| Model | Reference size HQ | Optimized size HQ | Optimized size LQ |
|---|---|---|---|
| Sofa A1 Leather | 80 MB | 3.4 MB | 0.24 MB |
| Sofa A1 fabric | 78 MB | 4.4 MB | 0.29 MB |
| Sofa A2 Leather | 80 MB | 3.3 MB | 0.22 MB |
| Sofa A2 fabric | 82 MB | 4.1 MB | 0.26 MB |
| Sofa B1 Leather | 102 MB | 3.3 MB | 0.26 MB |
| Sofa B1 fabric | 104 MB | 3.8 MB | 0.29 MB |
| Sofa B2 leather | 112 MB | 3.4 MB | 0.23 MB |
| Sofa B2 fabric | 113 MB | 4.0 MB | 0.27 MB |
As you can see, there is a huge difference in the download size between the original model and the optimized model. This comes at a price, and these models really aren't ideal for close inspection, but they represent meaningful viewing scenarios where the extra cost of a higher quality model is prohibitive. In all fairness, the reference model is not optimized for size, and there is a lot that can be done to make it smaller if a higher level of quality is required.
6.5 GPU draw calls
The optimized model combines textures for all materials into a single atlas, meaning they can be rendered as a single draw call.
The source models use 3 different material groups; therefore, most renderers will submit 3 draw calls for each model.
6.6 Overdraw
The internal geometry of the optimized model is cleaned up, which means there is very little unnecessary overdrawing.
As an example of a real-life situation where optimizing content can have an impact, I created an Adobe Aero project that includes all 8 models. One project uses the reference model and one project uses the optimization model.
The projects were created using the Adobe Aero beta desktop, then opened on an iPhone with a fast LTE connection to compare the differences in when the models were synced to the device so they were ready for use.
The first problem was that the two Sofa B1 and Sofa B2 were considered too heavy to be loaded in the project using the reference model, so they did not appear in the Aero iPhone app at all. Their source data has a significantly higher polygon count, and they are capped at model weight to ensure the application performs well.
6.7 Open time
Time to open and sync the project:
| model type | open time |
|---|---|
| reference | 4 minutes 50 seconds |
| optimization | 32 seconds |
6.8 Execution time Processing time
The optimized pipeline runs on a 6-core 2.6GHz Intel Core-7 CPU. This will generate a reference model and an optimized model for 2 material configurations for 4 different models.
| execution mode | execution time |
|---|---|
| Python native | 14 minutes 25 seconds |
| SCons parallel 6 cores | 5 minutes 50 seconds |
As you can see, the process is about twice as fast when run through SCons. This might be a bit surprising given that it can use 6 cores, but the reality is that many running processes are inherently multi-threaded - meaning you can't scale linearly by adding cores.
The real benefit of SCons comes when making minimal changes to assets and building incrementally, with turnaround times measured in seconds, as long as the operations performed are not time consuming and don't trigger bugs. A lot has changed downstream.
6.9 Execution time in context
The first thing to consider when looking at these numbers is to compare them to humans performing this work. Generating a low-resolution model is a time-consuming task in itself, potentially taking hours. This is also a task that needs to be redone to some extent every time the source model changes. Automating this process is a major win from a time consumption standpoint.
Another thing to keep in mind is that the process can be hidden from the user and run on a separate computer. This means it can run in the background of the build computer, ensuring the process doesn't slow down the user or tie up the workstation unnecessarily.
7. The future of work
The sample pipeline is very limited, but it should give you an overview of how pipelines allow you to optimize and deploy content more effectively. Here are some improvements that could be added to make it scalable and work better.
7.1 Texture Compression
The textures in the model are .png files. There are various ways to get smaller files:
- Implement pngquant in the pipeline. The pngquant tool is a lossy .png compressor that can significantly reduce the size of .png with very little loss of quality and does not require any special scaling in the viewer
- Work is currently underway on GPU hardware texture compression as a .gltf/.glb extension. Achieving this should allow for smaller files and faster load times.
7.2 Better Substance parameter support
The current material instance format is limited to work with scalars, vectors, booleans, etc. But it may error out if you try to use an enum string or a higher-level parameter type.
7.3 More optimization options/strategies
This optimization exposes only a single algorithm with a minimal set of parameters. There may be good reasons to introduce more options and algorithms to target a wider range of object types.
7.4 Mesh Compression
.gltf supports draco mesh compression. Its process is evaluated here, but since most of the data consists of textures, there is no significant impact on file size.
7.5 Grid Sharing
In this pipeline, the mesh is the same for different material changes. A setting that writes results to a .gltf file instead of a .glb allows mesh data to be shared between different files to use less disk space.
7.6 Decomposition operation
Both optimization and texture rendering are operations implemented by invoking multiple processes. If they are broken down into smaller operations, SCons will be able to find more parallelism in them and allow for finer-grained dependency tracking.
7.7 Normal map reduction
Normal maps are filtered using mipmaps. This is not the right filter for normal maps, normal details that are too fine to show up to the object's roughness can be shifted in order to improve quality. This is described in the paper on LEAN mapping.
7.8 Asset Reference
Current asset references (.sbsar files, meshes, etc.) are local file paths. This setup works fine on a single machine, but if you want to scale your pipeline to multiple machines, you may need to reference the files using some sort of URI scheme to ensure they can be referenced in a structured way across machines.
7.9 Setting overrides
In a real-world optimization process, there will be some assets that have issues or require special settings. A convenient way to make repeatable fixes (such as texture resolution, optimized quality, etc.) on a per-asset basis is to use settings overrides. This will allow users to override properties on a per-pipeline, per-asset basis; these will override the default values in the pipeline.
7.10 Larger scale
For large-scale deployments, such a build process can be distributed across multiple machines. However, this is not possible with SCons and requires a different build system.
Original Link: 3D Asset Optimization Pipeline Automation—BimAnt