Background information
The evolution of veImageX of the Volcano Engine mainly introduces the gradual evolution process of veImageX within Byte from 2012 with the growth of Byte. The evolution includes V1, V2, and V3 versions and is eventually exported to the industry; the entire evolution process includes services Introduce the experience and solutions in image processing compression, cost saving and experience optimization from multiple perspectives such as terminal, client, network library, business scenario and optimization;
This article focuses on the capabilities evolved and provided on the web side. Pictures are an important element in a Web site. The size, format, resolution and rendering method of pictures have a significant impact on user experience. Volcano Engine veImageX provides businesses with a flexible and efficient one-stop image solution and static material hosting solution, covering all links in the image production and consumption stages such as uploading, storage, processing, distribution, and evaluation.
solved problem
Pictures are widely used in Web scenarios, from traditional graphics to video covers. Picture experience is a very important part of user experience. The LCP and CLS indicators commonly used to measure site performance both list pictures as One of the most important elements. With the development of business, the increase in the number of users has also brought about a rapid increase in CDN bandwidth costs. The most important elements are pictures and videos. Therefore, the solution starts from experience and cost, aiming to improve user experience while reducing bandwidth costs.
User experience visualization
Picture experience problems usually include the following points:
Slow loading speed: Image size, network, CDN, processing time and other factors will affect the loading time;
High loading failure rate: There are many factors that lead to image loading failure. The focus is on how to locate the problem in time;
Poor rendering experience: including scenes where the image area is blank for a long time, the page jitters after loading, and there is no guarantee after making an error;
Developers often ignore the image experience and do not understand the impact of images on site performance. There is also a lack of quantifiable data to measure the image experience of the site. Referring to the Lighthouse performance optimization guide, the solution integrates image compression, image lazy loading, image stable layout, error recovery and other capabilities, and integrates data monitoring capabilities. It can be combined with the Volcano Engine veImageX console to view real-time market data to provide data reporting for the business. , data analysis, data tracking, data alarm and other full-link support.
Bandwidth cost issue
The following issues often result in additional bandwidth costs:
Image compression rate is low;
The original resolution of the image and the rendering resolution do not match;
Use traditional low compression rate formats such as PNG and JPEG;
Images are not lazy loaded;
In addition to image compression, the solution supports adaptive loading of high-compression image formats such as WebP and AVIF and adaptive loading of image resolutions to reduce image size as much as possible. At the same time, it integrates lazy loading of images to avoid loading images in invisible areas, reduces site CDN costs, and also improves the overall loading speed of the site. According to internal business data, image transmission bandwidth and image loading time can usually be reduced by more than 50%.
Solution architecture
The solution can generally be divided into two parts: image loading and data monitoring.
As shown in the figure, the image loading part supports resolution, format adaptation, lazy loading, stable layout and other features. The image processing part is based on the veImageX service of the Volcano engine, such as image transcoding, scaling, compression, etc. The SDK side generates the best image format and resolution in the current environment, obtains the corresponding image URL from the service, and uses cloud processing capabilities to dynamically generate the required images at runtime.
The data monitoring part can be divided into loading time monitoring, image detail monitoring, image quality evaluation, large image monitoring, and cloud control configuration. The monitoring SDK collects relevant data and reports the data according to the configuration issued by the cloud. The Volcano Engine veImageX service analyzes the data. After cleaning, you can view the data disk on the console side.
Module details
Image loading
Image format adaptive
Common image formats include PNG, JPEG, GIF, WebP, AVIF, HEIC, etc. Among them, high compression rate image formats such as WebP, AVIF, and HEIC can significantly reduce image size. However, since different browsers have different support for high compression formats, the image loading environment needs to be considered when applying. The compatibility of the three high compression rate formats on the web side is as follows:
WebP
AVIF
HERE
On the APP side, the SDK soft solution can be used to decode and render unsupported image formats. The performance on the Native side can ensure that the time-consuming and traffic saving of image decoding can achieve good benefits. On the Web side, due to browser performance limitations, veImageX internal performance testing shows that the benefits of SDK soft decoding in terms of overall image time consumption are not obvious, especially in multi-image scenarios, so the format adaptive solution is more suitable on the Web side. , that is, loading the relatively optimal image format based on the browser's support.
A common approach is to use tags to achieve format adaptability. Tags have relatively good compatibility and support containing zero or more elements and one element to provide image versions for different browser environments. The browser will select from top to bottom. If there is no matching image that can be rendered, the image in the element will be selected as the backend. This solution was initially used to load the SDK, as follows:
<picture>
<source srcset="image1.webp" type="image/webp" />
<img src="image1.jpg" decoding="async" loading="lazy"/>
</picture>However, due to the large number of browser versions, many unexpected situations may occur in actual applications, such as:
Multiple image resources will be loaded at the same time, causing a waste of bandwidth;
Not all features of WebP are fully supported, and there may be loading failure scenarios;
Only supports AVIF still image format, not animated images;
...
In order to ensure the success rate of image loading, tags cannot be used directly in actual applications. The loading SDK currently uses a combination of format detection + to solve this problem. At the same time, because the HEIC support rate is too low, format adaptation currently only adapts to WebP and AVIF. Under the same quality, WebP can reduce the image size by 30% compared to JPEG, and AVIF can reduce it by 20% based on WebP. %;
Image resolution adaptive
Resolution adaptation means that the client obtains images with corresponding resolutions based on the actual rendered width and height, thereby reducing the image size. A common approach is that we can use the native srcset attribute in HTML to define the image set and the application scenario of each image. It consists of the following three parts:
file name
space
Image descriptor, there are two description methods
The width descriptor w, describes the intrinsic width of the image, in pixels. For example, 480w indicates the image resource that should be used when the browser requires an image 480 pixels wide.
The pixel density descriptor x describes the correspondence between the pixel density of the display and the image resource.
window.devicePixelRatioThe pixel density of the display can be queried through
sizes defines a set of media conditions, such as screen width. And specifies the optimal image size when the media condition is true. Each condition consists of the following three parts:
A media condition, for example
max-width:480px, indicating that the width of the visible window does not exceed 480 pixelsspace
The image size that should be used when the media condition is true
You can combine tags with srcset attributes to achieve format and resolution adaptation, as follows:
<picture>
<source
srcset="image1.webp 200w,
image2.webp 600w"
sizes="100vw"
type="image/webp"
/>
<img
srcset="image1.jpg 200w,
image2.jpg 600w"
sizes="100vw"
decoding="async"
loading="lazy"
/>
</picture>However, in practice we will face some problems, such as:
Specifying multiple srcsets will increase the size of the HTML file, especially when there are multiple scenarios;
Media query conditions can only be screen width and pixel density, which cannot accurately reflect the actual image rendering situation;
When srcset is used with sizes, the cost of understanding is relatively high;
...
In practical applications, in some cases, the image rendering size or the size of the area where the image is located can be known in advance. Combined with several built-in layout methods and device pixel density and other information, the SDK can be loaded internally to analyze and select the best rendering method for the current module. Best resolution.
Image Stability Layout
The web side is usually based on the CLS (Cumulative Layout Shift) indicator to measure the visual stability of the page layout. Layout offsets occur when the position of visible elements changes during the page life cycle.
There are many factors that cause layout offset (such as dynamically inserted elements, iframe loading), and unsized images are one of the important factors that affect the CLS indicator. For example, among the two pages below, the page on the right that specifies the width and height of the image is more stable than the page on the left that does not specify the width and height of the image.
Inspired by next/image, the loading SDK has four built-in stable layout methods: intrinsic, responsive, fixed, and fill. By generating a stable dom structure, it improves visual stability and reduces the amount of business development. The effect is as follows:
intrinsic: If the specified width is smaller than the container width, the image will be rendered according to the specified width and height; otherwise, the image width will be the container width, and the image height will be reduced proportionally;
Responsive: The image rendering width is equal to the container width, and the height is scaled proportionally;
fixed: render the image according to the specified width and height;
fill: The image is scaled to fill the container. The objectFit and objectPosition attributes can be passed in to represent different filling modes;
Lazy loading of images
The simplest method for lazy loading of images is based on the native attribute loading="lazy", but two problems have also been found in actual applications:
The compatibility of this attribute is not up to standard and most browsers do not support it;
There is a bug on some Safari browsers that may cause image loading to be blocked;
Therefore, the SDK is internally implemented based on the IntersectionObserver API, which is relatively more controllable and can set attributes such as lazy loading distance and target element.
Data monitoring
The overall link of data monitoring is:
Listen to the global Load and Error events and filter out the parts that belong to the image;
Monitor the loading of image resources based on PerformanceObserver. In this event callback, you can get the indicators related to the image loading time, such as DNS, TCP, SSL, request, and download time in each stage. You can also monitor the loading of image resources in CSS based on this API. ;
For information such as image format, status code, image quality score, etc., we rely on the Response Header. The only way to get the Response Header is to request the resource. Therefore, after the resource is loaded, request the information in the local cache. At the same time, in order to avoid concurrent requests Affecting other types of HTTP requests, the SDK will read the cache of images that need to be reported when idle based on the sampling rate, current request volume and other information;
Integrate all original data and report to veImageX data service according to the sampling rate, and the data service cleans the original data;
After back-end service processing, you can finally view it on the veImageX quality monitoring dashboard. The specific supported indicators and dimensions are as shown in the figure below:
Downstream network monitoring
Customer status monitoring
Solution evolution
The solution is committed to providing the ultimate image loading experience for Web scenes, while also continuously improving stability and scene coverage.
Lower error rate
As mentioned above, under some browsers, there will be scenarios where some WebP and AVIF images fail to load. After monitoring such scenarios, loading the SDK based on format detection solves such problems at the lowest cost while ensuring performance.
For example: Safari browser under iOS 14.3 & 14.4 fails to load part of WebP, and the tag does not detect WebP support. It accepts all incoming WebP formats, and the SDK cannot detect all The incoming image is detected, so this problem can only be avoided by constructing a specific image and detecting it before loading the business image, as follows:
const checkWebP = () => {
const pro: Promise<boolean> = new Promise<boolean>((resolve) => {
if(typeof window === 'undefined') resolve(false);
if (window['__support_webp__'] !== undefined) {
resolve(!!window['__support_webp__']);
} else {
const img = new Image();
img.onload = () => {
window['__support_webp__'] = true;
resolve(true);
};
img.onerror = () => {
window['__support_webp__'] = false;
resolve(false);
};
img.src = 'error image';
}
});
return pro;
};More scene coverage
The current solution supports React, Vue2, Vue3 and small programs. In order to ensure the consistency of experience and reduce maintenance costs, the loading SDK has been designed in a layered manner, extracting the core Core layer for use by each framework, and configuring each framework. This capability has been plug-inized.
summary
With the iteration of the solution, we are also trying to cover more business scenarios, such as encrypted image rendering, Hybrid HEIC rendering, etc. The volcano engine veImageX hopes to bring customers a comprehensive, stable and smooth picture experience, while also bringing benefits to the business. Ultimate cost benefit.
We encapsulate the above capabilities into a simple webSDK and export it to the industry. This SDK can be obtained and used for free. More advanced capabilities can also be used with veImageX;
webSDK access address: https://www.volcengine.com/docs/508/177943