Introduction: what to save you, flutter memory
Author: Busy fish technology - Yasushi book
background
The picture scheme used by Xianyu is a self-developed external texture scheme:
- The SurfaceTexture is created on the Android side, registered in the Flutter engine through FlutterJNI, and finally the texture id is returned to the Flutter application layer. The application layer uses the Texture Widget and the texture id to display the image texture.
- The texture data is on the Android side. The image texture is written to SurfaceTexture through OpenGL, and then the texture data is passed to the application layer through the shared memory in the Flutter engine, and finally handed over to Skia for rendering.
There are problems here:
The texture data of the Flutter application layer is not cached, and the Bitmap data needs to be re-rendered into a texture each time, and then handed over to the Flutter application layer for use. Native image loading will be cached in memory, and the image library provided by Flutter itself also has a cache. These two caches are isolated from each other and occupy a lot of memory space. Moreover, Flutter image caches are basically stored local resource maps, and most of our Flutter pages are actually external texture images downloaded from the network, resulting in low utilization of cache resources.
analysis
In view of the above three problems, let's put aside the technical implementation first, and if we want to solve these three problems, what is the most ideal solution:
- The texture is not cached, so we can add a texture memory cache to the application layer to solve it.
- When the upper application layer has cached textures, the memory cache of Bitmap on the Native side can also be removed, leaving only the disk cache of image resources.
- The memory cache of the entire App, only texture cache, Flutter ImageCache cache, in order to avoid the waste of memory resources, these two caches are combined into one
So the ideal solution:
there is only one memory cache in the entire App, and it can cache both textures and image data loaded by Flutter's Image Widget.
solution
ImageCache is officially provided, we can't remove it, and there are some places in Xianyu App that use Image Widget. The solution now becomes:
put the texture data in the ImageCache cache. When using texture, first fetch it from imageCache.
Let’s first look at the existing Flutter image loading logic and how the image is cached
As you can see from the figure, Flutter's image loading will call the ImageCache.putIfAbsent method. The cache is taken through this method. If the cache is not hit, the incoming loader method will be used to construct the corresponding ImageStreamCompleter, and the ImageStreamCompleter will complete the image loading. Logic.
When the cache is hit, the putIfAbsent method will directly return ImageStreamCompleter, which holds the imageInfo, and ImageWidget directly takes the ui.Image of the imageInfo to render.
Solution 1: Expand ImageCache, cache texture
ImageCache provides external caching method just a putIfAbsent 
At the beginning, what we thought was to construct the corresponding key, loader, and ImageStreamCompleter according to the method parameters, and then also use the putIfAbsent method to fetch the cache.
After trying, it doesn’t work. As shown in the figure below, when the picture is downloaded and decoded successfully, the listener method will be called back, in which the picture will be stored in the ImageCache cache
This listener callback has 2 parameters, ImageInfo stores the image data ui.Image.
Our application layer has no way to construct ui.Image, because this class is set to the application layer after the bottom layer of the Flutter engine completes the image decoding. The application layer has no way to actively set the value. As a result, in the listener, the value of imageSize cannot be calculated, and naturally there is no way to store it in the cache.
Option 2: Customize ImageCache
Because the cache queue of ImageCache is private, only the putIfAbsent method can store data in it. Then we only have another way, start with ImageCache's source code, customize imageCache, and then expand its functions.
Replace ImageCache with our custom
Because the ImageCache provided by Flutter cannot modify the code, we directly copy the source code of ImageCache, inherit ImageCache, and then replace the imageCache of PaintingBinding with a custom one.
As shown in the figure: Flutter's PaintingBinding has a method to expose createImageCache. We inherit WidgetsFlutterBinding and override this method to return our own ImageCache. In addition, we can also set various cache sizes for ImageCache here.
Function extension of ImageCache
In order not to modify the ImageCache code as much as possible, we directly define a new method of caching textures, aligning the logic of the putIfAbsent method, and the core code logic is as follows:
This method is mainly implemented by referring to the logic of putIfAbsent. In order to also cache the texture into ImageCache, the following key extensions are mainly made:
- TextureCacheKey is the key that uniquely identifies the texture. The key is mainly used to determine whether it is the same texture based on the width and height and url.
- TextureImageStreamCompleter is the texture management class. This class inherits ImageStreamCompleter and holds texture data and a callback for successful downloading. When the cache is hit, the object is returned to the application layer, and the texture id is obtained from it and handed to the Texture Widget for rendering
- When the cache is not hit, the incoming loader method is called to construct the TextureImageStreamCompleter, and the texture loading logic is executed. At the same time, a listener callback will be constructed and registered into TextureImageStreamCompleter.
- When the texture is loaded successfully, the listener method callback is executed. The method mainly calculates the texture size and puts it in the cache queue to check whether the cache size exceeds the maximum value. If it exceeds, the texture that has not been used for a long time is eliminated.
One point to note here is that
because ordinary pictures are dart objects, they will be automatically recycled by Dart VM, but the real data of our texture objects are in the shared memory of Engine, so we need to manually manage the release of textures. By the way, the reference count will only be released when there is no widget holding the texture and the reference count is 0.
Similarly, when the upper Texture Widget is dispose, it will also call the interface provided by ImageCache to see if the currently used texture is cached or is being used. Only when no, the texture will really be released
effect
We use the search result page as the test page, which has many big baby pictures and various repeated small pictures of labels. Use Huawei Honor 20 to test the physical memory usage before and after optimization.
The operation steps are: open the app, enter the search result page, search for the same keyword and enter the search result page, then slide through 100 pieces of data after 10s of silence, and finally stop the operation. During this period, the physical memory is sampled once per second for a total of 100s, and the following data is obtained
The blue curve is the memory usage before optimization, and the orange curve is after optimization. You can see that the memory usage is basically the same when you enter. The decrease in memory usage during sliding is caused by the GC that starts to reclaim the App's memory. On the whole, the total memory usage after optimization is less than before optimization, because the glitch caused by GC is also less than before optimization.
Outlook
Although the above scheme implements a memory cache in an App, and stores both textures and Flutter images, which saves memory space and improves memory usage, it still invades the ImageCache source code, and subsequent flutter engine upgrades and code maintenance , Additional work is required.
In addition, because the Flutter side loads the original image, all use the putIfAbsent method, and because the original image is loaded when the original image is loaded, this situation exists from time to time in our app. An image may occupy several M of memory, so we directly A large image monitoring method is added to putIfAbsent. When it is found that the size of the loaded image exceeds 2M, the data will be reported, including the image url, image usage information, and image size. In this way, we found several cases of improper use of images: directly using Image.network to load the original image, or Image.asset to load a large local resource.
Original link: https://developer.aliyun.com/article/776520?
Copyright statement: The content of this article is contributed spontaneously by Alibaba Cloud real-name registered users. The copyright belongs to the original author. The Alibaba Cloud developer community does not own its copyright and does not assume corresponding legal responsibilities. Please refer to the "Alibaba Cloud Developer Community User Service Agreement" and "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines" for specific rules. If you find that there is suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.