Author: Busy fish technology - cloud o
background
In doing pictures, videos related functions when the album is a not open around the topic, because we have acquired basic needs pictures or video from the album. The most direct way is to call the Safe interface basic function is met, some advanced features to die, such as custom UI, multiple choice pictures.
We investigated the official image_picker, it also calls the system interface to handle the album, customizable level is not high, can not meet our requirements. So we choose to develop their own album Flutter components.
Our components need to have the following features:
- Complete the picture within the app, select the video, no need to rely on the system components albums
- The total number of pictures can be multiple-choice, supports the specified selected image
- When the multiple choice selection of the number reflect the UI.
- You can control the video, select the picture. For example: only allow the user to select a video, the picture is gray.
- Photo preview time can zoom, can also be added directly to the selection list.
Design ideas
API easy to use, feature-rich and flexible, with high resistance to order. Service may choose full access component can also select to customize a UI on the modules.
Flutter do UI presentation layer, specific data provided by the Native platform. This model, are isolated from the native UI code and data codes engineering. We often use the MVC architecture in the development of a native component of the time. Ideas developed Flutter components are basically similar. The overall structure is as follows:
As can be seen in the side Flutter is a typical MVC architecture, here is the Widget View, View and Model bindings, when the Model View will change to reflect changes to re-build the Model. View event triggers Native Controller to get the data and update the Model. Native and Flutter by Method Channel communication, there is no strong dependency between the two layers, simply press a communication protocol can be agreed.
Part of Native side, UIAdapter primarily responsible for model adaptation, bangs screen, full-screen identification and the like. Permission is responsible for media processing applications read and write permissions. Cache Cache is responsible for GPU texture, improve response time in the big picture preview. Decoder is responsible for parsing Bitmap, OpenGL Bitmap turn responsible for the texture.
It should be noted: This is a realization of our dependence on external flutter texture. In most of the pictures to see the whole album is a component GPU texture, so to java heap memory usage relative to the previous albums have achieved a significant reduction. If you use the machine in the low-end top album native system, due to memory, app risk of being killed systems. Phenomenon is the return from the system album, app restart. Use Flutter albums components, has changed in the low-end machines will experience above.
Some details
1 Page Load
Album list needs to load a lot of pictures, Flutter the GridView component has several constructors, relatively easy mistake is to use the first function, which requires the provision of a large number of widget at the outset. Should choose the second constructor, GridView will slide when the callback IndexedWidgetBuilder to get the widget, the equivalent of one kind of lazy loading.
GridView.builder({
...
List<Widget> children = const <Widget>[],
...
})GridView.builder({
...
@required IndexedWidgetBuilder itemBuilder,
int itemCount,
...
})Sliding process, the picture glide, which is not visible when you want to recover resources, we are here to here is the corresponding textures deleted. Continuous slide GridView, after rising in the memory will be in a stable, not always grow. If the rapid slide back and forth repeatedly texture creation and deletion, so there will shake the memory of the experience is not very good.
Thus, we maintain a picture of the state machine states are None, Loading, Loaded, Wait_Dispose, Disposed. Beginning loaded into the state from None Loading, this time the user sees is a blank placeholder or drawing, when the data status callback will come back to this time Loaded will re-build widget tree to display the picture icon, when the user take the time to slip into the state Wait_Dispose, this time not Dispose immediately, if you will come back and slippery from Wait_Dispose into the Loaded state, will not continue to Dispose. If the user does not slide back from the state will enter Disposed Wait_Dispose. When entering Disposed state, then the picture to be displayed when you need to re-take the loading process.
2 album Big Picture Show:
When you click the GridView of certain images will conduct this picture Enlarge the image more clearly user-friendly viewing. We know that camera images are very high resolution, if fully loaded, will be a great memory overhead, so we have been scaled when Decode Bitmap, and only up to 1080p. Enlarge the image can be summarized in three steps.
- 1 From the Bitmap file Decode
- 2 Bitmap converted into textures, and release Bitmap
- 3 texture to Flutter on display
In Step. 1, Bitmap Decode experience Android native apply equally to the Decode Bitmap width and height, and then to be calculated from the size of the display scaling factor, and the Bitmap Decode required.
Android photo album is mostly the angle of rotation, if not addressed directly displayed, there will rotate the picture 90 degrees issue, it is necessary to Bitmap rotation, a rotation using Matrix 1080p picture above in my test machine takes about 200ms , if the OpenGL texture coordinates is rotated, only greater than about 10ms, so the use of texture OpenGl rotation is a better choice.
During the big picture when preview will enter a horizontal sliding PageView, Flutter of PageView generally not take the initiative to load the adjacent page. For example, the display index is page 5 to page 4,6 when the index is not created in advance. There is a tricky way for viewportFraction we can set the parameters PageController become 0.9999. For the previous example, is displayed in the page index is 5 time, as the page index also needs to be displayed. 4, 6 0.0001. Such index is less than 1 pixel display page 4,6, substantially not see:
PageController(viewportFraction=0.9999)
There is another way to do that is pre-loaded in the Native side. For example: In the first five pictures loaded when the adjacent picture 4,6 textures loaded in advance, when the slide to 4,6 when the direct use of texture cache.
After the texture cache, a direct question: When will the release of textures? Until the time of the release of exit preview page displays all the textures are not very appropriate, if the user has browse memory will grow indefinitely. Therefore, we maintain a 5 LRU cache textures, during sliding, the oldest of the texture will be released. When pages exit the entire LRU cache will be destroyed.
3 About Memory
Album pictures using the GPU texture, will significantly reduce the Java heap memory usage, there is a certain improvement on the performance of the entire app. Note that, GPU memory is limited need for timely deleted after use, otherwise there will be risk of memory leaks. In addition, the Android platform delete texture when the need to ensure that in the GPU threads, or delete no effect.
Huawei P8, Android5.0 above tests were compared, Flutter original native albums and albums overall memory footprint basically the same, the GridView list page, add about 13M maximum memory. The difference is that the original native albums using the Java heap memory, Flutter album using Native memory.
to sum up
Albums component API is simple, easy to use, highly customizable. Flutter side structured, there is demand for customized UI Widget can be rewritten to achieve their goals. This album is another component that does not depend on the system of the album itself is complete, to maintain consistency UI, interaction and existing app. At the same time as the rear support more and albums related to play in future.
Next Steps
Since we are using the GPU texture, can be considered to support high-definition 4K display pictures, and client memory will not have too much pressure. But 4k Bitmap image texture turn consumes more time, UI interaction needs to be done to support the above loading conditions.
Components feature-rich, stable, be open, to give back to the community.