About the Author
Leo, a senior mobile development engineer at Ctrip, focuses on cross-end technology and is committed to efficient and high-performance development.
Jarmon is a senior mobile development engineer at Ctrip, focusing on Flutter and iOS development.
1. Background
With the vigorous development of various multi-terminal technologies, the main body of projects has evolved from pure Native projects, to Native+RN, and now to Native+RN+Flutter. Since our business is based on the Flutter technology stack, this requires us to nest the display map. Currently, there are two main solutions for implementing nested display maps:
When connecting to the official Flutter map plug-in , the main problems faced are:
The officially provided plug-ins are not mature enough, and some existing Native APIs are not supported by Flutter;
Currently, there are very few applications that are connected to the Flutter map plug-in, so we need to make a trip.
Since the official adaptation is a pure Flutter project, the hybrid project may encounter many unknown and difficult problems.
Display the Native map directly on the Flutter page :
The Native map is mature and will not encounter big pitfalls;
The main problem is that the business is on Flutter, and Flutter needs a lot of interaction with map components, requesting data, and linkage. A large number of bridge methods are needed to transfer operation data;
To nest Native maps, you need to customize the container. Android and IOS have to implement one-pass bridge, container and map logic respectively, which increases maintenance costs.
Considering the maintenance cost and after careful consideration, we still chose to connect to the Flutter map plug-in. In order to better customize some APIs and more quickly fix some problems that are not officially updated in time. We use source code to access the Flutter map plug-in. This article will focus on the hybrid project based on flutter-boost and the problems and solutions encountered in connecting the Flutter map plug-in in single-engine mode.
2. How to integrate source code
Integrating plug-ins in hybrid projects is mainly divided into flutter and native sides. When integrating Flutter plug-ins, the source code of the plug-in can be directly downloaded from the official demo . This article uses the example of accessing the flutter map plug-in version 3.3.1.
2.1 Flutter-side integration
After obtaining the official demo, execute flutter pub get in the directory, then go to the flutter SDK to find the pub-cache dependency cache file directory, and import the code in the src file of each plug-in into the flutter project according to business needs.
2.2 IOS side integration
After executing flutter pub get, import the code in the iOS/Classes/ directory of each plug-in into the project as needed.
2.3 Android side integration
The integration on the Native side of Android is similar to that on the IOS side. Create a new map module in the Native project. Just put the Android part of the map plug-in source code in the map Demo into the project.
3. Map plug-in implementation principle: platformView
The map plug-in is divided into modules such as Map, Search, and Util according to its functions. Its basic implementation is similar. MethodChannel is used to communicate with native. We take Map as an example to analyze its implementation. The plug-in uses PlatformView to embed the native map into the flutter page. The flutter layer is UIKitView and AndroidView. The native initializes the BMFMapViewController based on the viewId after generating the map, including the corresponding MethodChannel. BMFMapViewController aggregates map operations and dispatches them to different modules to call map native methods.
3.1 What is PlatformView
PlatformView is a technology that allows native components to be embedded into Flutter pages. It allows us to display some native mature components, maps, WebView and other components that are difficult to implement with the Flutter UI framework in Flutter pages.
Flutter provides two ways to implement PlatformView: Virtual Display and Hybrid Composition. Virtual Display mode loads the native view into memory and renders it together with the flutter Widget. Hybrid Composition mode directly adds the native view to the flutter view layer. iOS adopts Hybrid Composition mode, and Android adopts Virtual Display and Hybrid Composition modes.
3.2 PlatformView implementation principle
1) flutter rendering process
Before introducing the implementation of Hybrid Composition, let’s first have an overview of the rendering process of flutter through the following figure.
After receiving the VSync signal, the Dart layer completes the update and generation of the Widget Tree, Element Tree, and RenderObject Tree in the UI Thread, and then generates a layer Tree containing drawing information and hands it to the Engine for rendering. Finally, it goes through the Compositor, Skia renders the flutter view.
2) Hybrid Composition mode analysis
Taking iOS as an example, we will step by step analyze the execution process of Hybird Composition mode. First, the Dart layer provides the UIKitView component to display the native view. In the didChangeDependencies method, the native view is initialized through the channel, a viewId that uniquely identifies the native view is generated, and the native view is cached in root_views_. When actually assembling the layer layer, the dart layer will be transmitted to the engine to display the coordinates and size of the native view, and a PlatformViewLayer will be generated. In other words, the position and size information of the native view are controlled by the dart layer.
void FlutterPlatformViewsController::OnCreate(FlutterMethodCall* call, FlutterResult& result) {
NSDictionary<NSString*, id>* args = [call arguments];
long viewId = [args[@"id"] longValue];
NSObject<FlutterPlatformView>* embedded_view = [factory createWithFrame:CGRectZero viewIdentifier:viewId arguments:params]; // 初始化
UIView* platform_view = [embedded_view view];
FlutterTouchInterceptingView* touch_interceptor = [[[FlutterTouchInterceptingView alloc]
initWithEmbeddedView:platform_view
platformViewsController:GetWeakPtr()
gestureRecognizersBlockingPolicy:gesture_recognizers_blocking_policies[viewType]]
autorelease];
ChildClippingView* clipping_view =
[[[ChildClippingView alloc] initWithFrame:CGRectZero] autorelease];
[clipping_view addSubview:touch_interceptor];
root_views_[viewId] = fml::scoped_nsobject<UIView>([clipping_view retain]); // 缓存
}After generating the Layer Tree of the current frame, it will enter the Rasterizer process. First, BeginFrame is called to render a frame, PlatformViewLayer::Preroll is triggered, PlatformViewLayer marks that the current frame has a PlatformView, and then FlutterPlatformViewsController::PrerollCompositeEmbeddedView is called to update view_params_, including Platform View coordinates, size and other information, and finally the native view is taken out in the SubmitFrame method and added to In flutter view, rendering is completed.
void PlatformViewLayer::Preroll(PrerollContext* context,
const SkMatrix& matrix) {
set_paint_bounds(SkRect::MakeXYWH(offset_.x(), offset_.y(), size_.width(),
size_.height()));
context->has_platform_view = true;
set_subtree_has_platform_view(true); // 标记当前帧存在Platform View
std::unique_ptr<EmbeddedViewParams> params =
std::make_unique<EmbeddedViewParams>(matrix, size_,
context->mutators_stack); context->view_embedder->PrerollCompositeEmbeddedView(view_id_,
std::move(params));
}3.3 How does PlatformView achieve frame synchronization?
In native development, we know that UI operations cannot be executed in other threads, and frames will be out of sync. There are four threads in the flutter engine: platform, ui, raster, and io. The native view is rendered on the Platform Thread (main thread), while flutter rendering is normally executed on the Raster Thread. How does flutter ensure frame synchronization?
Flutter solves frame synchronization through thread merging. In the PostPrerollAction method of the Raster process in the figure above, it will be judged that if there is a PlatformView, the Raster Thread and the Platform Thread will be merged in the subsequent drawing process, and the Raster queue task will be placed in the Platform queue. In this way, all rendering tasks are executed in Platform Thread, ensuring picture synchronization.
PostPrerollResult FlutterPlatformViewsController::PostPrerollAction(
fml::RefPtr<fml::RasterThreadMerger> raster_thread_merger) {
if (!HasPlatformViewThisOrNextFrame()) { // 没有Platform View不用处理
return PostPrerollResult::kSuccess;
}
if (!raster_thread_merger->IsMerged()) { // 线程还没有并不用处理
CancelFrame(); // 取消绘制当前帧
return PostPrerollResult::kSkipAndRetryFrame; // 合并后完成当前帧
}
BeginCATransaction();
raster_thread_merger->ExtendLeaseTo(kDefaultMergedLeaseDuration);
return PostPrerollResult::kSuccess;
}
// 合并队列
bool MessageLoopTaskQueues::Merge(TaskQueueId owner, TaskQueueId subsumed) {
if (owner == subsumed) {
return true;
}
std::lock_guard guard(queue_mutex_);
auto& owner_entry = queue_entries_.at(owner);
auto& subsumed_entry = queue_entries_.at(subsumed);
auto& subsumed_set = owner_entry->owner_of;
if (subsumed_set.find(subsumed) != subsumed_set.end()) {
return true;
}
owner_entry->owner_of.insert(subsumed);
subsumed_entry->subsumed_by = owner;
if (HasPendingTasksUnlocked(owner)) {
WakeUpUnlocked(owner, GetNextWakeTimeUnlocked(owner));
}
return true;
}4. Problems and Solutions
4.1 IOS page switching Map component white screen problem
When using flutter_boost hybrid development, when platformview is used in page A and a new container is opened to jump to page flutter B, a brief white screen will appear on platformView, and the native page will not appear when jumping from page A. Based on the appearance, I first guessed that it was caused by a single engine. When flutter A page jumps to other pages, SceneBuilder::pushTransform will be triggered to re-render page A.
void SceneBuilder::pushTransform(Dart_Handle layer_handle,
tonic::Float64List& matrix4,
fml::RefPtr<EngineLayer> oldLayer) {
SkMatrix sk_matrix = ToSkMatrix(matrix4);
auto layer = std::make_shared<flutter::TransformLayer>(sk_matrix);
PushLayer(layer);
// matrix4 has to be released before we can return another Dart object
matrix4.Release();
EngineLayer::MakeRetained(layer_handle, layer);
if (oldLayer && oldLayer->Layer()) {
layer->AssignOldLayer(oldLayer->Layer().get());
}
}When flutter A page creates a new container and pushes to flutter B page, viewDidLayoutSubviews will be triggered first. The viewController corresponding to the engine will be modified inside the method. SceneBuilder::pushTransform will be triggered after viewDidLayoutSubviews, while platformView is rendered in native and re-rendered. When entering page A, the corresponding platformView cannot be found, resulting in a white screen problem. SurfaceUpdated will not be triggered when pushing to a non-flutter page, so this problem will not occur.
- (void)viewDidLayoutSubviews {
...
if (firstViewBoundsUpdate && applicationIsActive && _engine) {
[self surfaceUpdated:YES];
}
...
}
- (void)surfaceUpdated:(BOOL)appeared {
if (appeared) {
[self installFirstFrameCallback];
[_engine.get() platformViewsController]->SetFlutterView(_flutterView.get());
[_engine.get() platformViewsController]->SetFlutterViewController(self);
[_engine.get() iosPlatformView]->NotifyCreated();
}
}The initial plan was to call sufaceUpdated in viewWillAppear, but it would get stuck in the release environment. Another solution is to change [super bridge_viewWillAppear:animated]; to [super viewWillAppear:animated]; [super viewWillAppear:animated]; which will call the parent class method, and the parent class method will call sufaceUpdated, which can solve the white screen problem.
4.2 Android map is stuck and cannot be operated.
1) Problem description
Page A has an embedded map and jumps to page B. Then return to page A and the map cannot slide.
Combined with the Flutter map plug-in mentioned above, the operation is actually passed to the Native map view for processing through MathChannel. We debugged the Native code and found that in the onTouch() method in the PlatformViewsController class, context reported an Attempt to invoke virtual method 'android.content.res.Resources android.content.Context.getResources()' on a null object reference.
public void onTouch(@NonNull PlatformViewsChannel.PlatformViewTouch touch) {
final float density = context.getResources().getDisplayMetrics().density;
}2) Analyze the problem
The error is reported because the context object is recycled. Now we can only find out the problem by analyzing why the context object is recycled. Reading the source code, we find that the context object is recycled only in the detach() method.
public void detach() {
context = null;
}Combined with the log output, it was indeed found that the attach() method was executed when returning to page A, but the detach() method was immediately executed. Now we need to find out why the PlatformViewsController of page A will execute datach().
从B页面 返回A页面
2022-08-22 15:13:08.126 21878-21878/ctrip.flutter.demo D/PlatformViewsController: B===>detach（）
2022-08-22 15:13:08.135 21878-21878/ctrip.flutter.demo D/PlatformViewsController: A====>attach（）
2022-08-22 15:13:08.249 21878-21878/ctrip.flutter.demo D/PlatformViewsController: A=====>detach（）View the call chain:
Reading the source code class by class, we found that in the OnDetach() method of FlutterActivityAndFragmentDelegate, the life cycle of the engine and the life cycle of the Activity are bound. When the page ends, the engine is destroyed.
void onDetach() {
if (host.shouldAttachEngineToActivity()) {
if (host.getActivity().isChangingConfigurations()) {
flutterEngine.getActivityControlSurface().detachFromActivityForConfigChanges();
} else {
flutterEngine.getActivityControlSurface().detachFromActivity();
}
}3) Solve the problem
Setting shouldAttachEngineToActivity returns false so that the Flutter engine will persist throughout the entire life cycle of the application and is independent of the Activity. When the Activity is destroyed, the Flutter engine will not be destroyed. The problem is solved. The reason for the problem is that our newly opened B page is created by opening a new container. The onDetach() method in the FlutterFragment of page B is executed after onAttach() of page A. Pure Flutter projects or using the Push method to open a new page can avoid this problem without opening a new container.
public boolean shouldAttachEngineToActivity() {
return false;
}4.3 Android map memory overflow problem
1) Problem description
If you open the Android Flutter map page multiple times, it will become more and more stuck. Later, the entire map will go black. It is obvious that there is a memory overflow. Through the memory tool Android Profiler that comes with Android Studio IDE, it can be clearly seen that every time a page is opened, the memory occupation will increase, and the memory of the end page will not be released.
2) Analyze the problem
Flutter Boost and the map plug-in have such a large amount of third-party code, how do we locate the problem? Is it caused by the plug-in or the framework? With the help of LeakCanary, you can easily find memory leaks.
Access is also very simple, introduce leakcanary in Android build.gradle.
debugImplementation'com.squareup.leakcanary:leakcanary-android:2.6'Then run the application and repeat the problem reproduction process until LeakCanary prompts. Check the stack information of leaks memory overflow. This is because SingleViewPresentation has always held the context object of the container TripFlutterActivity. I suspect there is something wrong with the MapView life cycle. Is dispose not executed? After debugging, the PlatformViewsHandler handler object is empty and subsequent processes will not be executed.
3) Solve the problem
Viewing the source code, only the PaltformViewsController detach() method will set handler to null.
public void detach() {
if (platformViewsChannel != null) {
platformViewsChannel.setPlatformViewsHandler(null);
}
}After debugging, the FlutterActivity container ends and PaltformViewsController detach() has been executed when the onDestroy() method is called. The container's onDestroy() precedes MapView's dispose, causing the handler object to become empty.
The idea to solve the problem is very simple. First retain the handler object during onDestroy(), and then find an opportunity to clear it. Use viewIdSet to maintain a copy of View data yourself. In the creat method, disposeArgs.get("id") deletes viewIdSet.remove(viewId) after executing the dispose method. If setPlatformViewsHandler is empty, check whether there is a view handler that executes dispose and do not recycle it first. as follows:
public void setPlatformViewsHandler(@Nullable PlatformViewsHandler handler) {
if(handler == null && viewIdSet != null && viewIdSet.size() > 0) {
needReset = true;
return;
}
this.handler = handler;
}Currently, when needReset is true when dispose is executed, handler will be set to null. Why is there no problem with the official demo? The main reason is that we have connected FlutterBoost, which is single-engine by default, and the official Demo is a pure Flutter project with multiple engines. When the page ends, the problem is covered by destroying the engine, so memory recycling is performed smoothly.
5. Custom text BitMap Marker
Customizing markers is a common requirement in the map business. Since the map is implemented through PlatformView, the easiest way to think of is to pass in the style ID corresponding to the marker and the data required for display through the Channel, and draw the marker on each end. This This approach will increase labor costs, and the styles may also be inconsistent, losing the advantages of the flutter framework.
The map plug-in in v3.0 (you need to implement it yourself before v3.0) provides the iconData parameter to pass in the image data information. On the flutter side, the text and images are drawn to generate a picture, and the generated image Data is passed to the native. This implementation is also There is no need to change the code at each end. When drawing, please note that the view size is physical pixels, not logical pixels.
Future<Uint8List?> customMark(String name, BuildContext context) async {
final scale = MediaQuery.of(context).devicePixelRatio;
final recorder = PictureRecorder();
final canvas = Canvas(recorder);
final paint = Paint();
final textPainter = TextPainter(textDirection: TextDirection.ltr);
...
final path = Path();
canvas.drawPath(path, paint);
// 绘制图片
final imageInfo = await UIImageLoader.imageInfoByAsset(HotelListImage.mapPoiMark);
paintImage(canvas: canvas,rect: rect,image: imageInfo.image);
// 生成绘制图片
final image = await recorder.endRecording().toImage(
width.toInt(), (textBgHeight + arrowHeight + iconHeight + 2).toInt());
final data = await image.toByteData(format: ImageByteFormat.png);
return data?.buffer.asUint8List();
}There was an episode when upgrading from flutter 2 to flutter 3. The process calling toImage in the iOS debug environment will be terminated. After flutter was upgraded, a thread check was performed on weak reference pointer calls. If the creation and use are not in the same thread, the debug environment process will be terminated. The fml::WeakPtr<SnapshotDelegate> snapshot_delegate weak reference pointer is used in the toImage() method. Since snapshot_delegate is created in the raster thread, the normal call should also be in the raster thread. When embedding PlatformView in a flutter page, in order to ensure rendering Consistency will merge the raster thread with the main thread, causing snapshot_delegate to be called on the main thread, triggering the thread check to terminate the process, but it will not affect the release environment.
class WeakPtr {
T* operator->() const {
CheckThreadSafety();
return get();
}
}
if (0 == pthread_getname_np(current_thread, actual_thread,
buffer_length) &&
0 == pthread_getname_np(self_, expected_thread, buffer_length)) {
FML_DLOG(ERROR) << "IsCreationThreadCurrent expected thread: '"
<< expected_thread << "' actual thread:'" // Object被创建的线程
<< actual_thread << "'"; // 实际执行线程
}6. Customize the Marker to be displayed in the visible range
After adding markers on the map, it is also a common requirement to display all the added markers within the visible range. The plug-in provides a showmarkers method that supports iOS, which obviously cannot meet the needs. We think about specifying the geographical range of the displayed map through setVisibleMapRectWithPadding. This method requires us to pass in the parameter visibleMapBounds and set the northeast and southwest coordinates of the geographical range. Since the longitude and latitude of the upper right corner and lower left corner are divided into the largest and smallest visible geographical ranges, the northeast and southwest coordinates can be obtained.
BMFCoordinateBounds? getMarkersVisibleMapBounds(List<BMFMarker> markers) {
if (markers.isEmpty) return null;
final firstPosition = markers.first.position;
double maxLatitude = firstPosition.latitude;
double minLatitude = firstPosition.latitude;
double maxLongitude = firstPosition.longitude;
double minLongitude = firstPosition.longitude;
for (final marker in markers) {
final lat = marker.position.latitude;
final lon = marker.position.longitude;
maxLatitude = max(maxLatitude, lat);
minLatitude = min(minLatitude, lat);
maxLongitude = max(maxLongitude, lon);
minLongitude = min(minLongitude, lon);
}
return BMFCoordinateBounds(
northeast: BMFCoordinate(maxLatitude, maxLongitude),
southwest: BMFCoordinate(minLatitude, minLongitude));
}As the business iterates, large maps need to be integrated into the list. In order to make the switching animation between the large map and the small map smoother, when the small map is loaded, the map size is actually rendered to the same size as the large map, and the lower part is blocked by the list. This means that the small map needs to set the offset of the visible range, but the inserts parameters are calculated in different ways for iOS and Android. iOS is calculated based on points, and Android is calculated based on pixels. A conversion is required to differentiate between platforms.
|
 |
Future<bool> setAllMarkersVisibleWithPadding(
List<BMFMarker> markers,
BuildContext context, {
EdgeInsets insets = const EdgeInsets.all(20.0),
}) async {
final bounds = getMarkersVisibleMapBounds(markers);
if (bounds == null) return false;
if (Util.isAndroid()) {
final scale = MediaQuery.of(context).devicePixelRatio;
insets = EdgeInsets.only(
top: insets.top * scale,
bottom: insets.bottom * scale,
left: insets.left * scale,
right: insets.right * scale);
}
return await setVisibleMapRectWithPadding(
visibleMapBounds: bounds, insets: insets, animated: true);
}7. Summary
The Flutter map plug-in is re-encapsulated based on the Native Map Android and iOS SDKs. It uses MethodChannel interaction in Flutter to realize functions such as map display, interaction, overlay drawing, and event response. Problems that are likely to occur when hybrid projects are connected to Flutter maps are basically concentrated in PlatformView. It is usually a matter of event and life cycle synchronization between the container and View.
This article mainly introduces the FlutterBoost hybrid project and various problems and solutions encountered when connecting to the Flutter map plug-in. The working principle of PlatformView is explained to facilitate us to better understand the Flutter map plug-in. At the same time, it also introduces how to use the tools that come with Android Studio to visually check memory abnormalities. We also recommend leakcanary to locate the classes and methods of memory overflow. I hope it will be helpful for you to access the Flutter map plug-in.
[Recommended reading]
Dry information | Ctrip Hotel Flutter performance optimization practice
Dry information | Ctrip air ticket App KMM iOS engineering configuration practice
Open source | Ctrip air ticket cross-end Kotlin DSL database framework SQLlin
Dry information | Reduce link time by 20%, Trip.com APP QUIC application and optimization practice
“Ctrip Technology” public account
Share, communicate, grow