DJI UAV MobileSDK (remote control/mobile terminal) development v4 version <1>

DJI drone

Just finished the project delivery, strike while the iron is hot and share the new things I encountered this time. First of all, let’s take a look at DJI’s drones, which can be roughly divided into three levels.

• Entry-level : suitable for novices, flying in the wild to take pictures of scenery and so on. The operation is also simple, you can basically understand the flight tutorial, and you don’t need too much professionalism, and the aircraft is relatively small and has a relatively high safety factor. Such as: some aircrafts of Mavic 2 series.

• Consumer grade : The body will be a little bigger than the entry-level one, but the aircraft is equipped with some high-precision things, such as RTK, high-precision cameras, infrared lenses, etc. This type of aircraft requires pilots to have certain professional knowledge, and they need to obtain DJI's junior pilot certificate before they can start flying. Such as: some aircrafts of Phantom series and Wu series.

• Professional grade : larger in size, and there will be some serious consequences if it is not operated properly. Its advantages lie in high battery life, high-precision positioning, thermal imaging lens, and third-party megaphone. It is generally used for large-scale engineering operations. To use this level of aircraft, you need to obtain a DJI professional pilot certificate. Generally, this will give you a training course when you buy a professional-level aircraft. Generally, you can pass the test after listening to all of them. Typical representatives of this type of aircraft are: M30 series, M300 series.

SDK development kit

DJI's SDK development kit is currently divided into two major versions:

1. v4 version

The Mobile SDK is a software development kit designed to give developers access to the rich functionality of DJI's drone and handheld camera products. The SDK simplifies the process of application development by taking care of lower-level functions such as flight stabilization, battery management, signal transmission and communication. In this way, developers do not need to have rich background knowledge of robotics or embedded systems, but can focus on the development of industry applications related to DJI products.

The SDK includes:

• A library/framework that can be imported into an Android or iOS application to access the functionality of DJI products
• Flight simulators and visualization tools
• Debugging tool and remote logger for iOS
• Sample Code and Tutorials
• Developer Guide and API Documentation

Functional Overview

Developers can access the functions of many DJI products through the SDK. Developers can realize autonomous flight, control cameras and gimbals, receive real-time video image transmission and sensor data, download saved media files, and monitor the status of other components.

flight control

Mobile SDK provides three ways to control drone flight:

Manual control: The user uses the remote control to control the drone, and the SDK supports monitoring real-time video streams and sensor data.
Virtual Joystick Commands: The SDK supports generating control commands that simulate the joystick of the remote control.
Smart Mission: Convenient and easy to achieve advanced control of the drone. For example, waypoint missions can be used to allow the drone to fly along a predefined flight path.
Virtual joystick commands and smart missions allow simple yet powerful autonomous flight control of DJI drones.

camera

Both camera and gimbal functions support programming calls, for example:

Camera Mode: Video and still image capture
Exposure: Shutter, ISO, Aperture and Exposure Compensation are all customizable for maximum flexibility
Image Parameters: Aspect Ratio, Contrast, Hue, Sharpness, Saturation and Filters
Video Parameters: Resolution and Frame Rate
Orientation: Camera orientation and movement can be automatically controlled when using a gimbal

live video streaming

Developers can obtain the real-time video stream of the drone's main camera through the Mobile SDK. Get a live video stream even while the camera is capturing images or video to storage media.

sensor data

Developers can obtain rich sensor data through the SDK. GPS position, compass, barometer, flight speed and altitude are some of the sensor data acquired through the Mobile SDK with a frequency up to 10 Hz.

download media files

Developers can view and download photos and videos saved in the camera storage medium (SD card or solid state drive) through the Mobile SDK. Both preview and full image data can be accessed.

Remote control, batteries and wireless link

The remote, battery and wireless link are all accessible through the SDK. Typically, these components provide relevant state information, but developers can also exercise some control over them.

Connect Apps and Products

The figure below illustrates how the Mobile SDK integrates with the mobile application and connects with the DJI aircraft.

For handheld camera products, the remote control has been replaced with a handheld controller and there is no aircraft or other wireless link.

Mobile Apps are built with Mobile SDK, Platform SDK (iOS or Android), and run on mobile devices (Apple iPhone, iPad, Nexus Phone, Nexus Tablet, etc.).

Mobile devices can be connected to DJI products wirelessly through WiFi, or connected to DJI products through USB cables.

2. v5 version
The v5 version is only applicable to the M30 and M300 series, and it is still being updated continuously. Because it is a new version of the SDK, DJI engineers are constantly improving the content in it. I won’t go into details here. The next article will talk about the v5 version of the SDK in detail.

v4 version sdk secondary development

Don't talk too much, don't chatter, and serve the main course. "If a worker wants to do a good job, he must first sharpen his tools." First, prepare the development software. Because we use the Android version of the SDK, we use Android Studio to develop the software. I use the Android Studio Fox version. The versions feel the same and there is no difference in development. After preparing the tools, first go to the official website of the DJI drone developer to register a developer account, and register your own application, and get the special key value of the sdk. Give everyone an address [ DJI Developer Official Website ] Let’s see for yourself, it’s quite simple.

Register as a DJI developer

During the registration process, you are required to provide your email information and credit card or mobile phone number for registration verification. Any credit card information you provide will be used for verification only and will not be charged.

This guide assumes you are using Xcode 7.3 and Android Studio 2.1.1+.

Generate App Key

Each application requires a unique application key (App Key) to initialize the SDK.

To create an application App Key:

Please visit the developer center of the DJI developer website

• Select "Apps" on the left column.
• Select the Create App button on the right.
• Enter the application name, development platform, Package Name, category and description information.
• You will receive an application activation email to complete the generation of the App Key.
• You can find the AppKey in the developer center, copy and paste it into the application configuration.

Android sample code configuration

Download or clone the Android sample code project on Github .

Open the project in Android Studio, and paste the generated App Key string into the android:value under the "com.dji.sdk.API_KEY" meda-data element in the "AndroidManifest.xml" file.

<!--
    ADD API_KEY HERE and make sure you
    are connected to the Internet before
    the app is launched
-->
<meta-data
    android:name="com.dji.sdk.API_KEY"
    android:value="" />

Android Studio project integration

The screenshots in this section were generated using Android Studio 4.1.

Create a new application

A new application is available to demonstrate how to integrate the DJI SDK into an Android Studio project.

• Open Android Studio and select Start a new Android Studio project on the splash screen .

• On the New Project interface:
  • Set the Application name to "ImportSDKDemo".
  • Set the Company Domain and Package name to "com.dji.ImportSDKDemo".

Note: Package name is the identification string required to generate App Key. In this project, the Package name is "com.dji.ImportSDKDemo"

• On the Target Android Devices interface:

  • Select Phone and Tablet size.
  • Select API 23: Android 6.0 (Marshmallow).
    

• Select Empty Activity on the Add an Activity to Mobile interface.

• In the Configure Activity interface:
• Set Activity Name: to "MainActivity".
• Make sure to check Generate Layout File.
• Set Layout Name: to "activity_main".
• Click Finish.
  

Configure Gradle scripts

• Double-click build.gradle (Module: app) in Gradle Scripts
  

Update with the following:

apply plugin: 'com.android.application'

android {
    
    

    ...
    defaultConfig {
    
    
        ...
    }

    ...

    packagingOptions{
    
    
        doNotStrip "*/*/libdjivideo.so"
        doNotStrip "*/*/libSDKRelativeJNI.so"
        doNotStrip "*/*/libFlyForbid.so"
        doNotStrip "*/*/libduml_vision_bokeh.so"
        doNotStrip "*/*/libyuv2.so"
        doNotStrip "*/*/libGroudStation.so"
        doNotStrip "*/*/libFRCorkscrew.so"
        doNotStrip "*/*/libUpgradeVerify.so"
        doNotStrip "*/*/libFR.so"
        doNotStrip "*/*/libDJIFlySafeCore.so"
        doNotStrip "*/*/libdjifs_jni.so"
        doNotStrip "*/*/libsfjni.so"
        exclude 'META-INF/rxjava.properties'
    }
}

dependencies {
    
    
   ...
    implementation('com.dji:dji-sdk:4.15', {
    
    
    
    exclude module: 'library-anti-distortion'
    
    })
　　compileOnly 'com.dji:dji-sdk-provided:4.15
}

• The main changes are:

• Add packagingOptions to prevent unexpected app crashes.

• Add compile and provided dependencies to import the latest DJIAndroid SDK Maven dependencies.
  

  • Select Tools -> Android -> Sync Project with Gradle Files and wait for the Gradle project synchronization to complete.

• Confirm Maven dependencies again

  • Select File->Project Structure in the Android Studio menu to open the "Project Structure" interface. Then select the "app" module and click the Dependencies tab.
    

Implement application registration and SDK callback

Right click on com.dji.importSDKDemo and select New->Java Class to create a new Java class and name it "MApplication".

Open the MApplication.java file and replace the contents with the following:

package com.dji.importSDKDemo;

import android.app.Application;
import android.content.Context;

import com.secneo.sdk.Helper;

public class MApplication extends Application {
    
    

    @Override
    protected void attachBaseContext(Context paramContext) {
    
    
        super.attachBaseContext(paramContext);
        Helper.install(MApplication.this);
    }
}

• Here, rewrite the attachBaseContext () method, add Helper.install (MApplication.this); code.

Note: Since some SDK classes now need to be loaded before being used, the loading process is done by Helper.install(). Developers need to call this method before using any SDK functionality, otherwise unexpected crashes may result.

After the modification is completed, the Application name needs to be configured in the AndroidManifest.

Double-click MainActivity.java in the app module.

The MainActivity class needs to register the application to be authorized to use the Mobile SDK. It also needs to implement SDK callback methods.

• Start by modifying the MainActivity class to include several class variables, including mProduct, which is an object representing the DJI product connected to the mobile device.
• Also, the onCreate method will be modified to call the checkAndRequestPermissions method to check and request runtime permissions.
  Likewise, the checkAndRequestPermissions method will help to call the startSDKRegistration() method to register the application.
  Also, overriding the onRequestPermissionsResult method will help to check if the application has sufficient permissions, and if so, call the startSDKRegistration() method to register the application.
• Finally, replace the MainActivity class with:

public class MainActivity extends AppCompatActivity {
    
    

    private static final String TAG = MainActivity.class.getName();
    public static final String FLAG_CONNECTION_CHANGE = "dji_sdk_connection_change";
    private static BaseProduct mProduct;
    private Handler mHandler;

    private static final String[] REQUIRED_PERMISSION_LIST = new String[]{
    
    
            Manifest.permission.VIBRATE,
            Manifest.permission.INTERNET,
            Manifest.permission.ACCESS_WIFI_STATE,
            Manifest.permission.WAKE_LOCK,
            Manifest.permission.ACCESS_COARSE_LOCATION,
            Manifest.permission.ACCESS_NETWORK_STATE,
            Manifest.permission.ACCESS_FINE_LOCATION,
            Manifest.permission.CHANGE_WIFI_STATE,
            Manifest.permission.WRITE_EXTERNAL_STORAGE,
            Manifest.permission.BLUETOOTH,
            Manifest.permission.BLUETOOTH_ADMIN,
            Manifest.permission.READ_EXTERNAL_STORAGE,
            Manifest.permission.READ_PHONE_STATE,
    };
    private List<String> missingPermission = new ArrayList<>();
    private AtomicBoolean isRegistrationInProgress = new AtomicBoolean(false);
    private static final int REQUEST_PERMISSION_CODE = 12345;

    @Override
    protected void onCreate(Bundle savedInstanceState) {
    
    
        super.onCreate(savedInstanceState);

        // When the compile and target version is higher than 22, please request the following permission at runtime to ensure the SDK works well.
        if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
    
    
            checkAndRequestPermissions();
        }

        setContentView(R.layout.activity_main);

        //Initialize DJI SDK Manager
        mHandler = new Handler(Looper.getMainLooper());

    }

    /**
     * Checks if there is any missing permissions, and
     * requests runtime permission if needed.
     */
    private void checkAndRequestPermissions() {
    
    
        // Check for permissions
        for (String eachPermission : REQUIRED_PERMISSION_LIST) {
    
    
            if (ContextCompat.checkSelfPermission(this, eachPermission) != PackageManager.PERMISSION_GRANTED) {
    
    
                missingPermission.add(eachPermission);
            }
        }
        // Request for missing permissions
        if (missingPermission.isEmpty()) {
    
    
            startSDKRegistration();
        } else if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.M) {
    
    
            showToast("Need to grant the permissions!");
            ActivityCompat.requestPermissions(this,
                    missingPermission.toArray(new String[missingPermission.size()]),
                    REQUEST_PERMISSION_CODE);
        }

    }

    /**
     * Result of runtime permission request
     */
    @Override
    public void onRequestPermissionsResult(int requestCode,
                                           @NonNull String[] permissions,
                                           @NonNull int[] grantResults) {
    
    
        super.onRequestPermissionsResult(requestCode, permissions, grantResults);
        // Check for granted permission and remove from missing list
        if (requestCode == REQUEST_PERMISSION_CODE) {
    
    
            for (int i = grantResults.length - 1; i >= 0; i--) {
    
    
                if (grantResults[i] == PackageManager.PERMISSION_GRANTED) {
    
    
                    missingPermission.remove(permissions[i]);
                }
            }
        }
        // If there is enough permission, we will start the registration
        if (missingPermission.isEmpty()) {
    
    
            startSDKRegistration();
        } else {
    
    
            showToast("Missing permissions!!!");
        }
    }
}

The registerApp() method of DJISDKManager has a callback, which needs to handle two methods:
1. It is used to process the application registration result.
2. It is used to notify the connection changes of hardware products and mobile devices.

Go ahead and add the startSDKRegistration() method as shown below, and implement the onRegister(), onProductDisconnect(), onProductConnect(), onComponentChange(), onInitProcess() and onDatabaseDownloadProgress() and SDKManagerCallback methods:

private void startSDKRegistration() {
    
    
    if (isRegistrationInProgress.compareAndSet(false, true)) {
    
    
        AsyncTask.execute(new Runnable() {
    
    
            @Override
            public void run() {
    
    
                showToast("registering, pls wait...");
                DJISDKManager.getInstance().registerApp(MainActivity.this.getApplicationContext(), new DJISDKManager.SDKManagerCallback() {
    
    
                    @Override
                    public void onRegister(DJIError djiError) {
    
    
                        if (djiError == DJISDKError.REGISTRATION_SUCCESS) {
    
    
                            showToast("Register Success");
                            DJISDKManager.getInstance().startConnectionToProduct();
                        } else {
    
    
                            showToast("Register sdk fails, please check the bundle id and network connection!");
                        }
                        Log.v(TAG, djiError.getDescription());
                    }

                    @Override
                    public void onProductDisconnect() {
    
    
                        Log.d(TAG, "onProductDisconnect");
                        showToast("Product Disconnected");
                        notifyStatusChange();

                    }
                    @Override
                    public void onProductConnect(BaseProduct baseProduct) {
    
    
                        Log.d(TAG, String.format("onProductConnect newProduct:%s", baseProduct));
                        showToast("Product Connected");
                        notifyStatusChange();

                    }
                    @Override
                    public void onComponentChange(BaseProduct.ComponentKey componentKey, BaseComponent oldComponent,
                                                  BaseComponent newComponent) {
    
    

                        if (newComponent != null) {
    
    
                            newComponent.setComponentListener(new BaseComponent.ComponentListener() {
    
    

                                @Override
                                public void onConnectivityChange(boolean isConnected) {
    
    
                                    Log.d(TAG, "onComponentConnectivityChanged: " + isConnected);
                                    notifyStatusChange();
                                }
                            });
                        }
                        Log.d(TAG,
                                String.format("onComponentChange key:%s, oldComponent:%s, newComponent:%s",
                                        componentKey,
                                        oldComponent,
                                        newComponent));

                    }
                    @Override
                    public void onInitProcess(DJISDKInitEvent djisdkInitEvent, int i) {
    
    

                    }

                    @Override
                    public void onDatabaseDownloadProgress(long l, long l1) {
    
    

                    }
                });
            }
        });
    }
}

Finally, you need to implement the notifyStatusChange, Runnable and showToast methods:

private void notifyStatusChange() {
    
    
    mHandler.removeCallbacks(updateRunnable);
    mHandler.postDelayed(updateRunnable, 500);
}

private Runnable updateRunnable = new Runnable() {
    
    

    @Override
    public void run() {
    
    
        Intent intent = new Intent(FLAG_CONNECTION_CHANGE);
        sendBroadcast(intent);
    }
};

private void showToast(final String toastMsg) {
    
    

    Handler handler = new Handler(Looper.getMainLooper());
    handler.post(new Runnable() {
    
    
        @Override
        public void run() {
    
    
            Toast.makeText(getApplicationContext(), toastMsg, Toast.LENGTH_LONG).show();
        }
    });

}

App permissions must be granted for the DJI SDK to run.

• Double-click AndroidManifest.xml in the app module.

After package=com.dji.ImportSDKDemo, insert the following content before <application:

<!-- Permissions and features -->
<uses-permission android:name="android.permission.BLUETOOTH" />
<uses-permission android:name="android.permission.BLUETOOTH_ADMIN" />
<uses-permission android:name="android.permission.VIBRATE" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.WAKE_LOCK" />
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
<uses-permission android:name="android.permission.CHANGE_WIFI_STATE" />
<uses-permission android:name="android.permission.MOUNT_UNMOUNT_FILESYSTEMS" />
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />
<uses-permission android:name="android.permission.SYSTEM_ALERT_WINDOW" />
<uses-permission android:name="android.permission.READ_PHONE_STATE" />

<uses-feature android:name="android.hardware.camera" />
<uses-feature android:name="android.hardware.camera.autofocus" />
<uses-feature
    android:name="android.hardware.usb.host"
    android:required="false" />
<uses-feature
    android:name="android.hardware.usb.accessory"
    android:required="true" />

<!-- Permissions and features -->

• Add android:name=".MApplication" to the development of the application element:

<application
    android:name=".MApplication"
    android:allowBackup="true"
    android:icon="@mipmap/ic_launcher"
    android:label="@string/app_name"
    android:roundIcon="@mipmap/ic_launcher_round"
    android:supportsRtl="true"
    android:theme="@style/AppTheme" >

• After android:theme="@style/AppTheme">, insert the following code before:

<!-- DJI SDK -->
<uses-library android:name="com.android.future.usb.accessory" />
<meta-data
    android:name="com.dji.sdk.API_KEY"
    android:value="Please enter your App Key here." />
<activity
    android:name="dji.sdk.sdkmanager.DJIAoaControllerActivity"
    android:theme="@android:style/Theme.Translucent" >
    <intent-filter>
        <action android:name="android.hardware.usb.action.USB_ACCESSORY_ATTACHED" />
    </intent-filter>
    <meta-data
        android:name="android.hardware.usb.action.USB_ACCESSORY_ATTACHED"
        android:resource="@xml/accessory_filter" />
</activity>
<service android:name="dji.sdk.sdkmanager.DJIGlobalService" >
</service>
<!-- DJI SDK -->

• Insert android:configChanges="orientation" and android:screenOrientation="portrait" in the activity element as follows to prevent the activity from restarting when the screen orientation changes, and set the activity's screen orientation to portrait mode:

<activity android:name=".MainActivity"
          android:configChanges="orientation"
          android:screenOrientation="portrait">
    <intent-filter>
        <action android:name="android.intent.action.MAIN" />

        <category android:name="android.intent.category.LAUNCHER" />
    </intent-filter>
</activity>

Generate an App Key, and then replace the Please enter your App Key here. field in the AndroidManifest.xml file with the App Key string.