Development of HarmonyOS learning path - multimedia development (media data management and development)

1. Overview of Media Data Management Development

The HarmonyOS media data management module supports the development of functions related to multimedia data management. Common operations include: obtaining media metadata, intercepting frame data, etc.

Before developing applications, developers should understand the following basic concepts:

• PixelMap

  PixelMap is an uncompressed bitmap format after image decoding, which is used for image display or further processing.

• media metadata

  Media metadata is data used to describe multimedia data, such as data information such as media title and media duration.

Constraints and Restrictions

In order to release native resources in time, it is recommended to actively call the release() method after the media data management AVMetadataHelper object is used.

2. Media metadata acquisition and development

scene introduction

Media metadata is data describing multimedia data, such as media title, media duration, media frame data, and the like.

Interface Description

interface name	describe
setSource(String path)	Read the media file at the specified path and set it as the media source.
setSource(FileDescriptor fd)	Read the specified media file descriptor and set the media source.
setSource(FileDescriptor fd, long offset, long length)	Read the specified media file descriptor, the offset of the starting position of the read data and the length of the read data, and set the media source.
setSource(String uri, Map<String, String> headers)	Read the specified media file Uri and request header, and set the media source.
setSource(Context context, Uri uri)	Read the Uri and context of the specified media, and set the media source.
resolveMetadata(int keyCode)	Get the value corresponding to the specified keyCode in the media metadata.
fetchVideoScaledPixelMapByTime(long timeUs, int option, int dstWidth, int dstHeight)	Acquire frame data according to the timestamp in the video source, acquisition options, and image frame scaling.
fetchVideoPixelMapByTime(long timeUs, int option)	Acquire frame data according to the timestamp and acquisition options in the video source.
fetchVideoPixelMapByTime(long timeUs)	According to the timestamp in the video source, get the data of the frame closest to the timestamp.
fetchVideoPixelMapByTime()	Randomly obtain the data of a certain frame in the data source.
resolveImage()	Get the image data contained in the audio source, such as album cover, if there are multiple images, return the data of any image.
fetchVideoPixelMapByIndex(int frameIndex, PixelMapConfigs configs)	According to the frame index and the specified image pixel format option, get the data of a specified frame in the video source.
fetchVideoPixelMapByIndex(int frameIndex)	According to the frame index, get the data of a specified frame in the video source.
fetchVideoPixelMapByIndex(int frameIndex, int numFrames, PixelMapConfigs configs)	According to the start index of consecutive frames, the number of consecutive frames, and the specified image pixel format option, get the data of the specified consecutive frames in the video source.
fetchVideoPixelMapByIndex(int frameIndex, int numFrames)	According to the start index of consecutive frames and the number of consecutive frames, get the data of the specified consecutive frames in the video source.
fetchImagePixelMapByIndex(int imageIndex, PixelMapConfigs configs)	Gets the image specified in the source image according to the image index, the specified image pixel format option.
fetchImagePixelMapByIndex(int imageIndex)	According to the image index, get the image specified in the source image.
fetchImagePrimaryPixelMap(PixelMapConfigs configs)	Gets the default image in the source image according to the specified image pixel format option.
fetchImagePrimaryPixelMap()	Get the default image in the source image.
release()	Release the read media resource.

Development Steps for Acquiring Frame Data

1. Create a media data management AVMetadataHelper object. You can set the media file to be read through setSource. If it is not set or the setting is incorrect, subsequent operations cannot be performed.

AVMetadataHelper avMetadataHelper = new AVMetadataHelper ();
avMetadataHelper.setSource("/path/short_video.mp4");

2. Specify the options for obtaining frame data, as well as the time to obtain the frame, and obtain the frame data of the media source

PixelMap pixelMap = avMetadataHelper.fetchVideoPixelMapByTime(1000L, 0x00); 

3. After obtaining the PixelMap object and completing the relevant information processing, call the release() function to release the read media resources.

avMetadataHelper.release();

Development steps for getting media metadata

1. Create a media data management AVMetadataHelper object. You can set the media file to be read through setSource. If it is not set or the setting is incorrect, subsequent operations cannot be performed.

AVMetadataHelper avMetadataHelper= new AVMetadataHelper();
avMetadataHelper.setSource("/path/short_video.mp4");

2. Specify the key of the media metadata to be obtained to obtain the media metadata. The following code obtains the duration information of the media:

String result = avMetadataHelper.resolveMetadata(AVMetadataHelper.AV_KEY_DURATION); 

3. After obtaining the media metadata, call the release() function to release the read media resources.

avMetadataHelper.release();

Development Steps for Acquiring Image Data for Audio

1. Create the media data management AVMetadataHelper object. You can set the audio media file to be read through setSource. If it is not set or the setting is incorrect, subsequent operations cannot be performed.

AVMetadataHelper avMetadataHelper= new AVMetadataHelper();
avMetadataHelper.setSource("/path/short_video.mp4");

2. Get the image data of the audio.

byte[] data = avMetadataHelper.resolveImage();

3. After obtaining the image data, call the release() function to release the read media resources.

avMetadataHelper.release();

3. Media storage data operation open

scene introduction

Media storage provides Uri link information for manipulating metadata such as media images, videos, and audios.

Interface Description

interface name	describe
appendPendingResource(Uri uri)	Updates the given Uri for handling media items that contain pending tags.
appendRequireOriginalResource(Uri uri)	Update the given Uri for the caller to get the original file content.
fetchVolumeName(Uri uri)	Get the volume name to which the given Uri belongs.
fetchExternalVolumeNames(Context context)	Get a list of all the specified volume names that make up external.
fetchMediaResource(Context context, Uri documentUri)	Get the corresponding media-style Uri according to the document-style Uri.
fetchDocumentResource(Context context, Uri mediaUri)	Get the corresponding document-style Uri according to the media-style Uri.
fetchVersion(Context context)	获取卷名为external_primary的不透明版本信息。
fetchVersion(Context context, String volumeName)	获取指定卷名的不透明版本信息。
fetchLoggerResource()	获取用于查询媒体扫描状态的Uri。
Audio.convertNameToKey(String name)	将艺术家或者专辑名称转换为可用于分组，排序和搜索的“key”。
Audio.Media.fetchResource(String volumeName)	获取用于处理音频媒体信息的Uri。
Audio.Genres.fetchResource(String volumeName)	获取用于处理音频流派信息的Uri。
Audio.Genres.fetchResourceForAudioId(String volumeName, int audioId)	获取用户处理音频文件对应的流派信息的Uri。
Audio.Genres.Members.fetchResource(String volumeName, long genreId)	获取用于处理音频流派子目录的成员信息的Uri。
Audio.Playlists.fetchResource(String volumeName)	获取用于处理音频播放列表信息的Uri。
Audio.Playlists.Members.fetchResource(String volumeName, long playlistId)	获取用于处理音频播放列表子目录的成员信息的Uri。
Audio.Playlists.Members.updatePlaylistItem(DataAbilityHelper dataAbilityHelper, long playlistId, int oldLocation, int newLocation)	移动播放列表到新位置。
Audio.Albums.fetchResource(String volumeName)	获取用于处理音频专辑信息的Uri。
Audio.Artists.fetchResource(String volumeName)	获取用于处理音频艺术家信息的Uri。
Audio.Artists.Albums.fetchResource(String volumeName, long id)	获取用于处理所有专辑出现艺术家的歌曲信息的Uri。
Downloads.fetchResource(String volumeName)	获取用于处理下载条目信息的Uri。
Files.fetchResource(String volumeName)	获取用于处理媒体文件的Uri。
Images.Media.fetchResource(String volumeName)	获取用于处理图像媒体信息的Uri。
Video.Media.fetchResource(String volumeName)	获取用于处理视频媒体信息的Uri。

开发步骤

以播放视频文件为例：

获取媒体外部存储中的视频URI需要使用的预定义字段是：AVStorage.Video.Media.EXTERNAL_DATA_ABILITY_URI

查询数据前需要动态授予权限："ohos.permission.READ_MEDIA"

1. 根据媒体存储提供的Uri链接操作媒体元数据。

Context context = getContext();
DataAbilityHelper helper = DataAbilityHelper.creator(context);
try {
    DataAbilityPredicates dataAbilityPredicates = new DataAbilityPredicates("_id>?");
    dataAbilityPredicates.setWhereArgs(Arrays.asList("0"));
    Uri myUri = AVStorage.Video.Media.EXTERNAL_DATA_ABILITY_URI;
    // columns为null，查询记录所有字段，当前例子表示查询id字段
    ResultSet result = helper.query(myUri, null, dataAbilityPredicates);
    if (result == null) {
        return;
    }
    while (result.goToNextRow()) {
        int id = result.getInt(result.getColumnIndexForName(AVStorage.Video.Media.ID));// 获取id字段的值
    } catch (DataAbilityRemoteException e) {
        // ...
    }
}

2. 获取到媒体ID后，即可通过设置媒体源来进行业务操作，如：播放。

Uri uri = Uri.appendEncodedPathToUri​(AVStorage.Video.Media.EXTERNAL_DATA_ABILITY_URI, String.valueOf(id)); // id为步骤1获取到的id
Context context = getContext();
Player player = new Player(context);
DataAbilityHelper helper = DataAbilityHelper.creator(context);
player.setSource(new Source(helper.openFile(uri, "r")));
player.prepare();
player.play();

四、媒体扫描服务操作开发

场景介绍

媒体扫描服务从新创建或下载的媒体文件中读取元数据，并将文件添加到媒体数据库中。

接口说明

接口名	描述
performLoggerFile(String path, String mimeType)	请求通过文件的路径和类型扫描一个媒体文件。
performLoggerFile(Context context, String[] paths, String[] mimeTypes, AVLogCompletedListener callback)	一次扫描多个媒体文件。
connect()	连接到扫描服务。
disconnect()	从扫描服务断开连接。
isConnected()	检查扫描服务是否已连接。

开发步骤

媒体扫描服务分为动态调用和静态调用，以扫描文件为例：

动态调用

1. 初始化AVLoggerConnection，并注册回调函数。

public class ScannerDemo implements AVLoggerConnectionClient {
    private AVLoggerConnection scanConn;
    public ScannerDemo(Context context) {
        // 实例化  
        scanConn = new AVLoggerConnection(context, this);    
    }

    @Override
    public void onLoggerConnected() {
        String path = "/path/";
        String mimeType = "video/mp4";
        scanConn.performLoggerFile(path, mimeType); // 服务回调执行扫描，指定要扫描的路径和文件类型    
    }

    @Override
    public void onLogCompleted(String path, Uri uri) {
         // 回调函数返回扫描到的URI和path的值
         scanConn.disconnect(); // 断开扫描服务        
    }
    // ...
}

2. 连接媒体扫描服务。

ScannerDemo zScanConn = new ScannerDemo(context);
zScanConn.connect(); // 连接扫描服务

3. 在onLoggerConnected回调函数中执行扫描，开发者通过自定义文件的路径和类型扫描指定媒体文件。

@Override
public void onLoggerConnected() {
    scanConn.performLoggerFile(filePaths[i], mimeTypes[i]); // 服务回调执行扫描，指定要扫描的路径和文件类型    
}

4. 在onLogCompleted回调函数中通知扫描结果。

@Override
public void onLogCompleted(String path, Uri uri) {
     // 回调函数返回扫描到的URI和path的值
     scanConn.disconnect(); // 断开扫描服务        
}

静态调用

1. AVLoggerConnection静态方法performLoggerFile, 扫描结果在onLogCompleted中通知。

AVLoggerConnection.performLoggerFile(this, filePaths, null, new AVLogCompletedListener(){
    @Override    
    public void onLogCompleted(String path, Uri uri) {

    }
});

五、视频与图像缩略图获取开发

场景介绍

用于应用获取视频文件或图像文件的缩略图。

接口说明

接口名	描述
createVideoThumbnail(File file, Size size)	根据文件和指定缩略图大小，创建指定视频中代表性关键帧的缩略图。
createImageThumbnail(File file, Size size)	根据文件和指定缩略图大小，创建指定图像的缩略图。

开发步骤

获取视频文件的缩略图。

File file = new File("/path/short_video.mp4");
Size size = new Size(400, 400);
PixelMap resMap = AVThumbnailUtils.createVideoThumbnail(videoFile, size);

获取图片文件的缩略图。

File file = new File("/path/short_video.mp4");
Size size = new Size(400, 400);
PixelMap resMap = AVThumbnailUtils.createImageThumbnail(imageFile, size);