introduction
For privacy permissions such as reading and writing external storage, reading contacts, and sending text messages, Android started to perform dynamic authorization in the 6.0 system. However, in my country, it is not enough to only prompt the user with the authorization box. The Ministry of Industry and Information Technology issued a special rectification app for eight types of infringement reviews in early November 19, which clearly addresses the following eight types of issues:
1. Collect personal information privately;
2. Collect personal information beyond the scope;
3. Share information privately with third-party users;
4. Force users to use the targeted push function;
5. Do not give permission for use;
6. Frequently apply for permissions;
7 . Excessive request for permissions;
8. Set up barriers for user account logout.
Unfortunately, the online newspaper noticed criticism: the privacy authority was verified in the old version of our company's APP, but it was not effectively explained in the privacy document. After receiving the notice, the team immediately scanned the permissions and found that the APP had verified three privacy permissions in the AndroidManifest, but the actual process did not use them (some took a lot of mistakes). I believe that many teams face the same problem as us. Under multi-team development, there is no effective monitoring mechanism for the introduction of permissions. In order to avoid similar problems from happening again, this article gives a simple and effective code compilation layer interception scheme.
Before talking about the principle of the scheme, we first assume that the detection scheme is to scan the user-related privacy permissions verified in the APP AndroidManifest.xml file, and then compare the privacy documents and actual usage scenarios to make judgments. Faced with the detection scheme, we give solutions:
After the processApplicationManifest task runs in the compilation phase, scan the Merged Manifest Log file. If new permissions are used, a packaging error will be thrown until the problem is solved;
Source code brief reading
After the Android Gradle Plugin compiles the APP, it will generate a text file named [manifest-merger-${variantname}-report.txt] in the build/outputs/logs directory.
Take the AGP 3.5.0 source code as an example, simply analyze how the ProcessApplicationManifest task generates the Merged Manifest Log file.
package com.android.build.gradle.tasks;
/** A task that processes the manifest */
@CacheableTask
public abstract class ProcessApplicationManifest extends ManifestProcessorTask {
@Override
@Internal
protected boolean getIncremental() {
return true;
}
@Override
protected void doFullTaskAction() throws IOException {
... ...
MergingReport mergingReport =
ManifestHelperKt.mergeManifestsForApplication(
getMainManifest(),
getManifestOverlays(),
computeFullProviderList(compatibleScreenManifestForSplit),
navigationXmls,
getFeatureName(),
moduleMetadata == null
? getPackageOverride()
: moduleMetadata.getApplicationId(),
moduleMetadata == null
? apkData.getVersionCode()
: Integer.parseInt(moduleMetadata.getVersionCode()),
moduleMetadata == null
? apkData.getVersionName()
: moduleMetadata.getVersionName(),
getMinSdkVersion(),
getTargetSdkVersion(),
getMaxSdkVersion(),
manifestOutputFile.getAbsolutePath(),
// no aapt friendly merged manifest file necessary for applications.
null /* aaptFriendlyManifestOutputFile */,
metadataFeatureManifestOutputFile.getAbsolutePath(),
bundleManifestOutputFile.getAbsolutePath(),
instantAppManifestOutputFile != null
? instantAppManifestOutputFile.getAbsolutePath()
: null,
ManifestMerger2.MergeType.APPLICATION,
variantConfiguration.getManifestPlaceholders(),
getOptionalFeatures(),
getReportFile(),
LoggerWrapper.getLogger(ProcessApplicationManifest.class));
... ...
}
public static class CreationAction
extends AnnotationProcessingTaskCreationAction<ProcessApplicationManifest> {
private File reportFile;
@Override
public void preConfigure(@NonNull String taskName) {
super.preConfigure(taskName);
//这里就【manifest-merger-${variantname}-report.txt】文件
reportFile =
FileUtils.join(
variantScope.getGlobalScope().getOutputsDir(),
"logs",
"manifest-merger-"
+ variantScope.getVariantConfiguration().getBaseName()
+ "-report.txt");
}
}
}
Through the code, you can find that ProcessApplicationManifest is handed over to the ManifestHelperKt.mergeManifestsForApplication method to merge all Manifests, and the Log is saved in the [manifest-merger-${variantname}-report.txt] file.
package com.android.build.gradle.internal.tasks.manifest
/** Invoke the Manifest Merger version 2. */
fun mergeManifestsForApplication(
mainManifest: File,
manifestOverlays: List<File>,
dependencies: List<ManifestProvider>,
navigationFiles: List<File>,
featureName: String?,
packageOverride: String?,
versionCode: Int,
versionName: String?,
minSdkVersion: String?,
targetSdkVersion: String?,
maxSdkVersion: Int?,
outManifestLocation: String,
outAaptSafeManifestLocation: String?,
outMetadataFeatureManifestLocation: String?,
outBundleManifestLocation: String?,
outInstantAppManifestLocation: String?,
mergeType: ManifestMerger2.MergeType,
placeHolders: Map<String, Any>,
optionalFeatures: Collection<ManifestMerger2.Invoker.Feature>,
reportFile: File?,
logger: ILogger
): MergingReport {
try {
//ManifestMerger2是 manifest-merger库提供的辅助类
val manifestMergerInvoker = ManifestMerger2.newMerger(mainManifest, logger, mergeType)
.setPlaceHolderValues(placeHolders)
.addFlavorAndBuildTypeManifests(*manifestOverlays.toTypedArray())
.addManifestProviders(dependencies)
.addNavigationFiles(navigationFiles)
.withFeatures(*optionalFeatures.toTypedArray())
.setMergeReportFile(reportFile)
.setFeatureName(featureName)
if (mergeType == ManifestMerger2.MergeType.APPLICATION) {
manifestMergerInvoker.withFeatures(ManifestMerger2.Invoker.Feature.REMOVE_TOOLS_DECLARATIONS)
}
if (outAaptSafeManifestLocation != null) {
manifestMergerInvoker.withFeatures(ManifestMerger2.Invoker.Feature.MAKE_AAPT_SAFE)
}
setInjectableValues(
manifestMergerInvoker,
packageOverride, versionCode, versionName,
minSdkVersion, targetSdkVersion, maxSdkVersion
)
//关注这里的调用
val mergingReport = manifestMergerInvoker.merge()
//省略其他对merge结果处理代码
... ...
return mergingReport
} catch (e: ManifestMerger2.MergeFailureException) {
// TODO: unacceptable.
throw RuntimeException(e)
}
}
接着看manifestMergerInvoker.merge()的实现
package com.android.manifmerger;
/**
* merges android manifest files, idempotent.
*/
@Immutable
public class ManifestMerger2 {
public static class Invoker<T extends Invoker<T>>{
@NonNull
public MergingReport merge() throws MergeFailureException {
// provide some free placeholders values.
ImmutableMap<ManifestSystemProperty, Object> systemProperties = mSystemProperties.build();
... ...
FileStreamProvider fileStreamProvider = mFileStreamProvider != null
? mFileStreamProvider : new FileStreamProvider();
ManifestMerger2 manifestMerger =
new ManifestMerger2(
mLogger,
mMainManifestFile,
mLibraryFilesBuilder.build(),
mFlavorsAndBuildTypeFiles.build(),
mFeaturesBuilder.build(),
mPlaceholders.build(),
new MapBasedKeyBasedValueResolver<ManifestSystemProperty>(
systemProperties),
mMergeType,
mDocumentType,
Optional.fromNullable(mReportFile),
mFeatureName,
fileStreamProvider,
mNavigationFilesBuilder.build());
//调用下面的 private MergingReport merge()方法
return manifestMerger.merge();
}
}
/**
* Perform high level ordering of files merging and delegates actual merging to
* {@link XmlDocument#merge(XmlDocument, com.android.manifmerger.MergingReport.Builder)}
*
* @return the merging activity report.
* @throws MergeFailureException if the merging cannot be completed (for instance, if xml
* files cannot be loaded).
*/
@NonNull
private MergingReport merge() throws MergeFailureException {
// initiate a new merging report
MergingReport.Builder mergingReportBuilder = new MergingReport.Builder(mLogger);
//一系列merge manifest规则处理
... ...
MergingReport mergingReport = mergingReportBuilder.build();
if (mReportFile.isPresent()) {
writeReport(mergingReport);
}
return mergingReport;
}
//最终写入Log文件方法
/**
* Creates the merging report file.
* @param mergingReport the merging activities report to serialize.
*/
private void writeReport(@NonNull MergingReport mergingReport) {
FileWriter fileWriter = null;
... ...
fileWriter = new FileWriter(mReportFile.get());
mergingReport.getActions().log(fileWriter);
}
}
So far, I have seen how the Log file is generated from the code level.
Scheme realization
[Manifest-merger-${variantname}-report.txt] The general content of the file is as follows:
-- Merging decision tree log ---
manifest
ADDED from /somepath/AndroidManifest.xml:x:x-xx:xx
MERGED from [dependencies sdk] /somepath/AndroidManifest.xml:x:x-xx:xx
INJECTED from /somepath/AndroidManifest.xml:x:x-xx:xx
...
uses-permission#android.permission.INTERNET
方案代码实现很简单：
1.自定义一个Extension，列出暂禁用的权限；
2.实现相应Plugin和Task；
Extension定义可以如下所示：
host{
//明确暂禁用的权限列表
forbiddenPermissions = ['android.permission.GET_ACCOUNTS',
'android.permission.SEND_SMS',
'android.permission.CALL_PHONE',
'android.permission.BLUETOOTH',
... ...]
}
Plugin简单示例：
public class HostPlugin implements Plugin<Project> {
@Override
final void apply(Project project) {
if (!project.getPlugins().hasPlugin('com.android.application') && !project.getPlugins().hasPlugin('com.android.library')) {
throw new GradleException('apply plugin: \'com.android.application\' or apply plugin: \'com.android.library\' is required')
}
HostExtension hostExtension = project.getExtensions().create('host', HostExtension.class)
project.afterEvaluate {
def variants = null;
if (project.plugins.hasPlugin('com.android.application')) {
variants = android.getApplicationVariants()
} else if (project.plugins.hasPlugin('com.android.library')) {
variants = android.getLibraryVariants()
}
variants?.all { BaseVariant variant ->
MergeHostManifestTask taskConfiguration= new MergeHostManifestTask.CreationAction()
project.getTasks().create(taskConfiguration.getName(), taskConfiguration.getType(), taskConfiguration)
}
}
}
}
Task简单示例：
import org.gradle.util.GFileUtils
import com.android.utils.FileUtils
class MergeHostManifestTask extends DefaultTask {
List<String> forbiddenPermissions //禁用的权限列表
VariantScope scope
@TaskAction
def doFullTaskAction() {
File logFile = FileUtils.join(
scope.getGlobalScope().getOutputsDir(),
"logs",
"manifest-permissions-validate-"
+ scope.getVariantConfiguration().getBaseName()
+ "-report.txt")
GFileUtils.mkdirs(logFile.getParentFile())
GFileUtils.deleteQuietly(logFile)
checkHostManifest(forbiddenPermissions,logFile,scope)
if (logFile.exists() && logFile.length() > 0) {
throw new GradleException("Has forbidden permissions in host, please check it in file ${logFile.getAbsolutePath()}")
}
}
/**
* 检测host manifest 是否含有禁用权限列表
* @param forbiddenPermissions
* @param logFile
* @param variantScope
*/
public static void checkHostManifest(List<String> forbiddenPermissions, File logFile, def variantScope) {
if (forbiddenPermissions == null || forbiddenPermissions.isEmpty()) {
return
}
File reportFile =
FileUtils.join(
variantScope.getGlobalScope().getOutputsDir(),
"logs",
"manifest-merger-"
+ variantScope.getVariantConfiguration().getBaseName()
+ "-report.txt")
if (!reportFile.exists()) {
return
}
reportFile.withReader { reader ->
String line
while ((line = reader.readLine()) != null) {
forbiddenPermissions.each { p ->
if (line.contains("uses-permission#${p.trim()}")) {
logFile.append("${p.trim()}\n")
logFile.append(reader.readLine())
logFile.append("\n")
}
}
}
}
}
public static class CreationAction
extends TaskConfiguration<MergeHostManifestTask> {
BaseVariant variant
Project project
public CreationAction(Project project,BaseVariant variant){
this.project= project
this.variant=variant
}
@Override
void execute(MergeHostManifestTask task) {
... ...
HostExtension hostExtension = project.getExtensions().findByType(HostExtension.class)
task.forbiddenPermissions = hostExtension.getForbiddenPermissions()
task.scope= variant.getMetaClass().getProperty(variant, 'variantData').getScope()
task.dependsOn getProcessManifestTask()
}
private Task getProcessManifestTaskCompat() {
try {
//>=3.3.0
String taskName = variant.getMetaClass().getProperty(variant, 'variantData').getScope().getTaskContainer().getProcessManifestTask().getName()
return project.getTasks().findByName(taskName)
} catch (Exception e) {
}
}
}
If the APP or its dependent SDK has introduced disabled permissions, a compilation exception will be thrown, and the content of the generated [manifest-permissions-validate-${variantname}-report.txt] file is similar to the following:
android.permission.SEND_SMS
ADDED from /../app/src/main/AndroidManifest.xml:9:5-67
android.permission.BLUETOOTH
ADDED from /../app/src/main/AndroidManifest.xml:11:5-68
Concluding remarks
Regarding the list of privacy permissions, the relevant departments did not allow a complete list. It is recommended that the team use all dynamic permissions not described in the privacy document as a disabled permission until the privacy document is synchronized.
reference
1.Android Gradle Plugin：https://android.googlesource.com/platform/tools/base/+/studio-master-dev/build-system/gradle-core
This article has been included in the open source project: https://github.com/Android-Alvin/Android-LearningNotes , which contains self-learning programming routes in different directions, interview question collection/face sutras, and a series of technical articles, etc. The resources are continuously being updated …