Pre-check of the shelf package

1. Summary of common rejection reasons on iOS

The app contains distribution, download and distribution functions (guide users to download apps, etc.).
The provided test account cannot view the actual function
The interface returns a boolean value to determine whether the app is upgraded, but the interface does not request during the audit
Audit account, any time you log in at any ip, you will see the audit version.
The login account and password provided are incorrect, and you cannot log in
There is a problem with the marketing keywords filled in by the operation
Metadata problem, the iPhone case in the iPhoneX screenshot is from iPhone7, it should be iPhoneX
Explain the role of privacy permissions.
Marketing text, certain competencies require qualifications. Such features are turned off during the review period
Modify the copywriting related to privacy rights, so that the reviewers can understand it, and achieve "intellectual and elegant"
App can't log in, it belongs to bug level
App is not compatible with ipad.
Privacy - Data Collection and Storage, indicating that the App does not collect privacy permissions.
There is a problem accessing the h5 page. at bug level
The App integrates the device fingerprint SDK, and will upload the list of installed applications on the user's device. Solution: Remove the device fingerprint SDK, successfully launched

2. Summary of App Rejection Reasons

Judging from some information about the rejection of Android and iOS 2 apps, the reasons for rejection are generally divided into the following categories:

During an audit, both resources and configurations should be adjusted to audit mode
App contains certain keywords
Audit-related metadata issues (screenshots do not match the actual content, models and screenshots do not match, and the account and password provided for auditing cannot be logged in)
The privacy rights used must be stated, and the copy description must be clear
There is a bug in the app (the account cannot be logged in, the ipad is not adapted, and the h5 cannot be opened)
Induce users to open and view more apps
Android apps are not hardened
The application lacks relevant qualifications and certificates

3. Scheme

There are many reasons for common audit failures, a large proportion of which is that there are some sensitive words in the code or text, so this article focuses on keyword scanning. For example, the screenshots set on the shelf do not match the current device, and the provided account cannot use the function

3.1 Who will collect the word cloud?

Generally speaking, each company has more than one business line, so the app situation and content of each business line are different, so the sensitive words are also very different. The collection of sensitive words should be collected by the developers who are mainly responsible for the App in the business line. According to the usual listing situation, Apple's rejected emails should be sorted out.

3.2 Scheme design

The company's self-developed tool cli (iOS SDK, iOS App, Android SDK, Android App, RN, Node, React dependency analysis, construction, packaging, testing, hot repair, burying, and construction), each end uses "templates" to provide capability. Contains several sub-projects, each sub-project is a so-called " template ", each template is actually a Node project, an npm module, mainly responsible for the following functions: the directory structure of a specific project type, custom commands for development, construction, etc. Templates are continuously updated and patched.

Therefore, you can get the source code for scanning during the packaging and construction (each end submits the project to the packaging system, and the packaging system schedules the packaging machine according to the project language and platform). Based on this status quo, the solution is "scanning is based on source code scanning".

According to pod installthis , cocoapods reserves hooks for us: PreInstallHook.rb, PostInstallHook.rb, which allow us to do some custom operations for the project at different stages, so our iOS template design also refers to this idea, before packaging and building, during construction , provides hooks after build: prebuild, build, postbuild. After locating the problem, what we need to do is to perform the coding of keyword scanning in the prebuild.

Decide when to do something, and then discuss how it is appropriate to do it.

3.3 Technical solution selection

The string matching algorithm KMP was what I thought of at the beginning. I tested the timing of an app and found more than 50 sensitive words. The code scanning took 60 seconds, which was very unsatisfactory. There is no problem with the KMP algorithm. So go ahead and change your mind.

Because the template version is essentially a Node project, the glob module under Node just provides matching to the appropriate file according to the regular pattern, and can also match the strings in the file. Then continue to do the experiment, the data is as follows: 9 inscription words, 5967 code files, it takes 3.5 seconds

3.4 Complete solution

Business lines need to customize sensitive word clouds (because the keyword clouds of each business line are different)
Sensitive words need to be graded: error, warning. If an error is detected, the build needs to be stopped immediately, and a prompt will be prompted that "your source code has been scanned to contain sensitive words ***, there may be a possibility of failure to submit the review, please modify it and build again". In the case of warning, you don't need to stop the build immediately. After all the tasks are completed, a prompt will be given: "It has been scanned that there are sensitive words ***, ***... in your source code, there may be a possibility of failure to submit for review, please develop confirm by themselves"
The format scaner.ymlfile .

error: The format of the array. Write the keywords that need to be scanned later, and the level is error, which means that the construction will be stopped immediately when an error is scanned.
warning: The format of the array. Write the keywords that need to be scanned later, and the level is warning. The scan results do not affect the construction, and are only displayed in the end.
searchPath: String format. The line of business can customize the path that needs to be scanned.
fileType: Array format. Lines of business can customize the types of files that need to be scanned. The default issh|pch|json|xcconfig|mm|cpp|h|m
warningkeywordsScan: Boolean. Lines of business can set whether to scan warning-level keywords.
errorKeywordsScan: Boolean. The business line can set whether to scan for error-level keywords.

error:
  - checkSwitch
warning:
  - loan
  - online
  - ischeck
searchPath:
  ../fixtures
fileType:
  - h
  - m
  - cpp
  - mm
  - js
warningkeywordsScan: true
errorKeywordsScan: true

There are private apis on the iOS side, but not on the Android side. There are 70,111 private api files, and each file assumes 10 methods, so there are 700,000 apis in total. So I plan to find out the top 100. Go to scan the match, support the option of whether the business line is turned on

In fact, these problems are standard practices in the industry, and such capabilities must be reserved, so the format of custom rules can be determined by viewing the fields of the yml file above. Once it is clear what to do and the standards for doing it, it can be quickly developed and implemented.

'use strict'

const { Error, logger } = require('@company/BFF-utils')
const fs = require('fs-extra')
const glob = require('glob')
const YAML = require('yamljs')

module.exports = class PreBuildCommand {
  constructor(ctx) {
    this.ctx = ctx
    this.projectPath = ''
    this.fileNum = 0
    this.isExist = false
    this.errorFiles = []
    this.warningFiles = []
    this.keywordsObject = {}
    this.errorReg = null
    this.warningReg = null
    this.warningkeywordsScan = false
    this.errorKeywordsScan = false
    this.scanFileTypes = ''
  }

  async fetchCodeFiles(dirPath, fileType = 'sh|pch|json|xcconfig|mm|cpp|h|m') {
    return new Promise((resolve, reject) => {
      glob(`**/*.?(${fileType})`, { root: dirPath, cwd: dirPath, realpath: true }, (err, files) => {
        if (err) reject(err)
        resolve(files)
      })
    })
  }

  async scanConfigurationReader(keywordsPath) {
    return new Promise((resolve, reject) => {
      fs.readFile(keywordsPath, 'UTF-8', (err, data) => {
        if (!err) {
          let keywords = YAML.parse(data)
          resolve(keywords)
        } else {
          reject(err)
        }
      })
    })
  }

  async run() {
    const { argv } = this.ctx
    const buildParam = {
      scheme: argv.opts.scheme,
      cert: argv.opts.cert,
      env: argv.opts.env
    }

    // 处理包关键词扫描（敏感词汇 + 私有 api）
    this.keywordsObject = (await this.scanConfigurationReader(this.ctx.cwd + '/.scaner.yml')) || {}
    this.warningkeywordsScan = this.keywordsObject.warningkeywordsScan || false
    this.errorKeywordsScan = this.keywordsObject.errorKeywordsScan || false
    if (Array.isArray(this.keywordsObject.fileType)) {
      this.scanFileTypes = this.keywordsObject.fileType.join('|')
    }
    if (Array.isArray(this.keywordsObject.error)) {
      this.errorReg = this.keywordsObject.error.join('|')
    }
    if (Array.isArray(this.keywordsObject.warning)) {
      this.warningReg = this.keywordsObject.warning.join('|')
    }

    // 从指定目录下获取所有文件
    this.projectPath = this.keywordsObject ? this.keywordsObject.searchPath : this.ctx.cwd
    const files = await this.fetchCodeFiles(this.projectPath, this.scanFileTypes)

    if (this.errorReg && this.errorKeywordsScan) {
      await Promise.all(
        files.map(async file => {
          try {
            const content = await fs.readFile(file, 'utf-8')
            const result = await content.match(new RegExp(`(${this.errorReg})`, 'g'))
            if (result) {
              if (result.length > 0) {
                this.isExist = true
                this.fileNum++
                this.errorFiles.push(
                  `编号: ${this.fileNum}, 所在文件: ${file},  出现次数: ${result &&
                    (result.length || 0)}`
                )
              }
            }
          } catch (error) {
            throw error
          }
        })
      )
    }

    if (this.errorFiles.length > 0) {
      throw new Error(
        `从你的项目中扫描到了 error 级别的敏感词，建议你修改方法名称、属性名、方法注释、文档描述。\n敏感词有 「${
          this.errorReg
        }」\n存在问题的文件有 ${JSON.stringify(this.errorFiles, null, 2)}`
      )
    }

    // warning
    if (this.warningReg && !this.isExist && this.fileNum === 0 && this.warningkeywordsScan) {
      await Promise.all(
        files.map(async file => {
          try {
            const content = await fs.readFile(file, 'utf-8')
            const result = await content.match(new RegExp(`(${this.warningReg})`, 'g'))
            if (result) {
              if (result.length > 0) {
                this.isExist = true
                this.fileNum++
                this.warningFiles.push(
                  `编号: ${this.fileNum}, 所在文件: ${file},  出现次数: ${result &&
                    (result.length || 0)}`
                )
              }
            }
          } catch (error) {
            throw error
          }
        })
      )

      if (this.warningFiles.length > 0) {
        logger.info(
          `从你的项目中扫描到了 warning 级别的敏感词，建议你修改方法名称、属性名、方法注释、文档描述。\n敏感词有 「${
            this.warningReg
          }」。有问题的文件有${JSON.stringify(this.warningFiles, null, 2)}`
        )
      }
    }

    for (const key in buildParam) {
      if (!buildParam[key]) {
        throw new Error(`build: ${key} 参数缺失`)
      }
    }
  }
}