Author : Zhou Li (ask Jin), ants gold dress technical experts. In this paper, from Alipay service features, in-depth analysis of wireless payment experiments cluster evolution and development of the treasure, and explore how IoT and human-computer interaction and provide real time landing program.
Live video (copy the address into your browser open) :
http://t.cn/AiKDZg5G
0.5 Background
As the national level App, Alipay client needs to provide a wide range of services to one hundred million users, so the application of the stability and reliability of faces great challenges, need to continue to improve and optimize.
Today, let us stand round monitoring and optimization of the quality of the service point of view, started to explore migration path from terminal labs ants, from the use of open source by means of automation solutions, from research and to gradually improve wireless technology experimental cluster system, Alipay internal experienced a kind of business scenario exercises, as well as how to use the appropriate technical infrastructure for mobile development platform mPaaS external output.
1. development
Overall, the ants terminal laboratory from birth to now, experienced a total of three stages (tools, service and technology as well as in Taiwan), each stage has its characteristics and significance:
- Tools stage :
In this phase, the mainstream market using open source software based, open-source software such as client Appium, which covers the end of the iOS and Android; open source tool and in this way bind App testing processes, to mention measured quickly meet business needs side to help the business side automate testing in general (such as basic functionality testing, compatibility testing, etc.).
- Service stage :
Stage presence service is an important background: Payment end development process before and after the separation Bao started, and gradually precipitate out independent R & D process App end systems (R & D collaboration process with App build process). On a standalone basis App development processes and systems, the terminal laboratory support in the form of a service of App development and collaboration, process automation users to meet the daily work, but also play a continuous integration, daily automatic test package before release work etc; Also in the daily quality data were released to provide support, such as a client code coverage statistics.
- In Taiwan stage :
With the capability of the terminal laboratories and improve optimization, and gradually expand the scale of the test, and the service not only need to meet the ants gold dress System App (Alipay, word of mouth, network operators, banks, etc.) daily test requirements, but also need the ability to diffuse cover the entire Alibaba Group's business.
Followed by laboratory need to face a variety of business-side demand and customization capabilities, how diverse and complex business environment, the business side or the ability to build a complete system upstream? With this issue, the terminal labs to gradually settle and begin construction of the station platform: on the one hand to make universal service continues to sink, the other abstract standard SDK, which allows the business side specific capacity building according to their operational characteristics.
In addition, while building the platform of the terminal development laboratory fit together Alipay business scenarios, such as the construction of the laboratory network, the ability to code a series of real laboratory laboratory sweep.
Experienced several years of continuous development, the gradual completion of the laboratory terminal stage of transition, covers Android, iOS and IoT devices on its side, covering the general ability of the service, the applet access, research and development process construction, real machine Example rental and the like by control.
2. Technology Ecosystem
After completion of the course understand terminal lab, we are able to offer their services have a comprehensive understanding. When we went to summarize and analyze these services, you can put these specific capability divided into three parts: platform service capabilities, client SDK and laboratory capacity.
- Platform service capabilities
Service capacity of the target platform is focused "on how to build ants laboratory into a more open platform," so we need to consider how to get more business and the upstream side of the system with the ability to build participation, which will build a platform for ideas divided into two parts: the experimental device clusters and open SDK.
1. Device Cluster
Ants laboratory contains not only the thousands of public terminal equipment, covering the vast majority of mobile phone terminal market, to help students complete their daily business automated testing, but also provides a way of self-built laboratory users: Users need only according to their characteristic business scenarios procurement of equipment, lab deployment, they have the ability to run their own equipment on its own platform.
From the open and dynamic deployment of a platform perspective, the current device cluster to ensure that equipment ownership and business scenarios to achieve full isolation, to ensure that the business can use on the platform independent of each other. In addition, the face of Alibaba Group, a number of R & D center, also supports multi-device cluster deployed on the deployment, isolated from each other.
2. Open SDK
为了给上游系统和用户提供更为开放的能力，帮助业务方根据自身需求完成能力建设。终端实验室提供开放的 SDK 能力：上游系统只需在自己服务上接入 SDK，就能够完成任务构建链路，从用例管理、设备选择、任务执行，到执行结果回调，在此基础上用户就能够根据自身业务特点将业务数据进行多维度组合，形成自己的能力输出。
- 客户端 SDK
终端实验室经过几个阶段的发展，不仅提供 UI 自动化框架能力，而且在一些复杂场景做了深入研究和落地的工作。在这里我们以令大家头痛的“App 兼容性验证”作为切入点，结合目前常用的几种机器学习方案，分析方案的优缺点，最终形成了终端实验室的解决方案。
一方面伴随着移动互联网的快速发展，目前市面上手机的品牌和型号层出不穷，如何快速准确的验证 App 的功能在不同类型手机上运行有效性与稳定性，的确是件困难的事情；另一方面，目前针对图片的机器学习技术日益成熟，其图识别的准确性也完全能够满足日常兼容性的要求。
通常来说兼容性测试会采用两种方式：1.图像相似度计算；2. 无监督的异常点聚类。 这两种方式在使用方式和结果输出都有其优缺点：
- 对于“图像相似度计算”来说，其异常图片的识别成功率非常高，但其前提条件比较苛刻：用户需要对每一版 App 以及每一个业务点进行图片搜集和上传，而往往每条用例可能会包含少则几张图片多则十几张图片，对于几百、甚至几千条测试用例来说，就算是一版 App 的期望图片搜集工作都是巨大的，何况目前移动互联网普遍都是快速迭代发布，所以导致了这种预先处理图片的方式是不太可行的，下图是一般意义以图搜图的数据流：
- 另一种常用的方案是直接将同一业务场景下不同手机的一组截图交给无监督的异常点聚类算法处理，这种方案的优点比较明显：对于用户和平台来说，没有增加的额外的工作量，操作简单，但带来的问题是，计算出来的结果并不完全可信，特别是在一些极端情况下（如某一类异常图片总数较多的情况），少数正常的图片反而会被识别成异常图片，告知给业务方。
对比以上两种技术方案，终端实验室在兼容性异常图片发现上采用了更加灵活的方案，通过手机端“异常目标检测”和服务端“异常点聚类”相结合的方式完成目标。
首先，平台搜集常见异常图片，并训练成模型，植入手机端。
其次，当用户执行兼容性测试的时候，在手机端完成一部分“常见异常图片”的发现工作。
再次，当任务执行完后，服务端将剩下一部分图片交给““异常点聚类”处理，并进一步是被不同的图片。
最后，在整个执行任务结束后，平台就能有效识别异常图片，另外当异常图片未被有效识别的情况下，又可以在平台上快速提交异常图片，并交给算法逻辑继续学习，形成新的模型，从而在下一次任务执行过程中，就能把这种新发现的异常捕获住。
通过这种灵活的方案，一方面大大提升了异常图片检测结果的准确度，另一方面在整个异常图片的发现上形成了闭环，大大提升的兼容性测试的效能。
- 实验室能力
为了应对日益复杂的用户使用环境和不稳定的运行环境，终端实验室不断去构建各种专项实验室，尽可能在实验室环境里就把问题发现并推动研发流程去解决。同时伴随着 IoT 时代的到来，面对种类繁多的终端设备，如何能够通过实验技术的手段帮助研发同学提升效能，是一个新问题也是一个比较有挑战的问题：终端实验室通过托管 IoT 设备的方式，让用户快速方便寻找设备，并进行功能验证。具体技术方案是在原有的 Android/iOS 真机租用方案的基础上做了能力升级。
第一， 将终端实验室上某一款手机和 IoT 设备做关联，保证当浏览器通过 WS 远程操作手机打开摄像头就能够看到对应的 IoT 设备；
第二，通过 WS 读取 IoT 串口的 trace 信息，并将数据以 WS 的形式推送到用户浏览器端；
第三，在宿主机上集成 IoT 设备操作的 SDK，保证宿主机能够通过命令行或者 HTTP 方式操控 IoT 设备；
第四，宿主机集成语音转文字 SDK，这样当 IoT 设备发出声音时，就能够在页面上以文字的方式告诉用例。
通过这种远程 IoT 租用的方式，用户就能够快速做作一台远程设备，另外在给 IoT 设备发送指令的同时，可以看到设备的相应信息（视觉展示、声音展示以及实时日志信息），从而达到快速验证的目的。
- 机械臂扫码测试：
- 智能机柜支持真机云测
3. 借助 mPaaS 对外输出
以上介绍的蚂蚁金服终端实验室相应能力的构建与实践，目前已经通过移动开发平台 mPaaS 对外输出一部分能力。
在 mPaaS 平台上，我们将自动化测试框架，真机调度管理，场景化测试方案以及详尽的测试报告方案整合外部客户的现有业务场景和系统，从而覆盖 App 开发期的各个阶段，确保应用上线前获取充分测试，发现 bug，减少线上问题，提高整体用户体验。
目前，终端实验室不仅对内服务了包括蚂蚁金服体系下的支付宝 App、网商银行、口碑商家等，同时借助 mPaaS 与大量生态合作伙伴一同共建能力，包括常熟农商行、西安银行、泰隆银行等。由于篇幅限制，很多技术要点我们无法一一展开，欢迎大家通过技术文档或点击“阅读原文”进一步了解 mPaaS ：https://tech.antfin.com/docs/2/49549
| 活动推荐：MTSC 2019 测试开发大会
MTSC2019 第五届中国移动互联网测试开发会将于 6 月 28-29 日在北京国际会议中心举行，50+ 来自 Google，BAT，TMD 等一线互联网企业的测试大咖分享精彩议题，涵盖移动自动化测试、服务端测试、质量保障 QA、高新测试技术（AI+、大数据测试、IoT 测试）等专题。
蚂蚁金服多位技术专家将在大会上分享精彩议题，解密蚂蚁金服内部移动测试 2.0+ 演进之路、代码实时染色系统如何完成代码覆盖率检测等，期待与你交流。
往期阅读
《蚂蚁金服 mPaaS 服务端核心组件：亿级并发下的移动端到端网络接入架构解析》
《mPaaS 核心组件：支付宝如何为移动端产品构建舆情分析体系？》
关注我们公众号，获得第一手 mPaaS 技术实践干货
钉钉群：通过钉钉搜索群号“23124039”
期待你的加入～