The Smart Record and Replay Mechanism for Android

Authors: Kuei-Chun Liu, Yu-Yu Lai, Ching-Hong Wu, Hsiao-Han Huang

Abstract:

The number of Android applications (Apps) has increased rapidly in recent years. In order to get better programmatic control over Apps, we designed a record-and-replay mechanism to record Android input events and accessibility service events then make shortcuts. The shortcut is useful for complicated routine works and to Android beginners. We also generated graphical user interface (GUI) API by these shortcuts. GUI API helps developers make integrated Apps which can control other third-party Apps even if the official API is not offered by their providers. We demonstrated the usage of GUI API with two integrated Apps: Universal Bank App and Universal Communication App. Universal Bank App integrates three accounts from different banks and Universal Communication App integrates Line with WhatsApp. Both of them show the advantage of extendable GUI API. Furthermore, using our mechanism, shortcuts could replay almost all of the Top-100 Apps on Google Play correctly. In sum, the approach we present can help both Android developers and general users.

Keywords: graphical user interface, GUI API, record-and-replay, third-party apps