Igeekphone reported on February 4th that Google has begun to have Gemini control Android phones. Now the specific implementation details of this “screen automation” have been revealed.
According to 9to5google, the 17.4 beta version of Google applications includes a related string called “Complete Tasks with Gemini”. The internal code name for this laboratory function is “Bonobo”. The introduction text states: “Gemini can, through screen automation technology, help you complete various tasks such as placing orders and booking trips in the designated applications on your device.”
According to IT Information, this control function is also known as screen automation. Android 16 QPR3 has laid the technical foundation for it, and this function will be made available for use in designated applications.
Google stated: “Gemini may make operational errors”, and “You are responsible for the actions it performs on your behalf, so please closely monitor its operation.” Users can terminate the operation of this intelligent agent at any time and take over the task execution manually.
Privacy-related instructions:
“When Gemini interacts with the application, if the activity recording function is enabled, the captured screen images will be reviewed by professional auditors and used to optimize various services of Google.”
“Do not enter login credentials or payment information in the Gemini conversation interface. Avoid using screen automation functions to handle urgent matters or tasks involving sensitive information.”
In this test version, there is also a digital likeness function with the internal code name “Wasabi” and related strings integrated. It is worth noting that the Android XR system uses this code name to refer to the 3D virtual image currently used in Google Meet calls. One of the strings indicates that this function can be invoked through instructions.