Introducing the Gemini 2.5 Computer Use Model: Advancing AI-Powered User Interface Interactions
October 7, 2025 – Google DeepMind has officially launched the Gemini 2.5 Computer Use model, a breakthrough AI technology designed to enable software agents to interact directly with user interfaces (UIs). Delivered through the Gemini API and available on Google AI Studio and Vertex AI, this specialized model builds on the advanced visual understanding and reasoning capabilities of Gemini 2.5 Pro, empowering developers to create intelligent agents that can navigate and manipulate digital environments just like human users.
What is the Gemini 2.5 Computer Use Model?
Traditional AI software often interacts with systems via structured APIs, which limits capabilities in scenarios involving graphical interfaces where direct human-like interaction is necessary—such as filling out forms, clicking buttons, or navigating menus. Gemini 2.5 Computer Use addresses this gap by enabling agents to engage with UIs through typical user actions including clicking, typing, scrolling, and selecting dropdowns.
This iteration is optimized primarily for web browsers but also shows strong potential in controlling mobile user interfaces. While desktop OS-level interaction remains outside its current scope, the model excels in completing complex workflows that require visually guided decision-making in dynamic environments.
How It Works
The Gemini 2.5 Computer Use model operates through a feedback loop powered by the computer_use tool in the Gemini API. When a user submits a request, the model receives several inputs:
- A screenshot of the current UI environment
- The user’s command or task description
- A history of recent actions performed by the agent
Based on these inputs, the model generates function calls representing UI actions, such as clicking a button or typing text into a field. Sometimes it will request user confirmation for sensitive actions like purchases. Once the client-side system executes the action, the model receives updated screenshots and URLs to evaluate outcomes and determine next steps, continuing this iterative process until the task is finished or halted.
Developers can customize supported UI actions by specifying exclusions or including additional custom functions, enabling flexibility for specific workflows.
Demonstrations and Use Cases
Several demos illustrate the model’s ability to handle nuanced tasks at rapid speeds:
- Extracting pet details from a California residency form and entering this information into a spa CRM, followed by scheduling specialist appointments.
- Organizing virtual sticky notes for an art club’s event planning, rearranging tasks into designated categories to streamline project management.
These examples demonstrate the model’s capacity to interpret complex instructions, access multiple websites or apps, and execute multi-step processes autonomously.
Benchmark Performance
Gemini 2.5 Computer Use leads in both accuracy and response speed across multiple web and mobile control benchmarks. Evaluations conducted internally by Google, as well as externally by Browserbase, highlight its superior performance in controlling browsers with notably lower latency compared to other leading alternatives.
Results are publicly documented in detail within Google’s official evaluation reports and Browserbase’s blog, underscoring the model’s robust reliability for practical deployment.
Commitment to Safety
Recognizing the heightened risks inherent in AI systems that control computer interfaces—such as potential misuse, unexpected behavior, and exposure to malicious web content—Google has embedded comprehensive safety guardrails into Gemini 2.5 Computer Use.
These protective measures include:
- Integrated safety features in the model to mitigate misuse risks
- A per-step external safety service that reviews each proposed action before execution
- Developer-configurable system instructions requiring user confirmation or refusal on sensitive actions
- Controls preventing harmful interactions like security breaches, system compromise, bypassing CAPTCHAs, or unauthorized control of critical systems
Developers are encouraged to apply best practices and thoroughly test their implementations to ensure safe, responsible usage.
Early Adoption and Applications
Prior to its general availability, Google teams incorporated the Gemini 2.5 Computer Use model into real-world projects such as:
- Accelerating UI testing workflows to improve software development efficiency
- Enhancing Project Mariner’s automation capabilities
- Powering the Firebase Testing Agent
- Enabling agentic features in AI Mode within Google Search
Participants in the early access program have also experimented with Personal Assistants, workflow automation, and more effective UI testing, reporting promising outcomes that validate the model’s versatility.
Getting Started
Developers interested in harnessing the Gemini 2.5 Computer Use model can access it via the Gemini API on Google AI Studio and Vertex AI. Google welcomes feedback through the Developer Forum to continue refining and expanding the model’s capabilities.
The launch of the Gemini 2.5 Computer Use model marks a significant milestone in AI’s evolution toward seamless, natural interactions with software environments. By empowering agents to “use” computers as humans do, this technology promises to unlock innovative applications across industries — revolutionizing productivity, automation, and digital assistance.
For more information, demos, and technical documentation, visit Google DeepMind’s official site and the Gemini API resources.
