Google has moved “computer use” from a specialized model into Google Gemini 3.5 Flash, making agent-style control of browsers, apps, and desktop workflows a built-in capability instead of a separate product. That means Gemini can now see and interact with user interfaces, reason about what’s on a computer screen, and take direct actions. A Google DeepMind senior scientist recently warned that scaled AI agents create incentives “for malicious people to do malicious things.”
Developers can now build agents that do a lot more than call APIs. They can automate GUI-only workflows such as testing software, filling forms, navigating dashboards, or using legacy apps with no API access. This reduces bottlenecks for automation and expands what AI agents can realistically do in production.
If software has a graphical user interface (GUI) but no API, an AI agent can still use it. Agents can be told to log into a dashboard, export yesterday’s SEO reports to a spreadsheet, compare them with last week’s data, and email the user a summary. The workflow is handled with natural language instead of relying on custom scripts to connect the dashboard, spreadsheet, and email.
What It Means For SEO
SEO tools may become far more agentic in the near future. Instead of just surfacing data, AI could log into Google Search Console, audit sites, crawl a site with Screaming Frog, extract specific data points for comparison, and execute repetitive optimization workflows.
For site owners, it also carries the implication that another set of AI agents may act as “visitors,” which could affect how site owners interpret site interactions and engagement signals for site and sales optimization.
AI Agents Will Be Attacked
Google’s announcement is pretty upbeat but the “safety best practices” document it links to bears paying attention to because failure to get this part right may result in theft and other poor user experiences.
The document explains:
“Computer Use presents unique security and operational risks, as a model acting on a user’s behalf might encounter untrusted content on screens or make errors in executing actions.”
That “untrusted content on screens” may be reference to the “traps” set for AI agents that the senior scientist at Google DeepMind warned against.
Google recommends seven best practices when this new AI agent:
1. Human-in-the-Loop (HITL):
Enforce user confirmation: When the safety response indicates require_confirmation (or legacy safety decision requires it), prompt the user for approval.
Provide custom safety instructions: Implement a custom system instruction to define and enforce your own safety boundaries.2. Secure execution environment:
Run your agent in a secure, sandboxed environment to limit its potential impact. This can be a sandboxed virtual machine (VM), a container (e.g., Docker), or a dedicated browser profile with limited permissions3. Input sanitization:
Sanitize all user-generated text in prompts to mitigate the risk of unintended instructions or prompt injection. This is a helpful layer of security, but not a replacement for a secure execution environment.4. Content guardrails:
Use guardrails and content safety APIs to evaluate user inputs, tool inputs and outputs, and the agent’s responses for appropriateness, prompt injection, and jailbreak detection.5. Allowlists and blocklists:
Implement filtering mechanisms to control where the model can navigate and what it can do. A blocklist of prohibited websites is a good starting point, while a more restrictive allowlist is even more secure.6. Observability and logging:
Maintain detailed logs for debugging, auditing, and incident response. Your client should log prompts, screenshots, model-suggested actions (function_call), safety responses, and all actions ultimately executed by the client.7. Environment management:
Ensure the GUI environment is consistent. Unexpected pop-ups, notifications, or changes in layout can confuse the model. Start from a known, clean state for each new task if possible.
Beware Of Trap-Filled Websites
As attack surfaces grow, the greater the likelihood that hackers will seek to exploit them. What that means is that as the number of AI agents on the web proliferates, hackers will turn their attention to exploiting them. Websites become the battlefield from which attackers launch attacks on AI agents.
A senior scientist at Google DeepMind recently said that malicious actors are already setting traps to steal money from humans by targeting their AI agents.
That’s not an exaggeration. Just this month, a cybersecurity expert in California experienced illicit charges made to his credit card due to Anthropic Claude’s AI agent. According to the article, he appears to have downloaded a Skills.md file that may have contained an AI agent trap.
The article reports:
“…he found a problematic add-on connected to Claude, referred to as a “skill,” similar to a plug-in. ‘That basically told Claude to attempt to purchase different types of gift accounts on my stored information. So it was using the digital wallet that was on my computer for Claude to start to make these purchases…'”
Site owners may need stronger bot controls and the ability to identify when hackers have hidden prompt-injection instructions on their sites. But that’s not something website owners are looking for, which compounds the problem for users who are utilizing AI agents like the one that Google just released.
Read more: Google DeepMind: Traps For AI Agents Are Already Stealing Money
Featured Image by Shutterstock/blocberry
