This week, Microsoft introduced a new “computer use” feature in Copilot Studio, enabling AI agents to interact directly with both websites and desktop applications. The feature mirrors similar capabilities seen in OpenAI’s Operator and Anthropic’s Claude, allowing companies to create AI agents that use websites and apps like tools to automate tasks.
“Computer use allows agents to interact with websites and apps by clicking buttons, selecting menus, and typing in fields on the screen,” says Charles Lamanna, corporate vice president of Copilot for Business and Industry at Microsoft. “This allows agents to complete tasks even when there is no API to directly connect to the system. If a human can use the app, so can the agent.”
According to Microsoft, AI agents built in Copilot Studio will be able to handle routine tasks such as data entry, market research, and invoice processing. These agents can work with both web-based and desktop software interfaces by visually navigating through the interface like a human would. In addition, the agents will be capable of detecting changes in screens or buttons within the software or websites they interact with—helping them adapt and continue functioning without breaking down or needing reconfiguration. This makes them more resilient for real-world applications where interface elements frequently change.
Copilot Expands Its Capabilities
Earlier this month, Microsoft rolled out a similar function—called Actions—in its consumer-facing Copilot. Actions enables Copilot to run tasks in the background, such as booking restaurant tables, buying tickets, or purchasing products from online retailers. However, this feature is currently limited to a few selected partners.
Unlike Actions, the new “computer use” feature in Copilot Studio offers broader compatibility, giving businesses more flexibility when building automations that require interaction with third-party systems that don’t provide APIs, notes NIX Solutions. While Microsoft hasn’t shared the full list of compatible apps or platforms, it’s clear the focus is on increasing versatility and reducing barriers to automation.
This enhancement to Copilot Studio signals Microsoft’s continued investment in empowering users with more adaptable and capable AI tools. As more features roll out and additional integrations become available, we’ll keep you updated on what this could mean for productivity and business automation.