With the rise of AI in tech, Microsoft keeps injecting AI into Windows, and now even the firm itself is admitting there are safety risks in the new update.
Microsoft has introduced new agentic AI features for Windows 11 Insider users. These features allow AI to automate tasks such as sending emails and sorting files. By default, these options are turned off, and users must choose to enable them. However, Microsoft has published a security note on its website, cautioning that granting AI access to all of your files could pose security risks.
“As these capabilities are introduced, AI models still face functional limitations in terms of how they behave and occasionally may hallucinate and produce unexpected outputs,” Microsoft stated.
“Additionally, agentic AI applications introduce novel security risks, such as cross-prompt injection (XPIA), where malicious content embedded in UI elements or documents can override agent instructions, leading to unintended actions like data exfiltration or malware installation,” the note added.
In other words, it’s possible for something that’s meant to help users to harm them instead. This may very well be a super unlikely hypothetical edge case, but the fact that Microsoft felt compelled to say anything about it at all is alarming.
To address potential security concerns, Microsoft is introducing an experimental feature called “agent workspace.” This solution restricts the AI agent’s access to the PC, allowing it to operate in an isolated environment with scoped access to specific folders, rather than having full control over the user’s entire system.
We are still in the relatively early stages of all of this, so it will take some time to see how it shakes out. But the audience should be careful before turning on these features.