Windows 11 will now include a new image AI feature called Copilot, Microsoft announces

Microsoft is updating Windows Copilot to include a new feature that lets users capture a screen and ask AI to explain it. This new feature, “Add a Screenshot,” is currently being rolled out to the general public, but may not be available to everyone.

With the addition of a new button in the Copilot panel, users can capture a screen and upload it directly to the Copilot or Bing panel. Once uploaded, the screenshot can be discussed with Bing Chat, enabling users to ask questions about the screenshot.

This new feature is similar to the existing image upload feature on copilot.microsoft.com or Bing.com’s chat, but it has been optimized for screenshots. In a recent example, we asked Copilot to explain a screenshot of the Recycle Bin, and it promptly offered a short description.

Additionally, since Microsoft Edge powers Copilot, the screenshot feature allows users to draw over the screenshot or add specific instructions that Copilot can see.

Furthermore, for users who have access to ChatGPT-4 Turbo, the AI can explain emotions, including those depicted in abstract pictures. This feature is currently only available to select users who will receive the update at random. However, Microsoft plans to expand the rollout in the coming weeks.

Looking ahead, Microsoft is positioning Copilot as a central feature across its products, including Office and Windows. 2024 will see the addition of a dedicated Copilot button to Windows hardware as part of Microsoft’s efforts to “simplify people’s computing experience.”

Microsoft’s ambitions for Copilot are grand, with plans to integrate it into various areas, including the Windows keyboard. Older hardware can access Copilot using the Win+C keyboard shortcut, but Microsoft has confirmed plans to add a dedicated Copilot key to the Windows keyboard, marketing it as the most significant keyboard upgrade in nearly three decades. According to Microsoft, Copilot will be the entry point into the world of AI on the PC, and new-generation hardware will ship with the Copilot key.