A few weeks ago, Google Gemini taught me some new graphic design skills.
I was editing a screenshot in Photopea, a free online Photoshop alternative, and wanted to place the image over a colorful border with a drop shadow behind it. Instead of digging through documentation or looking for YouTube tutorials, I just shared a live view of my web browser with Gemini and asked for guidance. Google’s AI assistant proceeded to walk me through Photopea’s complex menus step by step.
This is the most underrated feature of Gemini’s Mac app, which launched in April. While other desktop AI apps have increasingly focused on taking direct control of your computer, Gemini’s app still sees the value of teaching you to do things on your own.
Looking over your shoulder
When you click the + button in Gemini’s Mac app, you’ll see a Share Window among Gemini’s list of tools. Dragging your cursor over this option brings up a list of open windows to share with Google’s AI assistant.
(This feature does require some extra privacy permissions, enabled under Settings > Privacy & Security > Screen & System Audio Recording. From there you can turn on the Gemini toggle so that the app can automatically take screenshots.)
Once you’ve shared a window with Gemini, it will take a screenshot of that window each time you post a question. That means you can use Gemini alongside your other apps and get help along the way.
While creating my image border in Photopea, for instance, I ran into some trouble applying a gradient effect to my background. In response, Gemini looked at which menu was open in Photopea and told me exactly which buttons to click from there, citing Photopea’s online documentation.
I’ve since used Gemini for guidance in couple of other software interactions. It helped me navigate the labyrinthine Fangraphs website while looking up some recent baseball statistics, and after I vibe coded a couple of Raycast scripts for window management, it led me through Raycast’s Settings menu to enable them.
Other desktop AI apps have their own built-in ways to share your screen, but the process is clunkier. ChatGPT and Claude both require you to manually add new screenshots when something changes on your screen, and in Claude you must click and drag to define the capture area each time. Gemini’s Share Window mode feels more like a teacher that looks over your shoulder and offers guidance as needed.
While there’s no desktop Gemini app for Windows, Google offers a separate Google app for desktop on Windows with a similar share screen feature. The main difference is that the conversation flows through Google Search’s AI Mode rather than Gemini. (Microsoft’s Copilot app also has a screen-sharing feature, though in my experience its instructions haven’t been as helpful.)
What’s next
Instead of teaching you to use your computer more effectively, Google’s rivals are focusing more on controlling your computer themselves.
Both Claude’s desktop app and OpenAI’s ChatGPT Codex app now offer Computer Use modes that can navigate through your desktop with virtual cursors and keyboards, using persistent screenshots to guide them along. The hope is that you’ll be able to automate complex computing tasks even when you’re not at the computer yourself.
Google seems likely to go down this path before long. While the Gemini app can’t control your computer today, Google started previewing a Computer Use model for Gemini last fall.
But full computer control has its downsides. Anthropic warns of security risks from malicious apps and web pages, which could ask Claude to override the user’s own instructions. It also cautions against letting AI make decisions with “meaningful real-world consequences,” at least not without seeking human confirmation first. AI is also just a lot slower at clicking through buttons and menus, and letting these companies see everything on your screen is a potential privacy nightmare.
My hope, then, is that even as computer use becomes a bigger focus, Google doesn’t give up on allowing AI to play the role of software tutor. Not every computing task needs to be automated away, and there’s always value in learning to do it yourself.
