Google has quietly rolled out a capability that changes how users interact with its AI inside the world’s most popular browser. The new “Select from screen” tool lets Gemini examine exactly what appears in a Chrome tab. Point at a chart, a block of text, a product image or a confusing diagram. The model receives it as context. No more copying. No more awkward descriptions. Just selection and query.
This arrives as part of Chrome 149. Digital Trends first detailed the rollout on June 24, 2026. Users access it through the Gemini side panel. Click “Ask Gemini,” open the add menu via the plus icon, choose “Select from screen,” then draw a box around the desired content. The highlighted portion attaches directly to the prompt. Simple. Effective.
But the implications stretch further. For months Google has pushed Gemini deeper into Chrome. Earlier this year the company moved the assistant from a floating window into a persistent sidebar. That shift, reported by TechCrunch in January 2026, allowed Gemini to draw context from multiple open tabs treated as a group. Price comparisons across sites suddenly became coherent. Research workflows gained continuity.
Now vision joins the mix. The timing coincides with another announcement. Google granted developers access to computer-use capabilities in Gemini 3.5 Flash. The model can see screens, reason about interfaces, and execute actions across browsers and apps. Digital Trends noted the parallel releases. One targets everyday users inside Chrome. The other equips builders to create agents that operate with screen awareness. Both point to the same direction.
Chrome Vice President Parisa Tabriz captured the intent in a blog post referenced by CNBC. “Chrome will remember context from past conversations so you get uniquely tailored answers to whatever you’re looking for across the web and you can already add specific instructions to Gemini to get more tailored responses.” The new screen tool takes that memory one step further. It adds immediate visual memory.
Consider the practical difference. A user stares at a dense financial table. Previously they might paste numbers into Gemini or describe trends in words. Accuracy suffered. Nuance got lost. With Select from screen the table itself becomes input. Gemini can analyze relationships, spot anomalies, or explain calculations without transcription errors. The same holds for code snippets, design mockups, news articles with embedded charts, or competitor product pages.
Yet this is not the first attempt at screen-aware AI. Competitors have experimented with similar ideas. What distinguishes Google’s approach is distribution. Chrome holds roughly two-thirds of the global browser market. The feature reaches hundreds of millions without requiring a separate app or subscription tier beyond existing Gemini access. Rollout began this week for signed-in users outside Incognito mode. Some simply restart the browser to activate it.
Support documentation updated rapidly. Google’s help page now walks through the exact steps: open Chrome, click Ask Gemini, select Add menu, click Select from screen, draw boxes. 9to5Google covered the documentation change hours after the feature appeared. The speed signals internal confidence. This is not a limited experiment.
Look back six months. In January Google layered additional capabilities into the same sidebar. Auto browse, an agentic function, lets Gemini perform multi-step tasks like researching flights or ordering groceries while requesting confirmation for sensitive actions. Personal Intelligence connects the assistant to a user’s Gmail, Photos, Calendar and YouTube history for personalized answers. CNBC reported those additions alongside image generation tools. Each layer builds on the last. Sidebar. Tab context. Personal data. Now direct screen selection.
The progression feels deliberate. Google isn’t bolting AI onto Chrome. It is redefining the browser as an intelligent surface. Every tab becomes potential training data, every visible element potential context. That raises questions about processing. How much computation happens locally versus in the cloud? The company has not disclosed exact architecture. Users notice only that responses arrive quickly and reference the selected content accurately.
Privacy considerations remain front of mind for enterprise adopters. Google has emphasized that Gemini in Chrome respects existing data governance policies when used in Workspace accounts. Still, the ability to feed screen content into a large model invites scrutiny. No major backlash has surfaced yet. Adoption appears enthusiastic on social platforms, where developers already test the computer-use API alongside the consumer feature.
And the developer side merits attention. Gemini 3.5 Flash with computer-use capabilities targets long-horizon tasks. Software testing. Workflow automation. Enterprise processes that span multiple applications. By combining screen understanding with action-taking, Google aims to reduce reliance on brittle scripting or separate vision models. One system sees. One system acts. Integration simplifies the stack.
Analysts watching the AI browser wars see this as Google’s counterpunch. Startups have pitched fully AI-native browsers. Perplexity and others experiment with conversational search layers. Google responds by infusing its incumbent product with the same intelligence. The sidebar doesn’t replace the address bar. It augments every page load.
Future iterations seem likely. What if Select from screen worked across multiple monitors? What if it captured dynamic video frames or interactive elements? What if agentic features used screen selection to decide where to click next? The foundation exists. Chrome 149 simply turns on the first switch.
Users testing the feature today report immediate productivity gains. One draws a box around an error message in a SaaS dashboard and receives troubleshooting steps tailored to the exact interface. Another selects a confusing graph from a research paper and gets a plain-language explanation plus potential flaws in the methodology. The assistant no longer guesses at relevance. It receives explicit visual direction.
That explicitness matters. Earlier multimodal models sometimes fixated on the wrong part of an image or page. By letting the user define the region of interest, Google reduces hallucination risk and improves output quality. The model still generates. The human still guides.
So the screen is no longer a barrier between user and AI. It becomes a shared canvas. Gemini doesn’t just chat about the web. It looks at the same pixels. Interprets them in context of the conversation. Acts on the understanding. This is how ambient computing evolves. Not through voice alone. Not through separate devices. Through the tool already open on every professional’s desktop.
Google continues to iterate at pace. The January features expanded Gemini’s memory. This week’s addition expands its sight. Both serve the same goal. Make the assistant an extension of the user’s intent rather than a separate oracle that requires perfect prompting. The less users must explain, the more they can accomplish.
Watch closely. The next updates may tighten the loop further. Persistent visual memory across sessions. Direct manipulation of selected elements. Deeper integration with Chrome’s own tools for reading, shopping, or productivity. Each step makes the browser feel less like software and more like a collaborator. One that sees what you see.
WebProNews is an iEntry Publication