Google’s experimental AI Mode can now examine and respond to queries about images.

Last month, Google unveiled an experimental feature called “AI Mode” within its Search platform, designed to handle intricate, multi-part questions and assist users with their most challenging inquiries. The company positioned this as akin to the AI-driven search capabilities seen in tools like ChatGPT and Perplexity AI, offering a conversational approach to finding answers.

Now, Google has taken this a step further by enhancing AI Mode with multimodal functionality, allowing users to incorporate visual elements into their queries. This update, announced in a blog post, enables the chatbot to process images—whether uploaded from a user’s existing collection or freshly captured via a camera—and respond to questions about them. According to Google, this new capability allows AI Mode to “understand the entire scene in an image,” factoring in the context of how objects interact, as well as their distinct attributes like materials, colors, shapes, and spatial arrangements.

This marks a significant departure from conventional search methods, which typically rely on a single text-based query. With the upgraded AI Mode, the system doesn’t just process one question at a time; it generates multiple queries based on the image, analyzing both the overall scene and the individual elements within it. This multi-layered approach delivers richer, more detailed responses compared to the simpler, one-dimensional answers of traditional searches. For example, rather than providing a basic description of an image, the AI Mode can break down complex relationships between objects and offer insights tailored to the user’s specific question, drawing from a broader understanding of the visual content.

The introduction of this multimodal feature reflects Google’s ongoing efforts to integrate advanced AI into its search ecosystem, making it more intuitive and versatile. By combining text and image analysis, AI Mode aims to tackle queries that require a deeper level of comprehension—such as identifying the purpose of an object in a specific setting or explaining how different items in a photo relate to one another. This builds on the initial promise of AI Mode, which was to go beyond standard search limitations and address complex, nuanced questions with greater precision.

In its blog post, Google emphasized that this enhancement is still experimental, suggesting that the feature is part of a broader testing phase to refine its capabilities. The addition of image-based questioning aligns with the growing trend of AI systems evolving into more interactive, context-aware tools, capable of bridging the gap between visual and textual information. For users, this means a more dynamic search experience—one where they can not only ask “what” or “why” about an image but also receive comprehensive answers that account for the intricacies of what they’re seeing. As AI Mode continues to develop, it could redefine how people interact with search engines, turning them into more holistic problem-solving platforms rather than mere repositories of information.