In the past year or so, Google has introduced a range of AI features, powered by Gemini and other models, across its suite of consumer-facing applications. These enhancements aim to improve user experiences in areas such as image editing, home automation, health tracking, photo management, communication, search, and more.
Pixel
A “Zoom Enhance” feature for Pixel 8 Pro, leveraging a generative AI image model, was teased at Made by Google 2023. This feature, designed to improve image quality when zooming in after a photo is taken, has been delayed and may debut on the Pixel 9 Pro instead.
Google Home
Generative AI will be integrated into the Google Home app, enabling it to provide summaries of recent events, answer questions about your home, and find video history clips. These features will be available to Nest Aware subscribers in 2024.
Fitbit
Fitbit Labs will allow Fitbit Premium users to test experimental AI capabilities, including a chatbot that can answer personalized questions about Fitbit data and provide actionable guidance based on fitness goals. This is powered by a new Personal Health LLM developed by Fitbit and Google Research, built on Gemini.
Google Photos
The “Ask Photos” feature, powered by Gemini, allows users to ask questions about their photo and video libraries, find specific images, and generate captions. This experimental feature is rolling out soon, with additional capabilities planned for the future.
Gmail + Google Workspace
Gmail for Android and iOS will feature a Gemini button for generating text, as well as Contextual Smart Replies that offer more customized suggestions. A voice prompting capability for “Help me write” and an “instant polish” feature are also in development for mobile Gmail. On desktop web, the Gemini side panel is available in Gmail, Google Drive, and Docs/Sheets/Slides. Gemini will also be integrated into Google Chat to summarize conversations and answer questions.
Google Maps
Google Maps will utilize LLMs to power an “Ask about” chatbot, enabling users to find places and get answers to follow-up questions based on details about 250 million places and user-submitted content.
Chrome
Gemini Nano will be integrated into desktop Chrome to power browser features like “Help me write” and should be available on most modern laptops and desktops.
Google Search
Several AI-powered features are coming to Google Search Labs, including options to simplify or break down AI Overviews, multi-step reasoning capabilities, meal and trip planning, AI-organized search results, and video search capabilities.
Android
Gemini Nano with Multimodality will launch on Pixel devices later this year, powering features like on-device/offline TalkBack descriptions and real-time scam alerts. Gemini on Android will soon be accessible through an overlay panel, and Dynamic Suggestions will use Gemini Nano with Multimodality to understand on-screen content. Gemini Extensions for Google Calendar, Tasks, and Keep will enable features like turning photos of dates into calendar events.
Gemini
Gemini Live will allow users to have two-way conversations with Gemini, with the option to choose from different voices. In the future, it will also include a live camera mode for asking questions about real-world objects.
Gems are customizable versions of Gemini, with pre-made Gems like Learning Coach available to all users and the ability for Gemini Advanced members to create custom ones.
Gemini Advanced users will also have access to an “immersive planner” that creates detailed itineraries based on travel details, interests, and recommendations from Google services.
