Vik
TL;DR: Google unveils major AI features for Android at I/O 2024, with Gemini as a more integrated virtual assistant, upgrades to Circle to Search for homework help, and Gemini Nano with Multimodality for enhanced interactions. These features will roll out to Android 15 and beyond, some launching first on Pixel devices.
At Google’s I/O 2024 developer conference, the company devoted most of its time to discussing artificial intelligence (AI), as anticipated. During the keynote, Google outlined how AI would integrate with Android in the future, with Gemini taking on the role of a virtual assistant, similar to Google Assistant, but in a more integrated and contextual manner.
Gemini for Android: A Suite of New Features
Since releasing the Gemini app in February, Google has been expanding Gemini for Android with a suite of new features that can incorporate AI into more aspects of your daily life.
While Google attempted to present these changes as a ground-up rethinking of Android, they currently serve more as complements to the broader Android experience.
Gemini’s Design Tweaks and Contextual Integration
Google is modifying Gemini’s design to float on top of whatever you’re doing, rather than occupying your entire screen as it does currently. This approach is similar to how Google Assistant presents itself. However, with Gemini, you have a large text field to type your prompts into, reducing the emphasis on voice prompts.
The new overlay is intended to represent deeper integration with the app you’re using and provide you with contextualized controls. For example, when watching a YouTube video, you can pull up Gemini and see a button that says “Ask this video,” allowing you to use the video’s knowledge base to ask questions or summarize the content. This feature also works with PDFs, provided you subscribe to Gemini Advanced, which has a longer context window.
Gemini will also flow more seamlessly within apps, such as when using drag-and-drop. During the keynote, Google demonstrated how you can ask the chatbot to generate an image, then drag the result into a messaging app, drop it, and send it to your friend.
Circle to Search Upgrades for Homework Help
Google is upgrading Circle to Search, which is already available on over 100 million Android devices, to assist with homework. The feature will be able to help students better understand complex physics and math word problems they’re struggling with, providing a detailed breakdown of how to work through the problem without needing to consult their digital info sheet or syllabus.
This new ability is powered by Google’s LearnLM model, which aims to make learning easier with AI. Google says that, over time, you’ll be able to use Circle to Search to solve more complex problems involving symbolic formulas, diagrams, and graphs.
Gemini Nano with Multimodality
Google also announced an upgrade to Gemini Nano, the model built directly into Android (albeit on very few devices), called “Gemini Nano with Multimodality.” The updated LLM will allow you to use various media inputs, such as text, photos, videos, and voice, to interact with Gemini and get answers to questions, information on queries, and more.
The model will power features like:
- TalkBack for text descriptions of images
- Real-time spam notifications during phone calls (helpful if you believe the unknown caller is actually the prince of Egypt and you must wire him 1 million dollars)
More AI Goodies to Come
These are just a few of the AI-powered features that Google will be introducing to Android 15 and beyond. Some will launch first on Pixel, while others will be available to anyone who downloads the Gemini app. How exactly it will all unfold remains to be seen, but it’s evident that if you own and use an Android phone, it’s about to become significantly more powerful.
Add comment