We are at a unique point in history when the latest AI-driven technologies are completely turning the phone into something new and exciting. Google AI, which is the technology basis of Android’s operating system, allows the billions of people who use Android to to their mobile devices in completely new ways.
we’re giving you a message with new features that let you feel the Google AI on Android.
Students can now use Circle to Search for homework assistance
With the Circle to Search feature, which is an in-built functionality in the user interface, you can look for anything you see on your phone with a simple gesture, without having to stop what you are doing or switch to a different app. Ever since its introduction at Samsung Unpacked, we have increased the features of Circle to Search, for instance, the introduction of the full-screen translation, and we have increased the coverage of more Pixel and Samsung devices.
Circle to Search can assist students with their homework, and thus, they acquire a better understanding of the topics, not only an answer but even directly from their phones and tablets. Students who circle a prompt they are stuck on will get step-by-step instructions to solve a bunch of physics and math word problems without having to leave their digital information sheet or syllabus.
In a few months, Circle to Search will be able to assist with more complicated problems which include symbolic formulas, diagrams, graphs, and more. This is all so because of our LearnLM project that is aimed at improving our models and products for learning.
Apart from that, Circle to Search is already available on more than 100 million devices now. We have the dream of the experience to be on all devices and thus, we are planning to double that by the end of the year.
Gemini will become even more adept at interpreting context to help you achieve your goals
Gemini on Android is a fresh kind of assistant that employs generative AI to assist you in being more creative and productive. Through this learning process, which is already part of Android, it is becoming better at recognizing the context of what you are looking at and the app you are using.
Shortly, you will be able to launch the overlay of Gemini on top of the app you are in to easily use the app in more ways. To illustrate, you can place generated images in Gmail, Google Messages, and other places, or tap “Ask this video” to find the information that you are looking for in a YouTube video.
If you own the Gemini Advanced, you will also have the opportunity to “Ask this PDF” so that you can get the answers you need without having to scan through many pages. This enhancement will be applied to millions of devices in the next few months.
Besides, we’ll keep working on Gemini so you can get more out of it and get real-time suggestions related to what’s on your screen.
Gemini Nano will get full multimodal capabilities soon
Android is the first mobile operating system to include a built-in, on-device foundation model. With Gemini Nano, we can offer you experiences in a short time and at the same time, keep your information strictly private to you. Pixel will be the first to do so in the current year, and then we will be providing our new model, Gemini Nano with Multimodality. Therefore, your phone will not be just processing text input but also be able to understand more information in a context like sights, sounds, and spoken language.
More understandable talkback descriptions
By the end of this year, the multimodal capabilities of Gemini Nano will be added to the TalkBack app, and thus, people with blindness or low vision will get more and better descriptions of the content in the images. The normal number of TalkBack users who find 90 unlabeled images per day is 90.
With this update, the missing information will be filled in by all the details that were not included in the photos that the family or friends sent or the style and cut of the clothes that could not be observed when shopping online. Given that Gemini Nano is on-device, these descriptions are done fast and they also work when there is no network connection.
Get notifications during phone calls about suspected scams
The report indicated that in 12 months, individuals lost more than $1 trillion to fraud. We’re experimenting with a new feature that makes use of the Gemini Nano to send alerts during a call if it identifies the patterns of a conversation that are frequently associated with the scams.
For instance, an alert would be sent to you once a “bank representative” asks you to urgently transfer funds, make a payment with a gift card, or request personal information like card PINs or passwords, which are not typical bank requests. This protection is on-device so your conversation is private to you. We will share more information about this opt-in feature later this year.
