Google Lens and Search gets visual with magic context
Earlier this year Google showed quite a few features that'd be coming to Google Lens in the future – now would appear to be the time these features will start to be available to the public. Google described their future in intelligent information parsing with their Multitask Unified Model, or MUM. The first pair of features for Google Lens are about to be available to the public using MUM to search for answers with Google in new ways.
Images and text in one search
One feature enhances the way in which a second-tier search is initiated. Sort of like when you do a search with Google Images and seek similar-looking images, Google Lens will soon be able to not only show you results based on an example image, it'll be able to expand your search.
Google showed an example of this expanded set of capabilities with an image of a shirt with a floral pattern. The user taps the Lens icon (Google Lens) and types "socks with this pattern", and Google understands what's being sought. This new search system uses a combination of image and text elements in a single query to get results.
Giving the user the option to combine images and text into a single search, Google says they are "making it easier to search visually and express your questions in more natural ways."
Point-and-ask search
Another example of a more powerful set of search results Google is releasing soon comes with video results. Using Google Lens to photograph a broken bit of your bicycle and typing "how to fix" can now result in not only webpage results with tutorials, but specific moments in video clips.
This is the "point-and-ask" mode of searching, as Google describes it. Google also has plans to expand how video content is parsed, with "related topics in a video". Google suggested this week that with MUM, they're able to show "related topics that aren't explicitly mentioned in the video."
Google suggested that the "first version of this feature" (the video context bit) will roll out "in the coming weeks." There'll also be more visual enhancements to this system in the following months.
Things to Know
Google Search will soon show expanded results with a section called "Things to know." With Things to know, Google uses their understanding of how people "typically explore" your topic of choice (your search terms). Take for example a search for "acrylic painting" – Google will show a Things to know section with titles like "step by step", "how to clean", and "styles" with links to articles, guides, and videos.
Google will also include "refine this search" and "broaden this search" sections with suggestions aplenty. These pieces of the search equation will launch in the coming months.
Launching today is a newly more vibrant looking set of search results. This new "visual results" system is designed to be more "browsable" than ever before, with expansive results from articles, images, video, and more with an emphasis on visual elements over text.