Can Gemini Make Google Home Less Awful? We Put Google's AI Update To The Test

Smart home ecosystems like Google Home and Amazon Alexa were supposed to herald a "Jetsons" style future when they arrived in the mid-2010s. Complete with voice assistants that allowed users to command their lights, thermostats, and security systems without moving a finger, it all looked like the utopian promises of the tech industry were finally bearing fruit.

Once people began to use smart home products in their actual homes, that Captain Kirk fantasy was given the red shirt treatment. Managing a smart home turned out to be far from frictionless, as users pleaded with their speakers to turn on the bedroom lights. Google Home was particularly messy. Every device added to a home seemed to cause new problems, and features that once worked reliably have steadily broken down or been removed.

Google finally seems to have noticed how dire the situation is, and it's attempting to remedy things with  — what else  — a big, AI-powered overhaul. But will throwing Gemini at the situation help, or will it make matters worse? I won't bury the lede here. Testing Gemini in Google Home for a week has been like living in my own, personal version of the Disney movie "Smart House." Many of the usual problems we've come to expect from AI carry over into this smart home integration. There's a glint of something promising amid the chaos, but this update needed a lot more time in the oven. Let's dive into my weeklong experience with Google's big bet on Gemini for Google Home, which began with the AI assistant gaslighting me about my own light bulbs.

Let's start with one of the best additions Gemini brings to Google Home: granular control and context-aware voice commands. Home's light grouping features have always been frustratingly anemic. Lights are grouped by room, but you can't easily make subgroups without relying on automation. For instance, my bedroom has three lamps and an LED light strip, the latter of which I don't use often. "Hey Google, turn on the bedroom lights," previously everything on, forcing me to follow up with, "Turn off the light strip." 

With Gemini, I can now specify, "Turn the light strip off, and set the remaining three bedroom lights to red, then dim those to 20%," and that's precisely what happens, most of the time (yes, that was blunt foreshadowing of horrors to come). It's hard to understate how amazing that level of voice control is. In general, you can use Gemini to do all sorts of useful things with that level of flexibility. Gone are the days of feeling like you're memorizing phrases from a wizard's spellbook just to adjust your thermostat.

There's one huge problem, though. Gemini frequently fails to accomplish tasks. Tell it to turn on your lights, and nothing happens around a quarter of the time. Even more annoying is that Gemini often insists it did things correctly, even when it didn't do anything at all. Call it out for the mistake, and it will sometimes claim your lights aren't even smart lights in the first place. Also, I once asked it to set my lights to white and got a dissertation on white people in response. That sort of unreliability is so frustrating that it cancels out the aforementioned natural language capabilities. Pictured above: a desperate attempt to set my desk lamp to white.

Another major improvement Gemini promises is an easier way to create automations. Instead of manually choosing which triggers and actions to include in an automation, you can simply describe the conditions to Gemini, which will build the routine itself. In theory, that makes home automation much more accessible for the average user. In practice, it's just as hit and miss as anything else Gemini is supposedly capable of.

One instance, I told Gemini, "When the TV is on, set the living room lamp to warm white and 30% brightness." It dutifully created that routine, but when I double-checked its work, things were off. Automating light colors has been a feature of Google Home automations as far back as I can recall, but it's completely gone in the new update (only retained for preexisting routines). Instead of telling me color adjustments were no longer available, Gemini tried to create the routine, which meant there was an error baked into it. Since the entire point of that routine would have been the warmer light for optimal TV viewing, I scrapped it entirely.

If you want something done right, do it yourself. That age-old wisdom became a persistent theme in my testing. Smart home systems are meant to be digital butlers, but if a human servant were to get things wrong as consistently as Gemini does in Google Home, they'd soon be out of a job. Google has installed the AI version of "Amelia Bedelia" in my apartment, and there's no way to get rid of it other than swapping to a different smart home system entirely.

By the time I tested Gemini inside of my Google Nest Doorbell, I was miffed. Did Google really expect me to entrust my front door camera, a crucial component of home security, to a robot that insists it's "under development" and "still learning" every time I call it out for a mistake? Sure enough, just a few minutes after enabling Gemini on my camera, it told me someone with a jump rope was outside. When I reviewed the video, it showed a neighbor walking past, holding a water bottle. A harmless error in this case, but what if he hadn't been a neighbor and the water bottle had been a gun? It was mostly accurate, though. I tested during Halloween, which led to a notification reading, "Person in grim reaper costume walks by."

Next, I asked Gemini if I had any packages, which is when I learned that Google thinks you'll pay money for this. My Google AI Pro plan is valued at $20 a month (six free months came bundled with my phone), so I thought I was covered — nope. Google charges an additional 10 bucks for something called Google Home Premium Advanced.

After a deep breath, I paid the fee, and dear reader, I did it for you. It took two days for my subscription to be recognized by Gemini, and Google support could not resolve the issue in that time. When things finally worked, I was able to ask Gemini basic questions, like whether any packages had been dropped off, when the dog was last walked, and so on. The error rate was mercifully low, answers take forever to load. Given the mounting misuse of AI video surveillance these days, I took the camera down after my week of testing.

Aside from controlling devices, smart home assistants should be able to handle a range of tasks, from setting timers and reminders to pulling up recipes and answering questions. Gemini replaces Google Assistant for those tasks, and everything seems to work well  — at least for those in the Google ecosystem. We do need to talk about Gemini's answers, though. 

When responding to a query with a web result, the outgoing Google Assistant always cited its sources, specifying which website it grabbed the answer from. Gemini is a large language model, which means that it synthesizes answers from its training data. When I asked Gemini to cite its source for a question about food safety, it was unable to do so. Given how frequently Gemini gets things wrong, it's best to do an old-fashioned web search in any situation where bad information could lead to bad outcomes. Plus, there are the many reasons not to trust Gemini, given Google's spotty track record with user privacy.

Lastly, the best thing about this update has been the app redesign. Google Home has never been as easy to navigate as it is now. Aside from dropping certain features in the new automation creator (which I hold out hope will be returned), it has never been easier to find and control my devices in the app. I especially like the new tabs on the devices page, where you can look at your lights, climate controls, and cameras separately. Favorites are also on their own tab. Gemini chat sits up top, along with a button to quickly add new devices, automations, apps, or home members.

In three years of testing generative AI products, what remains constant is the chasm between expectation and reality. In theory, I would love a personal assistant that lives ambiently in my home and makes sure everything is to my liking. It's what I've always wanted ever since seeing the voice-controlled computers on "Star Trek" as a child. But what Google has delivered here is only a faint echo of that sci-fi dream. The friction it adds to my smart home far outweighs the marginal benefits it brings to the table.

What Google Home has lacked for quite a while now is not a futuristic AI but basic functionality, but this update has removed even more of it. It's hard to communicate how irritating it is to find out that I can no longer automate the color of my lights, and nothing Gemini can do in its current form makes up for that loss. Meanwhile, I still can't make custom lighting groups. Google appears focused not on what can make its smart home system better, but rather on what will make it more AI-centric. After all, Google has fierce competition with other AI firms, and shareholders want to see user growth for Gemini. What better way to boost those numbers than by making people go through Gemini to use their household gadgets?

Well, I don't care about Google's stock price. Users rely on Google Home not just for convenience but for safety and security. In an actual emergency, my cameras need to be reliable, and I need to trust that I can turn the lights on without fiddling around in an app. Until it functions dependably, Gemini can take its own advice and eat rocks.