Why You Should Think Twice About Asking ChatGPT For Financial Advice

ChatGPT can be useful for a wide variety of tasks — including some unexpected uses — but it's definitely not a good idea to rely on it for financial advice. A study published in the Journal of Financial Planning put ChatGPT 3.5, Microsoft Bing's integrated GPT-4, and Google's Bard (now known as Gemini) to the test, asking them all a variety of questions covering a range of financial topics. The responses from every chatbot were pretty disastrous, with all three answering the majority of the questions wrong.

The chatbots consistently made mathematical mistakes when trying to grapple with questions about things like interest rates, mortgages, and Social Security benefits, often providing advice that was either extremely misleading or flat-out incorrect. Sometimes, they ignored certain parts of the prompt they were given, and other times, they made random assumptions without giving any reasoning as to where those assumptions came from.

Even worse, the chatbots were generally very confident in their incorrect answers. Anyone who isn't already a financial expert would find it scarily easy to assume these answers were accurate, since they wouldn't be able to manually check the calculations themselves. Of course, if you can do the calculations yourself — or you can pay an actual, human financial planner to do the calculations for you — then you no longer have any need to ask the chatbot in the first place.

Financial decisions such as making investments, purchasing property, and managing our pensions can all have potentially life-changing consequences if we get them wrong. That's why it's so crucial that any financial advisor, be it human or a robot, makes a balanced recommendation and clearly spells out the pros and cons of choosing each option for any given decision. 

However, a study published in the Artificial Intelligence journal found that users tend to trust chatbots that give overly confident financial advice more than they trust those that offer balanced advice. The study also found that the overly confident chatbot was more prone to dishing out incorrect advice. Even worse, the participants in the study weren't able to tell when they had been given that bad advice. Instead, they mostly just trusted that whatever the chatbot was telling them must be the right answer.

Another notable finding was that people who weren't as good at accurately describing their financial situation were given worse advice by the chatbot. In order for the chatbot to give them the best advice, participants needed to be able to clearly state things like their investing preferences and understand some financial jargon. It's pretty reasonable to assume that most people turning to chatbots for advice about money won't already be fluent in finance terminology, which makes it even more likely that they'll end up being given bad advice.

There are plenty of reasons that you might want to avoid ChatGPT altogether. It's known to fake some of its sources, it can occasionally give out some very creepy answers, and like all LLMs, it can amplify any biases found in its source material. However, it's still a useful resource for more than 700 million weekly users. The key to getting the most out of any chatbot is to understand its limitations, and chiefly, not to rely on it for any major financial decisions that could have significant real-world consequences. The chatbot might sound like it knows what it's talking about, but it lacks the ability to understand your individual financial needs — and in some cases, it seemingly can't do math either. 

While human financial advisors can be expensive, it's still best to consult a professional if you're looking for advice on a big financial decision. They'll be able to take the time to understand your situation and explain the advantages and disadvantages of any decision in detail. Even if seeking that advice costs more upfront, you'll benefit from being able to make the decision that works best for your financial situation in the long run.