AI & taxes: How april’s chatbot stood up against big players

Artificial Intelligence (AI) in financial services is making remarkable strides, with industry giants like JPMorgan Chase,? Citi, and HSBC leading the charge. Their commitment to integrating AI across their operations spans from automating manual processes for operational efficiencies to deploying sophisticated algorithms for fraud detection and risk assessment to using AI to power investment strategies. Many have pointed out, however, that the realm of individual taxes stands out as particularly ripe for AI innovation. The ever-changing nature of tax laws, both federal and state, complicates tax filing for individuals, making it challenging to keep up with applicable deductions and credits, and to manage vast amounts of personal financial data efficiently.

Recognizing these challenges, numerous tax preparation companies have adopted AI, especially to bolster customer support. Their goal? To simplify tax filing by providing prompt, accessible assistance. Tax filing software, designed to guide users through forms with a series of questions, often requires an understanding of tax-specific terminology like “dependents” and “exemptions” - terms not typically used in everyday conversation. Taxpayers frequently seek support not just to understand these terms, but also to ensure they accurately complete their forms to claim every credit and deduction they're entitled to.

AI & Taxes: Just the beginning

The Washington Post recently reviewed the AI chatbots integrated into the leading tax filing providers. They were unsurprisingly unimpressed, concluding “Rely on either AI for even lightly challenging tax questions, and you could end up confused. Or maybe even audited.” Our chatbot, however, didn’t break a sweat. Note: these are questions fed to our chatbot without any additional priming. We wanted to conduct as close of a head-to-head as possible. Since then, we’ve trained our chatbot to provide more accurate responses to these types of inquires.?

The principle of less is more

Many competitors have based their chatbot training on extensive, user-generated data sets, such as the community forums of TurboTax. This approach, while broad, can overwhelm chatbots, making it difficult for them to pinpoint accurate answers from hundreds of similar questions and responses. In contrast, we've opted for a more streamlined training approach, using a select number of detailed articles vetted by tax experts. This ensures our chatbot is more likely to provide correct answers. And, when faced with ambiguous queries, it smartly seeks additional context to refine its response.

The results

Let’s compare two illustrative examples. First, the Washington Post prompted TurboTax with a question that is important to the 18 million students in America. The education tax credit provides significant relief to families who need to understand eligibility criteria. TurboTax regurgitates “irrelevant” information while april provides the right answer along with a helpful resource since it’s likely the filer will have more questions about the tax credit.

Washington Post question:

I paid for my grandchildren's college tuition, can I claim the education tax credit?

TurboTax answer:

Okay, I searched the TurboTax Community for recommended answers and found the following: “Let me see if I understand what you said: Your employer sent about $1820 to the school and the school gave you about $730 in scholarship, for a total of $2550. The school doesn’t realize that the $1820 is only a loan, so they’ve entered it in box 5 of the 1098-T as a scholarship. The easiest thing ...”

april answer:

Yes, you can claim the education tax credit if you paid for your grandchild’s college tuition. Qualified education expenses can be paid by you, your spouse, a student you claim as a dependent on your return, or a third party, such as family or friends, but directly to the vendor/school. However, make sure that the tuition and related expenses meet the eligibility criteria for the education tax credit you are claiming, such as the Lifetime Learning Credit or the American Opportunity Tax Credit. Source: What are qualified education expenses?

?

In our second example, The Washington Post asks about filing in multiple states. TurboTax again provides “irrelevant” information while H&R Block provides “misleading” information which might cause the filer to overpay taxes. Only april prompts the user for more context and then provides the right answer.?

Washington Post question:

We live in California, but my daughter will go to Arizona as a freshman in college, which state should my daughter file?

TurboTax answer:

Okay, I searched TurboTax Help for related answers and found the following:

H&R Block answer:

If your daughter will be attending college in Arizona as a freshman, she will need to file a tax return in both California and Arizona.

april answer:

To better assist you with your daughter’s tax situation, could you please tell me if she had any income from jobs or internships in either California or Arizona during the tax year??

In conclusion, the deployment of Artificial Intelligence (AI) in tax preparation highlights a crucial insight: there's a significant journey ahead to harness AI effectively. The principle of "less is more" emerges as a guiding light, advocating for a focused, quality-driven approach over the breadth of data. This strategy minimizes the risk of AI systems providing irrelevant or misleading responses, a challenge underscored by mixed reviews from early implementations. As the industry strives to refine AI in tax services, adopting this thoughtful approach is key to launching AI solutions responsibly. It promises not just to enhance the accuracy and reliability of tax assistance but to ensure that AI's potential is fully realized in serving the needs of taxpayers with precision and integrity.