When lawyers at the firm of Levidow, Levidow & Oberman used ChatGPT to prepare their case against Avianca Airlines, the AI chatbot they used quickly compiled a 10-page dossier with cases and decisions strong enough, they believed, to take to court.
There was only one problem. The cases ChatGPT cited didn’t actually exist.
The AI chatbot made up entire legal decisions, including names of cases, presiding judges, docket numbers, and dates. Some had long quotes of legal discourse. Even the lawyers believed they were real.
The lawyer directly responsible for the document now faces a disciplinary hearing for bringing false information to court. ChatGPT, he told the judge in his defense, is “a source that has revealed itself to be unreliable.”
This case is the most dramatic yet of the risks involved in over-reliance on ChatGPT for research in areas such as law, health, taxes, and finance, where information changes quickly and unpredictably and absolute accuracy is paramount.
When it comes to compliance in global payroll and payments, with its myriad of ever-changing labor laws and tax rates, even the brilliance of artificial intelligence is still no match for trained and licensed experts.
Why AI Chatbots are Not Reliable for Compliance Questions
With well over 100 million users since its debut last November, ChatGPT has truly taken the world by storm. And with good reason. Making the power of AI available to everyone can completely transform workforce productivity.
The chatbot is already capable of saving time and effort in day-to-day tasks. It can compose effective emails in no time, turn a set of notes into a full slide deck, or provide a succinct summary of long industry reports. It can write an executive summary for your quarterly financial reports or even spot a bug in the computer code your engineering team is developing.
In most cases, its only limit is the user’s own imagination – and how effectively that user can create a prompt that will generate the desired results.
The advantages can be even greater on a company-wide level. Chatbots were already in widespread use, but ChatGPT is a quantum leap forward and beyond, taking interactions to a level that was unimaginable even one year earlier. According to McKinsey, “CEOs should consider exploration of generative AI a must, not a maybe,” adding that those who fail to do so risk falling behind competitors.
At the same time, those who come to rely on ChatGPT for research on legal and tax questions across the globe face a different risk.
Here’s why any global payroll compliance research on ChatGPT or other AI chatbots requires support from real experts.
1. ChatGPT is not up to date
ChatGPT lists its knowledge cutoff date as September 2021. That makes it dangerously outdated for issues relating to labor law and taxes. Even simple and straightforward questions on topics like regional minimum wage will produce information that is unusable.
ChatGPT’s answer “as of September 2021” may be accurate but it’s useless to anyone who needs to hire people in the country today. The current minimum wage in Colombia is 1,160,000 Colombian pesos with an additional 140,000 per month as a transportation allowance.
The correct answer about paternity leave in Armenia is that fathers are entitled to five days of paternity leave in the first 30 days after the birth of a child. They can also request an additional 5 days of parental leave.
2. The pool of data is not 100% accurate
ChatGPT was trained on an enormous amount of data, enough to provide examples of how to respond effectively in most situations. Much of the data was scraped from across the internet, meaning that the original source data may not have adhered to the high standard of accuracy required for compliance information.
Don’t let the confident tone fool you. While ChatGPT’s breadth of knowledge is extraordinary, its specific knowledge of the fine details and sources used by professionals is questionable.
Queries about taxes, for example, could be both out of date (changes in social contributions were instituted as recently as April 2023) and imprecise.
While the contribution for employee pensions is indeed 18.3% of employee salaries, it fails to mention that the contribution is evenly split between employer and employee, each paying 9.15%. Other figures are either outdated, imprecise, or range too broadly to be reliable.
As ChatGPT itself says at the end of its response, “seek professional advice to ensure accurate and up-to-date information.”
3. ChatGPT makes things up
As the legal team in the Avianca Airlines case described above learned the hard way, ChatGPT (like other generative AI tools) has a tendency to… hallucinate.
For people seeking answers to questions with consequences, it’s virtually impossible to distinguish between real and fake information. The AI bots have the ability to generate extremely realistic-sounding answers.
Sometimes they will admit to the fabrication if pressed:
It’s not just ChatGPT that has this improbable drawback. Google’s Bard chatbot may also fictionalize answers on occasion:
So beware of very realistic sounding but false information making its way into your data research. While most ChatGPT information can provide an effective starting point, the tendency to fabricate may leave you chasing false leads, wasting more time than you save. re time than it saves.
Limitations Are Part of the Deal
To be fair, ChatGPT is clear about its limitations on its main page:
- It may occasionally generate incorrect information
- It may occasionally produce harmful instructions or biased content
- It has limited knowledge of world and events after 2021
Many of the answers it generates include disclaimers and recommend consulting with experts.
ChatGPT can help in many ways.
But it’s still risky to rely on it for research on matters of compliance. A mistake can mean real consequences.
Blaming ChatGPT didn’t stand up in court. And it won’t save companies from compliance fines, either.
Papaya has its own team of global experts in employment law, payroll compliance, and cross-border payments. It’s a team you can trust.
You can also consult our CountryPedia, an invaluable resource on labor laws and tax rates around the world, updated regularly.
Contact us to learn more about how Papaya’s experts can provide what you need for compliance.
