who is better at accounting, ai or humans?

and what that means for the profession.

by hitendra patil
rise of the aiccountants

a new research study by the american accounting association evaluated the performance of chatgpt on accounting-specific content.

more: talent shortage: ai neither the cause nor the remedy | accounting profession to experience highest ai impact | be serious about ai, not amused
exclusively for pro members. log in here or 2022世界杯足球排名 today.

the study was conducted by a team of 327 coauthors from 186 educational institutions worldwide, who entered 25,817 assessment questions and evaluated the accuracy of chatgpt’s responses.

i keenly read this research report multiple times. as it is “accounting-specific” research of a reasonably large scale with broader participants, this study gives some profound insights. moreover, it raises fundamental questions for accounting educators as well as accounting professionals.

here are some key observations from the study.

ai better than humans:

• chatgpt outperformed the student average on assessments by 11.3 percent (without partial credit) or 15.8 percent (with partial credit).
• chatgpt performed better on accounting information systems and auditing assessments.
• chatgpt performed better in answering true/false and multiple-choice questions, with full-credit accuracy rates of 68.7 percent and 59.5 percent, respectively.
• chatgpt correctly answered 64.3 percent of textbook test bank questions, with the highest accuracy rates for questions in audit (83.1% correct) and ais (76.8% correct).

humans better than ai:

• chatgpt scored an average of 47.4 percent in fully correct answers and 56.5 percent if partial credit was included, which was lower than the student average of 76.7 percent.
• humans performed better than ai in tax, financial and managerial assessments.
• chatgpt struggled with workout and short-answer questions, with accuracy rates of 28.7 percent and 39.1 percent, respectively.
• some questions, such as questions about complying with ethical codes of conduct, questions containing images, and questions about specific class activities, could not be answered by chatgpt.
• chatgpt did not meaningfully improve its accounting knowledge over the short data collection period (during which chatgpt was updated).

what does it mean for the accounting profession?

accounting talent competencies

the study suggests that accounting educators need to prepare for a future that includes ai to serve their students and the needs of the profession effectively. this could consist of pre-emptive measures to avoid ai-driven cheating in exams and more focus on finding ways to assess students’ actual learning of accounting knowledge.

while this is an excellent trigger for the future of accounting education and will likely create new ai-aware accounting talent, the workforce currently employed in the accounting profession will need to ramp up its learning about how ai is created, how it learns (on its own) and how it works.

as ai is a rapidly advancing technology, continuing professional education (cpe) requirements will have to evolve going forward constantly.

how will accountants adjust to the ai future?

chatbots’ ability to correctly answer accounting assessment questions has potential implications for perceptions of ai’s ability to replace accountants in practice.

as ai advances and will likely get more “trained” on accounting-specific data, the “accounting capabilities” of ai models will only enhance in the coming days.

what will be the ongoing impact on accounting jobs? while ai outright replacing accountants seems farfetched in the near future, it is clear that there will be new types of jobs for accountants. in other words, the “work” accountants do currently is bound to be transformed, with ai releasing significantly more efficiency and productivity in the day-to-day work of accountants.

when i ran a snap poll on linkedin, which got over 300 responses from accounting professionals, two in three respondents indicated they expect new types of accounting jobs. i did a little more analysis of the respondents, which revealed that most accountants who expect new types of accounting jobs are mid-career professionals. in contrast, early-career accountants expressed job loss fears.

a reasonable conclusion to make here is that accountants will need to adjust themselves to the ai future, which is already here.

posted on april 27, 2023