Caelyn Pender, a Bloomberg reporter, tests the Presto Automation drive-thru chatbot at a Del Taco restaurant in Riverside, California, on June 29. Mark Abramson/Bloomberg

Checkers and Carl’s Jr. are among U.S. fast-food chains hailing AI-powered drive-thrus as labor-zapping wizards that speed up service. But a popular provider of these systems recently revealed a crucial part of how it gets so many orders right: humans.

Presto Automation Inc. pitched a restaurant industry that’s desperate to combat rising wages on a talking chatbot that could take orders with almost no human intervention. The firm touted OpenAI’s Sam Altman as an early investor. And it has used the firm’s technology to improve its system as it aims to triple deployments to 1,200 locations next year.

However, disclosures in recent filings with the U.S. Securities and Exchange Commission and changes to marketing suggest that the technology is less autonomous than it first appeared. The company, which went public last year, now says “off-site agents” working in locales such as the Philippines help during more than 70% of customer interactions to make sure its AI system doesn’t mess up.

“We felt the need to ensure this nuance was conveyed to market participants to enable them to understand our cost structure and to explain what makes our solution stand out,” a spokesperson for Presto said in an email. The company said using off-site workers helps train its system and should reduce human intervention over time.

The company was notified by the SEC in July that it’s under investigation over disclosures made “regarding certain aspects of its AI technology.” Meanwhile, Rajat Suri, its founder and CEO, left in March and was replaced internally by Xavier Casanova. The new boss has implemented several changes, including laying off 17% of the company’s staff and disclosing the human agents.

The SEC and Presto declined to comment on the nature or details of the probe.

Presto now touts what it calls “humans in the loop” as a competitive advantage that frees up restaurant staff to focus on preparing the food. It also points out the benefit of doing this behind the scenes, saying in a filing that the human element is integrated so well that diners are unaware.

“Our human agents enter, review, validate, and correct orders,” Casanova, who was promoted to Presto’s CEO from chief operating officer, told investors on a recent earnings call. “Human agents will always play a role in ensuring order accuracy.”

A sign at a Del Taco location in Riverside, Calif., tells customers to “order as you normally would” with the Presto ordering assistant. Mark Abramson/Bloomberg

Presto’s acknowledgment of human intervention fuels questions about how much public companies should disclose to investors as AI gets folded into more products. In Presto’s case, knowing that its drive-thru software is only taking orders fully on its own less than a third of the time would have been beneficial for shareholders to know earlier, said Brian Dobson, an analyst covering disruptive technologies for Chardan Capital Markets.

“It highlights the importance of investors understanding what an AI company can and cannot do,” said Dobson, who has the equivalent of a hold rating on Presto’s shares. Wall Street’s grasp of artificial intelligence is “relatively low.”

Presto shares fell almost 6% on Thursday. The stock has declined almost 70% this year through Wednesday’s close, pushing its market valuation below $50 million.

Suri, the tech firm’s founder, and customers Checkers and Del Taco declined to comment. A spokesperson for CKE Restaurants Holdings, owner of Carl’s Jr. and Hardee’s, said Presto is one of several vendors its franchisees use for AI ordering and referred questions to Presto.

AI’S HYPE

The promise of AI has tantalized a U.S. restaurant industry struggling with finding enough workers while paying more for the ones they do hire. Presto and other providers marketed voice chatbots to speed up ordering, sell extras like desserts, and improve accuracy. That’s appealing because drive-thrus might get as many as 14% of orders wrong, according to a study by Intouch Insight.

But for all the hype surrounding AI – especially since the emergence of OpenAI’s ChatGPT over the past year – the widespread swapping of chatbots for employees at the drive-thru intercom faces hurdles. McDonald’s, for example, has dabbled in the technology for years but has yet to broadly roll out a solution.

One issue is that it’s difficult for AI chatbots to discern the variety of accents, speech patterns, and noises they encounter at a drive-thru, including a car’s radio or traffic sounds. Orders could also be too complicated for AI to decipher.

When AI trips up or guests ask to speak to a person, many providers of this technology alert a worker at the restaurant, who takes over and talks to the customer directly to finish the transaction. But unlike some competitors, Presto also employs people off-site who check the chatbot to minimize the number of times guests get handed off to a staffer in the kitchen.

Valyant AI said it used to employ a similar technique but stopped earlier this year. Other providers, including IBM, Google, SoundHound, and ConverseNow, say they don’t employ this approach. These firms have focused on trying to develop fully automated order-taking. ConverseNow says its AI technology processes the “majority” of orders on its own, and that only orders that need human assistance are redirected to store workers or off-site agents.

ADDED EXPENSE

More broadly, Presto’s business model raises questions about how much efficiency AI tools are creating for businesses. An additional issue is whether by adding overseas workers they are starting another wave of outsourcing customer service to cheaper labor markets.

Presto has said those off-site workers are an added expense that will keep rising as it expands to more locations. Still, the company, which has posted operating losses since going public last September, expects the need for humans to ease as its system gets smarter. OpenAI might help it get there, with Presto saying its development platform will improve its drive-thru chatbot on several fronts, including sounding more natural.

Shelly Palmer, who runs a tech consulting firm for big companies, said that in the rush to employ AI, much of the business world is missing the finer details.

“There’s so much hype around AI that everyone is misunderstanding what this tool is,” Palmer said. “Everybody thinks that AI is some kind of magic.”