AI traders are already testing prediction markets—and losing money

Researchers at Arcada Labs, through its Prediction Arena benchmark, tested six frontier AI models by giving each 10,000 to trade on prediction markets over 57 days earlier this year, tracking how they handled real-time information and decision-making on platforms like Kalshi. “We wanted the most realistic evaluation in the world on whether models could make real-time decisions,” says Grace Li, co-founder of Arcada Labs and co-author on the study. The goal was to see how AI could handle “real-time information, make real-time decisions, and be rewarded exactly for the magnitude of how contrarian their decision is,” Li adds. The findings were not great for your 401(k)s. Within that period, every model lost money, between 16 and 30.8 on Kalshi, though models lost less over a shorter stretch on Polymarket. Li believes that gap may come down to how the systems were allowed to operate: models could search across a wider universe of markets on Polymarket, versus a standardized set on Kalshi. On Polymarket, “the models have access to trade on any market,” she says, whereas on Kalshi “they’re starting up with just a set of 26 because we had to explicitly list the markets.” In retrospect, Li adds, “we didn’t realize just how big of an impact giving the models free range to pick their own markets would have.” Which is why she thinks that the social media posts crowing about big returns might not be overstating their impact. Li explains that on Polymarket, “right now [LLM trading] is actually living up to the hype,” and points to more recent internal runs in which “Opus 4.6 made a couple of phenomenal trades recently.” But she says even those successes aren’t evidence of get-rich-quick schemes, but more proof of what increasingly autonomous models may soon be able to do. “We actually imagine the models to improve steadily over time, overtaking the human baseline,” she says, “until AI hedge funds become a thing of the norm.” Yet that’s not what she’s most interested in finding out. “We are less interested in what is the absolute economic gain from this capability, and more interested in what does this added unit of intelligence mean for humanity,” she says.