OpenAI chief scientist discusses methods to limit AI development
Tech Desk
Jakub Pachocki, the chief scientist at OpenAI, has outlined a method for managing the pace of artificial intelligence advancement in a recent blog post. He suggests that a measurement known as perplexity could serve as a standard metric to establish limits on future development.
Advertisement
The discussion around slowing AI progress intensified after a former researcher from OpenAI and Anthropic claimed firms were “gambling with our lives.” Anthropic’s IPO filing also noted potential “catastrophic or existential risks to society.”
Dario Amodei, the CEO of Anthropic, previously proposed a strategy to “pace the frontier,” a plan that appeared to receive support from OpenAI CEO Sam Altman and SpaceX CEO Elon Musk. Amodei advocated for industry agreement on safety standards and restrictions on the speed of unchecked AI growth.
According to Pachocki, whose post was described by Sam Altman as “the most important thing OpenAI has published in a while,” perplexity represents a reliable way to gauge progress. Perplexity is defined as a measurement of “how well a model predicts the next token on average; lower perplexity means better predictions.”
Pachocki argued that while benchmarks measuring specific capabilities often become obsolete, perplexity remains “informative.” He provided data showing that the perplexity of OpenAI models has decreased since the release of GPT 3.5.
Advertisement
“GPT‑4, at ~0.91, was proficient at utilizing chain-of-thought reasoning productively, which motivated the subsequent targeted elicitation of this capability and the creation of modern reasoning models,” Pachocki wrote. “Now, as we have reached losses below 0.8, we have seen the emergence of transformative capabilities in fields like cybersecurity and mathematics.”
He proposed that this metric could form the foundation for “a shared benchmark for general intelligence.” Under this framework, companies would agree to pause development at a specific threshold “until our ability to understand and monitor AI behavior achieves a certain concrete milestone.”