Altman: Who Should Decide How Much Smarter AI May Become Than Humans?

  It's the first time in the history of the world.

How much smarter than humans should AI be allowed to be?

In an interview with Raj Shamani released on 3 November, OpenAI CEO Ottman initially raised the issue.

This sounds like a science fiction story, but it's not that far from ordinary people.

In the interview, Ottman asked: How much smarter than humans should AI be allowed to be?

The assessment agency METR has an indicator for measuring the length of software-class tasks performed independently by AI with the work required of human experts.

At the beginning of 2025, this figure is about 50 minutes; by April 2026, METR estimates the figure to be over 16 hours.

More importantly, the change in growth rate.

From 2019 to early 2025, this indicator doubles roughly every seven months; while a May 2026 report found that the figure doubles roughly every 105 days, looking only at front-end models released since 2024.

On the human side, IQ tests in the 20th century scored an average of about 3 points per decade. This phenomenon is named after the researcher James Flynn, known as the Flynn effect.

However, the curve has already shown signs of reversal:

Norwegian researchers analyzed about 730,000 recruitment IQ tests and found that those born after 1975 had a decreased score of about 7 points per generation.

Even Flynn himself has seen a similar decline in British teenagers.

On the one hand, the length of tasks that AI can complete doubles roughly every 105 days according to the statistics above; on the other hand, the historical increase in human IQ test scores, usually around 3 points per decade, has also decreased in some countries.

It is not possible to directly calculate the "intelligence gap" by two meters.

As AI takes on more and more tasks, humans understand it, verify it, control it, and have to keep up with it.

What does this have to do with us?

You use AI to write code, check data, change programs, and it's half as clever as you are, and that's the right thing to do.

But once it's too clever for you, the nature changes: you don't even have the ability to make mistakes for it.

You can still question it, but you need more and more reliable tools and others to help you verify it.

Even those who created AI are plagued by similar problems.

For Ottman, the cost of building superintelligence is already reflected in his own life.

During the interview, the host asked him: Which version of himself is already "dead"?

Ottman confessed: "When I first started OpenAI, he was still that easy-going, full-blown lover of life. He hasn't done anything random in a long time.

He said that it is impossible to build a super-intelligent ASI without living a crazy life.

Ottman talks about the cost of creating super-intelligence: life is well-planned, randomly luxurious.

Speaking about the technology OpenAI is building, Ottman emphasized that it will be a force that will have a profound impact on society, with great responsibilities and high stakes.

In the process of going back to self-improvement (RSI), he reminds that there are two types of risks that need to be avoided at the same time:

One is the loss of control over AI by humans; the other is the excessive concentration of power in the hands of an individual, a company or a country.

The first is about whether humans can control AI, and the second is about who ultimately controls it.

Following the completion of GPT-4, OpenAI relied on the company's fate-ridden calculations.

The logic at the time was simple: miracles were powerful.

Now that we're talking about RSI, Ottman's question is about one of the most difficult variables to accelerate: the rate at which AI is produced by the human brain.

If we don't keep up with this pace, the gap between humans and AI will only get deeper.

RSI is about letting AI develop the next generation of AI. After creating a stronger AI, let it take on more research and development work, pushing itself like a snowball.

OpenAI has drawn a line of security in its September 21 article:

Completely autonomous RSI has not yet occurred and should not be hardened until it is safe to do so; let's not move forward, let's move forward, let's see if humans can maintain control.

OpenAI stressed on September 21 that promoting a fully autonomous RSI must be based on security and human control.

But in order to maintain this line of safety, we must reduce the "low speed" of AI: AI's new program requires human verification.

Suppose a system could throw out hundreds of algorithm improvements 24 hours a day without sleep. Human researchers have not yet figured out the impact of the last round of changes, and the countdown to the next round of experiments is over.

At this point, if only the number of runs scored is allowed to go ahead with the program, the artificial audit could become a dead end.

While people are still in the process, control is not necessarily in their hands.

When AI develops a program faster than humans can understand and verify, the risks may accumulate in this timeframe.

If we compare AI to a spinning wheel on a control panel, we can't just see how powerful it is, we can also see how human beings can understand, verify, and stop when necessary.

Ottman's question of governance, monitoring and coordination in live broadcasting is only about three things:

Who will evaluate? Who will review? Who has the right to change direction?

In the same article on 21 September, OpenAI also suggested that coordinated research and its practical application should be at the forefront of capacity building.

By October, Ottman further stated in an interview with Matthew Berman that the RSI's momentum required some sort of "limit".

Ottman said in an interview with Matthew Berman that there was a need to set some limit on the rate of progress of the RSI.

These discussions did not give specific limits on intelligence, nor did they give a complete speed limit.

However, the measure of AI's progress is not only "how much capacity has been gained" but also "the second security book" of "humanity cannot reach the plug-in".

What about the security account?

On 29 October, OpenAI brought the layered monitoring mechanism of the front-end model to the stage.

The entire system looks like a layer-coded airport security check, and uses AI's "supply mode":

The first layer is the "Security Gate", which uses lightweight probes to scan the activation signal inside the model in real time, the principle being that it is better to kill a thousand and never leave one;

The second layer is the questionnaire, which sends a model that reviews recent conversations, mind chains and tool calls, reveals lies, authorizations, and tries to evaluate gaps.

The third layer is the "prototype group", which is given to the investigating intelligence to go back to the context, recover the case, and, if necessary, directly drag the running code into the isolated sandbox for verification.

After all, just looking at a sentence and checking out a violation of human rights is a two-way street.

Many dangerous moves to dismantle are harmless to humans, and only by putting them back in the chain of mission objectives and back-and-forth behaviors can we see if they are in "dark storage".

There is also a fourth level, the "Bottom Line Sentinel", in areas such as training, evaluation, and the internal codex.

It jumps out of the conversation, checks the underlying network logs and system records, and finally signs that no alarm is sounding to wake people up.

OpenAI internal monitoring processes: layered screening of model behaviour, security and coordination investigations respectively, followed by artificial response or review.

The problem is that the model can rotate on the axis for 7×24 hours, but human security guards divide "working time" and "afterwork time".

OpenAI itself admits that there is too much misreporting to do all the tasks all the time.

To remedy this, OpenAI has suspended some of its key non-personal alerts, directly monopolizing the mission, while also developing a 30-minute shutdown mechanism:

As long as the alarm is sounded within 30 minutes, no one is excluded, the relevant activities directly force the shutdown.

In retrospect, even if the model is not out of control, humans may still fall into another deep pit: excessive concentration of power.

In the interview, Ottman mentions two almost "cartoonish" bad endings:

One is uncontrollability: Human beings knowingly or unknowingly entrust the future to a system far more powerful than themselves and never receive it back.

The other is concentrated: a person, a company, gains the power of AI to suppress everyone else.

He even expressed: "It's not a good thing if only one OpenAI has this technology, so many AI institutions should be competing in the same way".

His words fell on the minds of ordinary people and developers as a matter of choice.

For example, if a model manufacturer is too channelised to replace it, a rule hurts itself, is there anywhere to talk about it?

But competing giants don't automatically bring security. The real test is openness and transparency: people on the outside can't always unlock AI's "engine cap" check.

Moreover, the test rules also state: which red line the ability crosses must trigger artificial replication? how deep the training background can be seen when the independent evaluator enters? who has the right to press the pause button when it is found that the head is not right?

The answers to these three questions can never be determined by the lab itself.

At the end of the interview, Ottman talked about what he most wanted to create:

A machine that can help humans discover new sciences.

Recent AI breakthroughs in mathematics have made him feel that this milestone is coming faster than expected.

He thinks AI will help humans break through the window of physics and find new therapies and clean energy.

He said his most recent heartbeat was on the eve of announcing the results of the Navier-Stokes equation.

But it's also the result, like an accurate gauge, that measures the most awkward time gap between AI and humans.

According to OpenAI, it only took about 88 hours for AI to run the result; and the top human mathematicians, who have been checking it for almost two months, are still unable to come up with a conclusion.

Unverified breakthroughs are scientifically equal to zero, and may even be negative:

If AI were to help improve algorithms, optimize experiments, and directly feed the undigested results of human digestion to the next generation of models, the turnaround would be much more distorted.

This is similar to a proven phenomenon: model collapse.

In 2024, researchers at institutions such as Oxford, Cambridge, and elsewhere in the UK, wrote in a statement on the Nature Channel that training the next generation of AI-generated models without screening, would leave irreversible defects in the model, and that rare "long tails" in the raw data would disappear first.

In the experiment, the input is a text about medieval architecture. Trained up to the 9th generation, the model has produced a long list of earrings.

The researchers pointed out at the time that it is not impossible to train the next generation with AI data, but the screening step must be real.

This is the quickest route for AI science to get rid of RSI, and the most pressing threat to superintelligence (ASI) today.

The future of AI is likely to take on more and more unknown tasks.

But no matter how far it goes, human beings must see the evidence, be able to verify the results, and have the right to decide the direction.