antistatic.exchange
Register Log in
Back to feed

Both AI R&D-4 and CBRN-4 are high thresholds to pass, although AI capabilities improvements have been carrying along very quickly. However, there are some variables outside of AI research that are important to remember when thinking about forecasting when these thresholds will be released:

  1. It's not inevitable that AI companies are financially able to carry on full-speed ahead with their research until the end of 2030, as there is a very real chance of a major market correction. OpenAI and Anthropic are burning massive amounts of money, and one or both of these companies might run out of money in the near future. Furthermore, AI companies are starved for compute to the point that it is affecting the reliability of their products (Claude's uptime has been quite weak over the past several months). While companies have moved away from scaling as the primary source of their gains, I think that it will still require increasing huge amounts of money to train new SoTA models.

  2. There is already starting to be a societal backlash to AI, and I expect this to significantly increase to the point where it influences how AI companies operate. I think this might influence the leading AI companies to be more cautious of pushing capabilities forward.

  3. Even without a backlash, the companies might be more cautious of pushing capabilities forward as their models become more powerful. And leading AI companies might decide it is best to focus more of their time/effort/resources on monetizing the products they already have instead of racing toward superintelligence.

With that said, the leading AI companies have made it clear that they are focused on automation of AI R&D, so I expect them to continue this goal until one or more of the other variables I mentioned makes it harder for them to do so. It's important to also look at their definition: "the ability to fully automate the work of an entry-level, remote-only Researcher at Anthropic." This is impressive, but these entry-level researchers are not necessarily the ones responsible for Anthropic's biggest gains in capabilities (although I have never worked there, so perhaps I am underestimating how fundamental their entry-level researchers are to Claude's capability gains).

The CBRN-4 forecast is a bit of a different situation, as the leading AI companies are concerned with releasing dangerous models, so I do not expect them to release a model to the general public if it's clear that the model crosses the CBRN-4 threshold. Perhaps such a model could be used internally for AI research or given to very select clients (such as the US government) for use, so I'm not completely discounting the fact that these companies do not release dangerous models at all, even if these releases are not available to the general public. Claude Mythos is a good example that perhaps foreshadows such a scenario.

Here are my current forecasts:

AI R&D-4:

By end of 2026: 10%

By end of 2027: 25%

By end of 2028: 40%

By end of 2029: 55%

By end of 2030: 65%

In general, I'm less confident than the crowd forecast that AI R&D-4 is right around the corner.

CBRN-4:

By end of 2026: 3%

By end of 2027: 10%

By end of 2028: 25%

By end of 2029: 40%

By end of 2030: 50%

I am higher than the crowd for CBRN-4 because I think Anthropic might decide just to release its more dangerous models to the US government, and I'm skeptical the AI safety research they do could fully prevent these models from having more dangerous capabilities as their overall capabilities continue to improve.

0 replies

No replies yet.