Lifeboat Foundation

Our final estimate of the achievable inter data center bandwidth by 2030 is 4 to 20 Pbps, which would allow for training runs of 3e29 to 2e31 FLOP. In light of this, bandwidth is unlikely to be a major constraint for a distributed training run compared to achieving the necessary power supply in the first place.

Expanding bandwidth capacity for distributed training networks presents a relatively straightforward engineering challenge, achievable through the deployment of additional fiber pairs between data centers. In the context of AI training runs potentially costing hundreds of billions of dollars, the financial investment required for such bandwidth expansion appears comparatively modest.⁴⁴

We conclude that training runs in 2030 supported by a local power supply could likely involve 1 to 5 GW and reach 1e28 to 3e29 FLOP by 2030. Meanwhile, geographically distributed training runs could amass a supply of 2 to 45 GW and achieve 4 to 20 Pbps connections between data center pairs, allowing for training runs of 2e28 to 2e30 FLOP.⁴⁵ All in all, it seems likely that training runs between 2e28 to 2e30 FLOP will be possible by 2030.⁴⁶ The assumptions behind these estimates can be found in Figure 3 below.

Enjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.

From AI-generated video advancements to new AI-powered AR glasses, here were the 5 biggest announcements.

A race is on to push artificial intelligence beyond today’s limits.

Only competition between countries will prevent progress from being stopped.

Beijing opted out of a pact to keep humans in control of decision-making on nuclear weapons, a move observers say underlines its dilemma.

Many industries are experiencing digital transformation because of AI. Here’s how AI promotes innovation.

Scientists are using ever more sophisticated AI algorithms trained on vast, unlabeled datasets to develop models that can ‘interpret’ biological data to help guide biomolecule design.

The brain dynamically transforms cognitive information. Here the authors build task-performing, functioning neural network models of sensorimotor transformations constrained by human brain data without the use of typical deep learning techniques.

To expand its GPT capabilities, OpenAI released its long-anticipated o1 model, in addition to a smaller, cheaper o1-mini version. Previously known as Strawberry, the company says these releases can “reason through complex tasks and solve harder problems than previous models in science, coding, and math.”

Although it’s still a preview, OpenAI states this is the first of this series in ChatGPT and on its API, with more to come.

The company says these models have been training to “spend more time thinking through problems before they respond, much like a person would. Through training, they learn to refine their thinking process, try different strategies, and recognize their mistakes.”