Lifeboat Foundation

Tesla has unveiled its new supercomputer, which is already the fifth most powerful in the world, and it’s going to be the predecessor of Tesla’s upcoming new Dojo supercomputer.

It is being used to train the neural nets powering Tesla’s Autopilot and upcoming self-driving AI.

Over the last few years, Tesla has had a clear focus on computing power both inside and outside its vehicles.

Circa 2019

As quantum computing enters the industrial sphere, questions about how to manufacture qubits at scale are becoming more pressing. Here, Fernando Gonzalez-Zalba, Tsung-Yeh Yang and Alessandro Rossi explain why decades of engineering may give silicon the edge.

In the past two decades, quantum computing has evolved from a speculative playground into an experimental race. The drive to build real machines that exploit the laws of quantum mechanics, and to use such machines to solve certain problems much faster than is possible with traditional computers, will have a major impact in several fields. These include speeding up drug discovery by efficiently simulating chemical reactions; better uses of “big data” thanks to faster searches in unstructured databases; and improved weather and financial-market forecasts via smart optimization protocols.

Continue reading “Manufacturing silicon qubits at scale” »

Scientists have long struggled to understand how common planets form. A new supercomputer simulation shows that the missing ingredient may be magnetism.

“These are novel living machines. They are not a traditional robot or a known species of animals. It is a new class of artifacts: a living and programmable organism,” says Joshua Bongard, an expert in computer science and robotics at the University of Vermont (UVM) and one of the leaders of the find.

As the scientist explains, these living bots do not look like traditional robots : they do not have shiny gears or robotic arms. Rather, they look more like a tiny blob of pink meat in motion, a biological machine that researchers say can accomplish things traditional robots cannot.

Xenobots are synthetic organisms designed automatically by a supercomputer to perform a specific task, using a process of trial and error (an evolutionary algorithm), and are built by a combination of different biological tissues.

Since the DeepSpeed optimization library was introduced last year, it has rolled out numerous novel optimizations for training large AI models—improving scale, speed, cost, and usability. As large models have quickly evolved over the last year, so too has DeepSpeed. Whether enabling researchers to create the 17-billion-parameter Microsoft Turing Natural Language Generation (Turing-NLG) with state-of-the-art accuracy, achieving the fastest BERT training record, or supporting 10x larger model training using a single GPU, DeepSpeed continues to tackle challenges in AI at Scale with the latest advancements for large-scale model training. Now, the novel memory optimization technology ZeRO (Zero Redundancy Optimizer), included in DeepSpeed, is undergoing a further transformation of its own. The improved ZeRO-Infinity offers the system capability to go beyond the GPU memory wall and train models with tens of trillions of parameters, an order of magnitude bigger than state-of-the-art systems can support. It also offers a promising path toward training 100-trillion-parameter models.

ZeRO-Infinity at a glance: ZeRO-Infinity is a novel deep learning (DL) training technology for scaling model training, from a single GPU to massive supercomputers with thousands of GPUs. It powers unprecedented model sizes by leveraging the full memory capacity of a system, concurrently exploiting all heterogeneous memory (GPU, CPU, and Non-Volatile Memory express or NVMe for short). Learn more in our paper, “ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning.” The highlights of ZeRO-Infinity include:

When Open AI’s GPT-3 model made its debut in May of 2020, its performance was widely considered to be the literal state of the art. Capable of generating text indiscernible from human-crafted prose, GPT-3 set a new standard in deep learning. But oh what a difference a year makes. Researchers from the Beijing Academy of Artificial Intelligence announced on Tuesday the release of their own generative deep learning model, Wu Dao, a mammoth AI seemingly capable of doing everything GPT-3 can do, and more.

First off, Wu Dao is flat out enormous. It’s been trained on 1.75 trillion parameters (essentially, the model’s self-selected coefficients) which is a full ten times larger than the 175 billion GPT-3 was trained on and 150 billion parameters larger than Google’s Switch Transformers.

In order to train a model on this many parameters and do so quickly — Wu Dao 2.0 arrived just three months after version 1.0’s release in March — the BAAI researchers first developed an open-source learning system akin to Google’s Mixture of Experts, dubbed FastMoE. This system, which is operable on PyTorch, enabled the model to be trained both on clusters of supercomputers and conventional GPUs. This gave FastMoE more flexibility than Google’s system since FastMoE doesn’t require proprietary hardware like Google’s TPUs and can therefore run on off-the-shelf hardware — supercomputing clusters notwithstanding.

A team at Stony Brook University used ORNL’s Summit supercomputer to model x-ray burst flames spreading across the surface of dense neutron stars.

At the heart of some of the smallest and densest stars in the universe lies nuclear matter that might exist in never-before-observed exotic phases. Neutron stars, which form when the cores of massive stars collapse in a luminous supernova explosion, are thought to contain matter at energies greater than what can be achieved in particle accelerator experiments, such as the ones at the Large Hadron Collider and the Relativistic Heavy Ion Collider.

Although scientists cannot recreate these extreme conditions on Earth, they can use neutron stars as ready-made laboratories to better understand exotic matter. Simulating neutron stars, many of which are only 12.5 miles in diameter but boast around 1.4 to 2 times the mass of our sun, can provide insight into the matter that might exist in their interiors and give clues as to how it behaves at such densities.

The Israeli military is calling Operation Guardian of the Walls the first artificial-intelligence war. the IDF established an advanced AI technological platform that centralized all data on terrorist groups in the Gaza Strip onto one system that enabled the analysis and extraction of the intelligence.

The IDF used artificial intelligence and supercomputing during the last conflict with Hamas in the Gaza Strip.

The US Department of Energy on Thursday is officially dedicating Perlmutter, a next-generation supercomputer that will deliver nearly four exaflops of AI performance. The system, based at the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory, is the world’s fastest on the 16-bit and 32-bit mixed-precision math used for AI.

The HPE Cray system is being installed in two phases. Each of Phase 1’s GPU-accelerated nodes has four Nvidia A100 Tensor Core GPUs, for a total of 6159 Nvidia A100 Tensor Core GPUs. Each Phase 1 node also has a single AMD Milan CPU.