ABOUT THE LECTURE
Animals and humans understand the physical world, have common sense, possess a persistent memory, can reason, and can plan complex sequences of subgoals and actions. These essential characteristics of intelligent behavior are still beyond the capabilities of today’s most powerful AI architectures, such as Auto-Regressive LLMs.
I will present a cognitive architecture that may constitute a path towards human-level AI. The centerpiece of the architecture is a predictive world model that allows the system to predict the consequences of its actions. and to plan sequences of actions that that fulfill a set of objectives. The objectives may include guardrails that guarantee the system’s controllability and safety. The world model employs a Joint Embedding Predictive Architecture (JEPA) trained with self-supervised learning, largely by observation.
The JEPA simultaneously learns an encoder, that extracts maximally-informative representations of the percepts, and a predictor that predicts the representation of the next percept from the representation of the current percept and an optional action variable.
We show that JEPAs trained on images and videos produce good representations for image and video understanding. We show that they can detect unphysical events in videos. Finally, we show that planning can be performed by searching for action sequences that produce predicted end state that match a given target state.
ABOUT THE SPEAKER yann lecun, VP & chief AI scientist, meta; professor, NYU; ACM turing award laureate.
Yann LeCun is VP & Chief AI Scientist at Meta and a Professor at NYU. He was the founding Director of Meta-FAIR and of the NYU Center for Data Science. After a PhD from Sorbonne Université and research positions at AT&T and NEC, he joined NYU in 2003 and Meta in 2013. He received the 2018 ACM Turing Award for his work on AI. He is a member of the US National Academies and the French Académie des Sciences.
Leave a reply