Lectures & Medias

Abstract: In the current AI landscape, reasoning is frequently equated with the generation of intermediate “thinking traces”. However, these traces are merely a mechanism, not the ultimate objective. Relying solely on the presence of a trace can be deceptive, as models often learn to mimic the format of reasoning while effectively overfitting to specific training distributions. To build more robust and versatile reasoners, we shift our focus to more specific structural properties of the thinking process, in particular compositionality (inductive CoT, AdaBack) and abstraction (AbstRaL).

L’Intelligence Artificielle n’est plus une promesse lointaine, mais une réalité qui transforme en profondeur nos manières de travailler. Où en est-elle aujourd’hui ? Et comment son évolution fulgurante dessine-t-elle notre avenir ? Cette présentation abordera les grandes caractéristiques de son développement, des avancées techniques aux défis liés au raisonnement. L’objectif : mieux cerner les enjeux majeurs pour élaborer des stratégies capables de faire de la disruption un véritable levier d’opportunités.

In 1948, Shannon used a probabilistic argument to show the existence of codes achieving channel capacity. In 1954, Muller and Reed introduced a simple deterministic code construction, conjectured shortly after to achieve channel capacity. Major progress was made towards establishing this conjecture over the last decades, with the involvement of various areas of discrete mathematics. In particular, the special case of the erasure channel was settled in 2015 by Kudekar et al., relying on Bourgain-Kalai’s sharp threshold theorem for symmetric monotone properties. The case of error channels remained however unsettled, due in particular to the property being non-monotone, and the absence of techniques to obtain fast local error decay. In this talk, we provide a proof of the conjecture. The proof circumvents the requirement of monotone properties to establish first a “weak” threshold property that relies solely on symmetries. From there on, the proof proceeds with a new boosting framework for coding, using sunflower structures and weight enumerator bounds from Sberlo-Shpilka to control the global error down to capacity. Joint work with Colin Sandon.

On November 11, 2022, the scientific community celebrated the inauguration of EPFL’s new Bernoulli Center for Fundamental. The event, which brought together illustrious mathematicians, physicists, and computer scientists, marked the kickoff of the center’s second incarnation, with an expanded scope, an updated mission, new facilities, and a revised governance structure. The Bernoulli Center for Fundamental Studies will host ambitious initiatives to promote major discoveries and excellence in fundamental sciences, raising EPFL’s standing as a central hub for fundamental sciences. 

Abstract: This talk investigates the fundamental limits of learning by training neural networks with gradient-based algorithms. We show that when the initialization is free and, (i) when SGD is used, this is as powerful as PAC, (ii) when GD is used, this is limited to statistical query learning, (iii) when mini-batch-SGD is used, there is a tradeoff between (i) and (ii) depending on the batch-size versus precision noise regime, (iv) irrespective of SGD/GD/batch-size, this is strictly stronger than kernel-based learning. We then show that when the initialization is constrained to be ‘homogeneous’, a new class of staircase functions seem to capture the fundamental limits of gradient-based learning. Based on several joint works with Boix, Brenner, Bresler, Kamath, Malach, Nagarj, Sandon, Srebro.

In this Computer Science / Discrete Mathematics Seminar talk given at the Institute for Advanced Study on November 28, 2016, Emmanuel Abbe (Princeton University) presents on stochastic block models and probabilistic reductions. The stochastic block model (SBM) is a foundational random graph model used to study community detection — the problem of recovering hidden cluster structure in networks. Abbe’s work in this area established sharp information-theoretic thresholds for when communities can be recovered exactly or partially, and the talk explores how probabilistic reductions can be used to relate the SBM to other inference problems and to derive these recovery limits.