AISTATS 2015 Keynote Presentations
May 9 (Saturday) 17:00 - 18:00
University of California, Irvine
The Ebb and Flow of Deep Learning: a Theory of Local Learning
In a physical neural system, where storage and processing are intertwined, the learning rules for adjusting synaptic weights can only depend on local variables, such as the activity of the pre- and post-synaptic neurons. Thus learning models must specify
two things: (1) which variables are to be considered local; and (2) which kind of function combines these local variables into a learning rule. We consider polynomial learning rules and analyze their behavior
and capabilities in both linear and non-linear networks. As a byproduct, this framework enables the discovery of new learning rules and important relationships between learning rules and group symmetries.
Stacking local learning rules in deep feedforward networks leads to deep local learning. While deep local learning can learn interesting representations, it cannot learn complex input-output functions, even when targets are available for the top layer. Learning complex input-output functions
requires instead local deep learning, where target information is transmitted to the deep layers, thereby raising two fundamental issues: (1) the nature of the transmission channel; and (2)
the nature and amount of information transmitted over this channel. This leads to the class of deep targets learning algorithms, which provide targets for the deep layers, and its stratification along the information spectrum,
illuminating the remarkable power and uniqueness of the backpropation algorithm. The theory clarifies the concept of Hebbian learning, what is learnable by Hebbian learning, and explains the sparsity of the space of learning rules discovered so far and the unique role
backpropagation plays in this space.
Pierre Baldi earned MS degrees in Mathematics and Psychology from the University of Paris, and a PhD in Mathematics from the California Institute of Technology. He is currently Chancellor's Professor in the Department of Computer Science,
Director of the Institute for Genomics and Bioinformatics, and Associate Director of the Center for Machine Learning and Intelligent Systems at the University of California Irvine. The long term focus of his research is on understanding
intelligence in brains and machines. He has pioneered the development and application of deep learning methods to problems in the natural sciences such as the detection of exotic particles in physics,
the prediction of reactions in chemistry, and the prediction of protein secondary and tertiary structure in biology. He is the recipient of the 1993 Lew Allen Award at JPL, the 1999 Laurel Wilkening Faculty Innovation Award at UCI, a 2006 Microsoft Research Award, the 2010 E. R. Caianiello Prize for research in
machine learning, and a 2014 Google Faculty Research Award. He is and Elected Fellow of the AAAS, AAAI, IEEE, ACM, and ISCB.
May 10 (Sunday) 08:30 - 09:30
How Deep Learning Transforms Chinese Internet Search
Since 2012, the Chinese internet giant Baidu has launched a significant effort to develop deep learning technologies to transform its search engine to the next stage. The company established Institute of Deep Learning (IDL), the first research team of Baidu and later expanded the research force to a bigger organization Baidu Research. By leveraging big data and massive parallel computation on CPU and GPU servers, Baidu researchers have achieved transformative improvements to Baidu's core business, such as web search, online advertising, speech recognition, computer vision, natural language processing, smart data center, robotics, etc.. In this talk I will showcase some of the highlights, and introduce the underlying technologies. These technologies have been shaping up a foundation for imaginative long-term innovation of Baidu. In the end I will discuss some thoughts about the future directions of deep learning.
Dr. Kai YU is the Deputy Managing Director of Baidu Research, the Head of Baidu Institute of Deep Learning (IDL), and a Senior Engineering Director of Baidu. In 2013 and 2014, his team has won 3 Highest Achievement Awards of Baidu, for significant contributions to speech recognition, computer vision, online ads, and web search ranking. Dr. Yu is elected to China "Top 1000 Talents", a prestigious group of technology leaders of the country. Dr. Yu has published a number of papers in NIPS, ICML, AISTAT, CVPR, ICCV, ECCV, SIGIR, SIGKDD, etc., with more than 8000 citations. He and his colleagues have received many awards including the Best Paper Runner-up Award of ICML-2013, the First Place of PASCAL VOC 2009, and the First Place of ImageNet Challenge 2010. He is an Adjunct Professor of Nanjing University, Beijing University of Posts and Telecommunications, and Institute of Computing Technology, Chinese Academy of Sciences. Before joining Baidu, he worked at Microsoft, Siemens, NEC Labs America, and was an Adjunct Faculty at Department of Computer Sciences at Stanford University, teaching "CS121: Introduction to AI". He received his B.Sc and M.Sc degrees from Nanjing University, China, and his Ph.D. degree from University of Munich, Germany.
May 11 (Monday) 08:30 - 09:30
Data and/as Journalism
As statisticians know well, data and data technologies have profound
social and political relevance -- almost every aspect of our lives can
"rendered" in data. These data tell us stories about who we are and
how we live, but, as products of human attention, innovation and
memory, their perspective is not neutral and the stories they tell are
often incomplete, open-ended and biased.
Data can either reinforce or challenge systems of power and so, in a
very real sense, our democracy depends on the public being able to
think critically about these technologies, telling the good stories
from the bad. As "truth-tellers, sense-makers and explainers of last
resort," to quote my colleague Emily Bell, journalists needs to
understand the working of data in society, and the ways in which data,
code and algorithms function.
In this talk, I will look at how journalism is adapting to a world of
"Big Data" and "Data Science." From a pedagogical perspective,
journalism acts as a kind of "integrator" of real world problems with
more technical statistical and computational perspectives. The
practice of journalism fills a gap between lived experience and data,
its analysis and its visualization -- and might even expand the way we
Mark Hansen is a Professor in the Columbia Graduate School of
Journalism, where he heads the Brown Institute for Media
Innovation. Prior to that he was a Professor in the Department of
Statistics at UCLA and a Member of the Technical Staff at Bell
Laboratories. He is also co-founder of the Office for Creative
Research, a design group that has exhibited work at the Museum of
Modern Art in New York, the Whitney Museum, the Centro de Arte Reina
Sofia, the London Science Museum, the Cartier Foundation in Paris, and
the lobbies of the New York Times building and the Public Theater
(permanent displays) in Manhattan. He received a B.S. in Applied Math
from UC Davis and a Ph.D. in Statistics from UC Berkeley.
May 12 (Tuesday) 08:30 - 09:30
Scaling Machine Learning and Statistics for Web Applications
Scaling web applications like recommendation systems, search and computational advertising is challenging. Such systems have to make astronomical number of decisions every day on what to serve users when they are using the website and/or the mobile app. Machine learning and statistical modeling approaches that can obtain insights by continuously processing large amounts of data emitted at very high frequency by these applications have emerged as the method of choice. However, there are three challenges to scale such methods : a) scientific b)infrastructure and c) organizational. I will provide an overview of these challenges and the strategies we have adopted at LinkedIn to address those. Throughout, I will illustrate with examples from real-world applications at LinkedIn.
Big data analyst with 15 years of experience developing and deploying state-of-the-art machine learning and statistical methods for improving the relevance of web applications. Have worked in various positions: chief scientist of large projects, managed small and highly technical teams and also experienced in managing large teams (almost all PhDs). Most recently, leading a team of 60+ scientists and developers at LinkedIn. Also experienced in conducting new scientific research to solve notoriously difficult big data problems, especially in the areas of recommender systems and computational advertising. Fellow of the American Statistical Association, Member Board of Directors for SIGKDD, program chair of KDD in the past, associate editor of two top-tier journals in Statistics, regularly serves on senior program committees of top-tier conferences like KDD, NIPS, CIKM, ICDM, SIGIR, WSDM.