Data Compression (and Some Learning) with Logarithmic Loss

Date: 10:00am, February 27, 2017
Location:  Costa Commons (CEPSR 750)
Speaker: Dr. Yanina Shkel, Fellow NSF Center for Science of Information

Abstract:  In this talk we discuss data compression, as well as learning from data, with a specific performance criterion: the logarithmic loss (log-loss). The log-loss measures distortion in settings when the reconstructed information is soft, that is, a distribution over possible values is provided. It is natural to construct data processing modules which interface by processing such soft information: consider, for example, the Belief Propagation algorithm that works by passing probabilities between nodes in order to perform statistical inference on graphical models. Moreover, log-loss has nice mathematical properties which allow for strong theoretical bounds for compression and learning. Thus, log-loss performance criterion is of particular interest in this setting. The focus of our talk will be on the non-asymptotic and universal fundamental limits of lossy compression where focusing on log-loss lets us obtain simple and elegant results that generalize those in prediction and lossless coding. Finally, we will discuss connections with other statistical learning paradigms such as the Information Bottleneck method and the Minimum Description Length principle.

  Dr. Shkel is a postdoctoral fellow with the NSF Center for Science of Information where she works with collaborators at University of Illinois at Urbana-Champaign and at Princeton University. She has B.S. degrees in Mathematics and in Computer Science, as well as a Ph.D. degree in Electrical and Computer Engineering from University of Wisconsin-Madison. Before attending graduate school Dr. Shkel also worked as a developer for a financial company Morningstar, Inc. More recently, she spent time as an intern at 3M Corporate Research Labs and a visiting graduate student at University of Toronto. Dr. Shkel is broadly interested in identifying laws which govern the behavior of information in both engineered and naturally occurring systems, and using these laws to better understand the capabilities of such systems. Her most recent research focuses on points of connection between machine learning, statistics, and information theory.

500 W. 120th St., Mudd 1310, New York, NY 10027    212-854-3105               
©2014 Columbia University