Carnegie Mellon University

Electrical and Computer Engineering

College of Engineering

Course Information

18-753: Information Theory Measures for Artificial and Natural Intelligence Systems




We will discuss how, through examples, counterexamples, thought experiments, and desirable properties, one can systematically arrive at new measures of information relevant in Fairness, Accountability, Transparency, and Explainability (FATE) of machine learning. We will use data and examples from the real world to explore and understand these issues. The focus will be on defining measures -- classic and novel -- as well as the interplay between causality and information that is increasingly important when deciding whether and how to intervene on a decision-making system. The course introduces the measures of entropy, mutual information, partial information decomposition, causal inference, and their operational meaning. We will also discuss examples of how causal inference and existing measures come together. The course will discuss FATE of ML, information flow and interventions in neuroengineering and neuroscience, and making inferences in computational/dynamical systems (e.g. those in AI and neuroscience), and will engage with hands-on data analyses and real world examples.

Last Modified: 2024-01-19 12:49PM

Semesters offered:

  • Spring 2024
  • Spring 2022
  • Spring 2021
  • Spring 2020
  • Spring 2018
  • Spring 2016
  • Spring 2014
  • Spring 2012
  • Spring 2010
  • Spring 2008
  • Spring 2006
  • Spring 2004
  • Fall 2001