Carnegie Mellon University

Electrical and Computer Engineering

College of Engineering

Course Information

18-753: Information Theory




The first half of the course comprises of the concepts of entropy, mutual information, the Asymptotic Equipartition property, applications to source coding (data compression), applications to channel capacity (channel coding), differential entropy and its application to waveform channel capacities, and a subset of advanced topics such as network information theory, or rate-distortion theory, as time permits. The second half of the course comprises finite-field algebra, Hamming codes, cyclic codes (CRC and BCH codes), a brief introduction to Reed-Solomon codes, and perhaps universal codes (Lempel-Ziv coding).

Prerequisites: 36-217 and senior or graduate standing.

Last Modified: 2020-12-01 1:06PM

Semesters offered:

  • Spring 2021
  • Spring 2020
  • Spring 2018
  • Spring 2016
  • Spring 2014
  • Spring 2012
  • Spring 2010
  • Spring 2008
  • Spring 2006
  • Spring 2004
  • Fall 2001