Academics
  /  
Courses
  /  
Descriptions
ELEC_ENG 328, 428: Information Theory and Learning


VIEW ALL COURSE TIMES AND SESSIONS

Description

Fall 2020 course will meet in Tech Auditorium

This course gives students analytical tools to quantify information, perform inference, and study the relationship of information and learning. The course covers information measures, the source and the channel coding theorems, statistical inference, and learning with neural networks. In particular, the course explores a common set of models and tools used by both machine learning and state-of-the-art data compression and error-control codes. This course is aimed at undergraduate students in engineering, science, mathematics, and computing. It expects familiarity with undergraduate-level calculus, probability theory, and linear algebra. 

Prerequisites by course: ELEC_ENG 302 Probabilistic Systems or equivalent. 

Prerequisites by topic: Good understanding of basic probability. 

REQUIRED TEXT: 
J. MacKay, Information Theory, Inference and Learning Algorithms, Cambridge, 2004
https://www.inference.org.uk/itprnn/book.pdf 

REFERENCE TEXTS:
Cover & Thomas, Elements of Information Theory, 2nd ed., Wiley, 2006. 

COURSE DIRECTOR: Prof. Dongning Guo 

There will be weekly problem sets. 

GRADES:
Problem sets: 50%
Midterm exam: 20%
Final Exam: 30%