# InfoCoBuild

## An Introduction to Information Theory

An Introduction to Information Theory. Instructor: Prof. Adrish Banerjee, Department of Electrical Engineering, IIT Kanpur. Information Theory answers two fundamental questions: what is the maximum data rate at which we can transmit over a communication link, and what is the fundamental limit of data compression. In this course we will explore answers to these two questions. We will study some practical source compression algorithms. We will also study how to compute channel capacity of simple channels. (from nptel.ac.in)

 Lecture 23 - Blahut-Arimoto Algorithm

In this lecture, we first describe alternating optimization algorithm in a general settings. Then we describe Blahut Arimoto algorithm for computing channel capacity as well as rate distortion function.

Go to the Course Home or watch other lectures:

 Lecture 01 - Introduction Lecture 02 - Measure Information Lecture 03 - Information Inequalities Lecture 04 - Problem Solving Session I Lecture 05 - Block to Variable Length Coding: Prefix Free Code Lecture 06 - Block to Variable Length Coding: Bounds on Optimal Code Length Lecture 07 - Block to Variable Length Coding: Huffman Coding Lecture 08 - Variable to Block Length Coding Lecture 09 - The Asymptotic Equipartition Property Lecture 10 - Block to Block Coding of DMS (Discrete Memoryless Source) Lecture 11 - Problem Solving Session II Lecture 12 - Universal Source Coding: Lempel-Ziv Algorithm - LZ77 Lecture 13 - Universal Source Coding: Lempel-Ziv Welch Algorithm (LZW) Lecture 14 - Coding of Sources with Memory Lecture 15 - Channel Capacity Lecture 16 - Jointly Typical Sequences Lecture 17 - Noisy Channel Coding Theorem Lecture 18 - Differential Entropy Lecture 19 - Gaussian Channel Lecture 20 - Parallel Gaussian Channel Lecture 21 - Problem Solving Session III Lecture 22 - Rate Distortion Theory Lecture 23 - Blahut-Arimoto Algorithm Lecture 24 - Problem Solving Session IV