Statistics and Information Theory

During the course we will follow the lecture notes of Imre Csiszár and Paul C. Shields (supplements will be given, some parts will be omitted)

Sometimes I will supplement the material using the following sources:

Elements of Information Theory (written by Thomas M. Cover and Joy A. Thomas)

Information Theory: Coding Theorems for Discrete Memoryless Systems (witten by Imre Csiszár and János Körner)

Daniel Horvath drew my attention to the following article:

visualization based introduction to information measures and data compression

The homework is divided into 6 sets. Each set worth 10 points. I will collect your best 4 sets and I will sum the points.

First homework set

Second homework set

Third homework set

4th homework set

5th homework set

6th homework set

There will be written exam during the exam period. For the exam, you will have to learn everything that I will teach you during the semester (definitions, theorems, and proofs). All the questions will be related to the course. !4! pages long (!2! sheets of A4 paper) hand-written notes can be used during the exam. The exam is successful if your point is at least 30. In case of a successful exam, I will add the homework point and the exam point and I will give grade according to the following system: !!!45-59!!! (2), 60-69 (3), 70-79 (4), 80-100 (5).