Statistics and Information Theory

During the course we will follow the lecture notes of Imre Csiszár and Paul C. Shields (supplements will be given, some parts will be omitted)

Lecture notes


Sometimes I will supplement the material using the following sources:


Elements of Information Theory (written by Thomas M. Cover and Joy A. Thomas)

Information Theory: Coding Theorems for Discrete Memoryless Systems (witten by Imre Csiszár and János Körner)

Things to do - 9th class

The homework is divided into 6 sets. Each set is worth 10 points. I will collect your best 4 sets and I will sum the points. For the signature (without a signature you are not allowed to take an exam) this homework point must be at least 20.


There will be written exam during the exam period. The exam is worth 60 points. For the exam, you will have to learn everything that I will teach you during the semester (definitions, theorems, and proofs). All the questions will be related to the course. !4! pages long (!2! sheets of A4 paper) !!!hand-written!!! notes can be used during the exam. The exam is successful if your point is at least 30. In case of a successful exam, I will add the homework point and the exam point and I will give grade according to the following system: 50-59 (2), 60-69 (3), 70-79 (4), 80-100 (5).


First homework set

Second homework set