Nonparametric Statistics and Information Theory

During the first part of the course I will use the lecture notes of Imre Csiszár and Paul C. Shields

Lecture notes

During the second part of the course we will follow the following lecture notes:

Lecture notes (written by László Györfi)


Sometimes I will supplement the material using the following sources:


Elements of Information Theory (written by Thomas M. Cover and Joy A. Thomas)

Information Theory: Coding Theorems for Discrete Memoryless Systems (witten by Imre Csiszár and János Körner)

The homework is divided into 6 sets. Each set is worth 8 points. I will collect your best 4 sets and I will sum the points. For the signature (without a signature you are not allowed to take an exam) this homework point must be at least 12.


The exam is worth 68 points. For the exam, you will have to learn everything that I will teach you during the semester (definitions, theorems, and proofs). All the questions will be related to the course. The exam is successful if your point is at least 30. In case of a successful exam, I will add the homework point and the exam point and I will give grade according to the following system: 47-59 (2), 60-69 (3), 70-79 (4), 80-100 (5).