CSE 545, Error Correcting Codes: Combinatorics, Algorithms and Applications (original) (raw)

Instructor Atri Rudra Email: Phone: (716) 645-2464 Office: 123 Bell Hall Office Hours: By appointment	Class Meetings Monday, Wednesday, Friday. 2:00-2:50pm 110 Baldy Hall

Previous course offerings (with lecture notes): Fall 2007, Spring 2009

Course Blog

We will be using a blog for the course in lieu of a course newsgroup. All announcements will be made on the blog. If you are attending the course, you must check the blog regularly (and consider subscribing to the RSS feed or subscribe to receive email notifications for new posts.).

Homeworks

Homework 0. (No due date.)
Homework 1. (Due Feb 12.)
Homework 2. (Due Feb 26.)
Homework 3. (Due March 24.)
Homework 4. (Due April 2.)

Lectures

N.B.	The notes marked Draft of notes have not been reviewed by me at all. I hope to get them polished by summer 2010. --Atri

Use this style file for scribing notes. To get you started, here is a sample TeX file. (Both of these have been borrowed from Venkat Guruswami's style file from his coding theory course.)

Lecture 1: Introduction (Slides: ppt).
Lecture 2: Definitions. [Fall 07 notes: Lecture 1 and Lecture 2]. (Proof reader: Andrew Hughes)
Lecture 3: Error-Correction and Noise Models. [Fall 07 notes: Lecture 2 and Lecture 3]. (Proof reader: Devanshu Pandey)
Lecture 4: Minimum Distance. [Fall 07 notes: Lecture 3]. (Proof reader: Kushal Suryamohan)
Lecture 5: Hamming Code. [Fall 07 notes: Lecture 4]. (Proof reader: Shujie Liu)
Lecture 6: Hamming Bound. [Fall 07 notes: Lecture 5]. (Proof reader: Xinglei Zhu)
Lecture 7: Linear Codes. [Fall 07 notes: Lecture 5 and Lecture 6]. (Proof reader: Utkarsh Porwal)
Lecture 8: More on linear Codes. [Fall 07 notes: Lecture 6]. (Proof reader: Jiun-Jie Wang)
Lecture 9: Dual of linear Codes. [Fall 07 notes: Lecture 7]. (Proof reader: Sarah Lynn Karpie)
Lecture 10: Group Testing. (Scribe: Devnashu) [Draft of Notes]
Lecture 11: Group Testing Bounds. (Scribe: Andrew) [Draft of Notes]
Lecture 12: Bounds on the Volume of a Hamming Ball. [Fall 07 notes: Lecture 9 and Lecture 15]. (Proof Reader: Andrew)
Lecture 13: Gilbert-Varshamov Bound. [Fall 07 notes: Lecture 15, Spring 09 notes: Lecture 4]. (Proof Reader: Devanshu)
Lecture 14: Singleton and Plotkin Bounds. [Fall 07 notes: Lecture 11 and Lecture 16]. (Proof Reader: Kushal)
Lecture 15: Proof of Plotkin Bound. [Fall 07 notes: Lecture 16 and Lecture 17]. (Proof Reader: Shujie)
Lecture 16: A Geometric Lemma. [Fall 07 notes: Lecture 17]. (Proof Reader: Xinglei)
Lecture 17: Reed-Solomon Codes. [Fall 07 notes: Lecture 12]. (Proof Reader: Utkarsh)
Lecture 18: Properties of Reed-Solomon Codes. [Fall 07 notes: Lecture 12 and Lecture 8, Spring 09 notes: Lecture 12]. (Proof Reader: Jiun-Jie)
Lecture 19: Shannon's Theorem. [Fall 07 notes: Lecture 8]. (Proof Reader: Sarah)
Lecture 20: Proof of Shannon's Theorem. [Fall 07 notes: Lecture 10]. (Proof Reader: Andrew)
Lecture 21: Proof of Shannon's Theorem-II. [Fall 07 notes: Lecture 10]. (Proof Reader: Devanshu)
Lecture 22: Converse of Shannon's Theorem [Fall 07 notes: Lecture 9 and Lecture 11]. (Proof Reader: Kushal)
Lecture 23: List Decoding [Fall 07 notes: Lecture 12 and Lecture 13]. (Proof Reader: Shujie)
Lecture 24: List Decoding Capacity [Fall 07 notes: Lecture 14]. (Proof Reader: Xinglei)
Lecture 25: List Decoding from Random Errors (Scribe: Kushal) [Draft of Notes]
Lecture 26: Elias-Bassalygo Bound [Fall 07 notes: Lecture 19]. (Proof Reader: Utkarsh)
Lecture 27: Code Concatenation [Fall 07 notes: Lecture 24]. (Proof Reader: Jiun-Jie)
Lecture 28: Disjunct Matrices (Scribe: Utkarsh) [Draft of Notes]
Lecture 29: Construction of Disjunct Matrices (Scribe: Sarah) [Draft of Notes]
Lecture 30: Justesen Codes [Fall 07 notes: Lecture 25]. (Proof Reader: Sarah)
Lecture 31: Decoding Concaetnated Codes [Fall 07 notes: Lecture 26]. (Proof Reader: Andrew)
Lecture 32: Generalized Minimum Distance Decoding [Fall 07 notes: Lecture 28]. (Proof Reader: Devanshu)
Lecture 33: Derandomized GMD Decoding [Fall 07 notes: Lecture 29]. (Proof Reader: Kushal)
Lecture 34: Achieving BSC Capacity [Fall 07 notes: Lecture 30]. (Proof Reader: Shujie)
Lecture 35: Berlekamp-Welch Decoder [Fall 07 notes: Lecture 26 and Lecture 27]. (Proof Reader: Xinglei)
Lecture 36: List Decoding RS Codes [Fall 07 notes: Lecture 37]. (Proof Reader: Utkarsh)
Lecture 37: Sudan's List Decoding Algorithm [Fall 07 notes: Lecture 37 and Lecture 38]. (Proof Reader: Jiun-Jie)
Lecture 38: Guruswami-Sudan List Decoding Algorithm [Fall 07 notes: Lecture 38 and Lecture 39]. (Proof Reader: Sarah)
Lecture 39: Data Stream Algorithms (Scribe: Xinglei).
Lecture 40: Hot Items problem (Scribe: Shujie).
Lecture 41: Group Testing and List Recovery (Scribe: Jiun-Jie).
Lecture 42: Wrap-up.

Handouts

Syllabus for the course, andFeedback form. (Jan 11)
Feedback form. (Feb 15)

Optional Reading

Claude E. Shannon, A mathematical theory of communication. Bell System Technical Journal, vol. 27, pp. 379-423 and 623-656, July and October, 1948.
R. W. Hamming, Error Detecting and Error Correcting Codes. Bell System Technical Journal, vol. 29, pp. 147-160, April, 1950.
Atri Rudra, Algorithmic Coding Theory. Book Chapter in CRC Handbook on Algorithms and Theory of Computation edited byMike Atallah andMarina Blanton, Second edition. (Single spaced version).
This will give an overview of the topics we will cover in the class. Comments are welcome.

Course Pre-requisites

There is no specific course pre-requisite for this course. However, some "mathematical maturity" will be essential. In particular, comfort with basics of linear algebra (vector spaces, basis, dual spaces); finite fields, field extensions and polynomials over finite fields; elementary probability; analysis of algorithms; and (some exposure to) computational complexity will be useful. Some of these topics (for example finite fields) can be learned on a need to know basis as the course progresses. Email the instructor if you have any questions on the pre-requisites.

Reference material

We will not follow any particular textbook. Instead, we will use lecture notes from the previous fall 2007 and spring 2009 offerings of the course.Note: Unfortunately, not all the lecture notes are polished (i.e. those marked as draft): I plan to polish them before the relevant lecture this semester.

Another good resource is the excellent set of lecture notes for Madhu Sudan's coding theory course at MIT: Notes from 2001, 2002, 2004 and 2008. You might also want to check out thenotes from Venkat Guruswami's coding theory course.

The basic material on codes that we will discuss in initial lectures can be found in one of many textbooks (some of the standard ones are listed below), but the recent algorithmic developments and applications in computer science are not covered in any of these:

Introduction to Coding Theory, by J. H. van Lint, GTM 86.
The Theory of Error Correcting Codes, by F. J. MacWilliams and N. J. A. Sloane, North-Holland, Amsterdam.
Algebraic codes for data transmission. by Richard E. Blahut. Though we won't cover much information theory in this course, if you are interested to know more on topics such as entropy, capacity theorems, source coding, etc., there is the classic information theory text
Elements of Information Theory, Thomas M. Cover and Joy A. Thomas, Wiley Series in Telecommunications, 1991.

Workload

The workload will be moderate: there will be no exams. The following will be the major components:

Scribing notes. A few lectures will cover topics that were not covered in the last offering: these lectures will need to be scribed. I don't expect this load to be more than one lecture/student. I will typically ask for a volunteer at the beginning of the class.
Proof-reading notes. For lectures that have already been covered before, I will ask a volunteer to go over the existing lecture notes and email me any comments that they might have (e.g. typos, something that needs better explanantion etc.). Depending on the number of students, each student might have to volunteer for 6-7 lectures. However, each of them should not be time consuming as all of these notes would have been proof-read by me at least once.
Updating Wikipedia. Every student will pick a coding theory topic that is either absent or not well documented on Wikipedia and write/edit the corresponding entry. You will have the opportunity to work on initial versions of the entry on an in-house wiki page before the entries are posted on Wikipedia (you'll need to get my OK before you post your entry on Wikipedia). See the syllabus for the various deadlines.
Homeworks. Either one long or roughly 3 short ones.

Grading Policy

Here is a rough split of grades:

Scribing/Proof-reading notes (30-40%). The scribed notes entries will be graded on and timeliness of completion.
Homeworks (40-25%). Collaboration in groups of size at most three is allowed (and encouraged). However, every student is expected to do their own writeups and clearly state the names of their collaborators. More details will be available when the homeworks are handed out.
Updating Wikipedia (30-35%). The grade will depend on the clarity/quality of the writeup. For more details see the syllabus.

Academic Honesty

I have zero tolerance for cheating and will follow the CSE Department Policies on Academic Integrity.

Acknowledgment

The material in this webpage is supported in part by the National Science Foundation under CAREER grant CCF-0844796. Any opinions, findings and conclusions or recomendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation (NSF).