Bryn Mawr College
CS 380: Recent Advances in Computer Science
Topic: Science of Information
BMC Class Number: 2283
Instructor: Deepak Kumar, 202 Park Hall, 526-7485
E-Mail: dkumar at cs brynmawr dot edu
Lecture Hours: Mondays & Wednesdays 11:40a to 1:00p
Office Hourse: Wednesdays 2:00 to 3:30p
Room: Park Science Building, Room 159
- Computer Science Lab Room 231 (Science Building), self scheduled.
Claude Shannon's foundations of information theory have paved the way for data storage, compression, encoding, and transmission for the Internet, CDs, DVDs, MP3 players, JPEGs, WiFi, iPODs, mobile phones, and a whole host of applications underlying today's information technologies. The past six decades have brought information theory to the crossroads of several traditional disciplines: mathematics, statistics, computer science, physics, neurobiology, and electrical engineering. This course introduces students to the fundamentals of Information Theory and leads them to a broader understanding of the concept of “information” that transcends boundaries between disciplines, especially between physical and life sciences, communication, and knowledge extraction from massive datasets. Students in several disciplines will be able to draw upon the latest discoveries from multiple disciplines, replicate and discuss recent research, and learn to apply the techniques and tools of information-based inquiry in their lives.
The course will run in a seminar format where students will engage by participating in and leading discussions and presenting results from readings and computational experiments. The course requires a Junior or Senior standing. Students from ALL disciplines are encouraged to enroll.
Texts & Software
|Information: A Very Short Introduction: By Luciano Floridi, Oxford University Press, 2010.
You do not need to buy this book. It is available for all Tri-co students for free in electronic form. Click here.
The following topics are planned to be covered in the course...
- What is information?
- The Information Revolution
- The Mathematical Theory of Information
- Semantic & Physical Information
- Biological Information
- Economic Infomration
- The Ethics of Information
- Information Retreival & Big Data
- Information Visualization
- Quantun Information
September 4: First Meeting
December 11: Last Meeting
- Assignment#1 is posted (Due on Monday, September 16): Click here for details
- Assignment#2 is posted (Due on Monday, September 30): Click here for details.
- Week 1 (September 4)
September 4: Course Introduction. What is information? A class discussion.
Read: Chapter 1 from Floridi.
Slides: What is Information?
- Week 2 (September 9, 11)
September 9: Information: An Overview. Defining Information: Information = Data + Meaning. Understanding Data. Types of Data. Floridi's taxonomy.
Read: Chapter from Floridi.
Slides: Understanding Data & Information.
September 11: Information: Broader Perspectives. Syntactic versus Semantic distinctions. The five E-s of Information (Entropy, Economics, Encryption, Extraction, Emission). Towards a science of information: structure, time, space, semantics, cooperation, etc. Center for Science of Information. A little history of Point-toPoint communication.
Read: Chapter 3 from Floridi.
Homework#1 (Due on Monday, September 16): Click here.
- Week 4 (September 16, 18)
September 16: Defining Shannon Information: Motivations and background. "Figure 1". Information. Entropy. Information Source.
Slides: Mathematical Theory of Information - Part1
Read: Chapter 3 from Floridi. Shannon 1948.
September 18: "Pop Quiz". Defining Entropy. Introduction to coding.
Slides: Mathematical Theory of Information - Part 2
- Week 4 (September 23, 25)
September 23: Information (I), Entropy (H), Source Coding basics and terminology. Source Coding Theorem (Shannon's First Theorem). Huffman Encoding.
Slides: Mathematics Theory of Information - Part 2 (B) [Note: There is some more material here that the slides from last class.]
Assignment#2 is posted (Due on Monday, September 30): Click here for details.
September 25: Lossless compression of English Text. Zipf's Law. Lempel-Ziv encoding.
Slides: Mathematical Theory of Information - Part 3.
- Week 5 (September 30, October 2)
September 30: Lempel-Ziv encoding - Hands-on exercises. Channels, noise, discrete channels, conditional and joint entropy, mutual information, Shannon's Second Theorem, Error correcting codes. Applications of error-correcting codes.
Slides: Mathematical Theory of Communication - Part 4A
Code Examples: Assignment#2: Python w/ dictionaries, Python w/ Counter
October 2: Channels, noise, discrete channels, conditional and joint entropy, mutual information, Shannon's Second Theorem, Error correcting codes.
Slides: Mathematical Theory of Communication - Part 4B
- Week 6 (October 7, 9)
October 7: Compression: Image Compression. Steganography example: hiding an image inside another image. Video Compression. Case Studies: Netflix and Music. Frontiers of Compression Research: Genomic Compression.
Read: The Desperate Quest for Genomic Compression Algorithms. Tidying Up (bits on the internet) by Anne Aaron. Neil Young's Lonely Quest to Save Music.
October 9: Compression: Case Studies: Netflix and Music (Neil Young). Frontiers of Compression Research: Genomic Compression. Watch Silicon Valley: Season 1, Episode 1.
- Week 7 (October 14, 16)
No classes, Fall Break!!
- Week 9 (October 21, 23)
- Week 10 (October 28, 30)
- Week 11 (November 4, 6)
- Week 12 (November 11, 13)
- Week 13 (November 18, 20)
- Week 14 (November 25, 27)
November 27: Happy Thanksgiving!!
- Week 15 (December 2, 4)
- Week 16 (December 9, 11)
All graded work will receive a grade, 4.0, 3.7, 3.3, 3.0, 2.7, 2.3, 2.0, 1.7,
1.3, 1.0, or 0.0.
This is a upper-level research-topic seminar course. You have to engage in class discussions: this means discussing with other students during class when prompted, asking questions, offering ideas, etc. Bulk of your grade is based on your active participation in the class at this level.
At the end of the semester, final grades will be calculated
as a weighted average of all grades according to the following weights:
Labs & Written Work: 25%
Active class participation: 55%
Created by dkumar at cs dot brynmawr dot edu on
August 29, 2019.