Bryn Mawr College
CS 380: Recent Advances in Computer Science
Topic: Science of Information
BMC Class Number: 2283
Instructor: Deepak Kumar, 202 Park Hall, 526-7485
E-Mail: dkumar at cs brynmawr dot edu
Lecture Hours: Mondays & Wednesdays 11:40a to 1:00p
Office Hourse: Wednesdays 2:00 to 3:30p
Room: Park Science Building, Room 159
- Computer Science Lab Room 231 (Science Building), self scheduled.
Claude Shannon's foundations of information theory have paved the way for data storage, compression, encoding, and transmission for the Internet, CDs, DVDs, MP3 players, JPEGs, WiFi, iPODs, mobile phones, and a whole host of applications underlying today's information technologies. The past six decades have brought information theory to the crossroads of several traditional disciplines: mathematics, statistics, computer science, physics, neurobiology, and electrical engineering. This course introduces students to the fundamentals of Information Theory and leads them to a broader understanding of the concept of “information” that transcends boundaries between disciplines, especially between physical and life sciences, communication, and knowledge extraction from massive datasets. Students in several disciplines will be able to draw upon the latest discoveries from multiple disciplines, replicate and discuss recent research, and learn to apply the techniques and tools of information-based inquiry in their lives.
The course will run in a seminar format where students will engage by participating in and leading discussions and presenting results from readings and computational experiments. The course requires a Junior or Senior standing. Students from ALL disciplines are encouraged to enroll.
Texts & Software
|Information: A Very Short Introduction: By Luciano Floridi, Oxford University Press, 2010.
You do not need to buy this book. It is available for all Tri-co students for free in electronic form. Click here.
The following topics are planned to be covered in the course...
- What is information?
- The Information Revolution
- The Mathematical Theory of Information
- Semantic & Physical Information
- Biological Information
- Economic Infomration
- The Ethics of Information
- Information Retreival & Big Data
- Information Visualization
- Quantun Information
September 4: First Meeting
December 11: Last Meeting
- Assignment#1 is posted (Due on Monday, September 16): Click here for details
- Assignment#2 is posted (Due on Monday, September 30): Click here for details.
- Assignment#3 is posted (Due on Friday at noon on November 8): Click here for details.
- Week 1 (September 4)
September 4: Course Introduction. What is information? A class discussion.
Read: Chapter 1 from Floridi.
Slides: What is Information?
- Week 2 (September 9, 11)
September 9: Information: An Overview. Defining Information: Information = Data + Meaning. Understanding Data. Types of Data. Floridi's taxonomy.
Read: Chapter from Floridi.
Slides: Understanding Data & Information.
September 11: Information: Broader Perspectives. Syntactic versus Semantic distinctions. The five E-s of Information (Entropy, Economics, Encryption, Extraction, Emission). Towards a science of information: structure, time, space, semantics, cooperation, etc. Center for Science of Information. A little history of Point-toPoint communication.
Read: Chapter 3 from Floridi.
Homework#1 (Due on Monday, September 16): Click here.
- Week 4 (September 16, 18)
September 16: Defining Shannon Information: Motivations and background. "Figure 1". Information. Entropy. Information Source.
Slides: Mathematical Theory of Information - Part1
Read: Chapter 3 from Floridi. Shannon 1948.
September 18: "Pop Quiz". Defining Entropy. Introduction to coding.
Slides: Mathematical Theory of Information - Part 2
- Week 4 (September 23, 25)
September 23: Information (I), Entropy (H), Source Coding basics and terminology. Source Coding Theorem (Shannon's First Theorem). Huffman Encoding.
Slides: Mathematics Theory of Information - Part 2 (B) [Note: There is some more material here that the slides from last class.]
Assignment#2 is posted (Due on Monday, September 30): Click here for details.
September 25: Lossless compression of English Text. Zipf's Law. Lempel-Ziv encoding.
Slides: Mathematical Theory of Information - Part 3.
- Week 5 (September 30, October 2)
September 30: Lempel-Ziv encoding - Hands-on exercises. Channels, noise, discrete channels, conditional and joint entropy, mutual information, Shannon's Second Theorem, Error correcting codes. Applications of error-correcting codes.
Slides: Mathematical Theory of Communication - Part 4A
Code Examples: Assignment#2: Python w/ dictionaries, Python w/ Counter
October 2: Channels, noise, discrete channels, conditional and joint entropy, mutual information, Shannon's Second Theorem, Error correcting codes.
Slides: Mathematical Theory of Communication - Part 4B
- Week 6 (October 7, 9)
October 7: Compression: Image Compression. Steganography example: hiding an image inside another image. Video Compression. Case Studies: Netflix and Music. Frontiers of Compression Research: Genomic Compression.
Read: The Desperate Quest for Genomic Compression Algorithms. Tidying Up (bits on the internet) by Anne Aaron. Neil Young's Lonely Quest to Save Music.
October 9: Compression: Case Studies: Netflix and Music (Neil Young). Frontiers of Compression Research: Genomic Compression. Watch Silicon Valley: Season 1, Episode 1.
- Week 7 (October 14, 16)
No classes, Fall Break!!
- Week 9 (October 21, 23)
October 21: Biological Information: Introduction to Molecular Biology.
Slides: Introduction to Molecular Biology, Part 1.
Read: Chapter 6 (Biological Information) from Floridi.
The original Watson & Crick paper (1953), Crick's Central Dogma article (1970)
October 23: Molecular Biology: Protein Synthesis: transcription, splicing, translation. Genome Sequencing: history and algorithms. Shotgun Sequencing. Genome Sequence assembly.
Slides: Molecular Biology, Part 2
RNA Video (Thanks, to Ruby)
- Week 10 (October 28, 30)
October 28: Neural Information Processing: The Thinking Machine (1961).
October 30:Biological Information: Neuroscience. Nervous system, neurons, action potentials, spikes, information coding in spikes. From single neurons to the entire brain: fMRI brain decoding. Flexible sensors.
Epidermal Electronics, Todd Coleman Ted Talk on Epidermal Electronics in Medicine, Reconstruction from brain activity, Human brain mapping & Decoding (Jack Gallant, ted Talk).
Assignment#3 is posted (Due on Friday at noon on November 8): Click here for details.
- Week 11 (November 4, 6)
November 4: Neuroscience, contd.: Decoding the brain, epidermal electronics. Information Retreival: Inverted Index for finding hits.
Slides: Information Rereival, Part 1.
November 6: Information Retreival, contd. Google's Page Rank algorithm.
Slides: Information Retreival, Part 2.
- Week 12 (November 11, 13)
November 11: Information Retreival, contd. Voice activated assistants, Deep Question & Answering: Watson. Microsoft Speech Translation Demo.
Presentations: Tentative List of topics.
Slides: Information Retreival, Part 3.
Read: How IBM Watson Overpromised and Undelivered on Ai Health Care, IEEE Spectrum, 02 April 2019.
November 13: Visual Information. What is Data Science? The Importance of Data Visualization. Data Visualization: History, definition, examples. How to read a chart. Types of charts. Active chart reading.
Slides: Information Visualization, Part 1.
- Week 13 (November 18, 20)
November 20: How to read a chart. Types of charts. Active chart reading.
November 22: How to read a chart. Types of charts. Active chart reading.
Slides: InformationVisualization - Part 2, Information Visualization Part - 3.
- Week 14 (November 25, 27)
November 27: Happy Thanksgiving!!
- Week 15 (December 2, 4)
December 2: Presentations - Part 1
Naomi, Marilyn, Sophia
Readings: Guide to Differential privacy in Social Network Analysis (Task & Clifton), Differential Privacy in the context of the 2020 Census, Differential privacy in Apple Products, Differential Privacy: Overview, Privacy Book.
Brain Computer Interfaces
Readings: A Feedback Information-Theoretic Approach to the Design of Brain-Computer Interfaces, Epidermal Electronics.
Ethics of Information?
Readings: Need a specific source to place here.
Economics & Information Theory
Economics & Information Theory.
December 4: Presentations - Part 2
Elia, Nisha, Lamiaa, Abbie
Readings: Its not about winning: Its about sending a message: Hiding Information in Games, Steganography in Sudoku Puzzles, Games Without Frontiers: Investigating Video Games as a Covert Channel.
Viktoria, Chen, Sonya, Haley
Readings: On compressing social networks, Compression Schemes for Similarity Queries, Compression without a common prior:an information-theoretic justification for ambiguity inlanguage. One reading is missing...
- Week 16 (December 9, 11)
December 9: Presentations - Part 3
Quantum Computing and Quantum Supremacy
Readings: (Need a title/link), Quantum Supremacy.
Entropy of Genomes
Reading: Entropy of Genomes.
How DNA Could Store All the World's Data
How DNA Could Store All the World's Data.
Bayesian Modeling and Control of Chemotherapeutics
Bayesian Modeling and Control of Chemotherapeutics.
Modeling Bird Flocking Behavior
Reading: Modeling Bird Flocking Behavior.
Forecasting User Visits for Online Display Advertising
Readings: Forecasting User Visits for Online Display Advertising.
Modeling Relationship Strengthin Online Social Networks
Reading: Modeling Relationship Strengthin Online Social Networks.
December 11: Course Wrap up.
All graded work will receive a grade, 4.0, 3.7, 3.3, 3.0, 2.7, 2.3, 2.0, 1.7,
1.3, 1.0, or 0.0.
This is a upper-level research-topic seminar course. You have to engage in class discussions: this means discussing with other students during class when prompted, asking questions, offering ideas, etc. Bulk of your grade is based on your active participation in the class at this level.
At the end of the semester, final grades will be calculated
as a weighted average of all grades according to the following weights:
Labs & Written Work: 25%
Active class participation: 55%
Created by dkumar at cs dot brynmawr dot edu on
August 29, 2019.