Bryn Mawr College
CS 380: Recent Advances in Computer Science
Topic: Science of Information
Fall 2019
BMC Class Number: 2283
Course Materials
General Information
Instructor: Deepak Kumar, 202 Park Hall, 526-7485
E-Mail: dkumar at cs brynmawr dot edu
Tweet: @bmcdeepak
WWW: http://cs.brynmawr.edu/~dkumar
Lecture Hours: Mondays & Wednesdays 11:40a to 1:00p
Office Hourse: Wednesdays 2:00 to 3:30p
Room: Park Science Building, Room 159
Laboratories:
- Computer Science Lab Room 231 (Science Building), self scheduled.
Course Description:
Claude Shannon's foundations of information theory have paved the way for data storage, compression, encoding, and transmission for the Internet, CDs, DVDs, MP3 players, JPEGs, WiFi, iPODs, mobile phones, and a whole host of applications underlying today's information technologies. The past six decades have brought information theory to the crossroads of several traditional disciplines: mathematics, statistics, computer science, physics, neurobiology, and electrical engineering. This course introduces students to the fundamentals of Information Theory and leads them to a broader understanding of the concept of “information” that transcends boundaries between disciplines, especially between physical and life sciences, communication, and knowledge extraction from massive datasets. Students in several disciplines will be able to draw upon the latest discoveries from multiple disciplines, replicate and discuss recent research, and learn to apply the techniques and tools of information-based inquiry in their lives.
The course will run in a seminar format where students will engage by participating in and leading discussions and presenting results from readings and computational experiments. The course requires a Junior or Senior standing. Students from ALL disciplines are encouraged to enroll.
Texts & Software
|
Information: A Very Short Introduction: By Luciano Floridi, Oxford University Press, 2010.
|
You do not need to buy this book. It is available for all Tri-co students for free in electronic form. Click here.
Topics
The following topics are planned to be covered in the course...
- What is information?
- The Information Revolution
- The Mathematical Theory of Information
- Semantic & Physical Information
- Biological Information
- Economic Infomration
- The Ethics of Information
- Information Retreival & Big Data
- Information Visualization
- Quantun Information
- ...
Readings
Important Dates
September 4: First Meeting
December 11: Last Meeting
Assignments
- Assignment#1 is posted (Due on Monday, September 16): Click here for details
- Assignment#2 is posted (Due on Monday, September 30): Click here for details.
- Assignment#3 is posted (Due on Friday at noon on November 8): Click here for details.
Lectures
- Week 1 (September 4)
September 4: Course Introduction. What is information? A class discussion.
Read: Chapter 1 from Floridi.
Slides: What is Information?
- Week 2 (September 9, 11)
September 9: Information: An Overview. Defining Information: Information = Data + Meaning. Understanding Data. Types of Data. Floridi's taxonomy.
Read: Chapter from Floridi.
Slides: Understanding Data & Information.
September 11: Information: Broader Perspectives. Syntactic versus Semantic distinctions. The five E-s of Information (Entropy, Economics, Encryption, Extraction, Emission). Towards a science of information: structure, time, space, semantics, cooperation, etc. Center for Science of Information. A little history of Point-toPoint communication.
Read: Chapter 3 from Floridi.
Homework#1 (Due on Monday, September 16): Click here.
Slides: BroaderPerspectives.
- Week 4 (September 16, 18)
September 16: Defining Shannon Information: Motivations and background. "Figure 1". Information. Entropy. Information Source.
Slides: Mathematical Theory of Information - Part1
Read: Chapter 3 from Floridi. Shannon 1948.
September 18: "Pop Quiz". Defining Entropy. Introduction to coding.
Slides: Mathematical Theory of Information - Part 2
- Week 4 (September 23, 25)
September 23: Information (I), Entropy (H), Source Coding basics and terminology. Source Coding Theorem (Shannon's First Theorem). Huffman Encoding.
Slides: Mathematics Theory of Information - Part 2 (B) [Note: There is some more material here that the slides from last class.]
Assignment#2 is posted (Due on Monday, September 30): Click here for details.
September 25: Lossless compression of English Text. Zipf's Law. Lempel-Ziv encoding.
Slides: Mathematical Theory of Information - Part 3.
- Week 5 (September 30, October 2)
September 30: Lempel-Ziv encoding - Hands-on exercises. Channels, noise, discrete channels, conditional and joint entropy, mutual information, Shannon's Second Theorem, Error correcting codes. Applications of error-correcting codes.
Slides: Mathematical Theory of Communication - Part 4A
Code Examples: Assignment#2: Python w/ dictionaries, Python w/ Counter
October 2: Channels, noise, discrete channels, conditional and joint entropy, mutual information, Shannon's Second Theorem, Error correcting codes.
Slides: Mathematical Theory of Communication - Part 4B
- Week 6 (October 7, 9)
October 7: Compression: Image Compression. Steganography example: hiding an image inside another image. Video Compression. Case Studies: Netflix and Music. Frontiers of Compression Research: Genomic Compression.
Slides: Compression
Read: The Desperate Quest for Genomic Compression Algorithms. Tidying Up (bits on the internet) by Anne Aaron. Neil Young's Lonely Quest to Save Music.
October 9: Compression: Case Studies: Netflix and Music (Neil Young). Frontiers of Compression Research: Genomic Compression. Watch Silicon Valley: Season 1, Episode 1.
- Week 7 (October 14, 16)
No classes, Fall Break!!
- Week 9 (October 21, 23)
October 21: Biological Information: Introduction to Molecular Biology.
Slides: Introduction to Molecular Biology, Part 1.
Read: Chapter 6 (Biological Information) from Floridi.
The original Watson & Crick paper (1953), Crick's Central Dogma article (1970)
October 23: Molecular Biology: Protein Synthesis: transcription, splicing, translation. Genome Sequencing: history and algorithms. Shotgun Sequencing. Genome Sequence assembly.
Slides: Molecular Biology, Part 2
RNA Video (Thanks, to Ruby)
- Week 10 (October 28, 30)
October 28: Neural Information Processing: The Thinking Machine (1961).
October 30:Biological Information: Neuroscience. Nervous system, neurons, action potentials, spikes, information coding in spikes. From single neurons to the entire brain: fMRI brain decoding. Flexible sensors.
Slides: Neuroscience.
Epidermal Electronics, Todd Coleman Ted Talk on Epidermal Electronics in Medicine, Reconstruction from brain activity, Human brain mapping & Decoding (Jack Gallant, ted Talk).
Assignment#3 is posted (Due on Friday at noon on November 8): Click here for details.
- Week 11 (November 4, 6)
November 4: Neuroscience, contd.: Decoding the brain, epidermal electronics. Information Retreival: Inverted Index for finding hits.
Slides: Information Rereival, Part 1.
November 6: Information Retreival, contd. Google's Page Rank algorithm.
Slides: Information Retreival, Part 2.
- Week 12 (November 11, 13)
November 11: Information Retreival, contd. Voice activated assistants, Deep Question & Answering: Watson. Microsoft Speech Translation Demo.
Presentations: Tentative List of topics.
Slides: Information Retreival, Part 3.
Read: How IBM Watson Overpromised and Undelivered on Ai Health Care, IEEE Spectrum, 02 April 2019.
November 13: Visual Information. What is Data Science? The Importance of Data Visualization. Data Visualization: History, definition, examples. How to read a chart. Types of charts. Active chart reading.
Slides: Information Visualization, Part 1.
- Week 13 (November 18, 20)
November 20: How to read a chart. Types of charts. Active chart reading.
November 22: How to read a chart. Types of charts. Active chart reading.
Slides: InformationVisualization - Part 2, Information Visualization Part - 3.
- Week 14 (November 25, 27)
November 25: Today, we'll watch the movie, The Bit Player (2018, Argot Pictures). Please try and arrive mins early so we can squeeze the entire film during class time.
November 27: Happy Thanksgiving!!
- Week 15 (December 2, 4)
December 2: Presentations - Part 1
Differential Privacy
Naomi, Marilyn, Sophia
Readings: Guide to Differential privacy in Social Network Analysis (Task & Clifton), Differential Privacy in the context of the 2020 Census, Differential privacy in Apple Products, Differential Privacy: Overview, Privacy Book, Issues Encountered Deploying Differential Privacy.
Brain Computer Interfaces
Linda, Nadine
Readings: A Feedback Information-Theoretic Approach to the Design of Brain-Computer Interfaces, Epidermal Electronics.
Visualizing Word Clouds
Eun Soo
Readings: Context Preserving Dynamic Word Cloud Visualization
December 4: Presentations - Part 2
Economics & Information Theory
Alison:
Readings: Economics & Information Theory.
Steganography
Elia, Nisha, Lamiaa, Abbie
Readings: Its not about winning: Its about sending a message: Hiding Information in Games, Steganography in Sudoku Puzzles, Games Without Frontiers: Investigating Video Games as a Covert Channel.
Compression
Viktoria, Chen, Sonya, Haley
Readings: On compressing social networks, Compression Schemes for Similarity Queries, Compression without a common prior:an information-theoretic justification for ambiguity inlanguage. One reading is missing...
Compression of Graphical Structures.
- Week 16 (December 9, 11)
December 9: Presentations - Part 3
Quantum Computing and Quantum Supremacy
Nina, Isabel
Readings: (Need a title/link), Quantum Supremacy.
Finding Borders between Coding and Non-Coding DNA Regions
Ruby
Reading: Entropy of Genomes.
How DNA Could Store All the World's Data
Esther
Reading:
How DNA Could Store All the World's Data.
Bayesian Modeling and Control of Chemotherapeutics
Tanjuma
Reading:
Bayesian Modeling and Control of Chemotherapeutics.
Modeling Bird Flocking Behavior
Emily
Reading: Modeling Bird Flocking Behavior.
Forecasting User Visits for Online Display Advertising
Jenny
Readings: Forecasting User Visits for Online Display Advertising.
Modeling Relationship Strengthin Online Social Networks
Lipi
Reading: Modeling Relationship Strengthin Online Social Networks.
December 11: Course Wrap up.
Grading
All graded work will receive a grade, 4.0, 3.7, 3.3, 3.0, 2.7, 2.3, 2.0, 1.7,
1.3, 1.0, or 0.0.
This is a upper-level research-topic seminar course. You have to engage in class discussions: this means discussing with other students during class when prompted, asking questions, offering ideas, etc. Bulk of your grade is based on your active participation in the class at this level.
At the end of the semester, final grades will be calculated
as a weighted average of all grades according to the following weights:
Labs & Written Work: 25%
Attendance: 20%
Active class participation: 55%
Total: 100%
Links
Created by dkumar at cs dot brynmawr dot edu on
August 29, 2019.
