CMSC 380: Machine Learning

Bryn Mawr College
CMSC 380: Machine Learning
Spring 2011

Syllabus and Schedule

Course Information

Instructor Information

Textbooks and Software

Course Policies

Additional Resources

Syllabus and Schedule

This is a tentative syllabus and schedule. Topics, reading assignments, and due dates are subject to change.
All homework assignments are due by the start of class (10:10am) on the day listed.

Wk	Date	Topic	Reading	Lab	Assignments	Comments
1	1/19	Introduction to Machine Learning	Alpaydin Ch. 1	No lab		kNN notes
2	1/24	Statistical Learning, Empirical Methodology, Decision Trees	Alpaydin Ch. 2 Mitchell Ch. 3	Weka tutorial	Assignment 1 out
2	1/26	Statistical Learning, Empirical Methodology, Decision Trees		Weka tutorial
3	1/31	Review of Linear Algebra, Linear Discriminants	Alpaydin 10.1-10.7	Matlab tutorial
3	2/2	Linear methods for classification and regression		Matlab tutorial		Optimization notes
4	2/7	Support Vector Machines and Kernel Methods	Bennett article	No lab	Assignment 1 due Assignment 2 out	SVM notes ionosphere.data ionosphere.names
4	2/9	No Class -- Eric out of town
5	2/14	SVMs & Kernel Methods (continued)	Alpaydin 13.1-13.7	Probability review
5	2/16	Generative Modeling, Probability Review	Alpaydin Appx. A, 3.1-3.5, 4.1-4.4	Probability review
6	2/21	Bayes Rule, Graphical Models, Basic Bayesian Nonparametrics Logistic Regression	Alpaydin 4.5-4.8, 16.1-16.3		Assignment 2 due Assignment 3 out
6	2/23		Alpaydin 8.1-8.4
7	2/28	Logistic Regression (continued), MLE and MAP Ensemble Methods	Alpaydin 5.1-5.5 Alpaydin 10.7
7	3/2		Alpaydin 12.8-12.9, 17.1-17.4, 17.6-17.9
8	3/7	No Class -- Spring Break
8	3/9	No Class -- Spring Break
9	3/14	Exam 1			Assignment 3 due	Exam 1 Equation Sheet
9	3/16	Semi-supervised and Unsupervised Learning	Alpaydin 7.1-7.4		Assignment 4 out
10	3/21	Semi-supervised and Unsupervised Learning (continued)	Alpaydin 7.5-7.8
10	3/23	Relational Learning Primer				Jen Neville's visit on Wednesday
11	3/28	Semi-supervised learning for text and image classification				Tutorial on Feature Learning for Images: Part 0, Part 1
11	3/30	Transfer and Lifelong Learning			Assignment 4 due Project Description
12	4/4	Transfer Learning (continued) / Dimensionality Reduction	Alpaydin 6.1-6.8
12	4/6	Dimensionality Reduction, Manifold Methods				Dimensionality Reduction Notes
13	4/11	Manifold Methods / Isomap
13	4/13	Reinforcement Learning and MDPs	Sutton & Barto			RL Notes
14	4/18	Reinforcement Learning and MDPs	Sutton & Barto
14	4/20	Reinforcement Learning and MDPs
15	4/25	Adv. Topics / Review
15	4/27	Exam 2				Exam 2 Equation Sheet
	TBA	Project Presentations during Final Exam slot			Final Project Due

General Course Information

Description: Machine learning has been essential to the success of many recent technologies, including autonomous vehicles, search engines, DNA sequencing, automated medical diagnosis, and facial recognition. This course will cover the fundamental concepts and algorithms that enable computers to learn from experience, and their practical application to real-world problems. Topics include decision trees, logistic regression, linear methods, support vector machines, kernel methods, genetic algorithms, reinforcement learning, probabilistic Bayesian methods, graphical models, mixture models, ensemble techniques, evaluation methodologies, and emerging developments in transfer and lifelong learning.

Prerequisites: CMSC B110, CMSC/MATH B231, or permission of instructor.

Lecture Hours: Mondays, Wednesdays, 10:00-11:30 am
Room: Park 230

Lab Hours: Wednesday 2-4pm
Lab Room: Park 232

Website: http://cs.brynmawr.edu/cs380/

Contact Information and Office Hours

Professor: Eric Eaton
Office: Park 249
Office Phone: 610-526-5061
E-Mail:
Website: http://cs.brynmawr.edu/~eeaton

E-mail is the best way to reach me, and I make a concerted effort to respond to all e-mails within 24 hours on weekdays and 48 hours on weekends (often, much less!).

Office hours: 1-2pm Tuesday/Thursday and by appointment

Textbooks & Software

Textbook:

Introduction to Machine Learning, 2nd edition by Ethem Alpaydin, MIT Press, 2010. Available in the campus bookstore.

Software:

The Weka Machine Learning Toolkit, a widely-used open-source ML toolkit built in java. You should download your own copy for either Linux, Windows, or Mac. Since it's java, you can download it to your home directory and run it. We will be using Weka for some of the early assignment only. You will not be permitted to use Weka for later homeworks or your projects.
Matlab -- Installed on the lab machines.

Recommended Resources:

The Elements of Statistical Learning, 2nd edition by Hastie, Tibshirani and Friedman, Springer-Verlag, 2008. PDF available online at: http://www-stat.stanford.edu/~hastie/local.ftp/Springer/ESLII_print4.pdf
Machine Learning by Tom Mitchell, McGraw Hill, 1997.
Pattern Classification, 2nd edition by Duda, Hart and Stork, Wiley-Interscience, 2000.
Reinforcement Learning: An Introduction by Sutton and Barto, MIT Press, 1998. Online version available at: http://webdocs.cs.ualberta.ca/~sutton/book/the-book.html
LaTeX: A Document Preparation System, 2nd edition by Leslie Lamport.

Course Policies

Communication

Attendance and active participation are expected in every class. Participation includes asking questions, contributing answers, proposing ideas, and providing constructive comments.

As you will discover, I am a proponent of two-way communication and I welcome feedback during the semester about the course. I am available to answer student questions, listen to concerns, and talk about any course-related topic (or otherwise!). Come to office hours! This helps me get to know you. You are welcome to stop by and chat. There are many more exciting topics to talk about that we won't have time to cover in-class.

Whenever you e-mail me, please use a meaningful subject line and include the phrase "CS380" at the beginning of that line. Your e-mail will catch my attention and I will respond quicker if you do this. I make an effort to respond to e-mails within 24 hours on weekdays and 48 hours on weekends.

Although computer science work can be intense and solitary, please stay in touch with me, particularly if you feel stuck on a topic or project and can't figure out how to proceed. Often a quick e-mail, phone call or face-to-face conference can reveal solutions to problems and generate renewed creative and scholarly energy. It is essential that you begin homeworks and projects early, since we will be covering a variety of challenging topics in this course.

Grading

At the end of the semester, final grades will be calculated as a weighted average of all grades according to the following weights:

Homeworks -- 40% (10% each)
Exam 1 -- 15%
Exam 2 -- 15%
Final Project -- 25%
Class Participation -- 5%

Incomplete grades will be given only for verifiable medical illness or other such dire circumstances.

Here is how the final weighted average will map to final letter grades:

Rounded Percentage	Letter grade	Rounded Percentage	Letter grade
97% -100%	A+ (4.0)	77% - 79%	C+ (2.3)
93% - 96%	A (4.0)	73% - 76%	C (2.0)
90% - 92%	A- (3.7)	70% - 72%	C- (1.7)
87% - 89%	B+ (3.3)	67% - 69%	D+ (1.3)
83% - 86%	B (3.0)	60% - 66%	D (1.0)
80% - 82%	B- (2.7)	0% - 59%	F (0.0)

The instructor reserves the right to adjust the percentage ranges for each letter grade upward in your favor.

Submission and Late Policy

All work must be turned in either in hard-copy or electronic submission, depending on the instructions given in the assignment. E-mail submissions, when permitted, should request a "delivery receipt" to document time and date of submission. Extensions will be given only in the case of verifiable medical excuses or other such dire circumstances, if requested in advance.

Late submissions will receive a penalty of 10% for every 0-24 hours it is past the due date and time (e.g., assignments turned in 25 hrs late will receive a penalty of 20%). Submissions received more than one week late will not be accepted.

That being said, everyone will receive 4 "free" late days that you can choose to use for any assignment (except the final project). "How do I choose when to use them?," you ask. You don't -- I will add them in at the end of the semester to maximize your grade. Unused late days will be worth some unspecified amount of benevolence points.

Exams

There will be two exams in this course. The exams will be closed-book and closed-notes. They will cover material from lectures, homeworks, and assigned readings (including topics not discussed in class). So, keep up with those readings!

Study Groups

I want to encourage you to discuss the material and work together to understand it. Here are my thoughts on collaborating with other students:

The readings and lecture topics are group work. Please discuss the readings and associated topics with each other. Work together to understand the material. I highly recommend forming a reading group to discuss the material -- we will explore many challenging ideas and it helps to have multiple people working together to understand them.
It is fine to discuss the topics covered in the homeworks, to discuss approaches to problems, and to sketch out general solutions. However, you MUST write up the homework answers, solutions, and programs individually without sharing specific solutions, mathematical results, program code, etc. If you made any notes or worked out something on a white board with another person while you were discussing the homework, you shouldn't use those notes while writing up your answer.
You are fully permitted to (and should!) discuss the project with your partners for any group assignments. I also encourage you to work outside of your team to understand the other topics in the course.
Under ABSOLUTELY NO circumstances should you share computer code with another student, unless it is your partner on a group project. Similarly, you are not permitted to use code found on the internet for any of your assignments.
Exams, of course, must be your own individual work.

If you have any questions as to what types of collaborations are allowed, please feel free to ask.

Additional Resources

LaTeX Resources

A basic introduction to LaTex (tex, pdf). This is from a class I taught a while ago, so ignore any course-specific items.
The sample document from the introduction (tex, pdf).
Math in LaTeX (tex, pdf).
LaTeX Websites
- LaTeX Project home page
- LaTeX Project FAQ
- CTAN: the Comprehensive TeX Archive Network
- The American Mathematical Society (AMS) maintains several widely used extensions of LaTeX. The amsmath package provides just about every math symbol you can imagine, and more.
- Simplified LaTeX, a beginner's guide with a nice tutorial section at the beginning.
- In order to run LaTeX on your own computer, you will need to install TeX and LaTeX. A number of freeware and shareware implementations are available. You can look through the CTAN or LaTeX Project websites above for pointers. If you're running Windows, you might want to try the MikTeX implementation. Excellent LaTeX-friendly text editors for Windows include WinEdt or TeXnicCenter. For Mac OS X, try TeXshop.

Tutorials

Feature learning for images

Thanks to Terran Lane, Tom Mitchell, Andrew Moore, and Tim Oates for making their course materials available. Many of the course materials for this class have been adapted from those sources.