Fall 2020 - CSC 249/449 Machine Vision

Spring 2021 - CSC 249/449: Machine Vision

Basic Information

All zoom meetings are password-protected. Please find the information in Blackboard.

Meet Time/Location: T/R 12:30-13:45 on Zoom

Instructor: Prof. Jiebo Luo
Instructor's Office Hours: T/R 15:30-17:00 (By Appointment) on Zoom

TAs and Office Hours:

Jie An, Graduate TA (Fri 16:00-17:30 on Zoom)
Zhaoyi Wan, Graduate TA (Tue 16:00-17:30 on Zoom)

Information Flow: We will use Blackboard to make announcements, posting lecture notes/videos and assignments, and submitting student work. The course ID is CSC249.1.SPRING2021ASE. This course website contains an outline of the course schedule and also serves for the purpose of a syllabus.

Credit Hours: This is a four-credit-hour course. The credit hours are met by weekly lectures (150 minutes), office hours, homework assignments, and the final course project.

Description

Fundamentals of computer vision, including image formation, elements of human vision, low-level image processing, and pattern recognition techniques. Advanced topics include modern visual features, graphical models, model-based and data-driven approaches, and contextual inference, as well as illustrative examples of successes and challenges in applications. CSC 449, a graduate-level course, typically requires additional readings and assignments. Based on past classes, CSC 249, an undergraduate-level course, has the same requirements with a 5% bonus for completing the same requirements as 449.

Rough Topic Outline

Image Formation and Multiple View Geometry: camera models, light and color, camera calibration, image alignment, stereo vision
Features and Filters: image transforms, linear filtering, edges, corners, feature extraction, feature matching, optical flow
Segmentation and Fitting: segmentation by clustering, interactive segmentation, robust fitting and RANSAC, Hough transform
Recognition: feature learning, convolutional and recurrent neural networks, advanced CNN architectures, attention mechanisms, feature visualization, image classification, object detection, instance segmentation, face recognition, video analysis
Description and Generation: image captioning, VQA, generative adversarial networks, image and video synthesis

Course Objective

The course is designed as an upper-level elective for Computer Science undergraduate students and an AI-area breath course for graduate students. Students will develop a strong understanding of formulating and solving problems in computer vision.

Prerequisites

Basic knowledge of probability, linear algebra (MTH 165 Strongly Recommended); data structures, algorithms; programming experience. For assignments and projects, the most common programming language is Python or Matlab.

Hardware/Software Requirements

You need a computer and a reliable network to access online lectures and course materials. Besides, many of the course assignments require you to have access to a powerful GPU system. This can usually be resolved by:

Using our provided CSUG account to ssh to shared GPU resources; or
Owning an NVIDIA GPU system with at least 6GB GPU memory; or
Renting a cloud GPU via cloud service providers e.g., Google Cloud.

By taking this course, you confirm that you have met the system requirements.

Textbook and Other Resources

Machine/Computer Vision is under rapid development over the past few years. There is not a perfect textbook that covers everything needed for an introductory computer vision course in today's context. However, we will use the following textbook, which is widely used in many other computer vision courses.

Computer Vision: A Modern Approach, 2nd Edition, Forsyth and Ponce, Pearson, 2012

In addition, I recommend the following textbooks as supplementary reading.

Computer Vision: Algorithms and Applications, Szeliski, Springer, 2011 [link]
Deep Learning, Goodfellow, Bengio and Courville, The MIT Press, 2016 [link]
Multiple View Geometry in Computer Vision, 2nd Edition, Hartley and Zisserman, Cambridge University Press, 2004

Schedule

This is a tentative schedule of the course and is subject to modifications over the term.
Most lectures will be given in-person and streamed simultaneously via Zoom. Please note all classes will be recorded. The recordings along with any other course materials will be posted on Blackboard for asynchronous viewing by students. Please note that the instructor owns the copyright of all recorded lectures and course materials, You should not post them on the Internet or sharing with a friend without my consent.

Dates	Tuesday (T)	Thursday (R)	Chapter	Project
2/2,4	Introduction	Image Formation / Camera Models	1,2
2/9,11	Color Vision	Low-level Image Processing	3,4
2/16,18	Texture	Review of Probability	notes
2/23,25	Pattern Recognition Techniques	CNN and Feature Visualization	notes
3/2,4	Modern Visual Features	Modern Visual Features	5,6	Teaming Starts
3/9,11	Fitting (Hough Transform)	Segmentation	9,10
3/16,18	Image Registration	Face and People Recognition I	12,notes
3/23,25	Face and People Recognition II	Midterm Exam
3/30,4/1	Deep Learning and CNN I	Deep Learning and CNN II	notes	Proposal Due
4/6,8	Special Lecture: Vision + Language	Special Lecture: Vision + Language	notes
4/13,15	Tracking	Video Analysis I	7,8,11,notes
4/20,22	Video Analysis II	GAN and Image Synthesis	18,notes
4/27,29	Special Lecture: Medical Image Computing	Special Lecture: Medical Image Computing	notes
5/4,6	Contextual Inference in Computer Vision	Project Presentation I
5/11,13	Project Presentation II	Project Presentation III		Presentation Due
5/18,20	No Class: Term Finished	No Class: Term Finished		Report Due

Grading (total 102%)

midterm exam: 30%
homework assignments: 35% (including a small programming project 10%). Each (worth 5%) will contain a mix of analytical questions and programming questions. They will be done independently by each student.
research project & presentation: 35% (proposal counts 5%, presentation counts 10%). Students will form teams with up to two members to implement a research project. The project will generate a minimum 4-page (excluding references) double-column report and a presentation at the end of the term. Expectation for the final project is something "new", e.g., (1) solving a new problem, (2) applying an existing algorithm to new data, or (3) applying a new algorithm (or a modified version of an existing algorithm) to the same data.
new findings from a comparative study of using different algorithms for the same problem
class participation and effort: 2%

Late Work Policy: Deduct 10% of total score of that assignment per day passing the deadline for each assignment. For example, you submit HW1 one day past the deadline, then the max score you will get is 90% of that assignment. We will not accept late submissions after one week past the deadline. All assignments should be submitted on Blackboard.

Regrading: Any questions about the grading of a piece of work must be raised within one week of the date that the work was returned. In other words, if you do not pick up your work in a timely fashion, you may forfeit your right to question the grading of your work.

Policy on Incomplete: A student will receive an Incomplete if s/he misses either the final project or any two or more of the assignments.

Other Policies

Code of Conduct: Check here for classroom "etiquette" expectations.

Academic Honesty: All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available here. You are expected to read, understand, and follow the policy. Additionally:

Homework Collaboration: In general, homework is to be completed independently, and any forms of collaboration are strongly discouraged. You may discuss homework problems with others (in this case, you are required to report the names of those with whom you discussed an assignment in your submission; failing to do so is a violation of academic honesty), but you must not retain written notes from your conversations with other students, or share data via computer files to be used in completing your homework. Your written work must be completed without reference to such notes, with the exception of class and recitation notes, which may be retained in written form.
Online Materials: You may look for the documentation, manuals, and tutorials for a programming language or library. You may study a systematic introduction and tutorial for a particular research topic or problem space. In all cases, you should cite the information sources. However, you should not intentionally search for homework solutions or solutions to any similar problems posted by past students or any others online. Failing to do so is a violation of academic honesty.
General Rule: When in doubt, cite.

Posting homework and project solutions to public repositories on sites like GitHub is a violation of the College's Academic Honesty Policy, Section V.B.2 Giving Unauthorized Aid.

Disabilities: Please see me about your required accommodations as early as possible in the term. The University of Rochester respects and welcomes students of all backgrounds and abilities. In the event you encounter any barrier(s) to full participation in this course due to the impact of a disability, please contact the Office of Disability Resources. The access coordinators in the Office of Disability Resources can meet with you to discuss the barriers you are experiencing and explain the eligibility process for establishing academic accommodations. You can reach the Office of Disability Resources at: disability@rochester.edu; (585) 276-5075; Taylor Hall; www.rochester.edu/college/disability.

Accommodations for COVID-related Illness: Please let me know asap. At the same time, You should contact the College Center for Advising Services (CCAS) or the Graduate Education and Postdoctoral Affairs office. These offices will be able to assist you and refer you to the right resources.