Spring 2022 - CSC 577 Advanced Topics in Computer Vision

Spring 2022 - CSC 577: Advanced Machine Vision

Basic Information

Meet Time & Location: Tue/Thu 09:40-10:55 (Hylan Building 203, in-person only)
Instructor: Prof. Jiebo Luo
Office Hours: T/R 14:00-15:00

The complete/accurate information of the course should be found on Blackboard.

Description

This course covers advanced research topics in computer vision. Approaches for learning from unimodal (e.g., images and videos) and multimodal data (e.g., vision and language) will be discussed, including topics from small data learning, video analytics, vision and language, 3D vision, image and video generation, trustworthy AI, vision for medicine, and multimedia. The course will be a mix of instructor-led lectures, student-led presentations, paper reading/critiquing, guest lectures, and a semester-long project (team of 2).

Prerequisites

CSC 249/449 or CSC 246/446 or CSC 298/578 (Deep Learning and Graphical Models) or permission of the instructor.

Course Objectives

PhD Students: enriched research experience in or related to computer vision; MS Students: strengthened training for a software engineer career in computer vision and artificial intelligence.

Exposure to a variety of current topics and state-of-the-art techniques
Training for reading and evaluating other peoples' work
Hands-on experience in a semester-long research project (should aim for a research publication)

Schedule

This is a tentative schedule of the course and is subject to changes over the semester.
Course materials will be posted in Blackboard Learning Modules.

W	Date	Topic	Comment
1	Thu 1/13	Course Introduction and Overview	CV++
2	Tue 1/18	T1 Vision and Language: Lecture by Prof. Jiebo Luo
	Thu 1/20	T1 Video-aided Unsupervised Grammar Induction, Songyang Zhang, Linfeng Song, Lifeng Jin, Kun Xu, Dong Yu, Jiebo Luo, NAACL 2021 [Best Long Paper]	Songyang Zhang
3	Tue 1/25	T1 Improving Vision-and-Language Navigation with Image-Text Pairs from the Web. Arjun Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh, Dhruv Batra. 2020. arXiv	TBD
	Thu 1/27	T2 Image and Video Generation	Wei Xiong
4	Tue 2/1	T2 Advanced generation guided by semantics and sentiments	Haitian Zheng, Jie An
	Thu 2/3	T2 Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering. Yuxuan Zhang, Wenzheng Chen, Huan Ling, Jun Gao, Yinan Zhang, Antonio Torralba, Sanja Fidler. 2020. arXiv	TBD
5	Tue 2/8	T3 3D Pose and Motion: Lecture by Dr. Junsong Yuan (UB)
	Thu 2/10	T3 HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation. arXiv	Jingyuan Chen
6	Tue 2/15	T4 Few-shot and Representation Learning: Lecture by Prof. Jiebo Luo
	Thu 2/17	T4 Lecture by Prof. Xi Peng (UDel)	Domain Generalization
7	Tue 2/22	T4 When Does Self-supervision Improve Few-shot Learning? Jong-Chyi Su, Subhransu Maji, Bharath Hariharan. 2020. arXiv	TBD
	Thu 2/24	Project Proposal
8	Tue 3/1	T5 Computer Vision for Medicine: Lecture by Prof. Jiebo Luo
	Thu 3/3	Automated Bone Mineral Density Prediction and Fracture Risk Assessment using Plain Radiographs via Deep Learning, Chen-I Hsieh, Kang Zheng, Chihung Lin, Ling Mei, Le Lu, Weijian Li, Fang-Ping Chen, Yirui Wang, Xiaoyun Zhou, Fakai Wang, Guotong Xie, Jing Xiao, Shun Miao#, Chang-Fu Kuo#, Nature Communications, 12, Article number: 5472 (2021)	Weijian Li
9	Tue 3/8	Spring Break
	Thu 3/10	Spring Break
10	Tue 3/15	T6 Fairness/Biases in Computer Vision	Wei Zhu
	Thu 3/17	T6 Learning from Failure: Training Debiased Classifier from Biased Classifier. Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, Jinwoo Shin. 2020. arXiv	TBD
11	Tue 3/22	Mid-term Project Discussion
	Thu 3/24	T7 Multimodal Machine Learning Multimodal Machine Learning: A Survey and Taxonomy, Tadas Baltrusaitis, Chaitanya Ahuja, Louis-Philippe Morency. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 41, Issue 2, February 2019 pp 423–443. https://doi.org/10.1109/TPAMI.2018.2798607
12	Tue 3/29	T7 Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation. Hang Zhou, Xudong Xu, Dahua Lin, Xiaogang Wang, Ziwei Liu. 2020. arXiv	Ashwin Ramesh
	Thu 3/31	T7 SoundSpaces: Audio-Visual Navigation in 3D Environments. Changan Chen, Unnat Jain, Carl Schissler, Sebastia Vicenc Amengual Gari, Ziad Al-Halah, Vamsi Krishna Ithapu, Philip Robinson, Kristen Grauman. 2020. arXiv	TBD
13	Tue 4/5	T8 Video Analytics: Lecture by Prof. Haibin Ling or Prof. Michael Ryoo (Stony Brook)
	Thu 4/7	T8 Procedure Planning in Instructional Videos. Chien-Yi Chang, De-An Huang, Danfei Xu, Ehsan Adeli, Li Fei-Fei, Juan Carlos Niebles. 2020. arXiv	TBD
14	Tue 4/12	T9 Learning with Unpaired Data: Lecture by Prof. Jiebo Luo
	Thu 4/14	Project Presentation	3
15	Tue 4/19	Project Presentation (cont.)	3
	Thu 4/21	Project Presentation (cont.)	3
16	Tue 4/26	Project Presentation (cont.)	3

Course Work and Evaluation

Essay (35%): Throughout the semester, we will read many papers and hear many CV seminar talks. You will write at least seven essays. Your essay will be expanding the ideas presented in the paper/talk and have at least 300 words, for example, you can present a different/better solution or problem formulation that overcomes one of the paper's limitations. Note that an essay is different from a typical paper review. You can write up to ten essays, and I will count the top-scored seven essays.
- If you are writing an essay for a paper, the essay is due before we discuss the paper in class.
- If you are writing an essay for a talk, the essay is due in 48 hours after the talk.
Presentation (15%): Depending on the number of students, you will present two papers under different research themes throughout the semester.
Project (35%): You will be working on a semester-long project. There will be a project proposal, a midway project discussion, and a final project presentation. You will need to generate a 6-page double-column project report in the CVPR format by the end of the semester.
In-Class Discussion (15%): Actively participate in the in-class discussion. Try to ask questions. Try to offer ideas and opinions.

All assignments should be submitted on Blackboard, and no late work without pre-approval will be accepted.

Other Policies

Code of Conduct: Check here for classroom "etiquette" expectations.

Academic Honesty: All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available here. You are expected to read, understand, and follow the policy.

Disabilities: Please see me about your required accommodations as early as possible. The University of Rochester respects and welcomes students of all backgrounds and abilities. In the event you encounter any barrier(s) to full participation in this course due to the impact of a disability, please contact the Office of Disability Resources. The access coordinators in the Office of Disability Resources can meet with you to discuss the barriers you are experiencing and explain the eligibility process for establishing academic accommodations. You can reach the Office of Disability Resources at: disability@rochester.edu; (585) 276-5075; Taylor Hall; link.

Accommodations for COVID-related Illness: Please let me know as early as possible. At the same time, You should contact the College Center for Advising Services (CCAS) or the Graduate Education and Postdoctoral Affairs office. These offices will be able to assist you and refer you to the right resources.