Chenliang Xu

Fall 2019 - CSC 577: Advanced Topics in Computer Vision

Time/Place: TR 09:40-10:55 in ~~315 B&L~~ 205 Meliora

Instructor: Prof. Chenliang Xu
Office Hours: see my homepage

TA: Lele Chen
Office Hours: WF 11:10-noon in 4205 Wegmans Hall

The syllabus is below.

Description

This course covers advanced research topics in computer vision with an emphasis on learning structured representations and embeddings. Approaches for learning from unimodal (e.g., images and videos), and multimodal data (e.g., vision and language, vision and audio) will be covered and include topics from structured predications, deep learning and others. The course will be a mix of lecture, student presentation and discussion. Prerequisites: CSC 249/449 or CSC 246/446 or CSC 298/578 (Deep Learning and Graphical Models) or permission of the instructor.

Goals

PhD Students: gain research experience in computer vision; MS Students: get prepared for a software engineer career in computer vision and artificial intelligence.

Get exposed to a variety of topics and become familiar with state-of-the-art techniques.
Learn to read and evaluate other peoples' work.
Get hands dirty on a semester-long research project. (Past project has won best paper award at research conference.)

Piazza

Find our class page here.

Schedule

This is a tentative schedule of the course.
Course materials will be posted on Blackboard.
Red: lecture; Blue: student presentation; Green: project presentation; Grey: no class or cancelled class.

W	Date	Topic	Presenter
1	R 8/29	Overview, Boot Camp: Neural Networks	Instructor
2	T 9/3	Boot Camp: Convolutional Neural Networks	Instructor
2	R 9/5	Boot Camp: Recurrent Neural Networks	Instructor
3	T 9/10	Tutorial: PyTorch, Google Cloud, Training GANs	TA
3	R 9/12	Chen et al. CVPR 2019: Hierarchical Cross-modal Talking Face Generation with Dynamic Pixel-wise Loss	TA
4	T 9/17	Selvaraju et al. ICCV 2017: Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization	Guofeng, Haomin
4	R 9/19	Fong and Vedaldi ICCV 2017: Interpretable Explanations of Black Boxes by Meaningful Perturbation	Max, Tushar
5	T 9/24	Bau et al. ICLR 2019: GAN Dissection: Visualizing and Understanding Generative Adversarial Networks	Sizhe, Zhaoxiong
5	R 9/26	Liu et al. CVPR 2019: Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation	Guo, Ibrahim
6	T 10/1	Xie et al. ICCV 2019: Exploring Randomly Wired Neural Networks for Image Recognition	Nageswara, Xiaoning
6	R 10/3	Cubuk et al. CVPR 2019: AutoAugment: Learning Augmentation Policies from Data	Tushar, Zhenfei
7	T 10/8	Shevlev and Avidan CVPR 2019: Co-Occurrence Neural Network	Max, Zhaoxiong
7	R 10/10	Li et al. CVPR 2019: Finding Task-Relevant Features for Few-Shot Learning by Category Traversal	Jianyuan, Purvanshi
8	T 10/15	Fall Break
8	R 10/17	Wang et al. CVPR 2019: TAFE-Net: Task-Aware Feature Embeddings for Low Shot Learning	Guofeng, Sizhe
9	T 10/22	Alfassy et al. CVPR 2019: LaSO: Label-Set Operations networks for multi-label few-shot learning	Xiaoning, Zhenfei
9	R 10/24	Wu et al. CVPR 2019: Large Scale Incremental Learning	Ibrahim, Max
10	T 10/29	Tung et al. CVPR 2019: Learning Spatial Common Sense with Geometry-Aware Recurrent Networks	Guo, Haomin
10	R 10/31	Li et al. ICCV 2019: Expectation-Maximization Attention Networks for Semantic Segmentation	Nageswara, Sizhe
11	T 11/5	Zellers et al. CVPR 2018: Neural motifs: Scene graph parsing with global context	Jianyuan, Zhaoxiong
11	R 11/7	Kolkin et al. CVPR 2019: Style Transfer by Relaxed Optimal Transport and Self-Similarity	Ibrahim, Xiaoning
12	T 11/12	Wu et al. ArXiv 2019: A Comprehensive Survey on Graph Neural Networks	Max, Purvanshi
12	R 11/14	Wang et al. CVPR 2019: Learning Correspondence from the Cycle-Consistency of Time	Haomin, Zhenfei
13	T 11/19	Dwibedi et al. CVPR 2019: Temporal Cycle-Consistency Learning	Jianyuan, Nageswara
13	R 11/21	Sun et al. ICCV 2019: VideoBERT: A Joint Model for Video and Language Representation Learning	Guo, Tushar
14	T 11/26	Zellers et al. CVPR 2019: From Recognition to Cognition: Visual Commonsense Reasoning	Guofeng, Purvanshi
14	R 11/28	Thanksgiving Recess
15	T 12/3	Mao et al. ICLR 2019: The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences from Natural Supervision	Sizhe, Zhenfei
15	R 12/5	Project Presentation Group 1: Purvanshi, Max, Guo, Sizhe, Tushar, Xiaoning, Zhenfei
16	T 12/10	Project Presentation Group 2: Guofeng, Ibrahim, Jianyuan, Negaswara, Zhaoxiong

Work and Grade

Paper Summary and Review (25%): Students are expected to read two papers per week and write one short paper summary and one in-depth paper review. They are due before each class.
Presentation (25%): Present 2-4 papers during the semester (depending on the size of the class). You will have at least two weeks to prepare. Slides are due 8AM of your presentation day. Each paper will be presented by two presenters and class time should be evenly split between them.
- Background Presenter presents the problem addressed, surveys related work and delivers the necessary background for understanding the methods proposed and compared in the paper.
- Method Presenter introduces the authors, presents the proposed method and experiments in depth, and more importantly, prepares/facilitates classroom discussion of the paper.
By the end of the semester, each student should have at least one Background presentation and one Method presentation experiences. Students are responsible to coordinate with their presentation partner early in the semester to meet this requirement.
Project (25%): Complete a state-of-the-art research project on a topic subject to the instructor's approval. It consists of a mid-term proposal, a final-term presentation and a written report.
In-Class Discussion (25%): Actively participate in in-class discussion. Try to ask at least one good question each lecture. They are crucial in terms of your final course grade. Don't be shy to speak!

All assignments should be submitted on Blackboard and no late work will be accepted.

Textbooks: There are no required textbooks. We will read papers from CVPR, NIPS etc.

Code of Conduct: Check here for classroom "etiquette" expectations.

Academic honesty: All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available at here. You are expected to read, understand and follow the policy.

Disabilities: If you need an accommodation based on the impact of a disability, please contact me privately during the first two weeks to discuss your specific needs. Alternatively, contact Center for Excellence in Teaching and Learning at 585-275-9049 in room 1-154 Dewey Hall to coordinate reasonable accommodations.