All assignments and activities associated with this course must be performed in accordance with the University of Rochester's Academic Honesty Policy. More information is available at: http://www.rochester.edu/college/honesty.
The University of Rochester respects and welcomes students of all backgrounds and abilities. In the event you encounter any barriers(s) to full participation in this course due to the impact of a disability please contact the Office of Disability Resources. The access coordinators in the Office of Disability Resources can meet with you to discuss the barriers you are experiencing and explain the eligibility process for estabilishing academic accommodations. You can reach the Office of Disability Resources at disability@rochester.edu, (585) 276-5075; Taylor Hall.
To go from a model written in PyTorch, Tensorflow, or the like to a running service used by millions requires overcoming significant systems challenges. Although services like ChatGPT are used widely, the costs for training these models and running them in production remains too high for them to truly become pervasive.
In this course, we will study the computer systems responsible for the success of these AI models with the aim of understanding the open challenges remaining for deploying AI at scale. Although primarily aimed at graduate students, the course is suitable for advanced undergraduates with a strong interest in computer systems, perhaps looking to go on to graduate school for research. This is not an AI course.
We will survey techniques for creating and deploying high-performance and efficient large-scale machine learning (ML) systems. Topics chosen from the current research literature will span ML applications (e.g. LLMs), ML programming models (e.g. PyTorch), code generation and optimization (e.g. Triton), automatic parallelization, performance modeling, systems-level challenges, and specialized AI hardware. A background in CSC 252 (or Computer Organization) is essential, and having taken CSC 255 (Compiler optimization) and/or CSC 258 (Parallel and Distributed Systems) will be helpful.
Students who take this course will learn:
Letter grades are assigned using a standard scale based on the absolute score. Here is the scale I will use, although I might choose to use a different scale which will not be stricter than this:
| Score | CSC290 | CSC571 |
|---|---|---|
| >= 96 | A | A |
| >= 92 | A- | A- |
| >= 88 | B+ | B+ |
| >= 84 | B | B |
| >= 80 | B- | B- |
| >= 76 | C+ | C |
| >= 72 | C | C |
| >= 70 | C | C |
| >= 68 | C- | E |
| >= 60 | D | |
| < 60 | E |
In case we encounter difficulties due to COVID, I may eliminate the mid-term.
Students taking the 5xx version of this course should expect to read more, and expect to solve different problems, and be graded on different criteria.
Inform the instructor in advance if circumstances beyond your control will make you miss a deadline.
Late submissions are not accepted otherwise
Here are links to similar courses elsewhere that you may find inspiring. This course focuses on systems.
These resources may be helpful to acquire background knowledge, but are not required for this course.