The purpose of this course is two-fold. First, we want to increase your knowledge of the subject matter. We hope that by the end of the class, you will be able to comfortably talk about how today's mobile visual computing systems work, both in theory and principle as well as some practical aspects. Second, just as important as learning new material, we want to provide an environment that promotes and rewards creative and deep thinking. In the end, we want you to identify new research challenges in mobile visual computing and have a reasonably clear idea how to go about addressing them.

There are many references on various topics that might be helpful to you, which I list them below. Many of them are either made publicly available by the authors (annotated with OPEN) or are licensed by UR (annotated with UR), in which case you will have access either using the campus network or, when at home, log in using your NetID. We will assign readings from some of the books below. The assigned readings will be accessible in one of the approaches above (i.e., you don't need to buy any book if you don't want to). Some readings are advanced, but don't get intimated by them; they are provided to give you a "different voice" from my lectures, and to provide more detail about each topic in case you are interested.

Pleasant General Reading

Books here are just absolute pleasant to read. In terms of technical levels, I'd say they are in-between popular science and academic textbooks: they may use equations but focus on developing intuitions and connecting to real worlds rather than being "pedantic".

• QED: The Strange Theory of Light and Matter (OPEN). This book is basically the transcript of a lecture series that Feynman gave to a non-scientific audience; a version of the lectures is available on YouTube. It's just an incredibly inspirational book that shows how so much can be derived from so little. It talks about what light really is and light matter interaction at the fundamental level—without a single equation! If you are interested in the macroscopic behaviors of light and light-matter interaction discussed in the course and are wondering how things take place at the microscopic level, this book is a good start.
• The Optics of Life: A Biologist's Guide to Light in Nature. This book is breathtaking. Once you start you won't want to put it down until you finish it. It mainly talks about the birth (emission), life (scattering), and death (absorption) of light, and demystifies a great deal of things you might be wondering in day to day life. Many other books on the same topic tend to over-simplify things to the point that you can't really reason about new phenomena from what you are told. This book is free from such nonsense; it lays out the fundamental principles—in equations. But instead of focusing on deriving the equations, the focus is to use the equations to quantitatively explain and reason about things. Real-world biological examples throughout the book will put a smile on your face. It is written in a conversational style so it's very accessible. The author's wittiness is rampant throughout the exposition.
• Animal Eyes (UR)
• Virtual Reality (OPEN)

Vision Science and Color

• Foundations of Vision (OPEN)
• Measuring Colour (UR)
• Billmeyer and Saltzman's Principles of Color Technology (UR)
• Color Appearance Models (UR)
• Color for Science, Art and Technology (UR)
• Color Science in the Examination of Museum Objects - Nondestructive Procedures (OPEN). This book mainly talks about color science in paintings. Chapter 3 and 4 on colorants and colorant mixtures provide lots of examples from the author's hands-on experience working with paintings. So even though the book is not meant to be entirely rigorous, the empirical observations and conclusions are just fascinating to know and, once you develop a rigorous understanding of light matter interaction, to "prove".
• Basic Principles of Textile Coloration (UR). Pretty technical, and you might not be interested in dyes, but if you are this book is good. Chapter 22.3 is what led me to the book. It talks about, in the dying context, how to calculate the correct amount of dye mixtures to match a given color—something painters have to do, too!

Camera Imaging

• Physics of Digital Photography (UR). Not the most well known book, but it's the book I'd recommend on digital camera imaging. It covers optics, sensor, and raw data processing in sufficient detail. It starts from first principles and provide rigorous physical modeling of the entire process. It also discusses things that real cameras do that perhaps deviate away from the theories.
• Ultra Low Noise CMOS Image Sensors. The best resource on CMOS image sensor noise analysis.
• Canon - EF Lens Work III-The Eye of EOS, 8e (2006) (OPEN)
• Image Sensors and Signal Processing for Digital Still Cameras (UR)
• Digital Color Imaging Handbook (UR)
• Digital Video and HD : Algorithms and Interfaces (UR)

Optics, Display, and Lighting Systems

• Discovering Light: Fun Experiments with Optics (OPEN). I like this book a lot. Each chapter first tells you some fundamental concepts about lights or optics in a popular science way, and then invite you to a set of hands-on experiments, at the level of a ten year old in terms of difficulty, that you can do at home by yourself. It really gets you thinking.
• Optics, Fifth Edition (UR). A classic. The chapters on geometric optics will help you develop good intuitions. The chapter on Fourier optics is brief but gives you an overall idea.
• Displays: Fundamentals and Applications (UR). It's more of a survey that tells you what's out there rather than a textbook that teaches you things from ground up. But it's a very comprehensive survey for a wide variety of topics in the display world.
• OLED Display Fundamentals and Applications (UR). Specific to OLEDs, which are prevalent today. Chapter 4 provides a detailed description of OLED designs. Chapter 9 talks about using OLEDs as lighting systems rather than display systems. It provides a good platform for you to think about the connections and differences between display and lighting systems.
• Human Factors in Lighting (UR). Lots of books have been written about light, optics, and displays, but this book is a rare find that focuses on lighting systems.

Computer Graphics

• Physically Based Rendering: From Theory To Implementation (OPEN). Being the golden reference of physically based rendering, it is perhaps the most understandable book, because it has code. Each chapter is literally a walk-through of a part of the code interleaved with theoretical explanations. It takes the perspective that fundamentally graphics (rendering) is about simulating physics (light transport and light matter interaction), so the book is unconcerned with the traditional rasterization pipeline and many empirical shading algorithms. If you like physics you will fall in love with the book (and the general notion of physically based rendering).
• Principles of Digital Image Synthesis (OPEN). A bit dated but a classic. The book is written in a true textbook style: Andrew Glassner repeats the same concept over and over again and gives many examples to support a point; new knowledges are developed based on what you have already been told not just thrown in your face.
• Fundamentals of Computer Graphics (UR). Quite comprehensive and the exposition is concise and to the point.
• Digital Modeling of Material Appearance (UR). A bit dated perhaps, but the treatment of the basic is very good.
• Real-Time Rendering (UR)
• Light and Skin Interactions (UR)
• Physics for Game Developers (UR)

Math Foundations

• Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control (UR). If there is one book you want to buy, not just for this course but for your intellectual development in any STEM field, this book would definitely be one of my recommendations. Even better, the authors put their lectures on YouTube as well. I guarantee you, once you start you will find yourself binge watching, a word you probably wouldn't have thought that you would associate with math videos.
• Lecture Notes for the Fourier Transform and its Applications (OPEN)
• Numerical Algorithms (OPEN)
• Convex Optimizations (OPEN)
• Linear Algebra: Foundations to Frontiers (OPEN)

While the exact grading scheme is still being decided, the final grades will likely be assigned with these proportions. The percentages are roughly estimates/my best guesses as of now:

• Student Information Sheet: 3%

  The Student Information Sheet, a.k.a., Assignment 0, allows me to better know you: your interests, background, career goals, etc. Finish it within the first two weeks. You will receive full credit as long as you submit it and make reasonable effort.

• Written Assignments: 33%

  We will have four written assignments. The written assignments are designed to help you better understand the core principles discussed in the lectures. The assignments could ask you to work out some math or discuss things in the assigned readings.

• Programming Assignments: 24%

  We will have three relatively lightweight programming assignments, which mostly will ask you to implement some existing visual computing algorithms, but you don't have to be constrained by what you have learned and how they are implemented today. Use your imagination and be creative. Remember: the devil is in the detail!

• Final Project: 40%

  You will work on a final project that is completely chosen by you. It doesn't have to be a programming-related project. It could be arts (e.g., photography), design (e.g., 3D printing), systems building, or a written paper describing your investigation of a particular concept. Think of it as an independent study, where I am here just to guild you. You will need approval from me for your project within the first two months. Could be team of up to 2 people.

  If you don't feel like coming up with a project on your own, we have a list of potential projects for you to choose from. The benefit of choosing projects is that I might be able to help you more. The list will be made available a few weeks into the semester.

Everyone — the instructor, TAs, and students — must be respectful of everyone else in this class. All communication, in class and online, will be held to a high standard for inclusiveness: it may never target individuals or groups for harassment, and it may not exclude specific groups. That includes everything from outright animosity to the subtle ways we phrase things and even our timing.

For example: do not talk over other people; don't use male pronouns when you mean to refer to people of all genders; avoid explicit language that has a chance of seeming inappropriate to other people; and don't let strong emotions get in the way of calm, scientific communication.

If any of the communication in this class doesn't meet these standards, please don't escalate it by responding in kind. Instead, contact the instructor as early as possible. If you don't feel comfortable discussing something directly with the instructor — for example, if the instructor is the problem — please contact the advising office or the department chair.

Student conduct in CSC 292/572 is governed by the College Academic Honesty Policy and the Undergraduate Laboratory Policies of the Computer Science Department. The teaching staff will enforce them aggressively and strictly. The following are additional details specific to CSC 292/572.

Exams in CSC 292/572 must be strictly individual work.

Collaboration on assignments among team members is of course expected. Collaboration on assignments across teams (or among individuals on non-team-based assignments) is encouraged at the level of ideas. Feel free to ask each other questions, brainstorm on algorithms, or work together at a whiteboard. You may not claim work as your own, however, unless you transform the ideas into substance by yourself. This means you must leave any brainstorming sessions with no written notes.

Similarly, you are welcome to read anything you find on the web, but you must close all web pages before beginning to write your code. You are not permitted to repeatedly consult a source. You can read it, understand it, put it away, and write your own similar code, but you must not copy anything. Both electronic copy-and-paste and copying through short-term memory are expressly forbidden.

To minimize opportunities to steal code, all students must protect the directories in which they do their work.

For purposes of this class, academic dishonesty is defined as:

• Any attempt to pass off work on an exam that didn't come straight out of your own head.
• Any cross-team collaboration or copying of artifacts (substance) in which the collaborating parties don't clearly and prominently explain exactly who did what, at turn-in time.
• Any activity that has the effect of significantly impairing the ability of another student to learn. Examples here might include destroying the work of others, interfering with their access to resources, or deliberately providing them with misleading information. (Note too that grades in CSC 292/572 are assigned on the basis of individual merit, so there is no benefit, even a dishonest one, to be gained by sabotaging the work of others.)

Finally, if you have any questions about what is permitted and what is not, please ask!

---