Alumni Profiles

Russell with sharks

Daniel Russell PhD '84

Tech Lead for Search Quality and User Happiness at Google

Daniel Russell is the Über Tech Lead for Search Quality and User Happiness in Mountain View. He earned his PhD in computer science, specializing in artificial intelligence until he realized that magnifying and understanding human intelligence was his real passion. Twenty years ago he foreswore AI in favor of HI, and enjoys teaching, learning, running and music, preferably all in one day. He worked at Xerox PARC before it was, and was in the Advanced Technology Group at Apple, where he wrote the first 100 web pages for using SimpleText and a stone knife. He also worked at IBM running an HCI lab, and briefly at a startup that developed tablet computers before the iPad was a thing. His MOOCs have helped students become much more effective online searchers. His online course, has had ~500K students go through the content, meaning that somewhere on earth, an image of him has been on teaching search skills for more than 200 years.

Interview from 2014 Multicast Newsletter

From all accounts, the faculty and students at the fledgling URCS in the ‘70s were pioneers with great vision and energy. No rules, just research. What was that atmosphere like for you and how did a particular person or experience influence what you have done in your career?

DR: At the time, we were just kids with dreams. Remember, this was just when the ARPAnet was getting a start—the Internet as we know it didn't even exist except in science fiction, which I'm sure all of us thought was WAAY off in the future. We'd grown up on Star Trek and similar fare. I started in AI because I was deeply interested in figuring out how intelligence really worked—little did we know how much longer that particular road was going to be. But it was all part of our dream. So we worked until late at night, working away on our research projects, trying hard to implement our visions of AI or distributed systems (and yes, playing more than a few games of Trek—thanks Gene Ball for bringing that first truly networked game to the Ethernets in the CS lab).

But to answer your question, the atmosphere was just wonderful—lots of freedom, lots of classic grad school conversations with your peers, lots of hanging out with the faculty members.

Of the folks I talked with the most, many impressed me, but Jerry Feldman (my advisor and department chair at the time) had a great intellectual curiosity and a very laissez-faire approach to grad student management. That's probably for the good, as I changed my thesis topic a few times.

I was also profoundly influenced by folks in the neuropsychology department, especially Harry Whitaker (with whom I wrote my very first paper—a computational model of apraxia... of all things) and the other grad students over there who opened my eyes to what cognition (especially from a neuro perspective) was really all about. When I think back on it now from the perspective of a few decades, the CS department gave me a structure to work in (the AI model), and the neuropsychologists gave me data and a deep appreciation for how cognition actually operates. Both have served me well over the years.

Did the field of User Interface Design exist when you did your graduate work at UR? How did each company you worked at (Xerox, Apple, IBM and Google) strengthen your commitment to move in that direction?

DR: Here's the true story, never before written down.

As I said, my thesis work was in AI planning. There's been a long and illustrious line of work on planning ever since then (remember I graduated in 1984).

I took my background in planning and went to work at PARC with Richard Fikes, one of the founders of the whole planning area. And we had a *great* time working on a planner that would generate (provably correct!) plans to help humans run a complex Xerox copier. It really was complex, and the planner we worked on (called Bluebonnet) would help people through even the knottiest of jobs. Do you want to make 500 copies of a 50 page document with perfect binding, 2-sided copies for the first 100 pages, then copy onto blue page for the next 30 pages, then insert cardboard sections? Easy. Bluebonnet would walk you through the steps.

It was great. I wrote Lisp code to make the planner work, I wrote the serial driver to pull state from the copier (at the wire level), I wrote 3D shaded animated graphics (for the animated help system), etc etc. It really was a grad student's dream.

Then Lucy Suchman, an anthropologist at PARC, suggested that we watch what would happen when real people used Bluebonnet to do real tasks.

Great idea! Lets do that!

So we set up the cameras and brought in people to use Bluebonnet to solve their large, complex, copying tasks.

And it was a huge disaster. The system worked perfectly. The plans were created, the system state was correct.... and the UI was terrible. We hadn't paid much attention to it, so it looked very "industrial," and had no obvious flow. People literally didn't know where to look.

As we ran the tests, Lucy kept saying "This is really interesting..." And I kept thinking, "how did we find such terrible users?"

That is until the day Herb Simon and Alan Newell came to PARC on a visit. We managed to get them as test subjects for our user trial. And... THEY couldn't complete the task.

It was an epiphany for me. It really didn't matter how great, smart, or perfect the back-end planner was—if the UI didn't communicate what the human needed to know, it was so much computational junk.

And THAT'S the moment when I converted almost completely over to HCI. I still did AI as a day job for the next 10 years, but little by little I started doing work on HCI with Tom Moran, Peter Pirolli, Stuart Card, and Mark Stefik. That was really at the dawn of the discipline of HCI, when we were still very concerned with building predictive models of people, and the interest in the *design* of the UI was just emerging.

The Xerox Star was being developed as all this was going on, and it became abundantly clear that design REALLY matters—getting the iconography right, the appearance model, the information architecture, even the text of the error messages—it all matters.

So after 10 years I went to Apple to work in the area of AI + HCI. I ended up running half of Apple's HCI research lab, so it was a deeply broadening experience for me to work with truly great visual and interaction designers, as well as our own anthropologists, IR people, speech reco and TTS team.

After an amusing time returning to PARC for two years with the best people from Apple HCI labs, we found that Xerox was (still!) not ready to get into hardware, and I ended up at IBM running a fairly large HCI research lab for them. Although it was organizationally complicated, I learned a good deal of managerial skills, which has proven useful at Google.

In your position as research scientist in Search Quality and User Happiness at Google, you analyze the way different people use the Google search engine. Because of that, you have been called a search anthropologist for social media - part psychologist, part anthropologist, and part computer scientist. Which part is the most enjoyable for you or do you see them blending seamlessly?

DR: Blending seamlessly in the manner of a fruit smoothie is about the right model. There's a little of this, a little of that—my work draws on all of these skills, using tools and analysis methods from whatever field will help answer the questions. It's question-driven research, not methods- or field-driven research.

My job is to understand how and why people search the way they do. To understand this at Google scale requires blending the skills of a big data scientist (to analyze millions of log records), a field anthropologist (to conduct field studies about how people are thinking about their search process), and a usability lab experimenter (when we bring participants into our labs to do traditional studies or eyetracking analyses of their behaviors).

Although I was initially skeptical of the anthropological method, my early work with Lucy Suchman and later work with Victoria Bellotti (at Apple and PARC) convinced me that this is a deeply important part of understanding how people consider the artifacts that we (as engineers) construct. It's really something to visit someone in their home and have them RANT at the computer because of the inane way it's operating (and you, as the designer of that particular system, realize that you didn't consider this kind of person in your design).

What are some of the surprising results of your field studies of user's search techniques, and how has that impacted the way the Google search engine is evolving? What methods do you use to track the search behavior of subjects? Are there certain groups who show particular strengths and weaknesses?

DR: Probably the biggest surprise is how much of a gap there can be between logged behavior and actual behavior. In many ways, the rise of large amounts of log data has given researchers deep insights into what kinds of information people are really looking for, and how they respond to that. But there are interesting nuances that just clickstream data doesn't provide; the attitudinal response to our UX and product designs, as well as the mental model people have of our systems. We can impute some of that, but at the end of the day, you still have to go and look at people using the tools we've built.

The biggest surprise is the surprise that always happens—and that's the discrepancy between what people say they do, and then what they actually do. We see this repeatedly: People are not good describers of their own behavior, nor can they project forward very well to describe the tools they'd like to have. As Henry Ford reputedly said, "If I'd have asked people what they wanted, they would have said 'faster horses.' " We are in much the same boat. When we ask people what they want from search, they say "better results," but their response to "better results" is complex. What seems to be happening is that it's very hard to describe accurately what "better" really means. This is why we keep experimenting.

There are definitely different sets of search skills out there. Typically someone will be very skilled in a given domain, and not so great in another. MDs, for instance, are typically great at searching out medical information, but I've seen MDs fail at finding how-to information for a plumbing repair task. On the other hand, I've watched unemployed teens solve the plumbing search problem, but have zero idea how to find medical information. An interesting effect is that people "socially norm" their search skill estimates. That is, if you ask someone "who's the best Google searcher you know," it will typically be someone in their social group (usually defined by their working situation). A bus driver will point to another bus driver as the best searcher they know. That makes sense. But when you do studies across groups, you see each has domain depth, but cross-domains, very few people have broad search skills. (Information specialists, like reference librarians, almost always do—but that's their profession, and it's relatively rare.)

You have emerged as a leader in MOOCs (Massive Open Online Courses) such as A Google a Day and other Search Education tools. These reach millions of people each week. Will MOOCs reshape the educational systems of the future? What are the benefits and disadvantages to this type of learning?

DR: Learning in the Internet-age is changing rapidly. A motivated student with a search engine is a powerful force, and has access to a vastly larger set of resources than ever before.

MOOCs offer one way to reach a large number of students in a relatively short amount of time. My MOOCs ( have reached over 500,000 students in the past two years. I could have never done that in my lifetime. Even if I taught 500 students / year, that's a kilo-year of teaching. So in one way, MOOCs are incredibly efficient. And in studies I've done, I have been able to show that if you complete the MOOC, your search effectiveness basically doubles—and that improvement is persistent. Thus, the MOOCs can also be effective.

Will they revolutionize teaching? Yes, but I don't believe it will be the sudden collapse of academe that some predict. Instead, MOOCs will become just another tool in the educators box of tricks. The bigger shift is from knowledge-buried-in-books to discoverable knowledge online, and the ease with which students can search out, access, organize, and USE that knowledge. I see this in the classes I teach. Teaching programming or system-building is now nothing like what it was when I was a graduate student at Rochester. There are vast oceans of data, code, systems, and knowledge out there to be discovered and used.

At the same time, large MOOCs also provide a wealth of socialization for learning that I hadn't expected. When you have 100K students all taking the class together, it's a radically different experience than if there are 300 students. The chance that another student will be online AND be able to answer your question is very high. The chance that this student had exactly the same misunderstanding as you did is also quite high. A student who has just come to understand something in the class can give a much better explanation of the concept than I can.

How do we prevent the knowledge divide between the computer literate with access to unlimited information, and those in the world who are off the grid without access?

DR: The answer is obvious: provide access. The means of doing that are not obvious. There are many programs that are attempting to bring connectivity to the rest of the world. The one I know a bit about is Google's Loon project (to fly high-atmosphere balloons with regional wireless networking that are 6 miles above commercial aviation traffic), but this is a huge problem for which there probably isn't a single panacea.

Many of us have been victims of poor software design that we use in our daily work. Why aren’t there more “User Happiness Designers” like you throughout the software industry? Will user interface and need finding become a more important part of the end product in the future? Is the university pipeline currently producing enough HCI/UI graduates to meet the demand of this growing field?

DR: "User happiness" has been an (unstated) goal for UX designers for quite a while. The biggest change over the past few years is the shift towards quantification and careful data analysis about how people are actually using our designs. As I discovered years ago when working in the AI end of things, it doesn't matter how good the service is—if people can't understand it and use it, the product fails. The job of the designer to do create something that works with the mental models of the user AND has the correct affective response. My particular job as a User Experience Researcher is to bridge the gap between design and implementation. It's really a perfect job for someone like me who has a background in classical CS, a specialization in AI, a couple decades of practice in HCI work, all arranged over a set of broad ranging interests. HCI in general is a skill-set that's much in demand (that covers the needs from UI prototypers, designers, researchers, and data scientists), and my sense is that there's a fair bit of competition for these people—which suggests that we need to grow the pipeline from university to industry.

Is there too much personal information on the web? How can we protect ourselves from becoming vulnerable to criminals and scam artists as search abilities become more sophisticated? Is there a way to limit access to personal information and still allow public access to knowledge?

DR: This isn't my area of expertise, but my sense is that PII (personally identifiable information) is kind of a difficult battleground. There are too many repositories of information owned by too many different interests with too many levels of security and privacy. The biggest challenge I see now is to develop an awareness of what you're giving up when you sign up for this or that. People need to understand the basics of privacy, and I don't know where they're supposed to learn this—in school? That's not happening. Technologies change; scams change; best practices change—and yet people are, for the most part, immune to education about this. It worries me, which is part of the reason I've been spending so much time trying to figure out the best way to teach millions of people about how technology works, and how they can use it to change their lives (and incidentally, their personal information privacy practices).

Looking back on your career to this date, what contribution in your body of work gives you the most satisfaction?

DR: That's a hard question: Which of your children do you love best?

Probably the most influential from an academic perspective has been my work on sensemaking—that is, the processes people go through when they're searching for, finding, organizing, and understanding a complex body of information. I started that work at Xerox PARC back in the early 90s, and it's been a constant force in my research ever since.

But in terms of reaching people, my more recent educational work (with the MOOCs and various other teaching programs) has certainly affected more people in the world. I often get messages from people telling me that what they learned in one of my classes completely changed the way they work, or they were able to solve a problem of longstanding. That's very rewarding.

Knowing what you know now, what advice would you give to current Computer Science students at Rochester?

DR: When choosing a thesis topic, follow your heart, but be ready to listen to good advice. I changed my focus several times and found a topic I really, really enjoyed. But when my advisor said "stop already, just drop Chapter 13 from your thesis, you're done..." he was completely right. I was passionate about Chapter 13, but it was time to go pursue that chapter (which ultimately turned into the sensemaking work) elsewhere.

When you aren't working at making everyone else happy, what to you do for rest and relaxation?

DR: Family, running, bicycling, or scuba diving in exotic locations. And, truthfully, writing. (You'd have thought that all that writing for the thesis and all of those academic papers would have pounded it out of me, but you'd be wrong.)