First-Class Operating System Management of Computational Accelerators

Our research investigates operating system mechanisms to manage hardware accelerators and high-bandwidth devices in a safe, fair, and protected manner while maintaining high performance. Computational accelerators, including general-purpose graphical processing units (GP-GPUs) and engines for encryption, compression, media transcoding, pattern matching, parsing, etc., are increasingly common in modern computer systems. For the data center, extremely fast devices, including system area networks and massively parallel storage, are also increasingly common. For the sake of safety and fairness, such accelerators must be managed by the operating system, but for the sake of performance, they must be accessible either directly from user-level applications or in streamlined OS code, with little or no coordination across cores. In some cases (GP-GPUs in particular), the conflict between fairness and performance is exacerbated by the opacity of proprietary library/driver/hardware interfaces. Our research seeks a balanced solution to these conflicting goals through:

1. an operating system resource management architecture that allows direct user-level access in the common case, but intercedes in the existing accelerator access path when necessary to delay and re-order requests;
2. a scheduling discipline that minimizes re-apportionment of resources across cores, to avoid synchronization and cache misses in the common case;
3. a tool chain that uncovers hidden interface semantics required for resource management, together with a characterization of the information needed from vendors in the future;
4. an integrated management and scheduling strategy across the full set of computational resources in a given system.

Publications:

• J. Izraelevitz and M. L. Scott, "Generality and Speed in Nonblocking Dual Containers". ACM Transactions on Parallel Computing, to appear. 
  
  
• M. Graichen, J. Izraelevitz, and M. L. Scott, "An Unbounded Nonblocking Double-ended Queue". 2016 Intl. Conf. on Parallel Processing (ICPP), Philadelphia, PA, Aug. 2016. PDF version.
  
  
• Qi Zhu, Meng Zhu, Bo Wu, Xipeng Shen, Kai Shen, and Zhiying Wang, "Software Engagement with Sleeping CPUs". In Proc. of the 15th Workshop on Hot Topics in Operating Systems (HotOS XV), Kartause Ittingen, Switzerland, May 2015. PDF version. Conference talk slides in PDF.
  
  
• Konstantinos Menychtas, Kai Shen, and Michael L. Scott, "Disengaged Scheduling for Fair, Protected Access to Computational Accelerators". In Proc. of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'14), Pages 301--316, Salt Lake City, UT, Mar. 2014. Abstract. PDF version.
  
  
• Konstantinos Menychtas, Kai Shen, and Michael L. Scott, "Enabling OS Research by Inferring Interactions in the Black-Box GPU Stack". In Proc. of the USENIX Annual Technical Conference (USENIX ATC'13), San Jose, CA, June 2013. Abstract. PDF version.
  
  
• Joseph Izraelevitz and Michael L. Scott, "Fast Dual Ring Queues". Brief announcement, 26th ACM Symposium on Parallelism in Algorithms and Architectures (SPAA), Prague, Czech Republic, June 2014. Abstract. PDF version. Extended version published as TR 990, Computer Science Dept., Univ. of Rochester, Jan. 2014.
  
  
• Joseph Izraelevitz and Michael L. Scott, A Generic Construction for Nonblocking Dual Containers". Brief announcement, 33rd ACM Symposium on Principles of Distributed Computing (PODC), Paris, France, July 2014. Abstract. PDF version. Extended version published as TR 992, Computer Science Dept., Univ. of Rochester, May 2014.
  
  

Open-Source Software:

• Neon: A Disengaged Scheduling prototype for black-box GPUs

Project Members:

Alumni:

Support:

Note: