The InterWeave Project: Automatic Management of Distributed Shared State

Michael L. Scott, Sandhya Dwarkadas,
and, formerly,
Chunqiang Tang,
Umit Rencuzogullari,
DeQing Chen,
Srinivasan Parthasarathy,
Eduardo Pinheiro,
and Xiangchuan Chen.

Computer Science Department
University of Rochester

As a complement to message passing, Rochester’s InterWeave system allows programmers of distributed systems to map shared segments into programs spread across heterogeneous hardware and language platforms.  InterWeave represents a merger and extension of our previous Cashmere and InterAct projects, combining hardware coherence within small multiprocessors, Cashmere-style lazy release consistency within tightly coupled clusters, and InterAct-style version-based consistency for distributed shared segments. 

In InterWeave, each shared segment evolves through a series of consistent versions.  When beginning a read-only critical section on a given segment, InterWeave uses a program-specific predicate to determine whether the currently cached version, if any, is “recent enough” to use.  Automatic data conversions allow each program to employ its own natural data format, byte order, and alignment, with full support for intra- and inter-segment pointers.  Timestamping is used to determine and communicate only those pieces of a segment that are different from the cached copy. 

InterWeave currently runs on Alpha, Sparc, x86, MIPS, and Power series processors, under Tru64, Solaris, Linux, Irix, AIX, and Windows NT (XP).  Currently supported languages are C, C++, Java, Fortran 77, and Fortran 90.  Driving applications include data mining, intelligent distributed environments, multi-player games, and scientific visualization. 

Supported by NSF grants CCR-9705594, 6-1-1997 to 5-31-2001, and CCR-9988361, 6-1-2000 to 5-31-2003.  Additional support from NSF RI grant EIA-0080124, a cooperative Agreement between the Department of Energy Office of Inertial Confinement Fusion and the University of Rochester Laboratory for Laser Energetics, and equipment grants from IBM, Compaq, Intel, and Sun. 

InterWeave Publications and Presentations

• “Integrating Remote Invocation and Distributed Shared State,” by C. Tang, D. Chen, S. Dwarkadas, and M. L. Scott.  18th Intl. Parallel and Distributed Processing Symp., Apr. 2004. 
• “Exploiting High-Level Coherence Information to Optimize Distributed Shared State,” by D. Chen, C. Tang, B. Sanders, S. Dwarkadas, and M. L. Scott.  9th ACM Symp. on Principles and Practice of Parallel Programming, June 2003. 
• “Distributed Shared State,” by M. L. Scott, D. Chen, S. Dwarkadas, and C. Tang.  Intl. Workshop on Future Trends of Distributed Computing Systems, May 2003. 
• “Efficient Distributed Shared State for Heterogeneous Machine Architectures,” by C. Tang, D. Chen, S. Dwarkadas, and M. L. Scott.  Intl. Conf. on Distributed Computing Systems, May 2003.  Earlier but expanded version available as “Support for Machine and Language Heterogeneity in a Distributed Shared State System,” TR 783, Computer Science Department, Univ. of Rochester, June 2002. 
• “Programming Models for Parallel and Distributed Systems,” by M. L. Scott.  Presentation slides from panel session at 10th Intl. Conf. on Architectural Support for Programming Languages and Operating Systems, San Jose, CA, Oct. 2002. 
• “JVM for a Heterogeneous Shared Memory System,” by D. Chen, C. Tang, S. Dwarkadas, and M. L. Scott.  Workshop on Caching, Coherence, and Consistency (WC3 ’02), New York, NY, June 2002. 
• “Multi-level Shared State for Distributed Systems,” by D. Chen, C. Tang, X. Chen, S. Dwarkadas, and M. L. Scott.  ICPP 2002, Vancouver, BC, August 2002.  Winner, Best Paper Award.  Earlier but expanded version available as “Beyond S-DSM: Shared State for Distributed Systems,” TR 744, Computer Science Department, Univ. of Rochester, March 2001. 
• “Is S-DSM Dead?,” by M. L. Scott.  Invited keynote address.  2nd Workshop on Software Distributed Shared Memory, Santa Fe, NM, May 2000. 
• “S-DSM for Heterogeneous Machine Architectures,” by E. Pinheiro, D. Chen, S. Dwarkadas, S. Parthasarathy, and M. L. Scott.  2nd Workshop on Software Distributed Shared Memory, Santa Fe, NM, May 2000. 
• “InterWeave: A Middleware System for Distributed Shared State,” by D. Chen, S. Dwarkadas, S. Parthasarathy, E. Pinheiro, and M. L. Scott.  Proceedings, LCR 2000, Rochester, NY, May 2000.  Also on-line: presentation slides. 
• Work-in-Progress Talk, SOSP ’99, Charleston, SC, Dec. 1999. 
• “Shared State in Distributed Systems,” by M. L. Scott.  Position paper from the NSF Workshop on New Challenges and Directions for Systems Research, St. Louis, MO, July 1997.  Also available: Group report on Wide-Area Network Resource Management. 

Last Change: 20 February 2006 /