April 29, 2019, 09:30 AM
Linfeng Song: Tackling Graphical NLP problems with Graph Recurrent Networks
[Monday, April 29, 2019 at 9:30 AM in Wegmans Hall 2506]
How to properly model graphs is a long-existing and important problem in natural language processing, where several popular types of graphs are knowledge graphs, semantic graphs and dependency graphs. Comparing with other data structures, such as sequences and trees, graphs are generally more powerful in representing complex correlations among entities. For example, a knowledge graph stores real-word entities (such as Barack Obama and U.S.) and their relations (such as live in and lead by). Properly encoding a knowledge graph is beneﬁcial to user applications, such as question answering and knowledge discovery. Modeling graphs is also very challenging, probably be-cause graphs usually contain massive and cyclic relations. For instance, a tree with n nodes has n − 1 edges (relations), while a complete graph with n nodes can have O(n2) edges (relations).
Recent years have witnessed the success of deep learning, especially RNN-based models, on many NLP problems, including machine translation (Cho et al., 2014) and question answering (Shen et al., 2017). Besides, RNNs and their variations have been extensively studied on several graph problems and showed preliminary successes. Despite the successes that have been achieved, RNN-based models suffer from several major drawbacks. First, they can only consume sequential data, thus linearization is required to serialize input graphs, resulting in the loss of important structural information. In particular, originally closely located graph nodes can be very far away after linearization, and this introduces great challenge for RNNs to model their relation. Second, the serialization results are usually very long, so it takes a long time for RNNs to encode them.
In this thesis, we propose a novel graph neural network, named graph re-current network (GRN). GRN takes a hidden state for each graph node, and it relies on an iterative message passing framework to update these hidden states in parallel. Within each iteration, neighboring nodes exchange information be-tween each other, so that they absorb more global knowledge. Different from RNNs, which require absolute orders (such as left-to-right orders) for execution, our GRN only require relative neighboring information, making it very general and ﬂexible on a variety of data structures.
We study our GRN model on 4 very different tasks, such as machine read-ing comprehension, relation extraction and machine translation. Some tasks (such as machine translation) require generating sequences, while others only require one decision (classiﬁcation). Some take undirected graphs without edge labels, while the others have directed ones with edge labels. To consider these important differences, we gradually enhance our GRN model, such as further considering edge labels and adding an RNN decoder. Carefully designed experiments show the effectiveness of GRN on all these tasks.
Reception to follow at 12:30pm in Wegmans Hall 2506
Advisor: Prof. Daniel Gildea (Computer Science)
Committee: Prof. Jiebo Luo (Computer Science), Prof. Lenhart Schubert (Computer Science), Prof. Yue Zhang (Westlake University)
April 29, 2019, 04:00 PM
Sharanyan Srikanthan: Sharing-Aware Resource Management for Multicore Systems
[Monday, April 29, 2019 at 4:00 PM in Goergen 108]
Efficient resource management is a growing problem due to the ever increasing scale and complexity of computational systems and the applications that use them. Modern multicore systems offer abundant compute resources to exploit application-level parallelism. The multiple compute cores within a single system share resources such as processor pipelines, caches, interconnects, and memory, presenting opportunities for efficient data sharing and resource utilization. These multicore systems are regularly employed in large clusters and clouds, hosting a multitude of applications and services simultaneously. While a tremendous amount of research has been aimed at solving various problems in offering infrastructure as a service, managing multiple applications and achieving high utilization and efficiency remains a challenge. The quest to increase utilization may result in higher resource contention and correspondingly unpredictable and often significantly degraded performance. Moreover, the degree of data and resources sharing among different cores is nonuniform, and is dependent on the architecture and applications involved. Identifying and enabling the efficient and synergistic sharing of data and resources while also minimizing resource contention and saturation is the key to simultaneously achieving higher utilization and resource efficiency.
In this dissertation, we argue that it is possible to provide efficient and deterministic performance in both individual and distributed multicore systems using a holistic approach that simultaneously guides application resource acquisition and manages hardware resources and task placement. Our resource management strategy combines information from the execution environment with application-defined quality of service targets to achieve overall system efficiency while meeting individual application progress guarantees. We demonstrate that aggregate information from low overhead hardware performance counters is sufficient to characterize individual application resource demands and bottlenecks specific to the execution environment. Using this information, we develop a hierarchical resource management strategy that can: monitor performance critical architectural resources and control task placement for optimal use of these resources; understand application bottlenecks, scalability, and parallel efficiency to reallocate resources while guaranteeing quality of service; and consolidate the above information from individual machines into a shared state to guide resource reservation in a distributed setting in order to simultaneously improve utilization and efficiency by reducing resource contention and saturation.
Reception to follow on TUESDAY, April 30, 2019 at 12:00pm in Wegmans Hall 3rd Floor Atrium
Advisor: Prof. Sandhya Dwarkadas (Computer Science)
Committee: Prof. Michael Scott (Computer Science), Prof. Michael Huang (Electrical & Computer Engineering), Dr. Kai Shen (Google)