The ideal data structure for storing dynamic graphs would support fast updates as well as fast
range
queries which underlie graph traversals such as breadth-first search. The Packed Memory Array
(PMA)
seems like a good candidate for this setting because it supports fast updates as well as
cache-efficient range queries. Concurrently updating a PMA raises challenges, however, because
an
update may require rewriting the entire structure.

This paper introduces a parallel PMA with intra- and inter-operation parallelism and
deadlock-free
polylogarithmicspan operations. Our main observation is that the PMA is well-suited to
concurrent
updates despite occasionally requiring a rewrite of the entire structure because 1) most of the
updates only write to a small part of the structure and 2) the worst case is highly parallel and
cache-efficient.

To evaluate our data structure, we implemented Parallel Packed Compressed Sparse Row (PPCSR), a
dynamic-graph processing framework that extends the Ligra interface with graph updates. We show
that
PPCSR is on average about 1.6x faster on graph kernels than Aspen, a state-of-the-art
graph-streaming system. PPCSR achieves up to 80 million updates per second and is 2 – 5x faster
than
Aspen on most batch sizes. Finally, PPCSR is competitive with Ligra and Ligra+, two
state-of-the-art
static graph-processing frameworks.

The accuracy and computational efficiency of image alignment directly affects
the advancement of connectomics, a field which seeks to understand the
structure of the brain through electron microscopy.

We introduce the algorithms Quilter and Stacker that are designed to perform 2D
and 3D alignment respectively on petabyte-scale data sets from connectomics.
Quilter and Stacker are efficient, scalable, and simple to deploy on hardware
ranging from a researcher's laptop to a large computing cluster. On a single
18-core cloud machine each algorithm achieves throughputs of more than 1
TB/hr; when combined the algorithms produce an end-to-end alignment pipeline
that processes data at a rate of 0.82 TB/hr - an over 10x improvement
from previous systems. This efficiency comes from both traditional
optimizations and from the use of ``Frugal Snap Judgments'' to judiciously
exploit performance--accuracy trade offs.

A high-throughput image-alignment pipeline was implemented using the Quilter
and Stacker algorithms and its performance was evaluated using three datasets
whose size ranged from 550GB to 38TB. The full alignment pipeline achieved
a throughput of 0.6-0.8 TB/hr and 1.4-1.5 TB/hr on an 18-core and
112-core shared-memory multicore, respectively. On a supercomputing cluster
with 200 nodes and 1600 total cores, the pipeline achieved a throughput of
21.4 TB/hr.

This work won Best Student Paper at HPEC 2020.

This work was presented as a Poster at PPOPP 2019.

Perhaps the most popular sparse graph storage format is Compressed Sparse Row (CSR). CSR excels
at
storing graphs compactly with minimal overhead, allowing for fast traversals, lookups, and basic
graph computations such as PageRank. Since elements in CSR format are packed together, additions
and
deletions often require time linear in the size of the graph.

We introduce a new dynamic sparse
graph representation called Packed Compressed Sparse Row (PCSR), based on an array-based dynamic
data structure called the Packed Memory Array. PCSR is similar to CSR, but leaves spaces between
elements, allowing for asymptotically faster insertions and deletions in exchange for a constant
factor slowdown in traversals and a constant factor increase in space overhead.

Our contributions
are twofold. We describe PCSR and review the theoretical guarantees for update, insert, and
search
for PCSR. We also implemented PCSR as well as other basic graph storage formats and report our
findings on a variety of benchmarks. PCSR supports inserts orders of magnitude faster than CSR
and
is only a factor of two slower on graph traversals. Our results suggest that PCSR is a
lightweight
dynamic graph representation that supports fast inserts and competitive searches.

Energy efficiency is a key challenge for building modern sustainable societies. World’s energy
consumption is expected to grow annually by 1.6 %, increasing pressure for utilities and
governments
to fulfill demand and raising significant challenges in generation, distribution, and storage of
electricity. In this context, accurate predictions and understanding of population dynamics and
their relation to electricity demand dynamics is of high relevance.

We introduce a simple machine learning (ML) method for day-ahead predictions of hourly energy
consumption, based on population and electricity demand dynamics. We use anonymized mobile phone
records (CDRs) and historical energy records from a small European country. CDRs are large-scale
data that is collected passively and on a regular basis by mobile phone carriers, including time
and
location of calls and text messages, as well as phones’ countries of origin. We show that simple
support vector machine (SVM) autoregressive models are capable of baseline energy demand
predictions
with accuracies below 3 % percentage error and active population predictions below 10 %
percentage
error. Moreover, we show that population dynamics from mobile phone records contain information
additional to that of electricity demand records, which can be exploited to improve prediction
performance. Finally, we illustrate how the joint analysis of population and electricity
dynamics
elicits insights into the relation between population and electricity demand segments, allowing
for
potential demand management interventions and policies beyond reactive supply-side operations.

The goal of the research is to combine information from a Simplex-based system monitor with a Reinforcement Learning (RL) competence estimator to assure the safety of RL-controlled, city-scale critical infrastructure systems. The research explores the two issues above by producing (i) an easy-to-understand Black box monitor that avoids system failures by detecting possible breach of system’s correctness invariants, ignoring the details of the autonomic controller, and (ii) a Decision Module that takes input from the monitor to predict decisions that could result in the system entering a high-risk state and switching between the optimal RL-based autonomic controller and a sub-optimal safe controller.

Since the publication of PPCSR above, I have continued working on more efficient data structures and systems to improve the performance of dynamic graph analytics by using techniques including parallelization, data hierarchy, and compression.

For my senior research paper in Computer Science, I studied a variant of the Traveling Salesman Problem.

For my senior seminar paper in Discrete Math, I studied Anti-magic graphs.