Event Correlation with Algebraic Effects - Theory, Design and Implementation

• Main Author: Bračevac, Oliver
• Format: Others
• Language: en
• Published: 2019
• Online Access: https://tuprints.ulb.tu-darmstadt.de/9258/1/DissertationBracevac05112019.pdf; Bračevac, Oliver <http://tuprints.ulb.tu-darmstadt.de/view/person/Bra==010Devac=3AOliver=3A=3A.html> (2019): Event Correlation with Algebraic Effects - Theory, Design and Implementation.Darmstadt, Technische Universität, [Ph.D. Thesis]

Modern software systems are increasingly becoming distributed, reactive, and data intensive. In this context, events are an essential abstraction for representing and communicating interesting situations, enabling loosely-coupled and scalable systems. An integral part of these systems is event correlation, which is about computing relations between observed event notifications in space and time, e.g., to programmatically reason about the state of the system, for reacting to changes, for synchronization, for coordination or data processing. The importance of event correlation is witnessed by the considerable attention it has received from different research communities, under different names. This has led to the emergence of different "families" of event correlation approaches with a sheer staggering amount of specialized correlation semantics and software abstractions. We distinguish between four families: complex event processing, stream processing, reactive programming, and concurrent programming. We face a rather dire state of affairs: Existing event correlation approaches lack clarity, and most of them are not designed to be customizable or composable. Software systems evolve and their requirements change over time. Yet, no single approach is sufficiently adaptable to meet all requirements. We lack appropriate conceptual models and software abstractions that enable modular, extensible and cross-cutting compositions of features among the different event correlation families, fostering customizability and adaptability. It is time for a new generation of unifying, highly adaptable and customizable reactive software systems, by means of event correlation "à la carte". That is, we need language designs for event correlation that can express features from all families in a uniform, extensible and freely composable manner. This thesis makes event correlation "à la carte" a reality. We solve this issue by principled application of functional programming, in particular algebraic effects and effect handlers, which are a modern take on integrating side effects into functional languages in a modular and extensible way. The main statement of the thesis is that algebraic effects and effect handlers are a good programming abstraction for obtaining principled, versatile event correlation systems. To validate the thesis, we develop Cartesius, the first computational model that captures the essence of event correlation. It has an extensible and customiz- able semantics based on algebraic effects and effect handlers. We view instances of event correlation as cartesian products over streams of events, where user-defined effect handlers locally customize the control flow of the computation to obtain semantic variants that behave differently. Handlers compose freely and new kinds of correlation features can be added by introducing new user-defined side effects. Furthermore, we complement Cartesius with PolyJoin, which is an extensible programming language integration of declarative frontend syntax for event correlation systems. It can be deployed independently of Cartesius to provide a type-safe frontend for other event correlation systems in a programming language and subsumes mainstream language-integrated query techniques. To evaluate our approach, we conduct a survey, comparing the expressivity of Cartesius against representative systems from the above event correlation families. Our findings demonstrate that Cartesius captures the essence of other system's features and is fully customizable and adaptable, to a degree none of the surveyed event correlation approaches offer. Additionally, we evaluate the performance of our approach in terms of synthetic microbenchmarks on common event correlation variants. The benchmarks demonstrate that despite the modular decomposition into an expensive cartesian product and user-defined extensions, Cartesius' design is practical and achieves characteristic time and space complexity of the considered event correlation variants. Our "à la carte" approach based on algebraic effects and handlers exhibits a degree of unification, fine-grained customization and hybridization that no previous work on event correlation has attained.