GALOIS: A Hybrid and Platform-Agnostic Stream Processing Architecture

Abstract

With the increasing prevalence of IoT environments, the demand for processing massive distributed data streams has become a critical challenge. Data Stream Processing on the Edge (DSPoE) systems have emerged as a solution to address this challenge, but they often struggle to cope with the heterogeneity of hardware and platforms. To address this issue, we propose a new hybrid DSPoE architecture named GALOIS, which is based on WebAssembly (Wasm) and is hardware-, platform-, and language-agnostic. GALOIS employs a multi-layered approach that combines P2P and master-worker concepts for communication between components. We present experimental results showing that operators executed in Wasm outperform those in Docker in terms of energy and CPU consumption, making it a promising option for streaming operators in DSPoE. We therefore expect Wasm-based solutions to significantly improve the performance and resilience of DSPoE systems.

Publication
In Proceedings of the International Workshop on Big Data in Emergent Distributed Environments
Liam Tirpitz
Liam Tirpitz
Researcher / PhD Candidate

My research interests include …