NileOS: A Distributed Asymmetric Core-Based Micro-Kernel for Big Data Processing
National Science Foundation
Computer Science & Engineering Department
Find in your Library
Big Data applications have demanding expectations on computational resources front. Thus, general purpose operating systems are not a good fit. In this paper, we present a new special purpose distributed micro-kernel designed with big data applications' needs in mind. The new micro-kernel adopts a core-based Asymmetric Multiprocessing (AMP) approach. It optimizes interrupt management and I/O to suit the Map-Reduce model. The proposed micro-kernel design is based on Inter-processor Interrupts over Ethernet (IPIoE) frames and a BareMetal Operating System Markup Language (BOSML). A transparent deployment mechanism is presented to completely shield the developer of the micro-kernel service from the underlying distribution infrastructure and decouple the application implementation from its deployment perspective. Based on the initial prototype and the experiments presented, a considerable gain in performance of average 2.34 folds was achieved using the distributed TeraSort benchmark over Linux/Hadoop.
(2021). NileOS: A Distributed Asymmetric Core-Based Micro-Kernel for Big Data Processing. IEEE Access, 9, 3696–3711.
El-Rouby, Ahmad, et al.
"NileOS: A Distributed Asymmetric Core-Based Micro-Kernel for Big Data Processing." IEEE Access, vol. 9, 2021, pp. 3696–3711.