NileOS: A Distributed Asymmetric Core-Based Micro-Kernel for Big Data Processing

Funding Number

AGS-1265783

Funding Sponsor

National Science Foundation

Author's Department

Computer Science & Engineering Department

Find in your Library

https://doi.org/10.1109/ACCESS.2020.3048082

Document Type

Research Article

Publication Title

IEEE Access

Publication Date

1-1-2021

doi

10.1109/ACCESS.2020.3048082

Abstract

Big Data applications have demanding expectations on computational resources front. Thus, general purpose operating systems are not a good fit. In this paper, we present a new special purpose distributed micro-kernel designed with big data applications' needs in mind. The new micro-kernel adopts a core-based Asymmetric Multiprocessing (AMP) approach. It optimizes interrupt management and I/O to suit the Map-Reduce model. The proposed micro-kernel design is based on Inter-processor Interrupts over Ethernet (IPIoE) frames and a BareMetal Operating System Markup Language (BOSML). A transparent deployment mechanism is presented to completely shield the developer of the micro-kernel service from the underlying distribution infrastructure and decouple the application implementation from its deployment perspective. Based on the initial prototype and the experiments presented, a considerable gain in performance of average 2.34 folds was achieved using the distributed TeraSort benchmark over Linux/Hadoop.

First Page

3696

Last Page

3711

This document is currently not available here.

Find this item online

Share

COinS