Efficient Big Data Processing using Containerized Cloud Microservices

Abstract

The increasing scale and heterogeneity of data demand efficient and scalable processing frameworks beyond traditional monolithic systems. This paper proposes a containerized cloud microservices architecture for big data processing, where data pipelines are decomposed into loosely coupled services deployed via container orchestration. The approach enables dynamic scaling, fault isolation, and efficient resource utilization. A modular design integrating data ingestion, stream and batch processing, and distributed storage is developed with adaptive scheduling for varying workloads. The framework also supports rapid deployment, service portability, and simplified system maintenance through container abstraction. Experimental results show reduced latency and improved throughput compared to conventional architectures, demonstrating the effectiveness of the proposed framework for modern data-intensive applications.