Dynamic Resource Allocation for Efficient Parallel Data Processing Using RMI Protocol
Sushma K S1, Vinay Kumar V2
1Sushma K.S, M. Tech, Department of Computer Science & Engineering, VTU University, SIT, Mangalore, India.
2Vinay Kumar V, M. Tech, Department of Computer Science & Engineering, VTU University, SJCIT, Chickballapur, India.
Manuscript received on May 25, 2013. | Revised Manuscript received on June 17, 2013. | Manuscript published on June 30, 2013. | PP: 410-413 | Volume-2, Issue-5, June 2013. | Retrieval Number: E1886062513/2013©BEIESP
Open Access | Ethics and Policies | Cite
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: In recent years ad-hoc parallel data processing has emerged to be one of the killer applications for Infrastructure-asa-Service (IaaS) clouds. Major Cloud computing companies have started to integrate frameworks for parallel data processing in their product portfolio, making it easy for customers to access these services and to deploy their programs. the processing frameworks which are currently used have been designed for static, homogeneous cluster setups and disregard the particular nature of a cloud. Consequently, the allocated compute resources may be inadequate for big parts of the submitted job and unnecessarily increase processing time and cost. In this paper we discuss the opportunities and challenges for efficient parallel data processing in clouds and present our research project Nephele. Nephele is the first data processing framework to explicitly exploit the dynamic resource allocation offered by today’s IaaS clouds for both, task scheduling and execution. In this paper we discuss the opportunities and challenges for efficient parallel data processing Particular tasks of a processing job can be assigned to different types of virtual machines which are automatically instantiated and terminated during the job execution. Based on this new framework, we perform extended evaluations of MapReduce-inspired processing jobs on an IaaS cloud system and compare the results to the popular data processing framework Hadoop.
Keywords: Many-Task Computing, High-Throughput Computing, Loosely Coupled Applications, Cloud Computing.