In:
Applied Mechanics and Materials, Trans Tech Publications, Ltd., Vol. 513-517 ( 2014-02-06), p. 2281-2284
Abstract:
The problems of difference of nodes capabilities and the unevenly-distributed bandwidth of the network, widespread exist in the heterogeneous clouding environment. Together with the users randomness of submitting jobs, the problems above lead to server synchronization problems.Under the platform of Hadoop and the situations mentioned above, we come up with a method which is based on the native hadoop speculative algorithm to solve the problems. Through monitoring the load-balance in realtime, dynamically assessing the performance of the node and making the speculative tasks happened in high-performance node which meantime is the nearest node from input split, the algorithm effectively reduces the occupation of the network and accelerates executing speed. The experiment result shows that the method in the execution of which the speculative tasks has a high ratio, significantly improved the efficiency and throughput of the cluster.
Type of Medium:
Online Resource
ISSN:
1662-7482
DOI:
10.4028/www.scientific.net/AMM.513-517
DOI:
10.4028/www.scientific.net/AMM.513-517.2281
Language:
Unknown
Publisher:
Trans Tech Publications, Ltd.
Publication Date:
2014
detail.hit.zdb_id:
2251882-4
Permalink