kumar, Fahri (2022) An Efficient Replication Method for MapReduce Scaling. International Journal of Innovative Research in Engineering, 3 (4). pp. 96-100. ISSN 2582-8746
|
Text
paper_pdf_data (1).pdf Download (548kB) | Preview |
Abstract
Recently, research and utilization of distributed storage and processing systems for large-scale data processing and management are becoming important. Hadoop is widely used as a representative distributed storage and processing framework. Task assignment in MapReduce, which is performed based on the Hadoop distributed file system, is assigned as close as possible considering the locality of data. However, in the data analysis work in MapReduce, there is data that is frequently requested depending on the type of work. In this case, due to the low locality of the data, it causes problems of an increase in execution time and delay of data transmission. In this paper, we propose a Least Recently Used LRU-data replication technique according to the data access pattern to improve the processing speed of Map-Reduce. In the proposed method, data locality is improved by using a replica optimization algorithm for LRU-data showing high access frequency according to the data access pattern, and consequently the operation time is reduced. As a result of the performance evaluation, it was confirmed that the load of the access frequency was reduced compared to the existing technique.
| Item Type: | Article |
|---|---|
| Keywords (Kata Kunci): | Distributed Computing, Map-Reduce, Hadoop, Locality, LRU-Data |
| Subjects: | Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Divisions: | Fakultas Teknik |
| Depositing User: | Dr Tso Chen |
| Date Deposited: | 04 Jul 2023 23:17 |
| Last Modified: | 04 Jul 2023 23:17 |
| URI: | http://eprints.unram.ac.id/id/eprint/40837 |
Actions (login required)
 |
View Item |
