Perform wordcount Map-Reduce Job in Single Node Apache Hadoop cluster and compress data using Lempel-Ziv-Oberhumer (LZO) algorithm
Applications like Yahoo, Facebook, Twitter have huge data which has to be stored and retrieved as per client access. This huge data storage requires huge database leading to increase in physical storage and becomes complex for analysis required in business growth. This storage capacity can be reduced and distributed processing of huge data can be done using Apache Hadoop which uses Map-reduce algorithm and combines the repeating data so that entire data is stored in reduced format. The paper describes performing a wordcount Map-Reduce Job in Single Node Apache Hadoop cluster and compress data using Lempel-Ziv-Oberhumer (LZO) algorithm.
Keywords: Hadoop, Map-reduce, Hadoop Distributed file system HDFS, HBase, LZO
Download Full-Text
ABOUT THE AUTHORS
Nandan Nagarajappa Mirajkar
Nandan Nagarajappa Mirajkar is pursuing M.Tech in Advanced Information Technology with specialization in Software Technologies from IGNOU – I2IT Centre of Excellence for Advanced Education and Research, Pune, India. He is also Teaching Assistant in Advanced Software and Computing Technologies department. He has published one International Journal in IJCSI. He is IGNOU – I2IT Centre of Excellence for Advanced Education and Research Academic Scholarship holder. He has pursued B.E Electronics and Telecommunications from University of Mumbai. His research interests include Cloud computing, Databases and Networking.
Sandeep Bhujbal
Sandeep Bhujbal is Sr. Research Associate in Advanced Software and Computing Technologies department of IGNOU – I2IT Centre of Excellence for Advanced Education and Research, Pune, India. He has pursued M.C.S from University of Pune. His research interests include Operating systems, Compiler construction, Programming languages and Cloud computing.
Aaradhana Arvind Deshmukh
Prof. (Ms.) Aaradhana Arvind Deshmukh is Asst.Professor in Dept. of Computer Engineering , Smt. Kashibai Navale Collge of Engineering, Pune. She is pursuing PhD in Cloud Computing from Aalborg University, Denmark. She is visiting Faculty in Advanced Software and Computing Technologies department of IGNOU – I2IT Centre of Excellence for Advanced Education and Research, Pune, India. She obtained Masters [Computer Engineering ] , A.M.I.E. Computer Engineering , B.E. (Computer Engineering) M.A. (Economics) from Pune University. She is having 10 years experience in Teaching Profession and 2 ½ years R & D experience in various institutes under Pune University. She has published 43 papers , 13 in International Journals like ACM, IJCSI, ICFCA, IJCA etc, 16 in International Conferences like IEEE etc., 9 in National Conferences, 4 in symposiums . She has received Gold Medal at International level Paper Presentation on \"Neural Network\" as well as one more for \" UWB Technology based adhoc network\", in International Conferences. She is recipient of ‘Distinguished Alumni Award’ in 2011 from Inst. Of Engineers [India] , Gunawant Nagrik Puraskar for the year 2004 – 2005, ‘Anushka Purskar’ from Pimpri Chinchwad Municipal Corporation, and also won many Firodiya awards. She has organized many 15 multidisciplinary Short Term Training Program, workshops, conferences on National and International Level. Her research interests include Cloud computing, Databases and Networking, security.
Nandan Nagarajappa Mirajkar
Nandan Nagarajappa Mirajkar is pursuing M.Tech in Advanced Information Technology with specialization in Software Technologies from IGNOU – I2IT Centre of Excellence for Advanced Education and Research, Pune, India. He is also Teaching Assistant in Advanced Software and Computing Technologies department. He has published one International Journal in IJCSI. He is IGNOU – I2IT Centre of Excellence for Advanced Education and Research Academic Scholarship holder. He has pursued B.E Electronics and Telecommunications from University of Mumbai. His research interests include Cloud computing, Databases and Networking.
Sandeep Bhujbal
Sandeep Bhujbal is Sr. Research Associate in Advanced Software and Computing Technologies department of IGNOU – I2IT Centre of Excellence for Advanced Education and Research, Pune, India. He has pursued M.C.S from University of Pune. His research interests include Operating systems, Compiler construction, Programming languages and Cloud computing.
Aaradhana Arvind Deshmukh
Prof. (Ms.) Aaradhana Arvind Deshmukh is Asst.Professor in Dept. of Computer Engineering , Smt. Kashibai Navale Collge of Engineering, Pune. She is pursuing PhD in Cloud Computing from Aalborg University, Denmark. She is visiting Faculty in Advanced Software and Computing Technologies department of IGNOU – I2IT Centre of Excellence for Advanced Education and Research, Pune, India. She obtained Masters [Computer Engineering ] , A.M.I.E. Computer Engineering , B.E. (Computer Engineering) M.A. (Economics) from Pune University. She is having 10 years experience in Teaching Profession and 2 ½ years R & D experience in various institutes under Pune University. She has published 43 papers , 13 in International Journals like ACM, IJCSI, ICFCA, IJCA etc, 16 in International Conferences like IEEE etc., 9 in National Conferences, 4 in symposiums . She has received Gold Medal at International level Paper Presentation on \"Neural Network\" as well as one more for \" UWB Technology based adhoc network\", in International Conferences. She is recipient of ‘Distinguished Alumni Award’ in 2011 from Inst. Of Engineers [India] , Gunawant Nagrik Puraskar for the year 2004 – 2005, ‘Anushka Purskar’ from Pimpri Chinchwad Municipal Corporation, and also won many Firodiya awards. She has organized many 15 multidisciplinary Short Term Training Program, workshops, conferences on National and International Level. Her research interests include Cloud computing, Databases and Networking, security.
