Tuesday, July 22, 2014

Big Data- Installing Hadoop ( Single Node)

So finally we get to do Hadoop installation for single node.

Pre-Installation System:

-SSH server: Master communicate with the slaves with the SSH. So we need to get SSH server running. We need to generate public/private key pair. Than we take that key, that we generate on the master and send it to all our nodes.

-Java 6 or greater

Hadoop Installation:
-Download and install hadoop
-add paths to bashrc
-Configure hadoop environment
(env.sh)-> java.home/disable ipv6
-Configure.XML ( core -site ! mapred-site)
-Launch Hadoop Daemons




Now i will show you installation

Lets do SSH to the name node



lets start going with our prerequisites. The first prerequisite is to generate public-private key.

No problem, we just need to type the command 

#SSH Keygen


now it will ask for password. we are keeping the password blank



There we go, we have our key generated.

Now lets get JAVA installed.

We need to type the command
# sudo apt-get install openjdk-7-jdk

It will ask that do you want to download 218MB package, just say yes. So Java Installed !!

Now we shift focus on Hadoop.

First we download the hadoop from the website. We can download using command line or browser. Once its downloaded, we move to next course of action that to unzip the .tar file

#tar -zxvf hadoop-1.2.0-bin.tar.gz

now we copy this to user local area and name it hadoop



next is to add paths to bashrc and configure env.sh
#Sudo vi $HOME/.bashrc
go to the bottom of the file and insert the following entries and save


then run the command
#exec bash
This will run the command

now to configure the env.sh. We type the command

#sudo vi /usr/local/hadoop/conf/hadoop-env.sh

we go to 7th line from the top and remove the #

and edit the line to replace with our type of java






and disable IPV6 on the same file

Now we have to configure those XML files. 

#Sudo vi /usr/local/hadoop/conf/core-site.xml


#Sudo vi /usr/local/hadoop/conf/mapred-site.xml


this is the last step.

#hadoop namenode -format

then we say
#start-all.sh

Now we start Hadoop daemon and we are done


No comments:

Post a Comment

Featured Post

Amazon Route 53

Amazon Route 53 is a highly available and scalable Domain Name System (DNS) web service.Route 53  perform three main functions in any...