Skip to main content

Big Data- Installing Hadoop ( Single Node)

So finally we get to do Hadoop installation for single node.

Pre-Installation System:

-SSH server: Master communicate with the slaves with the SSH. So we need to get SSH server running. We need to generate public/private key pair. Than we take that key, that we generate on the master and send it to all our nodes.

-Java 6 or greater

Hadoop Installation:
-Download and install hadoop
-add paths to bashrc
-Configure hadoop environment
(> java.home/disable ipv6
-Configure.XML ( core -site ! mapred-site)
-Launch Hadoop Daemons

Now i will show you installation

Lets do SSH to the name node

lets start going with our prerequisites. The first prerequisite is to generate public-private key.

No problem, we just need to type the command 

#SSH Keygen

now it will ask for password. we are keeping the password blank

There we go, we have our key generated.

Now lets get JAVA installed.

We need to type the command
# sudo apt-get install openjdk-7-jdk

It will ask that do you want to download 218MB package, just say yes. So Java Installed !!

Now we shift focus on Hadoop.

First we download the hadoop from the website. We can download using command line or browser. Once its downloaded, we move to next course of action that to unzip the .tar file

#tar -zxvf hadoop-1.2.0-bin.tar.gz

now we copy this to user local area and name it hadoop

next is to add paths to bashrc and configure
#Sudo vi $HOME/.bashrc
go to the bottom of the file and insert the following entries and save

then run the command
#exec bash
This will run the command

now to configure the We type the command

#sudo vi /usr/local/hadoop/conf/

we go to 7th line from the top and remove the #

and edit the line to replace with our type of java

and disable IPV6 on the same file

Now we have to configure those XML files. 

#Sudo vi /usr/local/hadoop/conf/core-site.xml

#Sudo vi /usr/local/hadoop/conf/mapred-site.xml

this is the last step.

#hadoop namenode -format

then we say

Now we start Hadoop daemon and we are done


Popular posts from this blog

Data Center Migration

Note: This blog is written with the help of my friend Rajanikanth
Data Center Migrations / Data Center Consolidations
Data Center Consolidations, Migrations are complex projects which impact entire orgnization they support. They usually dont happen daily but once in a decade or two. It is imperative to plan carefully, leverage technology improvements, virtualization, optimizations.
The single most important factor for any migration project is to have high caliber, high performing, experienced technical team in place. You are migrating business applications from one data center to another and there is no scope for failure or broken application during migration. So testing startegy should be in place for enterprise business applications to be migrated.
Typical DCC and Migrations business objectives
Business Drivers
·Improve utilization of IT assets ·DC space & power peaked out - business growth impacted ·Improve service levels and responsiveness to new applications ·Reduce support complexi…

HP CSA Implementation

I know the above picture is little confusing but don’t worry I break it down and explain in detail. By the time I am done explaining you all will be happy. HARDWARE AND SOFTWARE REQUIREMENTS 1.VMware vSphere infrastructure / Microsoft Hyper V: For the sake of Simplicity we will use VMware vSphere. We Need vSphere 4.0 /5/5.5 and above and vCenter 4.0 and above ready and installed. This is the first step. 2.We need Software medias for HP Cloud Service Automation, 2.00, HP Server Automation, 9.02, HP Operations Orchestration (OO)9.00.04, HP Universal CMDB 9.00.02, HP Software Site Scope, 11.01,HP Insight Software6.2 Update 1 3.DNS, DHCP and NTP systems are already installed and configured. NTP information should be part of VM templates 4.SQL Server 2005 or Microsoft® SQL Server 2008 or Microsoft® SQL Server 2012 , Oracle 11g, both 32-bit and 64-bit versions may be used for CSA database.
5.We will install  HP Cloud Service Automation, 2.00, HP Server Automation, 9.02, HP Operations Orchestra…

Openstack- Its importance in Cloud. The HP Helion Boost

Every enterprise expects few things from cloud computing, mainly:

· Auto scaling: The workload should increase and decrease as needed by the IT environment.

· Automatic repair: If there is any fault or crash of the application or the server, it automatically fix it

· Fault tolerant: The application or underlying technology is intelligent enough to make itself fault torrent

· Integrated lifecycle: It should have integrated lifecycle

· Unified management: Its easy to manage all different aspects of technology

· Less cost

· Speed

Its year 2014. till now only 5% to 7% enterprises are using cloud computing. Such a small number. Its a huge opportunity and a vast majority for anyone who is interested in providing cloud computing services.
Current IT environment is very complex. You just cant solve all your problems with cloud computing.
There are legacy systems, databases, data processors, different hardware and software. You name it , there are so many technology available in just o…