How to Install Hadoop on EC2 Instance ?

We are going to install Hadoop on ec2 instance. We will be working here with the Amazon AWS system. It offers a fairly decent free tier service to set this up and the most important word is free doesn’t cost anything to get us going.

Step By Step Installation of Hadoop on EC2 Instance

STEP 1: On our EC2 dashboard we are working off the eastern US server set and we have 0 running instances 0 dedicated host 0 volume 0 key pairs and 0 placements, the only thing that we have is a security group and that’s the default security group.

STEP 2: So getting started here we’re going to click on the launch instance button now this is going to bring us to a page where we can set up our instance now we’re going to be setting up a four node system.

STEP 3: One name node and 3 data nodes from here we’re going to click on the free tier only we don’t want to be picking anything that’s not free from here. We have an Ubuntu server and we select that one as our choice here we’re going to be using the free tier system which is a t2 micro, it works well and there’s no problems with speed.

STEP 4: Then we click configure instance, so from here we’re going to select four instances that means it’s going to be created for four separate virtual machines for us. Don’t worry about the messages for auto-scaling we’re not concerned about that this is not a production environment either so there’s a lot of things that we are going to be setting up that are just for learning purposes.

STEP 5: Next we click on add storage we are looking at 8 gigabytes and that should be enough for our system to get started click Next tag instance now this gives us our each instance a name and because we’re creating for all at once it’s going to give them all the same name so it will all be node.

STEP 6: Now we configure the security group. We can create a new security group and what we’re going to do is we’re going to call this one open. And this is just open to every one simply from here we want all traffic and the type is all traffic you’ll see it sets all the defaults here in other words anybody that is working with our system they can access the system so you can access this system from outside as well.

STEP 7: We’ll go to review and launch now that we’ve set that up okay you’ll see here here are all the details we’ve set up there is there’s nothing that we really need to know here but I will go through it here is starting off with instance details quite simply from this we got our route our name is called node .

STEP 8: let’s move along click Launch okay now it’s asking us to create key pairs now we’re going to be working on a public and private key store so that we can access our servers using password lists SSH and which we mean we basically need a key to access that so here what we’re going to do is we’re going to create a key. Then go to click download key pair so what that will do and you’ll see it come out in the bottom. 

STEP 9: Then it says launch instances so from here I click launch instances now by the way guys this PEM file is going to be used later on so we will get back to it but right now we’re just going to click on launched instances so I can show you what happens. 

Install Hadoop on EC2 Instance with 4 Cluster

So once we’ve gone through this process it’s creating all of our servers for us tells us everything looks like it succeeded so why don’t we view instances and we’ll see what happened here. 

Once these things complete there refresh process we will get moving so it’s just saying it’s pending actually we can probably just go ahead and rename these so I’m going to name this I’m going to give it a node here okay this one is called name node next one we’re going to call this data node one okay and data node two and data node three. Now what we have is our four node cluster.

Leave a Comment