Install Hadoop and Hbase on AWS(Amazon Web Services) #1

1. Sign up to AWS http://aws.amazon.com (1 year free for now: Dec 09 2014) and choose EC2

스크린샷 2014-12-09 오전 9.50.16

스크린샷 2014-12-09 오전 9.50.45

스크린샷 2014-12-09 오전 9.51.09

스크린샷 2014-12-09 오전 9.51.21

2. Choose a server.

I’am selecting Ubuntu.

스크린샷 2014-12-09 오전 9.51.49

3. Choose an instance type.

스크린샷 2014-12-09 오전 9.52.01

4. Configure number of instances.

Change the number of instances to 4 and everything else will be default.

스크린샷 2014-12-09 오전 10.18.38

5. Add Storage.

Just let it be!

스크린샷 2014-12-09 오전 10.21.07

6. Give the instance name and description.

스크린샷 2014-12-09 오전 10.24.08

7. Define a Security Group.

스크린샷 2014-12-09 오전 10.25.26

8. Review Instance Launch.

Lets just launch. Once you click “Launch” 4 instance should be launched with “pending” state.

스크린샷 2014-12-09 오전 10.26.34

9. Create Security Pair.

스크린샷 2014-12-09 오전 10.28.48

10. Launching Instances and give names.

스크린샷 2014-12-09 오전 10.29.34

스크린샷 2014-12-09 오전 10.31.11

Once in “running” state we can give names like below.

HadoopNameNode (Master)
HadoopSecondaryNameNode
HadoopSlave1 (data node will reside here)
HaddopSlave2 (data node will reside here)

스크린샷 2014-12-09 오전 11.35.21

11. Let’s connect to the master server.

First, check the permission of the ‘pem’ file you received on ‘9. Create Security Pair.’ stage. You need to change permission to 600.

sudo chmod 600 ~/.ssh/name.pem

or you gonna see this error message.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

I’m using mac. To connect to the server.

ssh -l ubuntu -i ~/.ssh/name.pem YOUR_PUBLIC_SERVER_ADDRESS

Got it!

ubuntu@ip-xxx-xxx-xxx-xxx:~$ ls -al
total 28
drwxr-xr-x 4 ubuntu ubuntu 4096 Dec  9 10:08 .
drwxr-xr-x 3 root   root   4096 Dec  9 09:29 ..
-rw-r--r-- 1 ubuntu ubuntu  220 Apr  9  2014 .bash_logout
-rw-r--r-- 1 ubuntu ubuntu 3637 Apr  9  2014 .bashrc
drwx------ 2 ubuntu ubuntu 4096 Dec  9 10:08 .cache
-rw-r--r-- 1 ubuntu ubuntu  675 Apr  9  2014 .profile
drwx------ 2 ubuntu ubuntu 4096 Dec  9 09:29 .ssh
ubuntu@ip-xxx-xxx-xxx-xxx:~$ 

refer
http://java.dzone.com/articles/how-set-multi-node-hadoop
http://stackoverflow.com/questions/8193768/trying-to-ssh-into-an-amazon-ec2-instance-permission-error
http://stackoverflow.com/questions/201893/warning-unprotected-private-key-file-when-trying-to-ssh-into-amazon-ec2-instan

Published by

Raphael, Eom

Working these days with Meteor, Redux, MongoDB email: gblue1223@gmail.com

Leave a comment