Setting up an Cloudera Cluster on AWS EC2 - Part 2

Step #1 Cloudera Manager Installation 

Start NTPD services 
Note: Perform this step on all servers

service ntpd start


Install Cloudera manager
Navigate to m1 cloudera bin file and run below command to start the cluster installation

./cloudera-manager-installer.bin


click Next


click Next


Click on Yes to accept license


Note: Cloudera Manager Username and Password will be displayed as shown below


After successful installation, we need to setup Cloudera Manager console.

Step #2: Setting up cloudera manager console

Note: In web browser type m1 machines public IP address along with port 7180 to open cloudera manager console.


Accept the licensing terms and condition then click continue


Select Cloudera Enterprise Trial Edition and click Continue.


Search and add AWS Instances EC2 using Private DNS IP as shown below. Click Continue


Next step is to select repository for Cloudera Cluster Installation.

Choose method 
Use parcels
Select the version of CDH
CDH-5.8.0-1.cdh5.8.0.p0.42
Additional parcels
KAFKA-2.0.2-1.2.0.2.p0.5


Tick on check box to install Oracle Java Development Kit and click Continue



By default service processes runs as a distinct users on the system. click Continue


Import demopk.pem key and enter Username to install the cloudera packages on all host click Continue.


Cluster installation connects all three machines and brings in the packages related to java and cloudera manager agents.The packages and selected parcel are installed on all three machines. 
click on Continue


After successful cluster installation, selected parcels will be downloaded and installed on all the host in the cluster


Verify cluster installation. click Finish


Select Core Hadoop and click Continue.



Step #4: Customize and assigned the roles on the host 


HDFS


HDFS
Assigned Roles
Host
Name Node
ip-10-0-0-194.us-east-2.compute.internal
Secondary Node
ip-10-0-0-27.us-east-2.compute.internal
Balancer
ip-10-0-0-194.us-east-2.compute.internal
Data Node
All Host




Hive 


HIVE
Assigned Roles
Host
Gateways
All Host
Hive Megastore Server
ip-10-0-0-27.us-east-2.compute.internal
WebHCat Server
ip-10-0-0-27.us-east-2.compute.internal
HiveServer2
ip-10-0-0-27.us-east-2.compute.internal


Hue 


Hue
Assigned Roles
Host
Hue Server
ip-10-0-0-27.us-east-2.compute.internal
Load Balancer
ip-10-0-0-27.us-east-2.compute.internal


Cloud Management Service


Cloudera Management Service
Assigned Roles
Host
Service Monitor
ip-10-0-0-27.us-east-2.compute.internal
Activity Monitor
ip-10-0-0-27.us-east-2.compute.internal
Host Monitor
ip-10-0-0-27.us-east-2.compute.internal
Report Manger
ip-10-0-0-27.us-east-2.compute.internal
Event Server
ip-10-0-0-27.us-east-2.compute.internal


Oozie


Oozie
Assigned Roles
Host
Oozie Server
ip-10-0-0-27.us-east-2.compute.internal


YARN & Zookeeper


YARN (MR2 Included)
Assigned Roles
Host
Resource Manager
M2
JobHistory Server
M2
Node Manager
All Host
Zookeeper
Zookeeper Server
All Host

Keep default and click Continue



Below is a summary of roles distributed on each host



Keep default and click continue


Finally Cloudera server manager runs all the services in cluster



Cloudera Cluster setup has been configured successfully


No comments:

Post a Comment