CONFIGURE HADOOP BY ANSIBLE

🔰 11.1 Configure Hadoop and start cluster services using Ansible Playbook.

ANSIBLE

• Ansible is open-source software that is used for Configuration management, Provisioning, Orchestration, and many more.
• Ansible is agentless.
• Ansible runs on top of Python. (It is written in Python Language)
• Ansible uses YAML(Ain’t Markup Language) language for writing code.
• It is used to perform IT tasks.

Today I am using Ansible for Configuration of Hadoop.

Ansible will automatically setup Master Node, Slave Node, and Client.

— — — — — — — — — — — — — — — — — — — — — — — — -

CONFIGURE MASTER AND SLAVE NODE

Some setup is common between master, slave, and client.

• The first host will configure Master Node, Slave Node, and Client.
• This code will first search that JDK and Hadoop are already installed on nodes or not.
• The “ rpm “ command will search the software are present or not.
• “ stdout “ will show the software output.

• Copy the software to Master Node, Slave Node, and Client only if it is not present in the nodes.

• Now install tools on nodes only if it is not already installed.

CONFIGURE NAMENODE

• First, change the mode of SELinux to Permissive.
• Disable firewall.
• Copy hdfs-site.xml file to NameNode.
• Copy core-site.xml to Namenode.
• Format NameNode by non-interactive command.
• Start namenode service.
• Check the namenode service running or not by “jps” command.

CONFIGURE DATANODE

• First, change the mode of SELinux to Permissive.
• Disable firewall.
• Copy hdfs-site.xml file to DataNode.
• Copy core-site.xml to Datanode.
• Start Data node service.
• Check the Data node service running or not by “jps” command.

CONFIGURE CLIENT

• Ask for filename and path for uploading to cluster.
• First, change the mode of SELinux to Permissive.
• Disable firewall.
• Copy core-site.xml file to Client.
• Upload a file to Cluster.

RUN PLAYBOOK -

“ ansible-playbook -v file.yml ” is the command used to run Ansible Playbook.

CHECK NAMENODE AND DATANODE -

By using “ jps “ command check that NameNode and DataNode are running or not.

NAMENODE-

DATANODE -

• After running code the software are also copied to nodes.

NODES REPORT -

“ hadoop dfsadmin -report ” command is used to see that how many data nodes are connected to namenode.

CODE FOR core-site.xml FILE -

CODE FOR hdfs-site.xml FILE -

UPDATED core-site.xml -

DATANODE -

NAMENODE -

UPDATED hdfs-site.xml -

DATANODE -

NAMENODE -

INVENTORY -

SEE UPLOADED FILE -

BROWSER-

THANK YOU FOR READING !!

Github Link — https://github.com/manali1230/Hadoop-Setup-by-Ansible.git