Configure Hadoop and start cluster
services using Ansible Playbook, TASK-11 #Arth.

4 min readJan 3, 2021

🔰 11.1 Configure Hadoop and start cluster
services using Ansible Playbook

🔰 11.2 Create a Article, blog or Video on how
industries are solving challenges using Ansible.

🔰 11.3 Restarting HTTPD Service is not
idempotence in nature and also consume more
resources suggest a way to rectify this challenge
in Ansible playbook

Configuring up of Hadoop:

You need to have the jdk-version 8u121 or jdk-version 8u225 rpm. files and now the Hadoop suitable to that particular java version is installed.
Before installing Hadoop install java-jdk
Update the 2important files:

hdfs-site.xml . and . core-site.xml

4. After copying the desired files based on the requirement whether it is Name node or data node

Name-Node.yml:

- hosts: namenode
vars:
- direc_nn: /root/task11

tasks:
— name: Copying the hadoop .rpm files into the Target node
copy:
src: /root/ansible/hadoop-1.2.1–1.x86_64.rpm
dest: /root/

— name: Importing the hadoop dependency java JDK
copy:
src: /root/ansible/jdk-8u171-linux-x64.rpm
dest: /root/
— name: Installing the JDK
package:

name: jdk-8u171-linux-x64.rpm
disable_gpg_check: yes
# shell: rpm -ivh jdk-8u171-linux-x64.rpm — nobest
- name: Installing Hadoop
shell: rpm -ivh hadoop-1.2.1–1.x86_64.rpm — force
ignore_errors: yes

— name: we are going to Share the HDFS-site.xml file by using the variables
template:
src: /root/ansible/name-hdfs.txt
dest: /etc/hadoop/hdfs-site.xml

— name: Sending Core-site.xml file to the respective directory.
template:
src: /root/ansible/name-core.txt
dest: /etc/hadoop/core-site.xml
— name: Create directory to share over hdfs network
file:
path: “{{direc_nn}}”
state: directory

— name: Format the name node to start it
shell: echo Y | hadoop namenode -format
— name: Starting the Master node
shell: hadoop-daemon.sh start namenode
— name: Status of about Master node start
debug:
msg: “Name Node just Started if it appears in Green or yellow”

/etc/hadoop/hdfs-site.xml :

<configuration>
<property>
<name>dfs.name.dir</name>
<value>hdfs://{{ansible_facts[“default_ipv4”][“address”]}}:9001</value>
</property>
</configuration>

/etc/hadoop/core-site.xml:

<configuration>
<property>
<name>dfs.name.dir</name>
<value>hdfs://{{ansible_facts[“default_ipv4”][“address”]}}:9001</value>
</property>
</configuration>

Inventory (/root/ip.txt):

[namenode]
192.168.0.8 ansible_ssh_user=root ansible_ssh_pass=redhat123
[datanode]
192.168.0.7 ansible_ssh_user=root ansible_ssh_password=redhat123

Configuration file(/etc/ansible/ansible.cfg):

[defaults]
inventory = /root/ansible/ip.txt
host_key_checking = false

****************************************************************
Now we can configure the Data node similar to Name node but the core-site.xml, hdfs-site.xml
****************************************************************

Hdfs-site.xml:

<name>dfs.data.dir</name>
<value>hdfs://{{groups[‘namenode’][0]}}:9001</value>

</property>
</configuration>

Core-site.xml:

<configuration>
<property>
<name>fs.default.data</name>
<value>{{direc_nn}}</value>
</property>
</configuration>

#Now as per the datanode we don’t need to do the format same as the namenode.

#We can directly start the hadoop start.

Finally hadoop cluster is setup.

Github Url:
https://github.com/KVSSKPRADEEP/ArthTask7-_much_more/tree/main/Task11

Task 11.3 :

Restarting HTTPD Service is not
idempotence in nature and also consume more
resources suggest a way to rectify this challenge
in Ansible playbook

Solution:

According to the previous uses we have started using the in built services of ansible to start or stop the services. So we have the HANDLERS which are used to check the state of service and perform the operation based on the necessity of the DevOps admin operating the ansible tower or in this case ansible engine.