TEAM TASK NO 1

!! ?????????? ?????????????????????? !!

?? TASK DESCRIPTION:

?? Whenever the client uploads the file ( for ex - f.txt) of size 50MB and the replication is 3.

?? Does the client takes the entire data to the master or does the master provides the IP addresses of Datanodes so that client can upload the file to the Datanodes?

?? Question: Whom is the one uploading the file?

?? Answer: The client gets the IP from the Master and uploads the file to DataNode.

SOLUTION :

First, we have to go inside /etc/Hadoop directory to configure Hadoop configuration files.

Here in the slave hdfs-site.xml file, we have to put our dn directory to share the storage with the master.

For master, we have configured the same file also.

Now we configure core-site.xml in all nodes including the master, client, and slave.

In slave and client, give master public IP to do configuration.

In the master node, give 0.0.0.0 to allow all nodes within the cluster to contact the master.

Now we have to format the name node directory.

Then we have to start the namenode.

After starting the name node now we have to start the datanode also

From the client, the file is uploading.

Then we can run tcpdump command on all the nodes as well as in the client to capture all the packets.

In the client packets file, Here client first connects to the master to know which slave is available to use as the master has all slave nodes metadata.

Now the client can directly connect to slave 1 to upload the first block.

Again the client connects to slave 1 to upload the second block.

For the final block, once again client contact slave 1.

In the slave 1 packets file, slave 1 is first contacted by the client, and the file is uploaded in three blocks in slave 1 through the client.

Then slave 1 connect slave 2 to create a replica.

In slave 2 packets file, as shown slave 2 is contacted by slave 1, and a replica is created here.

Then slave 2 create another replica in slave 3.

Slave 3 gets data from slave 2 to upload and hence the third replica is also created in slave 3.

Here we successfully completed our Team_Task_No_1

Thank You For Reading !!!