Task 7.1

?? 7.1: Elasticity Task

A-Integrating LVM with Hadoop and providing Elasticity to DataNode Storage.

Add a hard disk:

Create a new hard disk:

BY dynamically fixing the size of your choice create your hard disk thereafter:

New hard disk thus created.

Now start your virtual machine and in the terminal procced with the following steps:

To see the virtual hard disk created

fdisk -l

Next step is to create a partition:

1. fdisk /dev/sdb
2. press n for creating new partition.
3. press p for primary
4. give size

and then press w to save.

After creating a partition we have to provide it with a driver by:

udevadm settle

And then we have to format partition that we created using ext4. extension type by:

mkfs.ext4 /dev/sdb. ( In mine case i have already completed these steps )

Now for mounting we first have to check that to which folder we have our data node connected in the above screenshot you can see that it is connected via /dn1 folder.

Therefore we have to mount the formatted partition that we have created with partition name along with this particular folder.

In mine case i have already mounted it before so its showing output like the above image.

To check the whether the data node is sharing the same size of partition or not we can go to namenode vm and there by starting the name node using:

hadoop-daemon.sh start namenode

and thereby checking the datanode size sharing report using:

hadoop dfsadmin -report

You will see that data node is sharing the the same size of partition that has been linked with dn1 folder.

Therefore we have successfully provided limited storage to namenode via concept called elasticity.

B-Increase or Decrease the Size of Static Partition in Linux.

All the steps till creating a partition is same as above in this section therefore i will starting from next step that is the step after we created a partition for our virtual harddisc.

FOR a reference i m inputing some data in dn1 folder that we have in above case so that we can check later on if the data is still there even after creating a different partition:

since in above case we have dn1 as our folder name therefore after creating a partition we have to unmount it using:

umount /dn1

Now in our next step we have to delete the created partition and create a new partition and then provide it with a different size:

for new partition of size 5 gb write:

after this step we have to remove the former meta information that was present in the inode table using:

e2fsck -f /dev/sdb1

our next step is to resize our recreated partition in such a way that it wont lose the information it earlier has in folder dn1 by anuj.txt and hi.txt name and this can be done by using the command :

resize2fs /dev/sdb1

The last and final step is to again mount it to same folder you unmount earlier using

mount /dev/sdb1 /dn1

And if you check here the file you created earlier you will find it here with same data you stored i.e in cat anuj.txt and hi.txt you will get your same output as you stored in there.

Therefore in this way we decreased the size of our partition without actually losing our data that we stored.

Thank You