Integrating LVM with Hadoop

Hello everyone here is my new article on Integration of Logical

Volume management with the Storage of data node in Hadoop.

What is Logical Volume Management(LVM)?

In Linux, Logical Volume Manager is a device mapper framework that provides logical volume management for the Linux kernel. Most modern Linux distributions are LVM-aware to the point of being able to have their root file systems on a logical volume.

LVM Architecture

• To create an LVM logical volume, the physical volumes are combined into a volume group (VG). This creates a pool of disk space out of which LVM logical volumes (LVs) can be allocated. This process is analogous to the way in which disks are divided into partitions. A logical volume is used by file systems and applications (such as databases).

????Task Description????

?? 7.1: Elasticity Task

??Integrating LVM with Hadoop and providing Elasticity to DataNode Storage

??Increase or Decrease the Size of Static Partition in Linux.

• Let's Attach a volume to your instance.

• Now, let's check if the volume is attached or not :

• So we attached two volumes of 4 Gib and 2 Gib
• Now let's create the Physical volumes(PV) of both volumes we attached.

First one :

pvcreate /dev/sdd

pvdisplay /dev/sdd

Second one :

pvcreate /dev/sdd

pvdisplay /dev/sdd

• We have To create a Volume Group(VG) of both the physical volumes

vgcreate task7.1 /dev/sdc /dev/sdd

vgdisplay task7.1

• Let's create a logical volume(LV) of 3Gib from the above volume group.

lvcreate --size 3G --name mylv1 task7.1

lvdisplay task7.1/mylv1

??

After creating the Logical volume we can use it as storage for the Datanode.

• To use it as a Storage for DataNode follow the following steps.

Step 1:Create a folder in the root folder.

mkdir task7.1

Step 2: Mount the above folder to the logical volume.

mount /dev/task7.1/mylv1 task7.1

• Successfully Mounted

After mounting the folder configure it in hdfs-site.xml file.

• As we configured with 3GiB of the logical volume.

Let's increase the size of the logical volume :

• Step 1: Extend it

lvextend --size +2G /dev/tasl7.1/mylv1

• Step 2: Resize it

resize2fs /dev/task7.1/mylv1

??

• The storage is Extended successfully on the fly.

• Let's decrease the size of the logical volume :

lvreduce -r -L 1G /dev/task7.1/Mylv1

• -r option defines to resize it

????Note: In the above image we selected to unmount it but you can keep it live by just inserting the option "n" when they ask.

??My Great team members??

1.Shrikant Luharia

2.Pritee Dharme

3.Suyog Shinde

4.Suraj Kumar

5.Vikas Rawat

?Hope you enjoyed it ?.

?Thank you ?

?? Keep learning!! Keep Sharing !!??