Logical Volume Management
Logical Volume Management:
Consider we are having two harddisk with size 10 GB and 20 GB, suppose we are having an scenario where we need to store a single file of 25 GB, can we store it? Answer yes, the solution is we can combine these harddisk. Then they wil be considered as single device. This concept is known as LVM (Logical Volume Management).
Lets start creating it....
Steps to be followed to create LVM :
- First make Physical Volume (PV) with the harddisks.
- Create a group of these physical volumes which is called as Volume Group (VG).
- Now we have to make partition using LV (Logical Volume).
- Then format it
- Finally mount.
Physical Volume :
I have two harddisks sdc, sdd of size 15 GiB, 10 GiB respectively.
First step is to make them as Physical Volume (PV) command used to create pv is pvcreate /disk i.e) pvcreate dev/sdc, pvcreate /dev/sdd.
We can check the details of Physical Volume by pvdisplay /disk command. Here it shows it is not allocated to any VG (Volume Group).
Volume Group :
Now we have to create a VG (Volume Group) with those PVs created about. Command: vgcreate <vg-name> <PV> and use vgdisplay <vg-name> to show the details.
Current PV is now 2 and VG size is approximately equal to 25 GiB (15+10).
Logical Volume :
Now we have to create LV(Logical Volume) of our desired size. Lets create a LV of 20 GiB size. Command : lvcreate --size <value> --name <lv-name> <vg-name>.
lvdisplay <lv path> to display details of the particular LV. Now we can see that a LV of size 20 GiB is created.
Format :
We have successfully created partition, the next step we have to do is formatting the partition. I have used ext4 file system for formatting.
Mount:
The last step we have to do is mounting the lv. Mount it on existing directory or create a new one and then mount it. command : mount /lv /directory.
Use df -h to see the report on disk space usage.
Finally LV is created Successfully.
Extending the size of LV:
We have created a LV os size 20 GiB, can we extend the size to 23 GiB? answer is yes because our VG has 5 GiB more space. For extending the LV we need to follow 2 steps
Steps:
- Extend the Size by lvextend.
- Format the extended part.
1. Lvextend :
command to extend the partition is lvextend --size +<value>G /lv
Now use the lvdisplay command to see the details. Here is it showing our LV size as 23 GiB.
Now lets use dh -f to see the disk usage. Why it is showing 20 GiB instead of 23GiB? It is because only the 20 GiB is formatted the extended partition 3 GiB is not formatted yet.
2. Format :
Now we have to use resize2fs to format the remaining part. Note : It will only format the part that is not formatted.
Now use df -h to see the details. Successfully our LV extended from 20 GiB to 23 GiB
Reducing the LV size:
We have successfully extended the LV size. Now lets try to reduce the size. Foe extending we need to follow 2 steps and for reducing we need to follow 5 steps.
Steps:
- Unmount
- Scan and clean
- Change the format size
- Reduce the partition
- Mount
1. Unmount and clean :
First step to reduce the LV size we need to unmont it and make it offline. Then it is good to clean the wanted data in Inode table. For this we are going to use e2fsck (examine errors and clean).
2. Change the format size :
We have already formatted 23 GiB, now we only to need Inode table for say 16 Gib and not 23 GiB. So use resize2fs to change the format size to 16 GiB. Now use lvdisplay command to see the details of lv. Technically we only changed the format size, but not removed the partition. Therefore 16 GiB it is not reflected. Still it shows 23 GiB.
3. Lvreduce :
Now we are going to reduce the partition using lvreduce. Command : lvreduce --size <value>G /lv
Now use lvdisplay command to see the change.
4. Mount
Finally mount it and use df -h to see the details. Successfully reduced the size of LV.
AUTOMATING LVM USING PYTHON SCRIPT:
Providing Elasticity to data node storage by LVM :
Before Creating LV and providing it to datanode directory. It takes whole space from / drive. Hadoop Master report :
For this we have created LV of name hadooplvm from VG task7 with the above steps.
Directory of Data node in our case is /datanode. Mount the LV to this directory.
Report in master node :
Finally we have provided elasticity to Data Node storage by LVM.
Increasing the size using Python Script :
Decrease the size by python script:
