How to contribute a limited/specific amount of storage as a slave to the cluster?

Hola Connections!

Whenever the Datanode shares the storage space to Master Node, it always shares the whole storage by default.

Ever got stuck up with the thought of how could you share limited / specific amount storage for example 10 GB from 20GB ? If yes, I am here to answer you how !

And if not, let us know how it could be useful ;)

• Imagine the situation where you have shared the whole storage of Datanode . Now your Datanode capacity has got full and it's hung up! Or just imagine the system crashed or the single system is overloaded !
• Or Just in case you don't wish to share the whole storage to Master !

Now, let us look at how to share specific amount of storage:

• This is the Datanode holding 10GB storage space

When storage is shared to the Namenode it shares whole storage like this:

Now I wish to Attach 5 GB of EBS storage externally to my instance and share only 2GB of it. So, I have attached volume as follows:

If you are unaware about how to create volume and attach it you can refer to this article of mine → Create and attach volume

Now we need to create partition. To do so we follow 4 steps:

Step 1: Confirm about how much partition we need and create its physical partition.

→ Here we need 2GB

Step 2: Load the driver

→ Command: udevadm settle

Step 3 :Format the Partition

→ Command: mkfs.ext4 disk_path

Step 4: Mount the partition to directory (To activate)

→ Command: mouth disk_path directory_path

Now as we have successfully created the partition of 2GB and mounted it to /limitshare, we need to edit our hdfs-site.xml with directory path as follows.

Now you can see the limited amount of storage is successfully shared.

Hoping of this was helpful....Thank you !