NameNode Server in HDFS

NameNode Server in HDFS

The main node in HDFS is that it maintains and manages the blocks on the DataNodes. NameNode is a very high-availability server that manages the file namespace and controls user access to files. The HDFS architecture is such that user data is not mounted on NameNode. These data are only mounted on NameNodes.

NameNode's mission are as follows:

  • A major process that maintains and manages DataNodes.
  • It records the metadata of all files stored in the cluster (for example, the location of stored blocks, file sizes, permissions, hierarchies, etc.). Two files associated with metadata are:
  1. FsImage: includes the full status of the file system namespace since the start of the NameNode.
  2. EditLogs: Includes all recent changes to the latest FsImage.
  • Any changes that occur on the file system metadata are logged. For example, if a file is deleted in HDFS, NameNode immediately logs it in EditLog.
  • It regularly receives heart beats and block's report from all DataNodes to ensure their viability.
  • Holds a record of all blocks in the HDFS, and there are nodes in which the blocks are located.
  • It is also responsible for taking care of all factors of the replication of all blocks.
  • In the event of a DataNode failure, it selects a new DataNode for duplication, use of disk balancing and traffic control for DataNode.


要查看或添加评论,请登录

Babak Rezaei Bastani的更多文章

  • HDFS Architecture (Basic concepts)

    HDFS Architecture (Basic concepts)

    HDFS is a blocked file system in which each file is split into blocks of predefined size. These blocks are stored in…

  • What is MapReduce?

    What is MapReduce?

    MapReduce is a processing method and a Java-based distribution model for distributed computing. The MapReduce algorithm…

  • HDFS goals

    HDFS goals

    Fault detection and recovery : Because HDFS contains a large number of commodity hardware, the probability of failure…

  • An overview of HDFS

    An overview of HDFS

    The Hadoop file system was developed using distributed file system design and runs on commodity hardware. Unlike other…

  • Introduction to Hadoop

    Introduction to Hadoop

    Hadoop is an apache-based open source framework written in Java programming language, which allows simple…

  • Data Science Processing Tools

    Data Science Processing Tools

    Once learned with data storage, you need to be familiar with data processing tools for converting data lakes to data…

  • Data Warehouse Bus Matrix

    Data Warehouse Bus Matrix

    The Enterprise Bus Matrix is a data warehouse planning tool developed by Ralph Kimball and is being used by numerous…

  • Data vault

    Data vault

    Data vault modeling, designed by Dan Linstedt, is a database modeling method that has been deliberately structured in…

  • Data Lake

    Data Lake

    A Data lake is a data storage tank for a large amount of raw data. Waiting for future needs, the data lake saves the…

  • Data Science Storage Tools

    Data Science Storage Tools

    The data science ecosystem has a set of tools that we use to build our solutions. The capabilities of this environment…

社区洞察

其他会员也浏览了