Tuesday 22 March 2022

What is block reporting in Hadoop hdfs?

A block report contains a metadata of data blocks that a Data node holds. Data nodes in a Hadoop cluster periodically sends this block report to the Namenode.

 

Why this block report is important?

Block report is important to make the system fault tolerant.

 

Let me explain it with one example. As you see the below image, data block ‘B1’ is replicated to 3 data nodes. Every data node periodically sends their block reports to Name node.

 

After certain time, Data Node 3 of Rack2 observed that the block ‘B1’ is corrupted.




When Name node receives the block report of Data Node 3 of Rack 2, name node immediately take action.

a.   By replicating the Block B1 to other data node.

b.   Delete the corrupted block from data node and

c.    Delete the corrupted block metadata from namenode.

 

 


a

Previous                                                 Next                                                 Home

No comments:

Post a Comment