datanode同namenode之间的几个心跳

heartBeat : dfs.heartbeat.interval 默认是3秒

blockreport:dfs.blockreport.intervalMsec 默认是6小时,第一次延迟时间dfs.blockreport.initialDelay 默认是0

cachereport:dfs.cachereport.intervalMsec 默认是10秒
触发是:org.apache.hadoop.hdfs.server.datanode.BPServiceActor#offerService 中
基本是按照时间间隔来判断是否需要上报一次cache

lifeline:dfs.datanode.lifeline.interval.seconds 默认是3*dfs.heartbeat.interval

badreport: 在写数据时候,发现由坏块,就上报 org.apache.hadoop.hdfs.server.datanode.DataNode#reportRemoteBadBlock

IBR report : 增量数据上报间隔
dfs.blockreport.incremental.intervalMsec 默认是0 具体是在 org.apache.hadoop.hdfs.server.datanode.IncrementalBlockReportManager 中管理的

heartBeat / blockReport / cacheReport 都会带回namenode中都命令
对应的处理类都在 org.apache.hadoop.hdfs.server.datanode.BPServiceActor#offerService 中

heartBeat


long startProcessCommands = monotonicNow();
if (!processCommand(resp.getCommands()))//处理命令
continue;

blockReport


if ((fullBlockReportLeaseId != 0) || forceFullBr) {
cmds = blockReport(fullBlockReportLeaseId);
fullBlockReportLeaseId = 0;
}
processCommand(cmds == null ? null : cmds.toArray(new DatanodeCommand[cmds.size()]));//处理命令

cacheReport


if (!dn.areCacheReportsDisabledForTests()) {
DatanodeCommand cmd = cacheReport();
processCommand(new DatanodeCommand[]{ cmd });//处理带回来的命令
}

badreport


public void reportRemoteBadBlock(DatanodeInfo srcDataNode, ExtendedBlock block)
throws IOException {
BPOfferService bpos = getBPOSForBlock(block);
bpos.reportRemoteBadBlock(srcDataNode, block);
}

 

参数设置后的日志

hadoop-hdfs-datanode-jssz-bigdata-datanode-200.log.9:2019-03-31 22:35:51,047 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: For namenode jssz-bigdata-namenode-01/10.69.8.34:9010 using BLOCKREPORT_INTERVAL of 21600000msec CACHEREPORT_INTERVAL of 10000msec Initial delay: 0msec; heartBeatInterval=3000; lifelineIntervalMs=9000
发表评论?

0 条评论。

发表评论


注意 - 你可以用以下 HTML tags and attributes:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>