мы настроили log4j службы диспетчера ресурсов в режиме DEBUG следующим образом:
yarn.server.resourcemanager.appsummary.logger=DEBUG,RMSUMMARY
а затем мы перезапускаем службу диспетчера ресурсов (из приборной панели Ambari)
из журналов мы по-прежнему видели уровень журнала как INFO вместо DEBUG
пример:
2021-07-02 06:00:53,620 INFO resourcemanager.ResourceTrackerService (ResourceTrackerService.java:registerNodeManager(458)) - NodeManager from node worker02.bd-gontamopigure-1.com(cmPort: 45454 httpPort: 8042) registered with capability: <memory:16384, vCores:3>, assigned nodeId worker02.bd-gontamopigure-1.com:45454
2021-07-02 06:00:53,620 INFO rmnode.RMNodeImpl (RMNodeImpl.java:handle(440)) - worker02.bd-gontamopigure-1.com:45454 Node Transitioned from NEW to RUNNING
2021-07-02 06:00:53,621 INFO capacity.CapacityScheduler (CapacityScheduler.java:addNode(1364)) - Added node worker02.bd-gontamopigure-1.com:45454 clusterResource: <memory:32768, vCores:6>
поэтому я делюсь конфигурацией log4j
любая идея, что не так с:
yarn.server.resourcemanager.appsummary.logger=DEBUG,RMSUMMARY
и почему мы получаем INFO несмотря на то, что мы настроили DEBUG
здесь вся конфигурация log4j
#Relative to Yarn Log Dir Prefix
yarn.log.dir=.
#
# Job Summary Appender
#
# Use following logger to send summary to separate file defined by
# hadoop.mapreduce.jobsummary.log.file rolled daily:
# hadoop.mapreduce.jobsummary.logger=INFO,JSA
#
hadoop.mapreduce.jobsummary.logger=${hadoop.root.logger}
hadoop.mapreduce.jobsummary.log.file=hadoop-mapreduce.jobsummary.log
log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
# Set the ResourceManager summary log filename
yarn.server.resourcemanager.appsummary.log.file=hadoop-mapreduce.jobsummary.log
# Set the ResourceManager summary log level and appender
yarn.server.resourcemanager.appsummary.logger=${hadoop.root.logger}
# To enable AppSummaryLogging for the RM,
# set yarn.server.resourcemanager.appsummary.logger to
# LEVEL,RMSUMMARY in hadoop-env.sh
yarn.server.resourcemanager.appsummary.logger=DEBUG,RMSUMMARY
# Appender for ResourceManager Application Summary Log
# Requires the following properties to be set
# - hadoop.log.dir (Hadoop Log directory)
# - yarn.server.resourcemanager.appsummary.log.file (resource manager app summary log filename)
# - yarn.server.resourcemanager.appsummary.logger (resource manager app summary log level and appender)
log4j.appender.RMSUMMARY=org.apache.log4j.RollingFileAppender
log4j.appender.RMSUMMARY.File=${yarn.log.dir}/${yarn.server.resourcemanager.appsummary.log.file}
log4j.appender.RMSUMMARY.MaxFileSize=256MB
log4j.appender.RMSUMMARY.MaxBackupIndex=20
log4j.appender.RMSUMMARY.layout=org.apache.log4j.PatternLayout
log4j.appender.RMSUMMARY.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
log4j.appender.JSA.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p %c{2}: %m%n
log4j.appender.JSA.DatePattern=.yyyy-MM-dd
log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary=${yarn.server.resourcemanager.appsummary.logger}
log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary=false
# Appender for viewing information for errors and warnings
yarn.ewma.cleanupInterval=300
yarn.ewma.messageAgeLimitSeconds=86400
yarn.ewma.maxUniqueMessages=250
log4j.appender.EWMA=org.apache.hadoop.yarn.util.Log4jWarningErrorMetricsAppender
log4j.appender.EWMA.cleanupInterval=${yarn.ewma.cleanupInterval}
log4j.appender.EWMA.messageAgeLimitSeconds=${yarn.ewma.messageAgeLimitSeconds}
log4j.appender.EWMA.maxUniqueMessages=${yarn.ewma.maxUniqueMessages}
# Audit logging for ResourceManager
rm.audit.logger=${hadoop.root.logger}
log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger=${rm.audit.logger}
log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger=false
log4j.appender.RMAUDIT=org.apache.log4j.DailyRollingFileAppender
log4j.appender.RMAUDIT.File=${yarn.log.dir}/rm-audit.log
log4j.appender.RMAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.RMAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
log4j.appender.RMAUDIT.DatePattern=.yyyy-MM-dd
# Audit logging for NodeManager
nm.audit.logger=${hadoop.root.logger}
log4j.logger.org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger=${nm.audit.logger}
log4j.additivity.org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger=false
log4j.appender.NMAUDIT=org.apache.log4j.DailyRollingFileAppender
log4j.appender.NMAUDIT.File=${yarn.log.dir}/nm-audit.log
log4j.appender.NMAUDIT.layout=org.apache.log4j.PatternLayout
log4j.appender.NMAUDIT.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
log4j.appender.NMAUDIT.DatePattern=.yyyy-MM-dd
