SAP HANA VORA - проблема с контроллером искры

Я пытаюсь установить запуск контроллера SAP HANA Spark на VORA 1.2 с помощью Ambari.

Однако, когда я запускаю свой контроллер Spark, я получаю следующее исключение.

Пожалуйста, помогите здесь ...

[hanaes@ip-172-30-2-218 bin]$ ./hanaes start 
Starting HANA Spark Controller ...  
Class path is /usr/sap/spark/controller/bin/../conf:/usr/hdp/2.3.4.7-4/hadoop/conf:/etc/hive/conf:../*:../lib/*:../lib/external/*:/usr/hdp/2.3.4.7-4/hadoop/*:/usr/hdp/2.3.4.7-4/hadoop/lib/*:/usr/hdp/2.3.4.7-4/hadoop-hdfs/*:/usr/hdp/2.3.4.7-4/hadoop-hdfs/lib/* 
STARTED 
[hanaes@ip-172-30-2-218 bin]$ clear 
[hanaes@ip-172-30-2-218 bin]$ tail -1000f /var/log/hanaes/hana_controller.log 
SLF4J: Class path contains multiple SLF4J bindings. 
SLF4J: Found binding in [jar:file:/usr/sap/spark/controller/lib/external/spark-assembly-1.5.2.2.3.4.7-4-hadoop2.7.1.2.3.4.7-4.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
SLF4J: Found binding in [jar:file:/usr/hdp/2.3.4.7-4/hadoop/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] 
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. 
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] 
16/05/09 12:04:43 INFO HanaESConfig: Loaded HANA Extended Store Configuration Found Spark Libraries. Proceeding with Current Class Path 
16/05/09 12:04:44 INFO Server: Starting Spark Controller 
16/05/09 12:04:52 ERROR SparkContext: Error initializing SparkContext. org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.waitForApplication(YarnClientSchedulerBackend.scala:125)
        at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend.start(YarnClientSchedulerBackend.scala:65)
        at org.apache.spark.scheduler.TaskSchedulerImpl.start(TaskSchedulerImpl.scala:144)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:523)
        at com.sap.hana.spark.network.CommandRouter.initializeHanaContext(CommandRouter.scala:125)
        at com.sap.hana.spark.network.CommandRouter.<init>(CommandRouter.scala:38)
        at com.sap.hana.spark.network.Server$$anonfun$1.apply(Server.scala:96)
        at com.sap.hana.spark.network.Server$$anonfun$1.apply(Server.scala:96)
        at akka.actor.TypedCreatorFunctionConsumer.produce(Props.scala:343)
        at akka.actor.Props.newActor(Props.scala:252)
        at akka.actor.ActorCell.newActor(ActorCell.scala:552)
        at akka.actor.ActorCell.create(ActorCell.scala:578)
        at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
        at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
        at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
        at akka.dispatch.Mailbox.run(Mailbox.scala:219)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) 16/05/09 12:04:52 ERROR Utils: Uncaught exception in thread SAPHanaSpark-akka.actor.default-dispatcher-2 java.lang.NullPointerException
        at org.apache.spark.network.netty.NettyBlockTransferService.close(NettyBlockTransferService.scala:152)
        at org.apache.spark.storage.BlockManager.stop(BlockManager.scala:1228)
        at org.apache.spark.SparkEnv.stop(SparkEnv.scala:100)
        at org.apache.spark.SparkContext$$anonfun$stop$12.apply$mcV$sp(SparkContext.scala:1749)
        at org.apache.spark.util.Utils$.tryLogNonFatalError(Utils.scala:1185)
        at org.apache.spark.SparkContext.stop(SparkContext.scala:1748)
        at org.apache.spark.SparkContext.<init>(SparkContext.scala:593)
        at com.sap.hana.spark.network.CommandRouter.initializeHanaContext(CommandRouter.scala:125)
        at com.sap.hana.spark.network.CommandRouter.<init>(CommandRouter.scala:38)
        at com.sap.hana.spark.network.Server$$anonfun$1.apply(Server.scala:96)
        at com.sap.hana.spark.network.Server$$anonfun$1.apply(Server.scala:96)
        at akka.actor.TypedCreatorFunctionConsumer.produce(Props.scala:343)
        at akka.actor.Props.newActor(Props.scala:252)
        at akka.actor.ActorCell.newActor(ActorCell.scala:552)
        at akka.actor.ActorCell.create(ActorCell.scala:578)
        at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
        at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
        at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
        at akka.dispatch.Mailbox.run(Mailbox.scala:219)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:397)
        at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
        at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
        at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
        at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

person Anand Agrawal    schedule 09.05.2016    source источник


Ответы (1)


Журнал контроллера искры указывает на проблему с Yarn, вам необходимо проверить журнал Yarn, который отвечает за неудачное задание контроллера искры:

Ambari -> Пряжа -> Быстрые ссылки -> Пользовательский интерфейс диспетчера ресурсов -> найдите неудачное задание Spark Controller -> щелкните идентификатор приложения слева -> щелкните «журналы»

person Daniel Zheng    schedule 09.05.2016
comment
Я добавил этот довольно распространенный вопрос под номером 27 в at Блог по устранению неполадок Vora - person Frank Legler; 10.05.2016