Я выполняю свою работу Spark, используя livy, однако я получаю исключение ниже
java.util.concurrent.ExecutionException: java.io.IOException: Internal Server Error: "java.util.concurrent.ExecutionException: org.apache.livy.rsc.rpc.RpcException: java.util.NoSuchElementException: cd1299a0-9c19-4db2-b81b-deba9bf5a594\norg.apache.livy.rsc.driver.RSCDriver.handle(RSCDriver.java:454)\nsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\nsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\njava.lang.reflect.Method.invoke(Method.java:497)\norg.apache.livy.rsc.rpc.RpcDispatcher.handleCall(RpcDispatcher.java:130)\norg.apache.livy.rsc.rpc.RpcDispatcher.channelRead0(RpcDispatcher.java:77)\nio.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)\nio.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)\nio.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)\nio.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)\nio.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)\nio.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)\nio.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:244)\nio.netty.handler.codec.ByteToMessageCodec.channelRead(ByteToMessageCodec.java:103)\nio.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)\nio.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)\nio.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:846)\nio.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)\nio.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)\nio.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)\nio.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)\nio.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)\nio.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)\njava.lang.Thread.run(Thread.java:745)"
глядя на журналы заданий Spark, в них нет ошибок или исключений, но он показывает
В журналах искры я не вижу никаких исключений, но вижу журналы ниже:
17/11/14 14:45:49 WARN NettyRpcEnv: RpcEnv already stopped.
17/11/14 14:45:49 INFO YarnAllocator: Completed container container_e60_1510219626098_0394_01_000013 on host: AUPER01-02-10-12-0.prod.vroc.com.au (state: COMPLETE, exit status: 0)
17/11/14 14:45:49 INFO YarnAllocator: Executor for container container_e60_1510219626098_0394_01_000013 exited because of a YARN event (e.g., pre-emption) and not because of an error in the running job.
17/11/14 14:45:49 WARN NettyRpcEnv: RpcEnv already stopped.
17/11/14 14:45:49 INFO YarnAllocator: Completed container container_e60_1510219626098_0394_01_000011 on host: AUPER01-01-10-13-0.prod.vroc.com.au (state: COMPLETE, exit status: 0)
17/11/14 14:45:49 INFO YarnAllocator: Executor for container container_e60_1510219626098_0394_01_000011 exited because of a YARN event (e.g., pre-emption) and not because of an error in the running job.
17/11/14 14:45:49 WARN NettyRpcEnv: RpcEnv already stopped.
17/11/14 14:45:49 INFO YarnAllocator: Completed container container_e60_1510219626098_0394_01_000005 on host: AUPER01-01-20-08-0.prod.vroc.com.au (state: COMPLETE, exit status: 0)
17/11/14 14:45:49 INFO YarnAllocator: Executor for container container_e60_1510219626098_0394_01_000005 exited because of a YARN event (e.g., pre-emption) and not because of an error in the running job.
17/11/14 14:45:49 WARN NettyRpcEnv: RpcEnv already stopped.
17/11/14 14:45:49 INFO YarnAllocator: Completed container container_e60_1510219626098_0394_01_000008 on host: AUPER01-02-30-12-1.prod.vroc.com.au (state: COMPLETE, exit status: 0)
17/11/14 14:45:49 INFO YarnAllocator: Executor for container container_e60_1510219626098_0394_01_000008 exited because of a YARN event (e.g., pre-emption) and not because of an error in the running job.
Я запускаю задания Spark 1.6.3 на HDP 2.6.3
