Pig和Hadoop连接错误

艾曼

当我在mapreduce模式下运行Pig时,出现ConnectionRefused错误。

详细信息:
我已经从tarball(pig-0.14)安装了Pig,并在bashrc中导出了类路径。
我已经启动并运行了所有Hadoop(hadoop-2.5)守护程序(由JPS确认)。

[root@localhost sbin]# jps
2272 Jps
2130 DataNode
2022 NameNode
2073 SecondaryNameNode
2238 NodeManager
2190 ResourceManager

我在mapreduce模式下运行Pig:

[root@localhost sbin]# pig
grunt> file = LOAD '/input/pig_input.csv' USING PigStorage(',') AS (col1,col2,col3);
grunt> dump file;  

然后我得到了错误:

java.io.IOException: java.net.ConnectException: Call From localhost.localdomain/127.0.0.1 to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
    at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:334)
    at org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:419)
    at org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:532)
    at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:183)
    at org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.getTaskReports(HadoopShims.java:231)
    at org.apache.pig.tools.pigstats.mapreduce.MRJobStats.addMapReduceStatistics(MRJobStats.java:352)
    at org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil.addSuccessJobStats(MRPigStatsUtil.java:233)
    at org.apache.pig.tools.pigstats.mapreduce.MRPigStatsUtil.accumulateStats(MRPigStatsUtil.java:165)
    at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:360)
    at org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:280)
    at org.apache.pig.PigServer.launchPlan(PigServer.java:1390)
    at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1375)
    at org.apache.pig.PigServer.storeEx(PigServer.java:1034)
    at org.apache.pig.PigServer.store(PigServer.java:997)
    at org.apache.pig.PigServer.openIterator(PigServer.java:910)
    at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:746)
    at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:372)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230)
    at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
    at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66)
    at org.apache.pig.Main.run(Main.java:558)
    at org.apache.pig.Main.main(Main.java:170)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.net.ConnectException: Call From localhost.localdomain/127.0.0.1 to 0.0.0.0:10020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:408)
    at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783)
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730)
    at org.apache.hadoop.ipc.Client.call(Client.java:1415)
    at org.apache.hadoop.ipc.Client.call(Client.java:1364)
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
    at com.sun.proxy.$Proxy15.getJobReport(Unknown Source)
    at org.apache.hadoop.mapreduce.v2.api.impl.pb.client.MRClientProtocolPBClientImpl.getJobReport(MRClientProtocolPBClientImpl.java:133)
    at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:320)
    ... 26 more
Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493)
    at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:606)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:700)
    at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:367)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1463)
    at org.apache.hadoop.ipc.Client.call(Client.java:1382)
    ... 34 more  

我尚未在Pig tarball中配置任何配置文件。是否需要对Pig配置进行任何更改以使其能够识别hadoop安装?在bashrc中,HADOOP_HOME已正确设置。
请帮忙。

尼斯

要启动它,请转到hadoop的sbin目录,然后键入命令

mr-jobhistory-server.sh start historyserver --config $HADOOP_CONF_DIR

$HADOOP_CONF_DIR是hadoop的配置文件(如hdfs-site.xmletc)所在的目录

本文收集自互联网,转载请注明来源。

如有侵权,请联系[email protected] 删除。

编辑于
0

我来说两句

0条评论
登录后参与评论

相关文章

来自分类Dev

spark错误和hadoop错误

来自分类Dev

在纱线“ org.apache.hadoop.ipc.Client-运行时重试时,apche Pig中出现错误-重试连接到服务器:tasktracker3 / 10.201.2.169:50000”

来自分类Dev

我不知道为什么SSH和Hadoop连接到错误的地方

来自分类Dev

我不知道为什么SSH和Hadoop连接到错误的地方

来自分类Dev

Hadoop和地理空间连接器

来自分类Dev

Hadoop 2.7.2上的Pig-0.16.0-错误1002:无法存储别名

来自分类Dev

MySQL和PHP连接错误

来自分类Dev

错误org.apache.pig.tools.grunt.Grunt-错误2998:未处理的内部错误。org / apache / hadoop / hbase / filter / WritableByteArrayComparable

来自分类Dev

启用hadoop和kerberos的Datastax Enterprise错误

来自分类Dev

Hadoop和Python:查看错误

来自分类Dev

Hadoop 错误启动 ResourceManager 和 NodeManager

来自分类Dev

使用Hadoop配置Pig关系

来自分类Dev

Hadoop Pig作业未运行

来自分类Dev

Python错误处理和连接错误

来自分类Dev

Pig错误:未处理的内部错误。找到了接口org.apache.hadoop.mapreduce.TaskAttemptContext,但是需要使用类

来自分类Dev

PIG:FLATTEN错误

来自分类Dev

错误 1070 PIG 至今

来自分类Dev

PDO连接上的CakePHP和XAMPP错误

来自分类Dev

Laravel Passport和PostgreSQL的连接错误

来自分类Dev

PostGIS和Django的Docker连接错误

来自分类Dev

Tensorflow错误连接Char和Word嵌入

来自分类Dev

PDO连接上的CakePHP和XAMPP错误

来自分类Dev

pyaudio和pocketsphinx中的连接被拒绝错误

来自分类Dev

连接和评估的VB语法错误

来自分类Dev

hadoop pig的长模式名称

来自分类Dev

Hadoop Pig-删除csv标头

来自分类Dev

Hadoop Pig的简单Python UDF问题

来自分类Dev

Hadoop Pig:使用STARTSWITH显示条目

来自分类Dev

Pig hadoop分组除法返回0

Related 相关文章

热门标签

归档