Apache Pig-说明命令错误

绿色的
]$ cat webccess.txt
mark,yahoo.com,6
sam,google.com,7
john,yahoo.com,3
patrick,cnn.com,8
mary,facebook.com,1
mark,yahoo.com,4
john,bbc.com,10
andrew,twitter.com,3
patrick,twitter.com,9

我在Cloudera Quick Vm Hue-Pig Shell(Grunt)中的任务下方运行

grunt> stage1 = LOAD '/user/cloudera/webaccess.txt' USING PigStorage(',') AS (name:chararray, website:chararray, access:int);
grunt> DUMP stage1;
grunt> stage2 = FILTER stage1 by access >= 8;
grunt> stage3 = GROUP stage1 by name;
grunt> stage4 = FOREACH stage3 GENERATE group as GROUPS, MAX(stage1.access);
grunt> DUMP stage4;

输出:

(sam,7)
(john,10)
(mark,6)
(mary,1)
(andrew,3)
(patrick,9)

直到这一切都很好。

当我在关联关卡stage4上使用ILLUSTRATE命令进行检查时,出现如下所示的错误,

grunt> ILLUSTRATE stage4;

2014-10-07 04:02:43,639 [main] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-10-07 04:02:43,642 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost.localdomain:8020
2014-10-07 04:02:43,643 [main] WARN org.apache.hadoop.conf.Configuration - io.bytes.per.checksum is deprecated. Instead, use dfs.bytes-per-checksum
2014-10-07 04:02:43,643 [main] WARN org.apache.hadoop.conf.Configuration - dfs.https.address is deprecated. Instead, use dfs.namenode.https-address
2014-10-07 04:02:43,643 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost.localdomain:8021
2014-10-07 04:02:43,799 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2014-10-07 04:02:43,800 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2014-10-07 04:02:43,800 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2014-10-07 04:02:43,804 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2014-10-07 04:02:43,805 [main] ERROR org.apache.pig.pen.ExampleGenerator - Error reading data. Internal error creating job configuration.
java.lang.RuntimeException: Internal error creating job configuration.
at org.apache.pig.pen.ExampleGenerator.getExamples(ExampleGenerator.java:160)
at org.apache.pig.PigServer.getExamples(PigServer.java:1182)
at org.apache.pig.tools.grunt.GruntParser.processIllustrate(GruntParser.java:739)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.Illustrate(PigScriptParser.java:626)
at org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:323)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
at org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:538)
at org.apache.pig.Main.main(Main.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
2014-10-07 04:02:43,868 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2997: Encountered IOException. Exception
Details at logfile: /dev/null

我正处于学习阶段,由于此错误,我无法移至下一个主题。

同样,在我第一次打开Hue-Pig Shell(Grunt)时开始执行此任务之前,我发现了以下警告。

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/util/PlatformName
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.util.PlatformName
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: org.apache.hadoop.util.PlatformName. Program will exit.
which: no hadoop in ((null))
which: no /usr/lib/hadoop/bin/hadoop in ((null))
dirname: missing operand
Try `dirname --help' for more information.
2014-10-07 03:18:27,802 [main] INFO org.apache.pig.Main - Apache Pig version 0.11.0-cdh4.7.0 (rexported) compiled May 28 2014, 11:05:48
2014-10-07 03:18:27,803 [main] INFO org.apache.pig.Main - Logging error messages to: /dev/null
2014-10-07 03:18:28,758 [main] INFO org.apache.pig.impl.util.Utils - Default bootup file /home/cloudera/.pigbootup not found
2014-10-07 03:18:30,436 [main] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-10-07 03:18:30,444 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to hadoop file system at: hdfs://localhost.localdomain:8020
2014-10-07 03:18:37,832 [main] INFO org.apache.pig.backend.hadoop.executionengine.HExecutionEngine - Connecting to map-reduce job tracker at: localhost.localdomain:8021
2014-10-07 03:18:37,842 [main] WARN org.apache.hadoop.conf.Configuration - fs.default.name is deprecated. Instead, use fs.defaultFS
锡瓦萨克(Sivasakthi jayaraman)

我没有遇到任何问题,说明命令运行正常。您可以先尝试在本地模式下执行吗?

    $pig -x local
    grunt> stage1 = LOAD 'input.txt' USING PigStorage(',') AS (name:chararray, website:chararray, access:int);
    grunt> stage2 = FILTER stage1 by access >= 8;
    grunt> stage3 = GROUP stage1 by name;
    grunt> stage4 = FOREACH stage3 GENERATE group as GROUPS, MAX(stage1.access);
    grunt> DUMP stage4;
    (sam,7)
    (john,10)
    (mark,6)
    (mary,1)
    (andrew,3)
    (patrick,9)
    grunt> ILLUSTRATE stage4;
    ----------------------------------------------------------------------------
    | stage1     | name:chararray     | website:chararray     | access:int     | 
    ----------------------------------------------------------------------------
    |            | john               | yahoo.com             | 3              | 
    |            | john               | bbc.com               | 10             | 
    ----------------------------------------------------------------------------
    --------------------------------------------------------------------------------------------------------------------------
    | stage3     | group:chararray     | stage1:bag{:tuple(name:chararray,website:chararray,access:int)}                     | 
    --------------------------------------------------------------------------------------------------------------------------
    |            | john                | {(john, yahoo.com, 3), (john, bbc.com, 10)}                                         | 
    |            | john                | {(john, yahoo.com, 3), (john, bbc.com, 10)}                                         | 
    --------------------------------------------------------------------------------------------------------------------------
    ------------------------------------------------
    | stage4     | GROUPS:chararray     | :int     | 
    ------------------------------------------------
    |            | john                 | 10       | 
    ------------------------------------------------

本文收集自互联网,转载请注明来源。

如有侵权,请联系[email protected] 删除。

编辑于
0

我来说两句

0条评论
登录后参与评论

相关文章

来自分类Dev

Apache Pig-说明命令错误

来自分类Dev

Apache Pig错误-无法跟踪

来自分类Dev

如何解决以下Apache Pig错误?

来自分类Dev

错误1070 Apache Pig,使用内置UDF

来自分类Dev

如何解决以下Apache Pig错误?

来自分类Dev

错误org.apache.pig.tools.grunt.Grunt-错误1000

来自分类Dev

Pig错误:无法找到或加载主类org.apache.pig.Main

来自分类Dev

apache-Pig map-reduce错误分组

来自分类Dev

Apache Pig浮点数SUM错误的精度

来自分类Dev

Apache Pig,编程问题

来自分类Dev

Apache Pig GROUP BY,ORDER BY

来自分类Dev

Apache Pig UDF解析问题

来自分类Dev

Apache Pig中的“ substr”语句

来自分类Dev

Apache Pig条件foreach生成

来自分类Dev

Apache Pig Rank函数的用法

来自分类Dev

Apache Pig转义列名称

来自分类Dev

从时间戳至今的Apache Pig

来自分类Dev

Apache PIG,JSON加载程序

来自分类Dev

在 Apache Pig 中过滤数据

来自分类Dev

使用 Apache Pig 过滤日期

来自分类Dev

错误1070:无法使用导入来解析Pigstorage:[,java.lang。,org.apache.pig.builtin。,org.apache.pig.impl.builtin。]

来自分类Dev

错误org.apache.pig.tools.grunt.Grunt-错误2998:未处理的内部错误。org / apache / hadoop / hbase / filter / WritableByteArrayComparable

来自分类Dev

失败的地图任务超出允许的限制,并显示错误:org.apache.pig.data.Typle.isNull()Z

来自分类Dev

Apache-PIG脚本:错误联接变量上的无效字段投影

来自分类Dev

Apache Pig错误-为什么它不接受chararray中的所有列

来自分类Dev

apache Pig查询在Pig Latin中花费的Cpu时间

来自分类Dev

PIG:FLATTEN错误

来自分类Dev

错误 1070 PIG 至今

来自分类Dev

Apache Pig关于排序前n个