为什么hadoop输出文件part-r-00000为空

蒂米

我的MR日志是:

[root@sicongli hadoop-2.4.1]# hadoop jar flowcount.jar   
cn.itheima.bigdata.hadoop.mr.flowcount.FlowCount /data/join.txt /out
16/04/13 23:32:20 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
16/04/13 23:32:22 INFO client.RMProxy: Connecting to ResourceManager at sicongli/192.168.218.111:8032

16/04/13 23:32:28 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
16/04/13 23:32:35 INFO input.FileInputFormat: Total input paths to process : 1
16/04/13 23:32:38 INFO mapreduce.JobSubmitter: number of splits:1
16/04/13 23:32:41 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1460601112521_0002
16/04/13 23:32:47 INFO impl.YarnClientImpl: Submitted application application_1460601112521_0002
16/04/13 23:32:47 INFO mapreduce.Job: The url to track the job: http://sicongli:8088/proxy/application_1460601112521_0002/
16/04/13 23:32:47 INFO mapreduce.Job: Running job: job_1460601112521_0002
16/04/13 23:35:20 INFO mapreduce.Job: Job job_1460601112521_0002 running in uber mode : false
16/04/13 23:35:28 INFO mapreduce.Job:  map 0% reduce 0%
16/04/13 23:36:47 INFO mapreduce.Job:  map 100% reduce 0%
16/04/13 23:37:25 INFO mapreduce.Job:  map 100% reduce 100%
16/04/13 23:37:48 INFO mapreduce.Job: Job job_1460601112521_0002 completed successfully
16/04/13 23:38:16 INFO mapreduce.Job: Counters: 49
    File System Counters
            FILE: Number of bytes read=6
            FILE: Number of bytes written=186579
            FILE: Number of read operations=0
            FILE: Number of large read operations=0
            FILE: Number of write operations=0
            HDFS: Number of bytes read=399
            HDFS: Number of bytes written=0
            HDFS: Number of read operations=6
            HDFS: Number of large read operations=0
            HDFS: Number of write operations=2
    Job Counters 
            Launched map tasks=1
            Launched reduce tasks=1
            Data-local map tasks=1
            Total time spent by all maps in occupied slots (ms)=17296
            Total time spent by all reduces in occupied slots (ms)=36727
            Total time spent by all map tasks (ms)=17296
            Total time spent by all reduce tasks (ms)=36727
            Total vcore-seconds taken by all map tasks=17296
            Total vcore-seconds taken by all reduce tasks=36727
            Total megabyte-seconds taken by all map tasks=17711104
            Total megabyte-seconds taken by all reduce tasks=37608448
    Map-Reduce Framework
            Map input records=23
            Map output records=0
            Map output bytes=0
            Map output materialized bytes=6
            Input split bytes=99
            Combine input records=0
            Combine output records=0
            Reduce input groups=0
            Reduce shuffle bytes=6
            Reduce input records=0
            Reduce output records=0
            Spilled Records=0
            Shuffled Maps =1
            Failed Shuffles=0
            Merged Map outputs=1
            GC time elapsed (ms)=217
            CPU time spent (ms)=1150
            Physical memory (bytes) snapshot=277962752
            Virtual memory (bytes) snapshot=1689296896
            Total committed heap usage (bytes)=127127552
    Shuffle Errors
            BAD_ID=0
            CONNECTION=0
            IO_ERROR=0
            WRONG_LENGTH=0
            WRONG_MAP=0
            WRONG_REDUCE=0
    File Input Format Counters 
            Bytes Read=300
    File Output Format Counters 
            Bytes Written=0
16/04/13 23:38:18 INFO ipc.Client: Retrying connect to server: sicongli/192.168.218.111:49806. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
16/04/13 23:38:19 INFO ipc.Client: Retrying connect to server: sicongli/192.168.218.111:49806. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
16/04/13 23:38:20 INFO ipc.Client: Retrying connect to server: sicongli/192.168.218.111:49806. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1000 MILLISECONDS)
16/04/13 23:38:23 INFO mapred.ClientServiceDelegate: Application state is completed. FinalApplicationStatus=SUCCEEDED. Redirecting to job history server

输出为:

[root@sicongli ~]# hadoop fs -ls /out
16/04/14 00:00:38 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Found 2 items
-rw-r--r--   3 root supergroup          0 2016-04-13 23:37 /out/_SUCCESS
-rw-r--r--   3 root supergroup          0 2016-04-13 23:37 /out/part-r-00000

而且我有两个问题:

一个:为什么输出文件part-r-0000为空

拖:为什么有警告:INFO ipc.Client:重试连接到服务器:sicongli / 192.168.218.111:49806。已经尝试了2次;重试策略是RetryUpToMaximumCountWithFixedSleep(maxRetries = 3,sleepTime = 1000 MILLISECONDS)

安德鲁·威廉姆森

问题1-阅读计数器:

地图输入记录= 23

地图输出记录= 0

Part-r-00000为空,因为没有任何内容使它脱离您的地图任务。如果您将地图任务的代码添加到帖子中,我们也许可以告诉您原因

问题2-阅读问题的答案,它们可能会对您有所帮助。

本文收集自互联网,转载请注明来源。

如有侵权,请联系[email protected] 删除。

编辑于
0

我来说两句

0条评论
登录后参与评论

相关文章

来自分类Dev

为什么数组为空

来自分类Dev

Shell脚本:如果输出文件为空,则在设置的时间后终止程序

来自分类Dev

为什么我的cron输出文件名的末尾有\ r?

来自分类Dev

R include目录为空

来自分类Dev

如何将输出文件名从reducer中的part-00000更改为inputfile名

来自分类Dev

打字稿:编译时输出文件为空

来自分类Dev

在R中输出文件而不会覆盖

来自分类Dev

为什么FormData为空?

来自分类Dev

R帮助为空

来自分类Dev

为什么JsonHttpContent的输出为空?

来自分类Dev

R:写入前清除输出文件

来自分类Dev

Nightwatch XML输出文件为空

来自分类Dev

当输出文件为/ dev / null时,为什么tar似乎会跳过文件内容?

来自分类Dev

为什么我的python脚本输出文件为空

来自分类Dev

什么是“破折号”-表示为ffmpeg输出文件名

来自分类Dev

Hadoop中的part-r-00000和part-m-00000文件有什么区别?

来自分类Dev

为什么打印为空?

来自分类Dev

当输出文件为/ dev / null时,为什么tar似乎会跳过文件内容?

来自分类Dev

“打开输出文件:权限被拒绝”,-rw-r--r--

来自分类Dev

为什么UITableView为空?

来自分类Dev

Shell脚本:如果输出文件为空,则在设置的时间后终止程序

来自分类Dev

R include目录为空

来自分类Dev

打字稿:编译时输出文件为空

来自分类Dev

为什么tcpdump输出文件为空?

来自分类Dev

为什么我的输出文件都命名为“ part-r-xxxxx”,即使我没有提到任何reducer类也是如此?

来自分类Dev

为什么OutputDebugString为空

来自分类Dev

为什么在访问simpleXML对象时输出为空?

来自分类Dev

jsPDF输出文件为空

来自分类Dev

为什么像在 Excel 中的 vlookup 一样在 bash 中使用 awk 会给出空的输出文件?

Related 相关文章

热门标签

归档