AWS Elastic mapreduce doesn't seem to be correctly converting the streaming to jar

Mittenchops

I have a mapper and reducer that work fine when I run them in the piped version:

cat data.csv | ./mapper.py | sort -k1,1 | ./reducer.py

I used the elastic mapreducer wizard, loaded inputs, outputs, bootstrap, etc. The bootstrap is successful, but I am still getting an error in execution.

This is the error I'm getting in my stderr for step 1...

+ /etc/init.d/hadoop-state-pusher-control stop
+ PID_FILE=/mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ LOG_FILE=/mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ SVC_FILE=/mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ case $1 in
+ stop
+ echo 0
/etc/init.d/hadoop-state-pusher-control: line 35: /mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher: No such file or directory
+ /etc/init.d/hadoop-state-pusher-control start
+ PID_FILE=/mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ LOG_FILE=/mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ SVC_FILE=/mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ case $1 in
+ start
++ dirname /mnt/var/lib/hadoop-state-pusher/run-hadoop-state-pusher
+ sudo -u hadoop mkdir -p /mnt/var/lib/hadoop-state-pusher
+ echo 1
++ dirname /mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ sudo -u hadoop mkdir -p /mnt/var/run/hadoop-state-pusher
++ dirname /mnt/var/log/hadoop-state-pusher/hadoop-state-pusher.out
+ sudo -u hadoop mkdir -p /mnt/var/log/hadoop-state-pusher
+ disown %1
+ sleep 5
+ sudo -u hadoop /usr/bin/hadoop-state-pusher -server --pidfile /mnt/var/run/hadoop-state-pusher/hadoop-state-pusher.pid
+ exit 0
Command exiting with ret '0'

This is cryptic. What on earth does this mean?

It seems to have a problem with mounting something? Which of the other log files might say something informative, where I should be looking?

I tried a solution I found here, in just making the instance bigger, but this did not work, same error message.

Mittenchops

I was looking in the wrong log file. There is a different (there were like 6?) that actually gave me some useful python debugging information. It turned out I had used a string interpolation.format("of this kind {}, not this kind with a digit {1}".vars(a,b)) that was unsupported in python < 2.7, which was what was installed by default on the EC2 image used in elastic mapreduce.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

Java Multithreading doesn't seem to be correctly working

From Dev

NodeJS clearTimeout() doesn't seem to work correctly?

From Dev

LZW compression doesn't seem to work correctly

From Dev

Gradle Jar doesn't compile the .jar correctly?

From Dev

Javascript doesn't seem to correctly handle hidden elements that are later displayed

From Dev

minItems doesn't seem to validate correctly in JSON schema

From Dev

jQuery after function doesn't seem to be working correctly for tables

From Dev

Wordpress doesn't seem to correctly map POST data

From Dev

Semaphore doesn't seem to work correctly with multiple threads

From Dev

The random.randint import doesn't seem to be working correctly

From Dev

jQuery after function doesn't seem to be working correctly for tables

From Dev

Nginx Log Rotation Doesn't Seem to Be Working Correctly

From Dev

TinyXML2 doesn't seem to load my file correctly?

From Dev

Table.keepRowsTogether() in iText 5.5.1 doesn't seem to work correctly

From Dev

SVG path doesn't seem to orientate itself correctly

From Dev

Semaphore doesn't seem to work correctly with multiple threads

From Dev

CSS calc() function doesn't seem to work correctly

From Dev

GNU datamash transpose example doesn't seem to work correctly

From Dev

Joda-time Period doesn't seem to calculate days correctly

From Dev

Background CSS property doesn't seem to work correctly

From Dev

data.table shift() doesn't seem to "lead" correctly

From Dev

JQuery converting anonymous function to named function doesn't works correctly

From Dev

JQuery converting anonymous function to named function doesn't works correctly

From Dev

HLS, AWS Elastic Encoder, and adaptive streaming

From Dev

invoke-command doesn't seem to work - Did I use winrm.cmd to configure TrustedHosts correctly?

From Dev

VS2013 doesn't seem to attach correctly - a debugger is attached to but not configured to debug this unhandled exception

From Dev

UIATarget.onAlert = function onAlert(alert) Issue - Script doesn't seem to go into block correctly

From Dev

My class doesn't seem to map correctly with an API's JSON response, not binding

From Dev

What is the difference between AWS Elastic MapReduce and AWS Redshift

Related Related

  1. 1

    Java Multithreading doesn't seem to be correctly working

  2. 2

    NodeJS clearTimeout() doesn't seem to work correctly?

  3. 3

    LZW compression doesn't seem to work correctly

  4. 4

    Gradle Jar doesn't compile the .jar correctly?

  5. 5

    Javascript doesn't seem to correctly handle hidden elements that are later displayed

  6. 6

    minItems doesn't seem to validate correctly in JSON schema

  7. 7

    jQuery after function doesn't seem to be working correctly for tables

  8. 8

    Wordpress doesn't seem to correctly map POST data

  9. 9

    Semaphore doesn't seem to work correctly with multiple threads

  10. 10

    The random.randint import doesn't seem to be working correctly

  11. 11

    jQuery after function doesn't seem to be working correctly for tables

  12. 12

    Nginx Log Rotation Doesn't Seem to Be Working Correctly

  13. 13

    TinyXML2 doesn't seem to load my file correctly?

  14. 14

    Table.keepRowsTogether() in iText 5.5.1 doesn't seem to work correctly

  15. 15

    SVG path doesn't seem to orientate itself correctly

  16. 16

    Semaphore doesn't seem to work correctly with multiple threads

  17. 17

    CSS calc() function doesn't seem to work correctly

  18. 18

    GNU datamash transpose example doesn't seem to work correctly

  19. 19

    Joda-time Period doesn't seem to calculate days correctly

  20. 20

    Background CSS property doesn't seem to work correctly

  21. 21

    data.table shift() doesn't seem to "lead" correctly

  22. 22

    JQuery converting anonymous function to named function doesn't works correctly

  23. 23

    JQuery converting anonymous function to named function doesn't works correctly

  24. 24

    HLS, AWS Elastic Encoder, and adaptive streaming

  25. 25

    invoke-command doesn't seem to work - Did I use winrm.cmd to configure TrustedHosts correctly?

  26. 26

    VS2013 doesn't seem to attach correctly - a debugger is attached to but not configured to debug this unhandled exception

  27. 27

    UIATarget.onAlert = function onAlert(alert) Issue - Script doesn't seem to go into block correctly

  28. 28

    My class doesn't seem to map correctly with an API's JSON response, not binding

  29. 29

    What is the difference between AWS Elastic MapReduce and AWS Redshift

HotTag

Archive