Is it possible to log a message from an Oozie Workflow without killing it

Stormcloud

I've got a somewhat complex Oozie work flow and I'd like to be able to see some of the properties at various points in the flow. The obvious solution is to simply write a message to the logs.

I've gone through the official documentation, and it appears that the only action that supports logging is the Kill action!

Is there anyway to log a set of properties without killing the workflow?

Update (to provide more details)

So imagine my (simplified) workflow looks something like:

<workflow-app name="myWorkflow" xmlns="uri:oozie:workflow:0.5">
  <start to="action1"/>

  <kill name="Kill">
    <message>Failed: error ${myField}</message>
  </kill>

  <action name='action1'>
    <java>
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <configuration>
           <property>
                <name>mapred.job.queue.name</name>
                <value>default</value>
            </property>
        </configuration>
        <main-class>my.Class</main-class>
        <arg>${myArgument}</arg>
        <capture-output/>
    </java>
    <ok to="action2" />
    <error to="Kill" />
  </action>

  <action name='action2'>
    <ok to="End" />
    <error to="Kill" />
  </action>

  <end name="End"/>
</workflow-app>

Inside the workflow I've got fields called ${myField}, ${myArgument}, ${jobTracker} and ${nameNode}. Some of these are properties set by the launching process others will be set by the Java class I call from 'action1'. In-between executing 'action1' and 'action2' I'd like log the values of these fields. I don't much care where the values are logged to, I just need to be able to see what the values are.

I do NOT want to call my 'kill' action to do the logging because it's important that 'action2' is run.

Bijoy

One way I could see is you can put a shell script at the end of your workflow with the values to be logged as parameters to the shell script. Inside the shell script write the values of those parameters in a file and copy the file to hdfs.

Once the job is done you can check the value of those properties in the file stored in hdfs.

Hope this helps.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

Cloudera - Oozie cannot get properties from workflow.xml

From Dev

Is it possible to retrieve the logged message as a string from boost::log::record_view?

From Dev

Getting current time in oozie workflow

From Dev

Oozie workflow.xml error

From Dev

Failure to return to calling function from NSURLSession delegate without killing the task

From Dev

Exit from REPL without killing background process

From Dev

Oozie coordinated workflow

From Dev

How to copy data from HDFS to Local FS using oozie workflow?

From Dev

Sqoop Export Oozie Workflow Fails with File Not Found, Works when ran from the console

From Dev

MapReduce oozie workflow using Hue

From Dev

Is it possible to stop/run Form workflow from a controller?

From Dev

How to pass parameters to Hadoop ToolRunner (or any MapReduce Action) from Oozie workflow

From Dev

How to terminate Linux tee command without killing application it is receiving from

From Dev

Cloudera - Oozie cannot get properties from workflow.xml

From Dev

Getting current time in oozie workflow

From Dev

Failure to return to calling function from NSURLSession delegate without killing the task

From Dev

Is it possible to use two "job.properties" file in a workflow oozie?

From Dev

Exit from REPL without killing background process

From Dev

Read logs from Logcat without killing the tablet

From Dev

Close program window from terminal without killing process

From Dev

Add CLASSPATH to Oozie workflow job

From Dev

Oozie coordinated workflow

From Dev

How to use a Oozie job property in a Oozie workflow EL function?

From Dev

Oozie Workflow with Pig, Hive and unix

From Dev

Plone/Workflow- Is it possible to set the state of a workflow without needing a transition?

From Dev

Killing a sub-subprocess from python/bash without leaving orphans

From Dev

java.lang.RuntimeException: Stream '/jars/oozie-examples.jar' was not found. while running spark workflow from oozie in CDH 5.8

From Dev

Case conversion from oozie workflow

From Dev

How can i let the oozie workflow run after killing the coordinator?

Related Related

  1. 1

    Cloudera - Oozie cannot get properties from workflow.xml

  2. 2

    Is it possible to retrieve the logged message as a string from boost::log::record_view?

  3. 3

    Getting current time in oozie workflow

  4. 4

    Oozie workflow.xml error

  5. 5

    Failure to return to calling function from NSURLSession delegate without killing the task

  6. 6

    Exit from REPL without killing background process

  7. 7

    Oozie coordinated workflow

  8. 8

    How to copy data from HDFS to Local FS using oozie workflow?

  9. 9

    Sqoop Export Oozie Workflow Fails with File Not Found, Works when ran from the console

  10. 10

    MapReduce oozie workflow using Hue

  11. 11

    Is it possible to stop/run Form workflow from a controller?

  12. 12

    How to pass parameters to Hadoop ToolRunner (or any MapReduce Action) from Oozie workflow

  13. 13

    How to terminate Linux tee command without killing application it is receiving from

  14. 14

    Cloudera - Oozie cannot get properties from workflow.xml

  15. 15

    Getting current time in oozie workflow

  16. 16

    Failure to return to calling function from NSURLSession delegate without killing the task

  17. 17

    Is it possible to use two "job.properties" file in a workflow oozie?

  18. 18

    Exit from REPL without killing background process

  19. 19

    Read logs from Logcat without killing the tablet

  20. 20

    Close program window from terminal without killing process

  21. 21

    Add CLASSPATH to Oozie workflow job

  22. 22

    Oozie coordinated workflow

  23. 23

    How to use a Oozie job property in a Oozie workflow EL function?

  24. 24

    Oozie Workflow with Pig, Hive and unix

  25. 25

    Plone/Workflow- Is it possible to set the state of a workflow without needing a transition?

  26. 26

    Killing a sub-subprocess from python/bash without leaving orphans

  27. 27

    java.lang.RuntimeException: Stream '/jars/oozie-examples.jar' was not found. while running spark workflow from oozie in CDH 5.8

  28. 28

    Case conversion from oozie workflow

  29. 29

    How can i let the oozie workflow run after killing the coordinator?

HotTag

Archive