How to pipe JSON output of curl to query a JSON value using python

LearnUntillDeath

I am writing a bash script where the curl command returns a JSON array of objects and then I need to filter the required values out of those objects. So I need to iterate through the objects to check then and parse them and finally get the result.

But inside my bash script if I do something like,

for i in 1 2 3 4
    do
        curl -XGET 'https://gitlab.com/user/api/v4/projects/1/pipelines/1/jobs' | python -c 'import sys, json; print(json.load(sys.stdin)[$i]["stage"])'
    done

I get the following error:

File "<string>", line 1
    import sys, json; print($i)
                            ^
SyntaxError: invalid syntax
100  3016  100  3016    0     0  18748      0 --:--:-- --:--:-- --:--:-- 19210
(23) Failed writing body

Everything Ok if I remove the python part after the pipe, but why can't I use python in combination with curl ? what am I doing wrong here ?

heemayl

In bash (and other shells), variables are not expanded when put inside single quotes.

Better approach than my original (as @chepner pointed out), pass the shell variable as command line argument to python command itself:

curl ... | python -c \
   'import sys, json; print(json.load(sys.stdin)[sys.argv[1]]["stage"])' "$i"

Original:

You can get around by using double quotes like:

curl ... | python -c \
            'import sys, json; print(json.load(sys.stdin)['"$i"']["stage"])'

i.e. terminating the single quote just before variable reference, and continuing afterwards.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

Using JSON output in curl bash script

From Dev

How to pretty json output from curl?

From Dev

How to POST 'shell output' as JSON data with Curl

From Dev

How to pretty json output from curl?

From Dev

How to filter json output from file using python?

From Dev

How to prevent curl output from printing a preceding pipe's output?

From Dev

Find a value in JSON using Python

From Dev

Select query using json format value

From Dev

Select query using json format value

From Dev

How can I query a table with key/value pairs into a JSON object using FOR JSON PATH?

From Dev

cURL GET query string is a JSON

From Dev

How to search for specific value in Json array using Python

From Dev

How to search for specific value in Json array using Python

From Dev

How to read a specific element's value from json using Python?

From Dev

How to curl with json postfields?

From Dev

How to pipe multi-line JSON Objects into separate python invocations

From Dev

How to do Json query in ansible with matched value

From Dev

How do I get the output of jq to pipe to curl

From Dev

How to output a live JSON feed in Python 3?

From Dev

How to parse json output through shell / Python

From Dev

How to output a live JSON feed in Python 3?

From Dev

How to make one json file as output in a python?

From Dev

Python-JSON - How to parse API output?

From Dev

How to parse ONLY the JSON value returned by CURL in PHP?

From Dev

AEM CURL - Need specific property and value shown in JSON response for Query without knowing the value

From Dev

How do I a retrieve specific value from JSON output in Python 2.7?

From Dev

JSON output in Ruby on Rails upon curl request

From Dev

Output bash variable with multiple lines to curl json

From Dev

Unable to filter JSON output using jq. I am using CURL

Related Related

  1. 1

    Using JSON output in curl bash script

  2. 2

    How to pretty json output from curl?

  3. 3

    How to POST 'shell output' as JSON data with Curl

  4. 4

    How to pretty json output from curl?

  5. 5

    How to filter json output from file using python?

  6. 6

    How to prevent curl output from printing a preceding pipe's output?

  7. 7

    Find a value in JSON using Python

  8. 8

    Select query using json format value

  9. 9

    Select query using json format value

  10. 10

    How can I query a table with key/value pairs into a JSON object using FOR JSON PATH?

  11. 11

    cURL GET query string is a JSON

  12. 12

    How to search for specific value in Json array using Python

  13. 13

    How to search for specific value in Json array using Python

  14. 14

    How to read a specific element's value from json using Python?

  15. 15

    How to curl with json postfields?

  16. 16

    How to pipe multi-line JSON Objects into separate python invocations

  17. 17

    How to do Json query in ansible with matched value

  18. 18

    How do I get the output of jq to pipe to curl

  19. 19

    How to output a live JSON feed in Python 3?

  20. 20

    How to parse json output through shell / Python

  21. 21

    How to output a live JSON feed in Python 3?

  22. 22

    How to make one json file as output in a python?

  23. 23

    Python-JSON - How to parse API output?

  24. 24

    How to parse ONLY the JSON value returned by CURL in PHP?

  25. 25

    AEM CURL - Need specific property and value shown in JSON response for Query without knowing the value

  26. 26

    How do I a retrieve specific value from JSON output in Python 2.7?

  27. 27

    JSON output in Ruby on Rails upon curl request

  28. 28

    Output bash variable with multiple lines to curl json

  29. 29

    Unable to filter JSON output using jq. I am using CURL

HotTag

Archive