Why does make ignore escape sequence if the output is piped

Sergei Morozov

In a Makefile, I want to print a byte represented as a hexadecimal number and pass it to STDIN of another program. For some reason, it doesn't work:

without-pipe:
    @printf '\x66\x6f\x6f'

with-bash-and-pipe:
    @/bin/bash -c "printf '\x66\x6f\x6f' | cat"

with-pipe:
    @printf '\x66\x6f\x6f' | cat

Running this file produces:

$ make without-pipe 
foo

$ make with-bash-and-pipe 
foo

$ make with-pipe 
\x66\x6f\x6f

What feature of make am I missing and what is the proper way to make the last target produce the same output. The one with-bash-and-pipe is a sort of workaround.

Kamil Maciorowski

Note: my testbed is Ubuntu 18.04.2 LTS.


Few steps are required to understand the behavior.

1. Make is somewhat smart

It looks like make determines if a shell is required. I did

strace -f -e execve make without-pipe

and the part of the output was:

[pid 17526] execve("/usr/local/sbin/printf", ["printf", "\\x66\\x6f\\x6f"], 0x5569b5a5b440 /* 67 vars */) = -1 ENOENT (No such file or directory)
[pid 17526] execve("/usr/local/bin/printf", ["printf", "\\x66\\x6f\\x6f"], 0x5569b5a5b440 /* 67 vars */) = -1 ENOENT (No such file or directory)
[pid 17526] execve("/usr/sbin/printf", ["printf", "\\x66\\x6f\\x6f"], 0x5569b5a5b440 /* 67 vars */) = -1 ENOENT (No such file or directory)
[pid 17526] execve("/usr/bin/printf", ["printf", "\\x66\\x6f\\x6f"], 0x5569b5a5b440 /* 67 vars */) = 0

So in this case make just probes possible locations of printf (according to $PATH) until the tool is found.

The behavior is different in the last case. This command

strace -f -e execve make with-pipe

yields (among other lines)

[pid 17592] execve("/bin/sh", ["/bin/sh", "-c", "printf '\\x66\\x6f\\x6f' | cat"], 0x561465476440 /* 67 vars */) = 0

The tool is smart enough to tell you want to run a pipe. If you do, a shell is used. The shell is sh.

2. The three cases use different printf implementations

  • As shown above, make without-pipe uses printf executable available in the OS.

  • printf is a builtin in bash (confirm with type -a printf), so make with-bash-and-pipe uses the builtin from bash.

  • printf is also a builtin in sh (confirm with (unset PATH; printf 'printf works\n'; ls); if it wasn't a builtin, sh couldn't find the executable like it cannot find ls), so make with-pipe uses the builtin from sh.

3. printf is not required to understand \x66 and such

POSIX specification of printf says:

The format operand shall be used as the format string described in XBD File Format Notation with the following exceptions:

[…]

Neither of the two linked documents says \xHH and such must be special. I'm not sure if the specification explicitly allows them to be special, but the fact is they are special for some printf-s in the question.

But not for the printf builtin in sh.


what is the proper way to make the last target produce the same output? The one with-bash-and-pipe is a sort of workaround.

You can make sure printf you run is not a builtin by specifying its full path:

/usr/bin/printf '\x66\x6f\x6f' | cat

however this requires you to know the path beforehand. To fix this issue you can use env:

env printf '\x66\x6f\x6f' | cat

env will run the executable found in $PATH. You can expect env to be present because it's required by POSIX.

On the other hand in general you can't be sure any printf executable (builtin or not) does support \xHH in the first place. Therefore the real fix is to rewrite your format string so it's properly understood by all POSIX-compliant implementations of printf. This seems useful:

In addition to the escape sequences shown in XBD File Format Notation ( \\, \a, \b, \f, \n, \r, \t, \v ), \ddd, where ddd is a one, two, or three-digit octal number, shall be written as a byte with the numeric value specified by the octal number.

Example:

printf '\146\157\157'

This approach works in all three cases.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

Why does -Verbose make Copy-Item ignore ErrorActionPreference?

From Dev

Why shell does not set a variable which is piped?

From Dev

ignore escape sequence c++

From Dev

Why does GLSL ignore return?

From Dev

Why does FloatToStr ignore ThousandSeparator?

From Dev

Why does setInterval() ignore errors?

From Dev

Why does make ignore my modifications?

From Dev

Octave's 'system' does not interpret a escape sequence

From Dev

Why does Git ignore these folders?

From Dev

Why don't some "for" commands work when the output is piped?

From Dev

Why does the error message appear when `IF` comparison is in the piped block command?

From Dev

Why does dereferencing an iterator make output-stream cut out?

From Dev

What does this bash escape sequence mean?

From Dev

In Perl, why does variable interpolation fail for a hexadecimal escape sequence?

From Dev

Why isn't apt-cache policy output piped?

From Dev

What does this bash escape sequence mean?

From Dev

Ping piped to txt does output nothing

From Dev

Why does sudo ignore aliases?

From Dev

Octave's 'system' does not interpret a escape sequence

From Dev

Why does the printf statement in this loop output the array out of sequence?

From Dev

Why does bash output data instead of executing, when a script is piped?

From Dev

Why does `cd` have no effect if output is piped?

From Dev

Why does piped FFMPEG output differ from an explicit file?

From Dev

why does the output of `ls` look different when piped through ``tr "\n" "\n"`?

From Dev

Escape puzzle: why does grep ignore escaping of single quote?

From Dev

How does a program output to a terminal when stdout is piped?

From Dev

Why won't journalctl show logs when the program output is piped?

From Dev

How can processes eliminate escape codes when its output is piped?

From Dev

How does a Unix program know to change output if it's being piped?

Related Related

  1. 1

    Why does -Verbose make Copy-Item ignore ErrorActionPreference?

  2. 2

    Why shell does not set a variable which is piped?

  3. 3

    ignore escape sequence c++

  4. 4

    Why does GLSL ignore return?

  5. 5

    Why does FloatToStr ignore ThousandSeparator?

  6. 6

    Why does setInterval() ignore errors?

  7. 7

    Why does make ignore my modifications?

  8. 8

    Octave's 'system' does not interpret a escape sequence

  9. 9

    Why does Git ignore these folders?

  10. 10

    Why don't some "for" commands work when the output is piped?

  11. 11

    Why does the error message appear when `IF` comparison is in the piped block command?

  12. 12

    Why does dereferencing an iterator make output-stream cut out?

  13. 13

    What does this bash escape sequence mean?

  14. 14

    In Perl, why does variable interpolation fail for a hexadecimal escape sequence?

  15. 15

    Why isn't apt-cache policy output piped?

  16. 16

    What does this bash escape sequence mean?

  17. 17

    Ping piped to txt does output nothing

  18. 18

    Why does sudo ignore aliases?

  19. 19

    Octave's 'system' does not interpret a escape sequence

  20. 20

    Why does the printf statement in this loop output the array out of sequence?

  21. 21

    Why does bash output data instead of executing, when a script is piped?

  22. 22

    Why does `cd` have no effect if output is piped?

  23. 23

    Why does piped FFMPEG output differ from an explicit file?

  24. 24

    why does the output of `ls` look different when piped through ``tr "\n" "\n"`?

  25. 25

    Escape puzzle: why does grep ignore escaping of single quote?

  26. 26

    How does a program output to a terminal when stdout is piped?

  27. 27

    Why won't journalctl show logs when the program output is piped?

  28. 28

    How can processes eliminate escape codes when its output is piped?

  29. 29

    How does a Unix program know to change output if it's being piped?

HotTag

Archive