How to do local IPC without leaking handles (cross platform)?

coldfix

How can I initiate IPC with a child process, without letting it inherit all handles? To make it more interesting, this shoud work on windows as well as unix.

The background: I am writing a library that interfaces with a 3rparty shared library (let's just call it IT) which in turn contains global data (that really should be objects!). I want to have multiple instances of this global data. As far as I understand, I have two options to solve this:

  1. create a cython module that links against a static variant of IT, then copy and import the module whenever I want a new instance. Analogously, I could copy IT but that's even more work to create a ctypes interface.

  2. spawn a subprocess that loads IT and establish an IPC connection to it.

There are a few reasons to use (2):

  • I am not sure, if (1) is reliable in any way and it feels like a bad idea (what happens with all the extra modules, when the application exits in an uncontrolled way?).

  • boxing IT into a separate process might actually be a good idea anyway for security considerations: IT deals with potentially unsafe input and IT's code quality isn't overly good. So, I'd rather not have any secure resources open when running it.

  • there is probably lot's of need for this kind of IPC in future applications

So what are my options? I have already looked into:

  • multiprocessing.Process at first looked nice, until I realized that the new process gets a copy of all my handles. Needless to say that this is quite problematic, since now resources cannot be reliably freed by closing them in the parent process + the security issues mentioned earlier.

  • Use os.closerange within a multiprocessing.Process to close to all handles manually - except for the Pipe I'm interested in. Does os.closerange close only files or does it take care of other types of resources as well? If so: how can I determine the range, given the Pipe object?

  • subprocess.Popen(.., close_fds=True, stdin=PIPE, stdout=PIPE) works fine on unix but isn't possible on win32.

  • Named pipes are very different on win32 and unix. Are their any libraries that their usage?

  • Sockets. Promising, especially since their are handy RPC libraries that can work with sockets. On the other hand, I fear that this may cause a whole bunch of security issues. Are sockets that I have determined to be of local origin (sock.getpeername()[0] == '127.0.0.1') secure against tempering?

Are there any possibilities that I have overlooked?

To round up: the main question is how to establish a secure IPC with a child process on windows+unix? But please don't hesitate to answer if you know any answers to only partial problems.

Thanks for taking the time to read it!

coldfix

It seems on python>=3.4 subprocess.Popen(..., stdin=PIPE, stdout=PIPE, close_fds=False) is a possible option. This is due to a patch that makes all opened file descriptors non-inheritable by default. To be more precise, they will be automatically closed on execv (so still can't use multiprocessing.Process), see PEP 446.

This is also a valid option for other python versions:

  • on windows, HANDLEs are created non-inheritable by default, so you will leak only handles that were made inheritable explicitly
  • on POSIX/python<=3.3 you can still use os.closerange to close open file descriptors after spawning the subprocess

for a corresponding example see:

https://github.com/coldfix/python-ipc-test

The most useful combinations are:

  1. stdio:pickle

    • pro: completely cross-platform in my tests
    • pro: fastest option (with 2)
    • con: stdin/stdout can not be redirected independently
  2. inherit_unidir:pickle

    • pro: you can redirect STDIO streams independently
    • pro: fastest option together with stdio:pickle
    • con: very low level platform specific code
  3. socket:sockpipe

    • pro: cross-platform with little effort
    • con: there is a short period when "attackers" may connect to the port, you could require a pass-phrase or something to prevent that from happening
    • con: slightly slower than alternatives on windows (factor 1.6 in my measurements)
    • when not using AF_UNIX there are unpredictable performance hits on linux

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

How do I use Haxe sockets in a cross-platform way without blocking?

From Dev

How to find which process is leaking file handles in Linux?

From Dev

How to create a cross-platform Ubuntu Touch app without qmlscene?

From Dev

iTunes cross-platform IAP subscriptions - how does Netflix do it?

From Dev

Crystal Reports ReportDocument leaking handles

From Dev

Crystal Reports ReportDocument leaking handles

From Dev

How Composer handles local changes

From Dev

How to get the json message from local using xamarin cross platform development

From Dev

How to @Provide an Activity for the MortarActivityScope, without leaking the Activity on orientation changes?

From Dev

Is possible to do sliding drawer in xamarin cross platform?

From Dev

How do I efficiently create a cross-platform 1000+ entry database system?

From Dev

Using a filesystem::path, how do you open a file in a cross-platform way?

From Dev

How do I efficiently create a cross-platform 1000+ entry database system?

From Dev

urllib2.urlopen a local file, cross platform

From Dev

Using local brew formula to install cross platform gcc compiler

From Dev

Get local IP address from "ip route" cross-platform

From Dev

How to implement cross platform file lock in GO

From Dev

How to use QtWinExtras in cross-platform application

From Dev

How to make cross platform mobile apps?

From Dev

Python how to join cross platform paths

From Dev

How to deploy cross-platform applications with Qt?

From Dev

Future of cross-platform mobile development without WebSQL

From Dev

Show Facebook Feeds in Cross Platform App Without Login

From Dev

Import monads doesnt work (leaking haskell platform)

From Dev

How do I use a local SPM package without a git repository?

From Dev

How can I reliably prevent my local IP address leaking in the web browsers?

From Dev

Where do I place an app icon? (cross-platform)

From Dev

Is there a cross-platform way to do PySide spontaneous resize event checking?

From Dev

How can I map types in a class library without mapped classes leaking from that library

Related Related

  1. 1

    How do I use Haxe sockets in a cross-platform way without blocking?

  2. 2

    How to find which process is leaking file handles in Linux?

  3. 3

    How to create a cross-platform Ubuntu Touch app without qmlscene?

  4. 4

    iTunes cross-platform IAP subscriptions - how does Netflix do it?

  5. 5

    Crystal Reports ReportDocument leaking handles

  6. 6

    Crystal Reports ReportDocument leaking handles

  7. 7

    How Composer handles local changes

  8. 8

    How to get the json message from local using xamarin cross platform development

  9. 9

    How to @Provide an Activity for the MortarActivityScope, without leaking the Activity on orientation changes?

  10. 10

    Is possible to do sliding drawer in xamarin cross platform?

  11. 11

    How do I efficiently create a cross-platform 1000+ entry database system?

  12. 12

    Using a filesystem::path, how do you open a file in a cross-platform way?

  13. 13

    How do I efficiently create a cross-platform 1000+ entry database system?

  14. 14

    urllib2.urlopen a local file, cross platform

  15. 15

    Using local brew formula to install cross platform gcc compiler

  16. 16

    Get local IP address from "ip route" cross-platform

  17. 17

    How to implement cross platform file lock in GO

  18. 18

    How to use QtWinExtras in cross-platform application

  19. 19

    How to make cross platform mobile apps?

  20. 20

    Python how to join cross platform paths

  21. 21

    How to deploy cross-platform applications with Qt?

  22. 22

    Future of cross-platform mobile development without WebSQL

  23. 23

    Show Facebook Feeds in Cross Platform App Without Login

  24. 24

    Import monads doesnt work (leaking haskell platform)

  25. 25

    How do I use a local SPM package without a git repository?

  26. 26

    How can I reliably prevent my local IP address leaking in the web browsers?

  27. 27

    Where do I place an app icon? (cross-platform)

  28. 28

    Is there a cross-platform way to do PySide spontaneous resize event checking?

  29. 29

    How can I map types in a class library without mapped classes leaking from that library

HotTag

Archive