classification
Title: subprocess.Popen causes socket to remain open after close
Type: resource usage Stage:
Components: Library (Lib) Versions: Python 3.0, Python 2.6
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: kevinwatters, mamulsow, ngrilly, pitrou
Priority: high Keywords:

Created on 2008-05-29 21:02 by mamulsow, last changed 2008-08-26 23:44 by ngrilly.

Files
File name Uploaded Description Edit Remove
socketTest.py mamulsow, 2008-05-29 21:02 socketTest.py
socketinherit.py kevinwatters, 2008-08-01 17:42 Snippet that patches socket.socket to create uninheritable sockets
Messages
msg67511 (view) Author: Matt Mulsow (mamulsow) Date: 2008-05-29 21:02
On Windows, when a suprocess.Popen command is issued while
a socket connection is being handled the socket connection
will not close until the output of the subprocess is consumed.
The connection remains open even though the request.close()
command returns successfully and the program starts listening
for the next connection.

On Windows, the request.close() call translates to the C function
closesocket. The closesocket function by default attempts to do
a graceful close in the background. By changing the linger structure,
the closesocket function can be made to do a hard close. That fixes
my problem, but then queued data may not be flushed before the
socket closes. I cannot figure out why the closesocket's graceful
shutdown is waiting for the Popen command to complete.

This problem does not show up with the equivalent os.popen command.

To reproduce:
    run socketTest.py
    use telnet to connect on port 6288
    watch for connection to close or not
msg70572 (view) Author: Kevin Watters (kevinwatters) Date: 2008-08-01 17:42
I found a workaround for this issue (attached) via the kernel32.dll
function SetHandleInformation. You can patch the socket class to set all
newly created sockets as uninheritable.

It's not perfect--another thread could still spawn a subprocess in
between.  We probably need some kind of API for setting socket inheritance.
msg71167 (view) Author: Antoine Pitrou (pitrou) Date: 2008-08-15 10:25
> It's not perfect--another thread could still spawn a subprocess in
> between

Then it could be done in socketmodule.c, when still holding the GIL.
Python code can change back the socket to inheritable afterwards if it
wants to.

> We probably need some kind of API for setting socket inheritance.

Right.
History
Date User Action Args
2008-08-26 23:44:23ngrillysetnosy: + ngrilly
2008-08-15 10:25:48pitrousetpriority: high
nosy: + pitrou
messages: + msg71167
versions: + Python 2.6, Python 3.0, - Python 2.5
2008-08-01 17:42:08kevinwatterssetfiles: + socketinherit.py
messages: + msg70572
2008-07-30 18:52:34kevinwatterssetnosy: + kevinwatters
2008-05-29 21:02:37mamulsowcreate