classification
Title: subprocess.call fails with unicode strings in command line
Type: Stage:
Components: Library (Lib) Versions: Python 2.5
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: andersjm, brotch, gregcouch, mclausch, ocean-city
Priority: normal Keywords: patch

Created on 2007-07-24 18:24 by mclausch, last changed 2008-10-01 19:40 by gregcouch.

Files
File name Uploaded Description Edit Remove
CreateProcessW.patch ocean-city, 2008-03-02 07:58
Python-2.5.2-subprocess.patch gregcouch, 2008-10-01 19:40 Alternate Python-only patch
Messages
msg32546 (view) Author: Matt (mclausch) Date: 2007-07-24 18:24
On Windows, subprocess.call() fails with an exception if either the executable or any of the arguments contain upper level characters. See below:

>>> cmd = [ u'test_\xc5_exec.bat', u'arg1', u'arg2' ]
>>> subprocess.call(cmd)

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python25\lib\subprocess.py", line 443, in call
    return Popen(*popenargs, **kwargs).wait()
  File "C:\Python25\lib\subprocess.py", line 593, in __init__
    errread, errwrite)
  File "C:\Python25\lib\subprocess.py", line 815, in _execute_child
    startupinfo)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xc5' in position 5: ordinal not in range(128)
msg32547 (view) Author: brotchie (brotch) Date: 2007-08-05 08:36
Python's default character coding is 'ascii' which can't convert unicode > 127 into chars.

Forcing the unicode string to encode as 'iso-8859-1' 

eg.
subprocess.call(cmd.encode('iso-8859-1')) 

resolves the problem and runs the correct command.
msg32548 (view) Author: Matt (mclausch) Date: 2007-08-20 21:12
Sorry, I should have been more specific. I'm looking for a general solution, not just one for characters in iso-8859-1. For instance, I need to execute a subprocess where the executable or the arguments may contain Japanese characters.

So another example would be:
cmd = [ u'test_\u65e5\u672c\u8a9e_exec.bat', u'arg1', u'arg2' ]
subprocess.call(cmd)
msg63176 (view) Author: Hirokazu Yamamoto (ocean-city) Date: 2008-03-02 07:58
I tried to fix this problem using CreateProcessW.
(environment variables are still ANSI)

I don't know Python C API well, maybe I'm doing
something wrong. (I confirmed test_subprocess.py
passes)
msg74142 (view) Author: Greg Couch (gregcouch) Date: 2008-10-01 19:40
We're having the same problem.  My quick fix was to patch subprocess.py
so the command line and executable are converted to the filesystem
encoding (mbcs).
History
Date User Action Args
2008-10-01 19:40:51gregcouchsetfiles: + Python-2.5.2-subprocess.patch
nosy: + gregcouch
messages: + msg74142
2008-08-26 13:35:10andersjmsetnosy: + andersjm
2008-03-02 07:58:17ocean-citysetfiles: + CreateProcessW.patch
keywords: + patch
messages: + msg63176
nosy: + ocean-city
2007-07-24 18:24:11mclauschcreate