Issue3675
Created on 2008-08-25 12:15 by hagen, last changed 2008-10-04 23:14 by alexandre.vassalotti.
| File name |
Uploaded |
Description |
Edit |
Remove |
|
pickle.diff
|
ajaksu2,
2008-08-28 17:51
|
Adds a 3.0:2.6 modules mapping and uses that in find_class |
|
|
| msg71916 (view) |
Author: Hagen Fürstenau (hagen) |
Date: 2008-08-25 12:15 |
|
After pickling a set of ints with Python 3.0 and pickle protocol 2:
[hagenf@chage ~]$ python3.0
Python 3.0b3 (r30b3:65927, Aug 21 2008, 11:48:29)
[GCC 4.1.0 20060304 (Red Hat 4.1.0-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> f = open("test", "wb")
>>> pickle.dump({1,2,3}, f, 2)
>>> f.close()
I get the following error when trying to read this with Python 2.6:
[hagenf@chage ~]$ python
Python 2.6b3 (r26b3:65922, Aug 21 2008, 11:42:25)
[GCC 4.1.0 20060304 (Red Hat 4.1.0-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> f = open("test", "rb")
>>> pickle.load(f)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/MC/hagenf/local/lib/python2.6/pickle.py", line 1370, in load
return Unpickler(file).load()
File "/home/MC/hagenf/local/lib/python2.6/pickle.py", line 858, in load
dispatch[key](self)
File "/home/MC/hagenf/local/lib/python2.6/pickle.py", line 1090, in
load_global
klass = self.find_class(module, name)
File "/home/MC/hagenf/local/lib/python2.6/pickle.py", line 1124, in
find_class
__import__(module)
ImportError: No module named builtins
|
| msg72026 (view) |
Author: Hagen Fürstenau (hagen) |
Date: 2008-08-27 13:03 |
|
Well, this is obviously caused by renaming "__builtin__" to "builtins"
and the fact that set (as well as frozenset) doesn't have its own opcode
and therefore gets looked up in "builtins". The problem therefore
extends to all builtin objects without opcode special casing (e.g.
object, slice, property, ...) I'm afraid that means we have to pickle
"builtins" as "__builtin__" for backwards compatibility in protocols <= 2.
But aside from that, wouldn't it be more consistent to have opcodes for
set/frozenset in protocol 3?
|
| msg72096 (view) |
Author: Daniel Diniz (ajaksu2) |
Date: 2008-08-28 16:40 |
|
Hagen,
does this simple patch (against 2.6) solve it for you?
Index: Lib/pickle.py
===================================================================
--- Lib/pickle.py (revision 66050)
+++ Lib/pickle.py (working copy)
@@ -1121,6 +1121,8 @@
def find_class(self, module, name):
# Subclasses may override this
+ if module == "builtins":
+ module = "__builtin__"
__import__(module)
mod = sys.modules[module]
klass = getattr(mod, name)
I think a dict mapping the moved modules would work better, perhaps
having it in PyPI would be enough?
|
| msg72097 (view) |
Author: Hagen Fürstenau (hagen) |
Date: 2008-08-28 16:52 |
|
Well, Python <= 2.5 still wouldn't be able to unpickle those built in
objects.
|
| msg72100 (view) |
Author: Daniel Diniz (ajaksu2) |
Date: 2008-08-28 17:51 |
|
FWIW, there's a mapping of 2.6:3.0 modules in lib2to3:
from lib2to3.fixes.fix_imports import MAPPING
The attached patch uses that for a quick and dirty way of loading 3.0
pickles in 2.6.
|
| msg72164 (view) |
Author: Terry J. Reedy (tjreedy) |
Date: 2008-08-29 21:54 |
|
Was it really intended that 3.0 pickles unpickle on 2.6?
What about other changes like moving something from one module to
another (reduce from built-in to functools), changing all classes to new
style (several examples), or adding methods to a built-in class (floats)?
|
| msg72165 (view) |
Author: Antoine Pitrou (pitrou) |
Date: 2008-08-29 22:02 |
|
> Was it really intended that 3.0 pickles unpickle on 2.6?
He used protocol 2, so he explicitly asked for something inpickleable
with 2.6. If it's not the intended behaviour, then protocols < 3 should
be deprecated.
|
| msg74328 (view) |
Author: Alexandre Vassalotti (alexandre.vassalotti) |
Date: 2008-10-04 23:14 |
|
I agree with Antoine, protocols <= 2 should remain compatible with
Python 2.x or be deprecated. Keeping compatibility will require a hack,
in addition to the proposed patch, in Pickler.save_global to map Python
3's module names to the ones of Python 2.
|
|
| Date |
User |
Action |
Args |
| 2008-10-04 23:14:44 | alexandre.vassalotti | set | messages:
+ msg74328 |
| 2008-08-29 22:02:11 | pitrou | set | nosy:
+ pitrou messages:
+ msg72165 |
| 2008-08-29 21:54:36 | tjreedy | set | nosy:
+ tjreedy messages:
+ msg72164 |
| 2008-08-28 17:51:42 | ajaksu2 | set | files:
+ pickle.diff keywords:
+ patch messages:
+ msg72100 |
| 2008-08-28 16:52:05 | hagen | set | messages:
+ msg72097 |
| 2008-08-28 16:41:00 | ajaksu2 | set | nosy:
+ ajaksu2 messages:
+ msg72096 |
| 2008-08-27 13:03:24 | hagen | set | messages:
+ msg72026 |
| 2008-08-25 15:58:24 | pitrou | set | priority: critical nosy:
+ alexandre.vassalotti type: behavior |
| 2008-08-25 12:15:41 | hagen | create | |
|