File gets truncated upon unpickling

[JanPokorny]

It seems that whenever a file is open for writing in the session, the (already closed) file object f gets unpickled in such a way that re-opens the file in writing mode, effectively truncating the file to zero length.

Is this expected behavior?

❯ python -c "
  with open('file.txt', 'w') as f:
    f.write('hello world')
  import dill
  dill.dump_module('session.pkl')
  "

❯ cat file.txt
hello world⏎                                                                                                                                                                                                                                                                                                                                                                                                   

❯ python -c "
  import dill
  dill.load_module('session.pkl')
  "

❯ cat file.txt

❯

[akiss-ic]

This issue was problematic for me even with the file that contained the session data. if I did something like

x = 1
y = 2
with open("foo.pckl", "wb") as file:
	dill.dump_session(file)

and then

with open("foo.pckl", "rb") as file:
	dill.load_session(file)

in another interpreter session/notebook I would see it start to load and then the pickle file go to zero bytes and a UnpicklingError: pickle data was truncated was thrown because the pickle file was changed underneath its feet (not to mention all the data I dumped to the pickle file is now gone!). It seems that if any file object instantiated with write permissions is dumped and then loaded, even if it was closed before dumping, it will re-open the file on loading and delete all the data it contains. This seems incredibly dangerous.

The way I got around this was to never instantiate a file object but rather pass the path object or path string to dill.dump_session() directly. This obviously works in this narrow case, but not in the general case.

[JanPokorny]

This could probably be solved by unpickling files open for write in "write append" mode. Seems like a more logical/rational/expected/intuitive behavior.

[akiss-ic]

Agreed, or to be extra safe open them in a read only mode and allow the caller to re-open in a write mode if needed. It isn't intuitive to me that a file object should be opened at all when un-pickled if it was closed prior to pickling.