Skip to content

Commit

Permalink
fix: close file properly in indexer_dbm.py:save_index()
Browse files Browse the repository at this point in the history
Fix this error found in debug logs of gentoo packaging of round 2.2.0.

/roundup/backends/indexer_dbm.py:253: ResourceWarning: unclosed file
<_io.BufferedWriter name='test-index/indexes/index.db-'>
    open(self.indexdb+'-', 'wb').write(zlib.compress(marshal.dumps(dbfil)))

Also added test that calls save_index(), reloads the index and tests
that the original item. I am not sure how Gentoo hit
this But they were missing a number of backends. So it's possible that
indexer_dbm.py is not getting fully tested depending on what is
installed on the system. Codecov from CI didnt show
indexer_dbm.py:save_index() being covered.
  • Loading branch information
rouilj committed Nov 2, 2023
1 parent 20d498c commit 4ca4e59
Show file tree
Hide file tree
Showing 3 changed files with 28 additions and 1 deletion.
1 change: 1 addition & 0 deletions CHANGES.txt
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ Fixed:
Rouillard)
- fix repeated password id with user.item.html in all templates except
jinja2. (John Rouillard)
- fix unclosed file when saving index in indexer_dbm.py. (John Rouillard)

Features:

Expand Down
4 changes: 3 additions & 1 deletion roundup/backends/indexer_dbm.py
Original file line number Diff line number Diff line change
Expand Up @@ -250,7 +250,9 @@ def save_index(self):

# First write the much simpler filename/fileid dictionaries
dbfil = {'WORDS': None, 'FILES': self.files, 'FILEIDS': self.fileids}
open(self.indexdb+'-', 'wb').write(zlib.compress(marshal.dumps(dbfil)))
marshal_fh = open(self.indexdb+'-', 'wb')
marshal_fh.write(zlib.compress(marshal.dumps(dbfil)))
marshal_fh.close()

# The hard part is splitting the word dictionary up, of course
letters = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ#_"
Expand Down
24 changes: 24 additions & 0 deletions test/test_indexer.py
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,30 @@ def test_basics(self):
self.assertSeqEqual(self.dex.find(['blah', 'hello']), [])
self.assertSeqEqual(self.dex.find([]), [])

def test_save_load(self):

# only run for anydbm test
if ( not type(self) is IndexerTest ):
pytest.skip("test_save_load tested only for anydbm backend")

self.dex.add_text(('test', '1', 'foo'), 'b the hello world')
self.assertSeqEqual(self.dex.find(['hello']), [('test', '1', 'foo')])
self.dex.save_index()

# reopen saved db.
from roundup.backends.indexer_dbm import Indexer
self.dex = Indexer(db)

# verify index is unloaded
self.assertEqual(self.dex.index_loaded(), False)

# add also calls load_index(), so it should load the first item.
self.dex.add_text(('test', '2', 'foo'), 'b the olleh world')

# note find also does a load_index() if not loaded.
self.assertSeqEqual(self.dex.find(['hello']), [('test', '1', 'foo')])
self.assertSeqEqual(self.dex.find(['olleh']), [('test', '2', 'foo')])

def test_change(self):
self.dex.add_text(('test', '1', 'foo'), 'a the hello world')
self.dex.add_text(('test', '2', 'foo'), 'blah blah the world')
Expand Down

0 comments on commit 4ca4e59

Please sign in to comment.