shelve —- Python object persistence

Source code:Lib/shelve.py

A "shelf" is a persistent, dictionary-like object. The difference with "dbm"databases is that the values (not the keys!) in a shelf can be essentiallyarbitrary Python objects —- anything that the pickle module can handle.This includes most class instances, recursive data types, and objects containinglots of shared sub-objects. The keys are ordinary strings.

  • shelve.open(filename, flag='c', protocol=None, writeback=False)
  • Open a persistent dictionary. The filename specified is the base filename forthe underlying database. As a side-effect, an extension may be added to thefilename and more than one file may be created. By default, the underlyingdatabase file is opened for reading and writing. The optional flag parameterhas the same interpretation as the flag parameter of dbm.open().

By default, version 3 pickles are used to serialize values. The version of thepickle protocol can be specified with the protocol parameter.

Because of Python semantics, a shelf cannot know when a mutablepersistent-dictionary entry is modified. By default modified objects arewritten only when assigned to the shelf (see 示例). If theoptional writeback parameter is set to True, all entries accessed are alsocached in memory, and written back on sync() andclose(); this can make it handier to mutate mutable entries inthe persistent dictionary, but, if many entries are accessed, it can consumevast amounts of memory for the cache, and it can make the close operationvery slow since all accessed entries are written back (there is no way todetermine which accessed entries are mutable, nor which ones were actuallymutated).

注解

Do not rely on the shelf being closed automatically; always callclose() explicitly when you don't need it any more, oruse shelve.open() as a context manager:

  1. with shelve.open('spam') as db:
  2.     db['eggs'] = 'eggs'

警告

Because the shelve module is backed by pickle, it is insecureto load a shelf from an untrusted source. Like with pickle, loading a shelfcan execute arbitrary code.

Shelf objects support all methods supported by dictionaries. This eases thetransition from dictionary based scripts to those requiring persistent storage.

Two additional methods are supported:

  • Shelf.sync()
  • Write back all entries in the cache if the shelf was opened with _writeback_set to True. Also empty the cache and synchronize the persistentdictionary on disk, if feasible. This is called automatically when the shelfis closed with close().
  • Shelf.close()
  • Synchronize and close the persistent dict object. Operations on a closedshelf will fail with a ValueError.

参见

Persistent dictionary recipewith widely supported storage formats and having the speed of nativedictionaries.

Restrictions

  • The choice of which database package will be used (such as dbm.ndbm ordbm.gnu) depends on which interface is available. Therefore it is notsafe to open the database directly using dbm. The database is also(unfortunately) subject to the limitations of dbm, if it is used —-this means that (the pickled representation of) the objects stored in thedatabase should be fairly small, and in rare cases key collisions may causethe database to refuse updates.

  • The shelve module does not support concurrent read/write access toshelved objects. (Multiple simultaneous read accesses are safe.) When aprogram has a shelf open for writing, no other program should have it open forreading or writing. Unix file locking can be used to solve this, but thisdiffers across Unix versions and requires knowledge about the databaseimplementation used.

  • class shelve.Shelf(dict, protocol=None, writeback=False, keyencoding='utf-8')
  • A subclass of collections.abc.MutableMapping which stores pickledvalues in the dict object.

By default, version 3 pickles are used to serialize values. The version of thepickle protocol can be specified with the protocol parameter. See thepickle documentation for a discussion of the pickle protocols.

If the writeback parameter is True, the object will hold a cache of allentries accessed and write them back to the dict at sync and close times.This allows natural operations on mutable entries, but can consume much morememory and make sync and close take a long time.

The keyencoding parameter is the encoding used to encode keys before theyare used with the underlying dict.

A Shelf object can also be used as a context manager, in whichcase it will be automatically closed when the with block ends.

在 3.2 版更改: Added the keyencoding parameter; previously, keys were always encoded inUTF-8.

在 3.4 版更改: 添加了上下文管理器支持。

  • class shelve.BsdDbShelf(dict, protocol=None, writeback=False, keyencoding='utf-8')
  • A subclass of Shelf which exposes first(), next(),previous(), last() and setlocation() which are availablein the third-party bsddb module from pybsddb but not in other databasemodules. The _dict object passed to the constructor must support thosemethods. This is generally accomplished by calling one ofbsddb.hashopen(), bsddb.btopen() or bsddb.rnopen(). Theoptional protocol, writeback, and keyencoding parameters have the sameinterpretation as for the Shelf class.
  • class shelve.DbfilenameShelf(filename, flag='c', protocol=None, writeback=False)
  • A subclass of Shelf which accepts a filename instead of a dict-likeobject. The underlying file will be opened using dbm.open(). Bydefault, the file will be created and opened for both read and write. Theoptional flag parameter has the same interpretation as for the open()function. The optional protocol and writeback parameters have the sameinterpretation as for the Shelf class.

示例

To summarize the interface (key is a string, data is an arbitraryobject):

  1. import shelve
  2.  
  3. d = shelve.open(filename)  # open -- file may get suffix added by low-level
  4.                            # library
  5.  
  6. d[key] = data              # store data at key (overwrites old data if
  7.                            # using an existing key)
  8. data = d[key]              # retrieve a COPY of data at key (raise KeyError
  9.                            # if no such key)
  10. del d[key]                 # delete data stored at key (raises KeyError
  11.                            # if no such key)
  12.  
  13. flag = key in d            # true if the key exists
  14. klist = list(d.keys())     # a list of all existing keys (slow!)
  15.  
  16. # as d was opened WITHOUT writeback=True, beware:
  17. d['xx'] = [0, 1, 2]        # this works as expected, but...
  18. d['xx'].append(3)          # *this doesn't!* -- d['xx'] is STILL [0, 1, 2]!
  19.  
  20. # having opened d without writeback=True, you need to code carefully:
  21. temp = d['xx']             # extracts the copy
  22. temp.append(5)             # mutates the copy
  23. d['xx'] = temp             # stores the copy right back, to persist it
  24.  
  25. # or, d=shelve.open(filename,writeback=True) would let you just code
  26. # d['xx'].append(5) and have it work as expected, BUT it would also
  27. # consume more memory and make the d.close() operation slower.
  28.  
  29. d.close()                  # close it

参见

  • Module dbm

  • Generic interface to dbm-style databases.

  • 模块 pickle

  • Object serialization used by shelve.