Navigation

dircache – Cache directory listings

Purpose:Cache directory listings, updating when the modification time of a directory changes.
Available In:1.4 and later

Listing Directory Contents

The main function in the dircache API is listdir(), a wrapper around os.listdir() that caches the results and returns the same list each time it is called with a given path, unless the modification date of the named directory changes.

import dircache

path = '.'
first = dircache.listdir(path)
second = dircache.listdir(path)

print 'Contents :', first
print 'Identical:', first is second
print 'Equal    :', first == second

It is important to recognize that the exact same list is returned each time, so it should not be modified in place.

$ python dircache_listdir.py

Contents : ['__init__.py', 'dircache_annotate.py', 'dircache_listdir.py', 'dircache_listdir_file_added.py', 'dircache_reset.py', 'index.rst']
Identical: True
Equal    : True

If the contents of the directory changes, it is rescanned.

import dircache
import os

path = '/tmp'
file_to_create = os.path.join(path, 'pymotw_tmp.txt')

# Look at the directory contents
first = dircache.listdir(path)

# Create the new file
open(file_to_create, 'wt').close()

# Rescan the directory
second = dircache.listdir(path)

# Remove the file we created
os.unlink(file_to_create)

print 'Identical :', first is second
print 'Equal     :', first == second
print 'Difference:', list(set(second) - set(first))

In this case the new file causes a new list to be constructed.

$ python dircache_listdir_file_added.py

Identical : False
Equal     : False
Difference: ['pymotw_tmp.txt']

It is also possible to reset the entire cache, discarding its contents so that each path will be rechecked.

import dircache

path = '/tmp'
first = dircache.listdir(path)
dircache.reset()
second = dircache.listdir(path)

print 'Identical :', first is second
print 'Equal     :', first == second
print 'Difference:', list(set(second) - set(first))

After resetting, a new list instance is returned.

$ python dircache_reset.py

Identical : False
Equal     : True
Difference: []

Annotated Listings

The other interesting function provided by the dircache module is annotate(). When called, annotate() modifies a list() such as is returned by listdir(), adding a '/' to the end of the names that represent directories.

import dircache
from pprint import pprint
import os

path = '../..'

contents = dircache.listdir(path)

annotated = contents[:]
dircache.annotate(path, annotated)

fmt = '%25s\t%25s'

print fmt % ('ORIGINAL', 'ANNOTATED')
print fmt % (('-' * 25,)*2)

for o, a in zip(contents, annotated):
    print fmt % (o, a)

Unfortunately for Windows users, although annotate() uses os.path.join() to construct names to test, it always appends a '/', not os.sep.

$ python dircache_annotate.py

                 ORIGINAL                       ANNOTATED
-------------------------       -------------------------
                      .hg                            .hg/
                 .hgcheck                       .hgcheck/
                .hgignore                       .hgignore
                  .hgtags                         .hgtags
              LICENSE.txt                     LICENSE.txt
              MANIFEST.in                     MANIFEST.in
                   PyMOTW                         PyMOTW/
          PyMOTW.egg-info                PyMOTW.egg-info/
               README.txt                      README.txt
                      bin                            bin/
                     dist                           dist/
                   module                          module
                     motw                            motw
              pavement.py                     pavement.py
             pavement.py~                    pavement.py~
        paver-minilib.zip               paver-minilib.zip
                 setup.py                        setup.py
   sitemap_gen_config.xml          sitemap_gen_config.xml
  sitemap_gen_config.xml~         sitemap_gen_config.xml~
                   sphinx                         sphinx/
                    utils                          utils/
                      web                            web/

See also

dircache
The standard library documentation for this module.

Navigation