fs.walk¶

Machinery for walking a filesystem.

Walking a filesystem means recursively visiting a directory and any sub-directories. It is a fairly common requirement for copying, searching etc. See Walking for details.

class fs.walk.BoundWalker(fs, walker_class=<class 'fs.walk.Walker'>)[source]¶

A class that binds a Walker instance to a FS instance.

You will typically not need to create instances of this class explicitly. Filesystems have a walk property which returns a BoundWalker object.

Example

>>> tmp_fs = fs.tempfs.TempFS()
>>> tmp_fs.walk
BoundWalker(TempFS())

A BoundWalker is callable. Calling it is an alias for the walk method.

__call__(path='/', namespaces=None, **kwargs)¶

Walk the directory structure of a filesystem.

Keyword Arguments:
Parameters:	path (str) – namespaces (list, optional) – A list of namespaces to include in the resource information, e.g. `['basic', 'access']` (defaults to `['basic']`).
	ignore_errors (bool) – If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable) – If `ignore_errors` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str) – If `'breadth'` then the directory will be walked top down. Set to `'depth'` to walk bottom up. filter (list) – If supplied, this parameter should be a list of file name patterns, e.g. `['.py']`. Files will only be returned if the final component matches one of the patterns. exclude* (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g. `['~', '.']`. Files matching any of these patterns will be removed from the walk. filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns. exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g. `['.svn', '.git']`. max_depth (int, optional) – Maximum directory depth to walk.
Returns:	an iterator of `(<path>, <dirs>, <files>)` named tuples, where `<path>` is an absolute path to a directory, and `<dirs>` and `<files>` are a list of `Info` objects for directories and files in `<path>`.
Return type:	Iterator

Example

>>> walker = Walker(filter=['*.py'])
>>> for path, dirs, files in walker.walk(my_fs, namespaces=['details']):
...     print("[{}]".format(path))
...     print("{} directories".format(len(dirs)))
...     total = sum(info.size for info in files)
...     print("{} bytes".format(total))
[/]
2 directories
55 bytes
...

This method invokes Walker.walk with bound FS object.

__init__(fs, walker_class=<class 'fs.walk.Walker'>)[source]¶

Create a new walker bound to the given filesystem.

Parameters:	fs (FS) – A filesystem instance. walker_class (type) – A `WalkerBase` sub-class. The default uses `Walker`.

dirs(path='/', **kwargs)[source]¶

Walk a filesystem, yielding absolute paths to directories.

Keyword Arguments:
Parameters:	path (str) – A path to a directory.
	ignore_errors (bool) – If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable) – If `ignore_errors` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str) – If `'breadth'` then the directory will be walked top down. Set to `'depth'` to walk bottom up. filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns. exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g. `['.svn', '.git']`. max_depth (int, optional) – Maximum directory depth to walk.
Returns:	an iterator over directory paths (absolute from the filesystem root).
Return type:	Iterator

This method invokes Walker.dirs with the bound FS object.

files(path='/', **kwargs)[source]¶

Walk a filesystem, yielding absolute paths to files.

Keyword Arguments:
Parameters:	path (str) – A path to a directory.
	ignore_errors (bool) – If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable) – If `ignore_errors` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str) – If `'breadth'` then the directory will be walked top down. Set to `'depth'` to walk bottom up. filter (list) – If supplied, this parameter should be a list of file name patterns, e.g. `['.py']`. Files will only be returned if the final component matches one of the patterns. exclude* (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g. `['~', '.']`. Files matching any of these patterns will be removed from the walk. filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns. exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g. `['.svn', '.git']`. max_depth (int, optional) – Maximum directory depth to walk.
Returns:	An iterator over file paths (absolute from the filesystem root).
Return type:	Iterator

This method invokes Walker.files with the bound FS object.

info(path='/', namespaces=None, **kwargs)[source]¶

Walk a filesystem, yielding path and Info of resources.

Keyword Arguments:
Parameters:	path (str) – A path to a directory. namespaces (list, optional) – A list of namespaces to include in the resource information, e.g. `['basic', 'access']` (defaults to `['basic']`).
	ignore_errors (bool) – If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable) – If `ignore_errors` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str) – If `'breadth'` then the directory will be walked top down. Set to `'depth'` to walk bottom up. filter (list) – If supplied, this parameter should be a list of file name patterns, e.g. `['.py']`. Files will only be returned if the final component matches one of the patterns. exclude* (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g. `['~', '.']`. Files matching any of these patterns will be removed from the walk. filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns. exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g. `['.svn', '.git']`. max_depth (int, optional) – Maximum directory depth to walk.
Returns:	an iterable yielding tuples of `(<absolute path>, <resource info>)`.
Return type:	Iterable

This method invokes Walker.info with the bound FS object.

walk(path='/', namespaces=None, **kwargs)[source]¶

Walk the directory structure of a filesystem.

Keyword Arguments:
Parameters:	path (str) – namespaces (list, optional) – A list of namespaces to include in the resource information, e.g. `['basic', 'access']` (defaults to `['basic']`).
	ignore_errors (bool) – If `True`, any errors reading a directory will be ignored, otherwise exceptions will be raised. on_error (callable) – If `ignore_errors` is `False`, then this callable will be invoked with a path and the exception object. It should return `True` to ignore the error, or `False` to re-raise it. search (str) – If `'breadth'` then the directory will be walked top down. Set to `'depth'` to walk bottom up. filter (list) – If supplied, this parameter should be a list of file name patterns, e.g. `['.py']`. Files will only be returned if the final component matches one of the patterns. exclude* (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g. `['~', '.']`. Files matching any of these patterns will be removed from the walk. filter_dirs (list, optional) – A list of patterns that will be used to match directories paths. The walk will only open directories that match at least one of these patterns. exclude_dirs (list) – A list of patterns that will be used to filter out directories from the walk, e.g. `['.svn', '.git']`. max_depth (int, optional) – Maximum directory depth to walk.
Returns:	an iterator of `(<path>, <dirs>, <files>)` named tuples, where `<path>` is an absolute path to a directory, and `<dirs>` and `<files>` are a list of `Info` objects for directories and files in `<path>`.
Return type:	Iterator

Example

>>> walker = Walker(filter=['*.py'])
>>> for path, dirs, files in walker.walk(my_fs, namespaces=['details']):
...     print("[{}]".format(path))
...     print("{} directories".format(len(dirs)))
...     total = sum(info.size for info in files)
...     print("{} bytes".format(total))
[/]
2 directories
55 bytes
...

This method invokes Walker.walk with bound FS object.

class fs.walk.Step(path, dirs, files)¶

type: a step in a directory walk.

dirs¶: Alias for field number 1

files¶: Alias for field number 2

path¶: Alias for field number 0

class fs.walk.Walker(ignore_errors=False, on_error=None, search='breadth', filter=None, exclude=None, filter_dirs=None, exclude_dirs=None, max_depth=None, filter_glob=None, exclude_glob=None)[source]¶

A walker object recursively lists directories in a filesystem.

__init__(ignore_errors=False, on_error=None, search='breadth', filter=None, exclude=None, filter_dirs=None, exclude_dirs=None, max_depth=None, filter_glob=None, exclude_glob=None)[source]¶

Create a new Walker instance.

Parameters:

ignore_errors (bool) – If True, any errors reading a directory will be ignored, otherwise exceptions will be raised.
on_error (callable, optional) – If ignore_errors is False, then this callable will be invoked for a path and the exception object. It should return True to ignore the error, or False to re-raise it.
search (str) – If "breadth" then the directory will be walked top down. Set to "depth" to walk bottom up.
filter (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g. ["*.py"]. Files will only be returned if the final component matches one of the patterns.
exclude (list, optional) – If supplied, this parameter should be a list of filename patterns, e.g. ["~*"]. Files matching any of these patterns will be removed from the walk.
filter_dirs (list, optional) – A list of patterns that will be used to match directories names. The walk will only open directories that match at least one of these patterns. Directories will only be returned if the final component matches one of the patterns.
exclude_dirs (list, optional) – A list of patterns that will be used to filter out directories from the walk. e.g. ['*.svn', '*.git']. Directory names matching any of these patterns will be removed from the walk.
max_depth (int, optional) – Maximum directory depth to walk.
filter_glob (list, optional) – If supplied, this parameter should be a list of path patterns e.g. ["foo/**/*.py"]. Resources will only be returned if their global path or an extension of it matches one of the patterns.
exclude_glob (list, optional) – If supplied, this parameter should be a list of path patterns e.g. ["foo/**/*.pyc"]. Resources will not be returned if their global path or an extension of it matches one of the patterns.

classmethod bind(fs)[source]¶

Bind a Walker instance to a given filesystem.

This binds in instance of the Walker to a given filesystem, so that you won’t need to explicitly provide the filesystem as a parameter.

Parameters:	fs (FS) – A filesystem object.
Returns:	a bound walker.
Return type:	BoundWalker

Examples

Use this method to explicitly bind a filesystem instance:

>>> walker = Walker.bind(my_fs)
>>> for path in walker.files(filter=['*.py']):
...     print(path)
/foo.py
/bar.py

Unless you have written a customized walker class, you will be unlikely to need to call this explicitly, as filesystem objects already have a walk attribute which is a bound walker object:

>>> for path in my_fs.walk.files(filter=['*.py']):
...     print(path)
/foo.py
/bar.py

check_file(fs, info)[source]¶

Check if a filename should be included.

Override to exclude files from the walk.

Parameters:	fs (FS) – A filesystem instance. info (Info) – A resource info object.
Returns:	`True` if the file should be included.
Return type:	bool

check_open_dir(fs, path, info)[source]¶

Check if a directory should be opened.

Override to exclude directories from the walk.

Parameters:	fs (FS) – A filesystem instance. path (str) – Path to directory. info (Info) – A resource info object for the directory.
Returns:	`True` if the directory should be opened.
Return type:	bool

check_scan_dir(fs, path, info)[source]¶

Check if a directory should be scanned.

Override to omit scanning of certain directories. If a directory is omitted, it will appear in the walk but its files and sub-directories will not.

Parameters:	fs (FS) – A filesystem instance. path (str) – Path to directory. info (Info) – A resource info object for the directory.
Returns:	`True` if the directory should be scanned.
Return type:	bool

dirs(fs, path='/')[source]¶

Walk a filesystem, yielding absolute paths to directories.

Parameters:	fs (FS) – A filesystem instance. path (str) – A path to a directory on the filesystem.
Yields:	str – absolute path to directories on the filesystem found recursively within the given directory.

files(fs, path='/')[source]¶

Walk a filesystem, yielding absolute paths to files.

Parameters:	fs (FS) – A filesystem instance. path (str) – A path to a directory on the filesystem.
Yields:	str – absolute path to files on the filesystem found recursively within the given directory.

info(fs, path='/', namespaces=None)[source]¶

Walk a filesystem, yielding tuples of (<path>, <info>).

Parameters:	fs (FS) – A filesystem instance. path (str) – A path to a directory on the filesystem. namespaces (list, optional) – A list of additional namespaces to add to the `Info` objects.
Yields:	(str, Info) – a tuple of `(<absolute path>, <resource info>)`.

walk(fs, path='/', namespaces=None)[source]¶

Walk the directory structure of a filesystem.

Parameters:	fs (FS) – A filesystem instance. path (str) – A path to a directory on the filesystem. namespaces (list, optional) – A list of additional namespaces to add to the `Info` objects.
Returns:	an iterator of `Step` instances.
Return type:	collections.Iterator

The return value is an iterator of (<path>, <dirs>, <files>) named tuples, where <path> is an absolute path to a directory, and <dirs> and <files> are a list of Info objects for directories and files in <path>.

Example

>>> walker = Walker(filter=['*.py'])
>>> for path, dirs, files in walker.walk(my_fs, namespaces=["details"]):
...    print("[{}]".format(path))
...    print("{} directories".format(len(dirs)))
...    total = sum(info.size for info in files)
...    print("{} bytes".format(total))
[/]
2 directories
55 bytes
...