Multi Filesystem¶
A MultiFS is a filesystem composed of a sequence of other filesystems, where the directory structure of each overlays the previous filesystem in the sequence.
One use for such a filesystem would be to selectively override a set of files, to customize behavior. For example, to create a filesystem that could be used to theme a web application. We start with the following directories:
`-- templates
|-- snippets
| `-- panel.html
|-- index.html
|-- profile.html
`-- base.html
`-- theme
|-- snippets
| |-- widget.html
| `-- extra.html
|-- index.html
`-- theme.html
And we want to create a single filesystem that will load a file from
templates/
only if it isn’t found in theme/
. Here’s how we could
do that:
from fs.osfs import OSFS
from fs.multifs import MultiFS
theme_fs = MultiFS()
theme_fs.add_fs('templates', OSFS('templates'))
theme_fs.add_fs('theme', OSFS('theme'))
Now we have a theme_fs
filesystem that presents a single view of both
directories:
|-- snippets
| |-- panel.html
| |-- widget.html
| `-- extra.html
|-- index.html
|-- profile.html
|-- base.html
`-- theme.html
-
class
fs.multifs.
MultiFS
(auto_close: bool = True)[source]¶ A filesystem that delegates to a sequence of other filesystems.
Operations on the MultiFS will try each ‘child’ filesystem in order, until it succeeds. In effect, creating a filesystem that combines the files and dirs of its children.
-
__init__
(auto_close: bool = True) → None[source]¶ Create a new MultiFS.
- Parameters
auto_close (bool) – If
True
(the default), the child filesystems will be closed whenMultiFS
is closed.
-
add_fs
(name: Text, fs: FS, write: bool = False, priority: int = 0) → None[source]¶ Add a filesystem to the MultiFS.
- Parameters
name (str) – A unique name to refer to the filesystem being added.
fs (FS or str) – The filesystem (instance or URL) to add.
write (bool) – If this value is True, then the
fs
will be used as the writeable FS (defaults to False).priority (int) – An integer that denotes the priority of the filesystem being added. Filesystems will be searched in descending priority order and then by the reverse order they were added. So by default, the most recently added filesystem will be looked at first.
-
close
() → None[source]¶ Close the filesystem and release any resources.
It is important to call this method when you have finished working with the filesystem. Some filesystems may not finalize changes until they are closed (archives for example). You may call this method explicitly (it is safe to call close multiple times), or you can use the filesystem as a context manager to automatically close.
Example
>>> with OSFS('~/Desktop') as desktop_fs: ... desktop_fs.writetext( ... 'note.txt', ... "Don't forget to tape Game of Thrones" ... )
If you attempt to use a filesystem that has been closed, a
FilesystemClosed
exception will be thrown.
-
download
(path: Text, file: BinaryIO, chunk_size: Optional[int] = None, **options: Any) → None[source]¶ Copy a file from the filesystem to a file-like object.
This may be more efficient that opening and copying files manually if the filesystem supplies an optimized method.
Note that the file object
file
will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).- Parameters
path (str) – Path to a resource.
file (file-like) – A file-like object open for writing in binary mode.
chunk_size (int, optional) – Number of bytes to read at a time, if a simple copy is used, or
None
to use sensible default.**options – Implementation specific options required to open the source file.
Example
>>> with open('starwars.mov', 'wb') as write_file: ... my_fs.download('/Videos/starwars.mov', write_file)
- Raises
fs.errors.ResourceNotFound – if
path
does not exist.
-
get_fs
(name: Text) → FS[source]¶ Get a filesystem from its name.
- Parameters
name (str) – The name of a filesystem previously added.
- Returns
the filesystem added as
name
previously.- Return type
- Raises
KeyError – If no filesystem with given
name
could be found.
-
getinfo
(path: Text, namespaces: Optional[Collection[Text]] = None) → Info[source]¶ Get information about a resource on a filesystem.
- Parameters
path (str) – A path to a resource on the filesystem.
namespaces (list, optional) – Info namespaces to query. The
"basic"
namespace is alway included in the returned info, whatever the value ofnamespaces
may be.
- Returns
resource information object.
- Return type
- Raises
fs.errors.ResourceNotFound – If
path
does not exist.
For more information regarding resource information, see Resource Info.
-
getsize
(path: Text) → int[source]¶ Get the size (in bytes) of a resource.
- Parameters
path (str) – A path to a resource.
- Returns
the size of the resource.
- Return type
int
- Raises
fs.errors.ResourceNotFound – if
path
does not exist.
The size of a file is the total number of readable bytes, which may not reflect the exact number of bytes of reserved disk space (or other storage medium).
The size of a directory is the number of bytes of overhead use to store the directory entry.
-
getsyspath
(path: Text) → Text[source]¶ Get the system path of a resource.
- Parameters
path (str) – A path on the filesystem.
- Returns
the system path of the resource, if any.
- Return type
str
- Raises
fs.errors.NoSysPath – If there is no corresponding system path.
A system path is one recognized by the OS, that may be used outside of PyFilesystem (in an application or a shell for example). This method will get the corresponding system path that would be referenced by
path
.Not all filesystems have associated system paths. Network and memory based filesystems, for example, may not physically store data anywhere the OS knows about. It is also possible for some paths to have a system path, whereas others don’t.
This method will always return a str on Py3.* and unicode on Py2.7. See
getospath
if you need to encode the path as bytes.If
path
doesn’t have a system path, aNoSysPath
exception will be thrown.Note
A filesystem may return a system path even if no resource is referenced by that path – as long as it can be certain what that system path would be.
-
gettype
(path: Text) → ResourceType[source]¶ Get the type of a resource.
- Parameters
path (str) – A path on the filesystem.
- Returns
the type of the resource.
- Return type
- Raises
fs.errors.ResourceNotFound – if
path
does not exist.
A type of a resource is an integer that identifies the what the resource references. The standard type integers may be one of the values in the
ResourceType
enumerations.The most common resource types, supported by virtually all filesystems are
directory
(1) andfile
(2), but the following types are also possible:ResourceType
value
unknown
0
directory
1
file
2
character
3
block_special_file
4
fifo
5
socket
6
symlink
7
Standard resource types are positive integers, negative values are reserved for implementation specific resource types.
-
geturl
(path: Text, purpose: Text = 'download') → Text[source]¶ Get the URL to a given resource.
- Parameters
path (str) – A path on the filesystem
purpose (str) – A short string that indicates which URL to retrieve for the given path (if there is more than one). The default is
'download'
, which should return a URL that serves the file. Other filesystems may support other values forpurpose
.
- Returns
a URL.
- Return type
str
- Raises
fs.errors.NoURL – If the path does not map to a URL.
-
hassyspath
(path: Text) → bool[source]¶ Check if a path maps to a system path.
- Parameters
path (str) – A path on the filesystem.
- Returns
True
if the resource atpath
has a syspath.- Return type
bool
-
hasurl
(path: Text, purpose: Text = 'download') → bool[source]¶ Check if a path has a corresponding URL.
- Parameters
path (str) – A path on the filesystem.
purpose (str) – A purpose parameter, as given in
geturl
.
- Returns
True
if an URL for the given purpose exists.- Return type
bool
-
isdir
(path: Text) → bool[source]¶ Check if a path maps to an existing directory.
- Parameters
path (str) – A path on the filesystem.
- Returns
True
ifpath
maps to a directory.- Return type
bool
-
isfile
(path: Text) → bool[source]¶ Check if a path maps to an existing file.
- Parameters
path (str) – A path on the filesystem.
- Returns
True
ifpath
maps to a file.- Return type
bool
-
iterate_fs
() → Iterator[Tuple[Text, FS]][source]¶ Get iterator that returns (name, fs) in priority order.
-
listdir
(path: Text) → List[Text][source]¶ Get a list of the resource names in a directory.
This method will return a list of the resources in a directory. A resource is a file, directory, or one of the other types defined in
ResourceType
.- Parameters
path (str) – A path to a directory on the filesystem
- Returns
list of names, relative to
path
.- Return type
list
- Raises
fs.errors.DirectoryExpected – If
path
is not a directory.fs.errors.ResourceNotFound – If
path
does not exist.
-
makedir
(path: Text, permissions: Optional[Permissions] = None, recreate: bool = False) → SubFS[FS][source]¶ Make a directory.
- Parameters
path (str) – Path to directory from root.
permissions (Permissions, optional) – a
Permissions
instance, orNone
to use default.recreate (bool) – Set to
True
to avoid raising an error if the directory already exists (defaults toFalse
).
- Returns
a filesystem whose root is the new directory.
- Return type
- Raises
fs.errors.DirectoryExists – If the path already exists.
fs.errors.ResourceNotFound – If the path is not found.
-
makedirs
(path: Text, permissions: Optional[Permissions] = None, recreate: bool = False) → SubFS[FS][source]¶ Make a directory, and any missing intermediate directories.
- Parameters
path (str) – Path to directory from root.
permissions (Permissions, optional) – Initial permissions, or
None
to use defaults.recreate (bool) – If
False
(the default), attempting to create an existing directory will raise an error. Set toTrue
to ignore existing directories.
- Returns
A sub-directory filesystem.
- Return type
- Raises
fs.errors.DirectoryExists – if the path is already a directory, and
recreate
isFalse
.fs.errors.DirectoryExpected – if one of the ancestors in the path is not a directory.
-
open
(path: Text, mode: Text = 'r', buffering: int = -1, encoding: Optional[Text] = None, errors: Optional[Text] = None, newline: Text = '', **kwargs: Any) → IO[source]¶ Open a file.
- Parameters
path (str) – A path to a file on the filesystem.
mode (str) – Mode to open the file object with (defaults to r).
buffering (int) – Buffering policy (-1 to use default buffering, 0 to disable buffering, 1 to select line buffering, of any positive integer to indicate a buffer size).
encoding (str) – Encoding for text files (defaults to
utf-8
)errors (str, optional) – What to do with unicode decode errors (see
codecs
module for more information).newline (str) – Newline parameter.
**options – keyword arguments for any additional information required by the filesystem (if any).
- Returns
a file-like object.
- Return type
io.IOBase
- Raises
fs.errors.FileExpected – If the path is not a file.
fs.errors.FileExists – If the file exists, and exclusive mode is specified (
x
in the mode).fs.errors.ResourceNotFound – If the path does not exist.
-
openbin
(path: Text, mode: Text = 'r', buffering: int = -1, **options: Any) → BinaryIO[source]¶ Open a binary file-like object.
- Parameters
path (str) – A path on the filesystem.
mode (str) – Mode to open file (must be a valid non-text mode, defaults to r). Since this method only opens binary files, the
b
in the mode string is implied.buffering (int) – Buffering policy (-1 to use default buffering, 0 to disable buffering, or any positive integer to indicate a buffer size).
**options – keyword arguments for any additional information required by the filesystem (if any).
- Returns
a file-like object.
- Return type
io.IOBase
- Raises
fs.errors.FileExpected – If
path
exists and is not a file.fs.errors.FileExists – If the
path
exists, and exclusive mode is specified (x
in the mode).fs.errors.ResourceNotFound – If
path
does not exist andmode
does not imply creating the file, or if any ancestor ofpath
does not exist.
-
readbytes
(path: Text) → bytes[source]¶ Get the contents of a file as bytes.
- Parameters
path (str) – A path to a readable file on the filesystem.
- Returns
the file contents.
- Return type
bytes
- Raises
fs.errors.FileExpected – if
path
exists but is not a file.fs.errors.ResourceNotFound – if
path
does not exist.
-
readtext
(path: Text, encoding: Optional[Text] = None, errors: Optional[Text] = None, newline: Text = '') → Text[source]¶ Get the contents of a file as a string.
- Parameters
path (str) – A path to a readable file on the filesystem.
encoding (str, optional) – Encoding to use when reading contents in text mode (defaults to
None
, reading in binary mode).errors (str, optional) – Unicode errors parameter.
newline (str) – Newlines parameter.
- Returns
file contents.
- Return type
str
- Raises
fs.errors.ResourceNotFound – If
path
does not exist.
-
remove
(path: Text) → None[source]¶ Remove a file from the filesystem.
- Parameters
path (str) – Path of the file to remove.
- Raises
fs.errors.FileExpected – If the path is a directory.
fs.errors.ResourceNotFound – If the path does not exist.
-
removedir
(path: Text) → None[source]¶ Remove a directory from the filesystem.
- Parameters
path (str) – Path of the directory to remove.
- Raises
fs.errors.DirectoryNotEmpty – If the directory is not empty ( see
removetree
for a way to remove the directory contents).fs.errors.DirectoryExpected – If the path does not refer to a directory.
fs.errors.ResourceNotFound – If no resource exists at the given path.
fs.errors.RemoveRootError – If an attempt is made to remove the root directory (i.e.
'/'
)
-
scandir
(path: Text, namespaces: Optional[Collection[Text]] = None, page: Optional[Tuple[int, int]] = None) → Iterator[Info][source]¶ Get an iterator of resource info.
- Parameters
path (str) – A path to a directory on the filesystem.
namespaces (list, optional) – A list of namespaces to include in the resource information, e.g.
['basic', 'access']
.page (tuple, optional) – May be a tuple of
(<start>, <end>)
indexes to return an iterator of a subset of the resource info, orNone
to iterate over the entire directory. Paging a directory scan may be necessary for very large directories.
- Returns
an iterator of
Info
objects.- Return type
Iterator
- Raises
fs.errors.DirectoryExpected – If
path
is not a directory.fs.errors.ResourceNotFound – If
path
does not exist.
-
setinfo
(path: Text, info: RawInfo) → None[source]¶ Set info on a resource.
This method is the complement to
getinfo
and is used to set info values on a resource.- Parameters
path (str) – Path to a resource on the filesystem.
info (dict) – Dictionary of resource info.
- Raises
fs.errors.ResourceNotFound – If
path
does not exist on the filesystem
The
info
dict should be in the same format as the raw info returned bygetinfo(file).raw
.Example
>>> details_info = {"details": { ... "modified": time.time() ... }} >>> my_fs.setinfo('file.txt', details_info)
-
upload
(path: Text, file: BinaryIO, chunk_size: Optional[int] = None, **options: Any) → None[source]¶ Set a file to the contents of a binary file object.
This method copies bytes from an open binary file to a file on the filesystem. If the destination exists, it will first be truncated.
- Parameters
path (str) – A path on the filesystem.
file (io.IOBase) – a file object open for reading in binary mode.
chunk_size (int, optional) – Number of bytes to read at a time, if a simple copy is used, or
None
to use sensible default.**options – Implementation specific options required to open the source file.
- Raises
fs.errors.ResourceNotFound – If a parent directory of
path
does not exist.
Note that the file object
file
will not be closed by this method. Take care to close it after this method completes (ideally with a context manager).Example
>>> with open('~/movies/starwars.mov', 'rb') as read_file: ... my_fs.upload('starwars.mov', read_file)
-
validatepath
(path: Text) → Text[source]¶ Validate a path, returning a normalized absolute path on sucess.
Many filesystems have restrictions on the format of paths they support. This method will check that
path
is valid on the underlaying storage mechanism and throw aInvalidPath
exception if it is not.- Parameters
path (str) – A path.
- Returns
A normalized, absolute path.
- Return type
str
- Raises
fs.errors.InvalidPath – If the path is invalid.
fs.errors.FilesystemClosed – if the filesystem is closed.
fs.errors.InvalidCharsInPath – If the path contains invalid characters.
-
which
(path: Text, mode: Text = 'r') → Tuple[Optional[Text], Optional[FS]][source]¶ Get a tuple of (name, fs) that the given path would map to.
- Parameters
path (str) – A path on the filesystem.
mode (str) – An
io.open
mode.
-
writebytes
(path: Text, contents: bytes) → None[source]¶ Copy binary data to a file.
- Parameters
path (str) – Destination path on the filesystem.
contents (bytes) – Data to be written.
- Raises
TypeError – if contents is not bytes.
-
writetext
(path: Text, contents: Text, encoding: Text = 'utf-8', errors: Optional[Text] = None, newline: Text = '') → None[source]¶ Create or replace a file with text.
- Parameters
path (str) – Destination path on the filesystem.
contents (str) – Text to be written.
encoding (str, optional) – Encoding of destination file (defaults to
'utf-8'
).errors (str, optional) – How encoding errors should be treated (same as
io.open
).newline (str) – Newline parameter (same as
io.open
).
- Raises
TypeError – if
contents
is not a unicode string.
-