dask.dataframe.read_hdf
dask.dataframe.read_hdf¶
- dask.dataframe.read_hdf(pattern, key, start=0, stop=None, columns=None, chunksize=1000000, sorted_index=False, lock=True, mode='r')[source]¶
Read HDF files into a Dask DataFrame
Read hdf files into a dask dataframe. This function is like
pandas.read_hdf, except it can read from a single large file, or from multiple files, or from multiple keys from the same file.- Parameters
- patternstring, pathlib.Path, list
File pattern (string), pathlib.Path, buffer to read from, or list of file paths. Can contain wildcards.
- keygroup identifier in the store. Can contain wildcards
- startoptional, integer (defaults to 0), row number to start at
- stopoptional, integer (defaults to None, the last row), row number to
stop at
- columnslist of columns, optional
A list of columns that if not None, will limit the return columns (default is None)
- chunksizepositive integer, optional
Maximal number of rows per partition (default is 1000000).
- sorted_indexboolean, optional
Option to specify whether or not the input hdf files have a sorted index (default is False).
- lockboolean, optional
Option to use a lock to prevent concurrency issues (default is True).
- mode{‘a’, ‘r’, ‘r+’}, default ‘r’. Mode to use when opening file(s).
- ‘r’
Read-only; no data can be modified.
- ‘a’
Append; an existing file is opened for reading and writing, and if the file does not exist it is created.
- ‘r+’
It is similar to ‘a’, but the file must already exist.
- Returns
- dask.DataFrame
Examples
Load single file
>>> dd.read_hdf('myfile.1.hdf5', '/x')
Load multiple files
>>> dd.read_hdf('myfile.*.hdf5', '/x')
>>> dd.read_hdf(['myfile.1.hdf5', 'myfile.2.hdf5'], '/x')
Load multiple datasets
>>> dd.read_hdf('myfile.1.hdf5', '/*')