datalad get(1) | General Commands Manual | datalad get(1) |
datalad get - get any dataset content (files/directories/subdatasets).
datalad get [-h] [-s LABEL] [-d PATH] [-r] [-R LEVELS] [-n] [-D DESCRIPTION] [--reckless [auto|ephemeral|shared-...]] [-J NJOBS] [PATH ...]
This command only operates on dataset content. To obtain a new independent dataset from some source use the CLONE command.
By default this command operates recursively within a dataset, but not across potential subdatasets, i.e. if a directory is provided, all files in the directory are obtained. Recursion into subdatasets is supported too. If enabled, relevant subdatasets are detected and installed in order to fulfill a request.
Known data locations for each requested file are evaluated and data are obtained from some available location (according to git-annex configuration and possibly assigned remote priorities), unless a specific source is specified.
Just as DataLad supports getting file content from more than one location, the same is supported for subdatasets, including a ranking of individual sources for prioritization.
The following location candidates are considered. For each candidate a cost is given in parenthesis, higher values indicate higher cost, and thus lower priority:
- URL of any configured superdataset remote that is known to have the desired submodule commit, with the submodule path appended to it. There can be more than one candidate (cost 500).
- In case .GITMODULES contains a relative path instead of a URL, the URL of any configured superdataset remote that is known to have the desired submodule commit, with this relative path appended to it. There can be more than one candidate (cost 500).
- A URL or absolute path recorded in .GITMODULES (cost 600).
- In case .GITMODULES contains a relative path as a URL, the absolute path of the superdataset, appended with this relative path (cost 900).
Additional candidate URLs can be generated based on templates specified as configuration variables with the pattern
DATALAD.GET.SUBDATASET-SOURCE-CANDIDATE-<NAME>
where NAME is an arbitrary identifier. If NAME starts with three digits (e.g. '400myserver') these will be interpreted as a cost, and the respective candidate will be sorted into the generated candidate list according to this cost. If no cost is given, a default of 700 is used.
A template string assigned to such a variable can utilize the Python format mini language and may reference a number of properties that are inferred from the parent dataset's knowledge about the target subdataset. Properties include any submodule property specified in the respective .GITMODULES record. For convenience, an existing DATALAD-ID record is made available under the shortened name ID.
Additionally, the URL of any configured remote that contains the respective submodule commit is available as REMOTE-<NAME> properties, where NAME is the configured remote name.
Lastly, all candidates are sorted according to their cost (lower values first), and duplicate URLs are stripped, while preserving the first item in the candidate list.
Get a single file::
% datalad get <path/to/file>
Get contents of a directory::
% datalad get <path/to/dir/>
Get all contents of the current dataset and its subdatasets::
% datalad get . -r
Get (clone) a registered subdataset, but don't retrieve data::
% datalad get -n <path/to/subds>
datalad is developed by The DataLad Team and Contributors <team@datalad.org>.
2021-02-04 | datalad get 0.14.0 |