Differences between revisions 1 and 2
Deletions are marked like this. Additions are marked like this.
Line 30: Line 30:
== Getting a data file ==

To retrieve the contents of a data file:

{{{
## Retrieve a specific file
git annex get mri_em_register/testdata.tar.gz

## Retrieve everything under the current directory (not recommended)
git annex get .
}}}

To retrieve everything (not recommended):

{{{
Line 38: Line 54:
To list the metadata of a given file: === Display metadata ===

To show all the metadata associated with a file:
Line 44: Line 62:
To assign a tag to an existing datafile: === Assign metadata ===

To assign a tag to an existing datafile.
Line 62: Line 83:

Contents

  1. Initial Setup
  2. Adding a data file
  3. Getting a data file
    1. Assign metadata

This page describes how to deal with adding and tagging data files in the freesurfer source code repository.

Initial Setup

Based in the information included in the Freesurfer_github page, the remotes of your freesurfer repo working directory should look something like: 

  • git remote -v 
 
  datasrc       file:///space/freesurfer/repo/annex.git (fetch)
  datasrc       file:///space/freesurfer/repo/annex.git (push)
  origin        git@github.com:zkaufman/freesurfer.git (fetch)
  origin        git@github.com:zkaufman/freesurfer.git (push)
  upstream      git@github.com:freesurfer/freesurfer.git (fetch)
  upstream      git@github.com:freesurfer/freesurfer.git (push)

Adding a data file

The following example assumes we want to add a data file called 'testdata.tar.gz' to the 'distribution' directory: 



Getting a data file


To retrieve the contents of a data file: 
## Retrieve a specific file
git annex get mri_em_register/testdata.tar.gz

## Retrieve everything under the current directory (not recommended)
git annex get .
To retrieve everything (not recommended): 
== Tagging a data file ==

The data files have been broken down into the following 3 categories, and it is essential that data files get the proper tag(s) so that our servers and diskspace is not overwhelmed when only a known subset of the data is required.:

 1. Those being required for build time checks (tagged '''makecheck''')
 1. Those required for a local installation (tagged '''makeinstall''')
 1. Everything else (untagged)

=== Display metadata ===

To show all the metadata associated with a file:

{{{
git annex metadata mri_em_register/testdata.tar.gz


Assign metadata


To assign a tag to an existing datafile.  
git annex metadata mri_em_register/testdata.tar.gz -s fstags=makecheck
git annex sync
We can also append tags: 
git annex metadata mri_em_register/testdata.tar.gz -s fstags+=makeinstall
To list all files with a given tag: 
git annex find --metadata fstags=makecheck
To download all datafiles with a given tag: 
## Get only the data files required for build time checks (1.9 GB)
git annex get --metadata fstags=makecheck .

## Get only the data files required for local installation (4.3 GB)
git annex get --metadata fstags=makeinstall .

## Just give me everything! Not Recommended (6.8 GB)
git annex get .