Buckner40 Testing

see also: TestPlan, and Buckner40Notes

Introduction

The 'Buckner40' is a name given to a small subset of subjects originating from Washington University, collected by Randy Buckner and colleagues. It consists of a group of 40 nondemented subjects, ranging in age from 18 to 93 (with no subjects between 30 to 65 years of age), 21 women and 19 men. These data are a subset of a larger open-access structural data set that can be obtained from the fMRIDC at: http://www.fmridc.org/f/fmridc, accession number 2-2004-1168X.

For Freesurfer, this dataset is especially nice as a test-set because of the wide subject age range, and the quality of the scans, which, while good, are not great, but are typical of what Freesurfer has to handle.

The Buckner40Notes wiki page documents the work of examining these brains, slice by slice, to verify their correctness, in terms of how well Freesurfer has reconstructed the white and pial surfaces, and segmented the subcortical structures.

It is important that Freesurfer's recon-all processing stream continue to process all of the subjects successfully, where that means the results are as good as or better than prior 'known-good' results (the reference set). (Note, if Freesurfer improves the results, then a new reference set is created, aka 'reset the standard').

Historical tests

The location of Buckner40 reference datasets with respect to previously released versions can be found at the following locations:

/space/freesurfer/subjects/test/buckner40/stable5_1_0
/space/freesurfer/subjects/test/buckner40/stable5_3_0
/space/freesurfer/subjects/test/buckner40/stable6

Monthly/weekly testing for the development branch should be stored here:

/space/freesurfer/subjects/test/buckner40/dev

Running the test

The 2 main scripts for running the Buckner40 and Adni60 tests are located here:

/space/freesurfer/subjects/test/scripts/comparison_test.sh
/space/freesurfer/subjects/test/scripts/comparison_utils.sh

To run the Buckner40 and generate groupstats, ssh to launchpad and run the comparison_test.sh as follows:

ssh launchpad
cd /space/freesurfer/subjects/test/scripts
comparison_test.sh \
  -fs /usr/local/freesurfer/dev \
  -subjects_dir /space/freesurfer/subjects/test/buckner40/dev \
  -generate_groupstats /space/freesurfer/subjects/test/buckner40/dev/groupstats_2016_07_30