Differences between revisions 13 and 14
Deletions are marked like this. Additions are marked like this.
Line 31: Line 31:
 * ADNI-HIPPO - [[http://www.hippocampal-protocol.net/SOPs/index.php|135 ADNI subjects with manual segmentations of hippocampus]]  * ADNI-HIPPO - [[http://www.hippocampal-protocol.net/SOPs/index.php|135 ADNI subjects with manual segmentations of hippocampus]] - original MINC labels [[http://hippocampal-protocol.net/SOPs/LINK_PAGE/FINAL_RELEASE/Released_ACPC_brainScans_MNC.zip|downloadable here]]

This page is readable only by those in the LcnGroup.

Author(s): Nick Schmansky

See: Samseg

Samseg Aseg Testing

This page documents one front of the work on Samseg, where testing of a version of Samseg operating on T1-weighted input and using the existing RB FS atlas, is conducted to compare its performance against the existing FreeSurfer v6.0 subcortical segmentation processing stream. Initial work was conducted in an evaluation task: SamsegEvaluationMarch2016. The current work pertaining to this page extends that work by:

  • Creation of a test cycle allowing the comparison of a subject run of Samseg (which is under algorithmic development) to the FS aseg results from v6 for that subject. The test cycle should generate reports on a run, allowing for debug, and evaluation of performance to meet, initially, the aims of the CorticoMetrics grant aims (ie, an accelerated FS based on T1 input only)

  • Creation of a data set which includes enough subjects to cover the wide variety of possible inputs. Starting with Buckner40 and ADNI60, and continuing with many others, particularly subjects having lots of neck in the FOV, as well as 'tough cases' both in terms of registration and segmentation, plus a variety of scanner model, and age and disease states.
  • Documentation of design decisions.

Those working on this project include: Doug Greve (DG), Nick Schmansky (NS), Andrew Hoopes (AH), Christian Larsen (CH) and Lee Tirrell (LT)

Test Data

The test subjects and scripts are located here: /cluster/fsm/users/samseg

Test sets used to compare samseg results will include:

  • Buckner39 - 39 manually labeled subjects used to create the FS subcortical atlas
  • Siemens13 - 13 manually labeled subjects scanned on Siemens Sonata (currently used in testing the FS atlas: AsegTestNotes)

  • GE14 - 14 manually labeled subjects scanned on GE Signa (currently used in testing the FS atlas: AsegTestNotes)

  • Buckner40 - public buckner40, processed by FS v6, where aseg's have been manually inspected as part of FS release testing
  • ADNI60 - processed by FS v6, where aseg's have been manually inspected as part of FS release testing
  • ADNI-HIPPO - 135 ADNI subjects with manual segmentations of hippocampus - original MINC labels downloadable here

  • ADNI-1.5T-FS-5.3 - 1.5T ADNI data processed by FS 5.3, and QA'd by Tian Ge (but asegs not each manually inspected)
  • ADNI-3T-FS-5.3 - 1.5T ADNI data processed by FS 5.3, and QA'd by Tian Ge (but asegs not each manually inspected)
  • Outliers - consisting of subject scans known to be problematic (lots of neck, skewed placement in scanner, etc.)
  • IXI, HCP, OASIS, HABS, BRAINS, Mindboggle-101, BGSP - all publicly available datasets where the intent is to process by FS v6 to include in the aseg test set.
  • ANDI-Long - test-retest set
  • Bammer120 - test-retest set
  • disorder sets - data from various disorder studies (outside of AD), eg. ms, schizophrenia, autism, tumor, child, addiction, etc.

Tasks

  • AH - determine reasons for ANDI60 failures
  • AH - Initial test data: create directories using the names Buckner40, Siemens13, GE14, Buckner40, ADNI60, ADNI-1.5T-FS-5.3 and ADNI-3T-FS-5.3, and copy from their respective source data directories only the input data necessary to run samseg, and also the reference seg file (symlinked to name 'ref_seg.mgz'), which will be the manual labels in Buckner39, Siemens13, GE14, the FS v6 aseg.mgz for Buckner40 and ADNI60, and FS v5.3 for ADNI-1.5T-FS-5.3 and ADNI-3T-FS-5.3)
  • AH - One-subject test run: refer to paper for testing methods. Create a script that does the following:

    • runs samseg on a subject
    • runs mri_compute_overlap against ref_seg.mgz
    • save a png of the dice of each structure and the overall, as shown in Figure 3 of SamsegEvaluationMarch2016.

  • AH - Multi-subject test run: create a script to run a set of subjects, and then produce a png of the dice results as shown in Figure 6 of SamsegEvaluationMarch2016.

  • AH - run multi-subject tests runs on the initial data sets
  • AH - document on this wiki page the location of these scripts and how to run them, including where to find the results

SamsegTesting (last edited 2021-04-20 18:17:35 by DevaniCordero)