Differences between revisions 17 and 109 (spanning 92 versions)

FreeSurfer 5.3 QA Tools

Deprecated

Note: The QA tools described in this page were developed for FreeSurfer 5.3. For versions 7+, use the following executable to compute QA stats:

qatools.py --help

Contents

FreeSurfer 5.3 QA Tools

The QA Tools are intended to be used in assessing the quality of one or more FreeSurfer recons. These scripts are capable of verifying all steps in the FreeSurfer recon-all stream were executed and in the correct order, as well as verifying all files exist and were created in the correct order. Additionally, these scripts can be used to detect potential outlier regions in the aseg.mgz within a dataset, calculate SNR and WM intensity values, and collect detailed snapshots of various volumes. The following people have been major contributors in developing these scripts: David Koh, Stephanie Lee, Jenni Pacheco, Vasanth Pappu, and Louis Vinke.

The QA Tools scripts are a work-in-progress. Any feedback, bug reports, or feature requests are appreciated (email vinke[at]nmr.mgh.harvard.edu).

Click here to view steps for an alternate QA procedure using Qdec.

Setup

Click the link below to download the QA Tools scripts. Extract all the contents of the tar file into a new directory. Source your local version of FreeSurfer, set your subjects directory variable, and set the QA Tools path:

<source freesurfer>

In tcsh:
setenv SUBJECTS_DIR /path/to/your/subjects_dir
setenv QA_TOOLS /path/to/QATools/scripts

In bash:
export SUBJECTS_DIR=/path/to/your/subjects_dir
export QA_TOOLS=/path/to/QATools/scripts

QAtools_v1.2.tar.gz

Dependencies

QA Tools requires:

ImageMagick Studio version 6.2.8 or higher when using the snapshots option. This can be downloaded here.
The GNU awk (gawk) utility. This can be downloaded here.

Getting Started

Run the following command to view explanations for all the QA Tools options:

$QA_TOOLS/recon_checker -help

By default QA-Tools will check the status log, the order files were created, the aseg for any outliers, create snapshots of slices throughout various volumes, and calculate SNR and WM measures. See the flags below if you want to disable one or more of these options.

Arguments

Required Flagged Arguments

-s <subject1> [<subject2>...]

Case ID for one or more subjects located in $SUBJECTS_DIR (use -s-file as alternative)

Optional Flagged Arguments

-subjid <subject1> [<subject2>...]	Specify subject(s) to process
-s-file	Specify a file with a list of subjects. May be used instead of -s or -subjid
-snaps-out	Full path and name of snapshots HTML file. Default: "$SUBJECTS_DIR/QA/QA_check.html"
-snaps-detailed	Take a more detailed set of snapshots
-snaps-overwrite	Take all snapshots, overwriting any existing snapshots
-no-snaps	Do not take snapshots
-snaps-only	Only take snapshots
-outputFOF <file-order list>	Specify a file which lists the proper order of output files
-asegLUT <file>	Specify a file containing aseg outlier lookup table
-gen-outputFOF	Generate the file order list from the first subject
-gen-asegLUT <file>	Generate an aseg look up table from your subjects that can be used to identify aseg outliers
-nocheck-aseg	Do not check for aseg outliers
-nocheck-status	Do not check status log file for each subject
-nocheck-outputFOF	Do not check output order of files for each subject
-nocheck-SNR-WM	Do not calculate SNR and WM measures for each subject
-completeStatusFile <file>	Compares each subject's status log file to the specified log file
-completeStatusFiles <file>	Compares each subject's status log file to each of the complete status logs listed in <file>
-ignore <process1> [<process2> ...]	Specify one or more FreeSurfer processes to ignore while checking status files

Future Directions

Implement -gen-status flag
Shorter alternative flags
Handle cases where FS ran multiple times (status_file_checker)
Generate status log file from specified subject
Create file order and status step lists for longitudinal QA
Snapshots taken using Freeview
Specify snapshots be taken of specific regions found in aparc+aseg.mgz with Freeview
Automatically adjust intensity/contrast of volumes before taking snapshots
Compile snapshots into GIF animations
Detect outliers in aparc and aparc2009 volumes
Handle cases where two files flip/flop in creation order (output_file_checker)
WM/SNR measurement outputs to seperate file (tab-delimited)
Make image & html links for snapshots relative so directory can be moved or renamed

Release Notes

v1.1 (3/5/2012)

Log and screen output modified
Created default file order list for *.long subject folders processed with FSv5.1 (default_FOF_FSv5.1_long)

Known Issues

To get the "-gen-asegLUT" flag of 'recon_checker' to work on FS 5.3 data, you need to change the helper 'gnicv' script to grep on "EstimatedTotalIntraCranialVol" rather than "ICV" (due to the name change of that variable in the stats/aseg.stats file relative to FS 5.1).
When taking snapshots, the tkmedit and tksurfer windows that appear can't be blocked by other windows, and the screensaver can't be activated while these scripts are running. Otherwise, the script will take snapshots of the screensaver or the interfering windows. The monitor can be physically turned off while the scripts are running, however, to save power.

For internal users

If you want to disable your screensaver so snaps can be taken overnight, set the value in

~/.gconf/desktop/gnome/session/%gconf.xml

to the number of minutes you want to pass before your screensaver activates (eg 960 for 16 hours). Also, go to System Preferences -> Screensaver and click on Power Management. In the "On AC Power" tab, choose Never in both of the pull down menus.

-  ⇤ ← Revision 17 as of 2006-09-21 16:32:05 → 
  Size: 4788
  Editor: JenniPacheco
  Comment:
+   ← Revision 109 as of 2021-08-12 10:51:08 → ⇥
  Size: 6169
  Editor: AndrewHoopes
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-#acl JenniPacheco:read,write,delete,revert BradDickerson:read DavidSalat:read
+= FreeSurfer 5.3 QA Tools =
 Line 3:
-= FreeSurfer QA Tools =
[[TableOfContents]]
 
== QA Notepad ==
+== Deprecated ==
-Line 8:
+Line 5:
-The QA Notepad program allows researchers to quickly take concise notes on subject or data derived from subjects in an organized, consistent manner. It is fully customizable and will work on an individual subject as well as a group of subjects.[[BR]]
#The program is located in the folder ~dkoh, and is named QAnotepad. The source files are located in ~dkoh/QAstream. [[BR]]
QA Notepad outputs text files with the extension .qa (unless you use the --noappend option).
+Note: The QA tools described in this page were developed for !FreeSurfer 5.3. For versions 7+, use the following executable to compute QA stats:
-Line 12:
+Line 7:
-==== The options for QA Notepad are: ====
-Line 14:
+Line 8:
-  -h, --help: displays a help message
  -f, --file <filename>: name of output file (default: QAnotes)
  -s, --subj <subject1> <subject2> ... : enters a list of subjects to choose
                                         from when recording data.
  --subjfile <subject file>: specifies a file with a list of subjects.
  -u, --user <username>: name of user.
  -c, --config <config file>: input config file to customize interface
                              It is required to enter a custom output filename
                              if you use this option.
  --noappend: ".qa" will not be appended to the filename.
  --dbupdate: if this flag is set, QAnotepad will automatically update the
              memage database via ~dkoh/scripts/perl/load_QA.pl.
+qatools.py --help
-Line 28:
+Line 11:
-If you already have a list of the subjects you'll be editing, you can enter that list into QA Notepad through the '''-s''' or '''--subj''' option. Alternatively, you can create a file with a list of subjects, and pass the filename in through the option '''--subjfile'''.
+<<TableOfContents>>
-Line 30:
+Line 13:
-==== Using Config Files ====
+The QA Tools are intended to be used in assessing the quality of one or more FreeSurfer recons.  These scripts are capable of verifying all steps in the FreeSurfer recon-all stream were executed and in the correct order, as well as verifying all files exist and were created in the correct order.  Additionally, these scripts can be used to detect potential outlier regions in the aseg.mgz within a dataset, calculate SNR and WM intensity values, and collect detailed snapshots of various volumes.  The following people have been major contributors in developing these scripts: David Koh, Stephanie Lee, Jenni Pacheco, Vasanth Pappu, and Louis Vinke.
-Line 32:
+Line 15:
-  You can customize the options in QA Notepad by using config files.  The config file should be formatted with the option header in the first column, followed by the desired options.  The option header will not appear unless it has at least one option.  Here's an example config file.  To pass in the config file, use the '''-c''' or '''--config''' option.  If you use a config file, you must also specify an output file name with the '''-f''' or '''--file''' option.
+The QA Tools scripts are a work-in-progress.  Any feedback, bug reports, or feature requests are appreciated (email vinke[at]nmr.mgh.harvard.edu).
-Line 34:
+Line 17:
-example.cfg:
+Click [[QuickQaQdec|here]] to view steps for an alternate QA procedure using Qdec.

== Setup ==
Click the link below to download the QA Tools scripts.  Extract all the contents of the tar file into a new directory.  Source your local version of FreeSurfer, set your subjects directory variable, and set the QA Tools path:
-Line 36:
+Line 23:
-  Reoriented Yes No
  Realigned Yes No
  Manual_Edits None A_Few Lots
  Automatic_Segmentation Good Bad Corrected Good_For_Vol
+<source freesurfer>

In tcsh:
setenv SUBJECTS_DIR /path/to/your/subjects_dir
setenv QA_TOOLS /path/to/QATools/scripts

In bash:
export SUBJECTS_DIR=/path/to/your/subjects_dir
export QA_TOOLS=/path/to/QATools/scripts
-Line 42:
+Line 34:
-This would create a panel with four buttons: Reoriented, Realigned, Manual_Edits, and Automatic_Segmentation. If you clicked on Automatic_Segmentation, you would get a new window with the options: Good, Bad, Corrected, Good_For_Vol. The underscores in place of spaces are mandatory, as the program separates options with spaces and won't be able to tell the difference. [[BR]]
attachment:QAnotepad_example.jpg [[BR]]
-Line 45:
+Line 35:
-If you want to have a free text field as an option for what to save, enter "$freetext" as an option after one of the option headers, like so:
+[[attachment:QAtools_v1.2.tar.gz]]

=== Dependencies ===
QA Tools requires:

 * !ImageMagick Studio version 6.2.8 or higher when using the snapshots option.  This can be downloaded [[http://www.imagemagick.org/script/index.php|here]].
 * The GNU awk (gawk) utility. This can be downloaded [[http://www.gnu.org/software/gawk/|here]].

== Getting Started ==
Run the following command to view explanations for all the QA Tools options:
-Line 48:
+Line 47:
-  Region_Edited Hippocampus Amygdala $freetext
+$QA_TOOLS/recon_checker -help
-Line 50:
+Line 49:
+By default QA-Tools will check the status log, the order files were created, the aseg for any outliers, create snapshots of slices throughout various volumes, and calculate SNR and WM measures.  See the flags below if you want to disable one or more of these options.
 Line 51:
-If you want to have a custom title for your window, just put the following text on it's own line: 
{{{
$title My Title Here
}}}
+== Arguments ==
=== Required Flagged Arguments ===
||-s <subject1> [<subject2>...] ||Case ID for one or more subjects located in $SUBJECTS_DIR (use -s-file as alternative) ||
-Line 56:
+Line 55:
-attachment:QA_notepad2.jpg  [[BR]]
-Line 58:
+Line 56:
-This will output a simple text file, that is tab delimited and contains all the information that you entered for each subject.
{{{
Subject      User          Reoriented    Realigned    Manual_Edits    Automatic_Segmentation
bert         jpacheco      Yes           Yes          A_Few           Good_For_Vol              your own text here!!
subject2     jpacheco      No            Yes          None            Good_For_Vol              Hippocampus
}}}
+=== Optional Flagged Arguments ===
||-subjid <subject1> [<subject2>...] ||Specify subject(s) to process ||
||-s-file ||Specify a file with a list of subjects.  May be used instead of -s or -subjid ||
||-snaps-out ||Full path and name of snapshots HTML file. Default: "$SUBJECTS_DIR/QA/QA_check.html" ||
||-snaps-detailed ||Take a more detailed set of snapshots ||
||-snaps-overwrite ||Take all snapshots, overwriting any existing snapshots ||
||-no-snaps ||Do not take snapshots ||
||-snaps-only ||Only take snapshots ||
||-outputFOF <file-order list> ||Specify a file which lists the proper order of output files ||
||-asegLUT <file> ||Specify a file containing aseg outlier lookup table ||
||-gen-outputFOF ||Generate the file order list from the first subject ||
||-gen-asegLUT <file> ||Generate an aseg look up table from your subjects that can be used to identify aseg outliers ||
||-nocheck-aseg ||Do not check for aseg outliers ||
||-nocheck-status ||Do not check status log file for each subject ||
||-nocheck-outputFOF ||Do not check output order of files for each subject ||
||-nocheck-SNR-WM ||Do not calculate SNR and WM measures for each subject ||
||-completeStatusFile <file> ||Compares each subject's status log file to the specified log file ||
||-completeStatusFiles <file> ||Compares each subject's status log file to each of the complete status logs listed in <file> ||
||-ignore <process1> [<process2> ...] ||Specify one or more FreeSurfer processes to ignore while checking status files ||
-Line 65:
+Line 76:
-==== Automatic Database Updates ====
-Line 67:
+Line 77:
-If you use the flag '''--dbupdate''' with the standard configuration of QA Notepad, it will automatically update the memage database to match your file. It will only update the subjects in your file that already exist in the database. It does so via the perl script ''load_QA.pl'', located at ''~dkoh/scripts/perl/load_QA.pl''.
+== Future Directions ==
 * Implement -gen-status flag
 * Shorter alternative flags
 * Handle cases where FS ran multiple times (status_file_checker)
 * Generate status log file from specified subject
 * Create file order and status step lists for longitudinal QA
 * Snapshots taken using Freeview
 * Specify snapshots be taken of specific regions found in aparc+aseg.mgz with Freeview
 * Automatically adjust intensity/contrast of volumes before taking snapshots
 * Compile snapshots into GIF animations
 * Detect outliers in aparc and aparc2009 volumes
 * Handle cases where two files flip/flop in creation order (output_file_checker)
 * WM/SNR measurement outputs to seperate file (tab-delimited)
 * Make image & html links for snapshots relative so directory can be moved or renamed
-Line 69:
+Line 92:
-Updating Custom Configurations
+== Release Notes ==
v1.1 (3/5/2012)
-Line 71:
+Line 95:
-If you want to have your custom configuration updated in the database, first, create a procedure with User Defined Select data that matches your custom configuration. Then, determine your procedure's id number. You can do this from one of two places. From the page where you edit the steps of your procedure, if you look at the url, there should be a part after the ? which says pprocedure=NUMBER. That number is your procedure's id. You can also find this from the page on which you edit the subject. Once you have your procedure id, put this on its own line in your config file:
+ * Log and screen output modified
 * Created default file order list for *.long subject folders processed with FSv5.1 (default_FOF_FSv5.1_long)
-Line 73:
+Line 98:
-  $procedure_id ID
+== Known Issues ==
 * To get the "-gen-asegLUT" flag of 'recon_checker' to work on FS 5.3 data, you need to change the helper 'gnicv' script to grep on "!EstimatedTotalIntraCranialVol" rather than "ICV" (due to the name change of that variable in the stats/aseg.stats file relative to FS 5.1).
 * When taking snapshots, the tkmedit and tksurfer windows that appear can't be blocked by other windows, and the screensaver can't be activated while these scripts are running.  Otherwise, the script will take snapshots of the screensaver or the interfering windows.  The monitor can be physically turned off while the scripts are running, however, to save power.
-Line 75:
+Line 102:
-Where ID is the procedure id that you found earlier. If you don't do this step, you will get an error message.
+== For internal users ==
If you want to disable your screensaver so snaps can be taken overnight, set the value in
-Line 77:
+Line 105:
-== Recon Checker ==
+ . ~/.gconf/desktop/gnome/session/%gconf.xml

to the number of minutes you want to pass before your screensaver activates (eg 960 for 16 hours).  Also, go to System Preferences -> Screensaver and click on Power Management.  In the "On AC Power" tab, choose Never in both of the pull down menus.