Differences between revisions 1 and 2
Deletions are marked like this. Additions are marked like this.
Line 8: Line 8:
== Chart is going to go here ==

xxx
Line 14: Line 10:
This version is more tractable to look at and print. Though an image of the full chart appears at the bottom of this page, it's more tractable to inspect (or print) your own copy of the Excel version, which you can download from this link:
Line 16: Line 12:
xxx attachment:gw_FS_FileIO_20070618.xls
Line 18: Line 14:
=== Data notes === == Data notes ==
Line 39: Line 35:
== Image of the Files vs Steps table ==
Line 40: Line 37:
attachment:gw_FS_FileIO_20070618.gif

Page status: 2007-06-18: Under construction!

This page describes and provides access to a table which shows the relationships between steps in recon-all, and files that are read, created, written or deleted at each step.

Applies to: Freesurfer version 3.0.5 2007-02-07

Download Excel version

Though an image of the full chart appears at the bottom of this page, it's more tractable to inspect (or print) your own copy of the Excel version, which you can download from this link:

attachment:gw_FS_FileIO_20070618.xls

Data notes

  1. To avoid clutter, I have omitted columns for most log files
  2. Unlike my previous effort of this type, in this chart I've retained steps and files from right as well as left processing. I did this to more clearly see steps that involve both left and right.

Method

In brief, this chart builds on directory data gathered during a run of recon-all on the "bert" sample subject. Gathering this data entails two parts:

  1. A "directory survey" script (FWIW in python) recursively navigates the Freesurfer directories, capturing write, access and directory-mod date-time information for each file.
  2. Commands added to the recon-all script to run the directory survey script before and after each existing recon-all command. More precisely, in recon-all, commands at each step are invoked using a pattern that builds the command string in variable "cmd", and then runs it by invoking "$cmd". I manually searched for each such location, and inserted my own command to invoke "before" and "after" versions of the directory survey script.

Once the run is complete, the directory-survey data is slurped into a database, cleaned up, and then a cross-tab query organizes the data in Files-vs-steps form.

Wrinkles

  1. The main wrinkle is that using the file system's access/write/dir-mod date-time feature is a bit iffy. First, this only works on some systems -- on others not all the data-time values work, or they may have granularity too coarse to be useful (despite being reported in seconds). I found that this did work OK on Centos 4 (RHEL 4, or FC something). It did not work on Centos 4 with the files on Mac OS X server accessed via NFS. Don't know whether that's a Mac problem or an NFS problem.
  2. Even on a system where directory timestamps work acceptably well, it's obviously necessary to make sure the successive steps in the recon-all process do not occur within one second of each other, otherwise the available timestamp resolution will be unable to report a difference in times. For that reason, the directory-survey script incorporates a delay of at least two seconds.
  3. Backup jobs running in the middle of the proceedings can log extra reads that are nothing to do with recon-all :-).

Image of the Files vs Steps table

attachment:gw_FS_FileIO_20070618.gif

ReconAllFilesVsSteps (last edited 2009-02-02 20:25:25 by GrahamWideman)