THIS PAGE HAS BEEN MOVED TO SHAREPOINT!
Please refer to this site/make edits here for the most updated information: https://partnershealthcare.sharepoint.com/sites/LCN-Exvivo/SitePages/Offline-Recon.aspx
#acl ExvivoGroup:read,write,delete,revert All: This page is readable only by those in the ExvivoGroup.
Streaming Scans for Offline Processing
Exvivo data collected with the 32 channel coils on the 7T scanner (Bay 5) is too big for the MRIR to reconstruct because the data overwrites itself on the RAID before the scan finishes, so the raw data must be streamed off of the RAID during scanning. To get images, the raw data must then be reconstructed offline (i.e. not on the scanner but on another machine). This wiki details the steps to reconstruct this raw data (assuming all necessary files were already collected) .
Instructions for setting up streaming on the scanner and running a streamed scan can be found at the path below. Make sure you copy all relevant header and dat files from twix as listed in the instructions:
Approximate size of raw data
Be sure you have enough empty space on the RAID the data is being streamed to!
- 150um hemi:
180GB per segment x 8 segments = 1440GB per run (1.41TB); 4320GB (4.22TB) = total for 3 runs, 5760GB (5.625TB) = total 4 runs
- 150um whole brain:
314GB per segment x 8 segments = 2512GB per run (2.51TB); 7536GB (7.53TB) = total for 3 runs, 10048 GB(10.04TB) = total for 4 runs
- 200um whole brain
Smaller brains: 210GB per segment x 4 segments = 840GB per run (0.820TB); 2520GB (2.46TB) for 3 runs, 3360GB (3.28TB) for 4 runs
Larger brains: 250GB per segment x 4 segments = 1000GB per run (0.977TB); 3000GB (2.92TB) for 3 runs, 4000GB (3.91TB) for 4 runs
The directory structure should consist of a parent directory (<subject name> for Fischl scan, <subject name>/7T for Edlow scans), which contains NOTES and protocol.pdf files, as well as directories called headers, raw, mri, scripts and snapshots. After raw data has been deleted, there should be a file in the raw directory called deleted_files containing a list of the segment names, who deleted them, and the date they were deleted. The mri directory should contain a subdirectory for each flip angle collected and for parameter_maps. An example directory structure is below:
<SUBJECT> headers meas_*_<scan name>.dat meas_*_<scan name>.hdr meas_*CoilCovariance.dat <other files>.dat mri FA?? <offline recon output files> FA?? <offline recon output files> <other FAs> parameter_maps <mri_msfitparms output files> NOTES protocol.pdf raw <streamed *.dat files> OR deleted_files scripts <scripts used for processing> snapshots <snapshots of parameter maps or scans of interest>
Documentation of files
Make a note file called NOTES which includes the following:
- Which scan segment matches which .dat file (e.g. meflash_2014_10_10_09_13_26.dat FA20_run1_seg0of3)
- size for the whole data set of the scan session (e.g. total size: 4.2TB)
- size for each run (e.g. FA20_run_1: 840 GB)
- size for each scan segment (e.g. FA20_run_1_seg?of3: 240GB)
Be sure to delete all the .hybrid files after they have been used to create satisfactory .mgh files! These take up a lot of space. After all processing is complete (or at a safe point in the processing ,if space is desperately needed), the raw data can be removed from the cluster. A note called deleted_files should be left in the raw directory including the file names, date and user who deleted the files. For special cases, the raw data may be stored off the cluster on a hard drive instead of being deleted.
Current Offline Recon Method
Below is what is currently being used to process an offline recon. There are several different methods: OfflineReconDylan, OfflineReconTwitzel, as well as Jon Polimeni's personal scripts which are not documented here. Also, if using an external RAID instead of oribi, follow the instructions on the OfflineReconSshfs wiki to process the images without needing to transfer data to the cluster space first.
If you also acquired non-streaming data during the scan session (i.e. data that was reconstructed "online", on the scanner, and saved as dicoms in the patient browser), don't forget to process that data as well. Instructions for processing those are here: ExvivoRecons