Photo Reconstruction Documentation

Citation

If you use these tools in your analysis, please cite:

• Gazula, H., Tregidgo, H.F., Billot, B., Balbastre, Y., Williams Ramirez, J., Herisse, R., Deden-Binder, L.J., Casamitjana, A., Melief, E.J., Latimer, C.S. and Kilgore, M.D., 2024. Machine learning of dissection photographs and surface scanning for quantitative 3D neuropathology. Elife, 12, p.RP91398. doi

The machine learning slice imputation in the reconstruction is described in:

• Crespo-Aguirre, M., Williams-Ramirez, J., Zemlyanker, D., Hu, X., Deden-Binder, L.J., Herisse, R., Montine, M., Connors, T.R., Mount, C., MacDonald, C.L., Keene, C.D., Latimer, C.S., Oakley, D.H., Hyman, B.T., Lawry-Aguila, A. and Iglesias, J.E., 2026. Improving neuropathological reconstruction fidelity via AI slice imputation. Acta Neuropathologica Communications. doi

Automatic segmentation model trained using nnUNet:

• Williams Ramirez, J., Zemlyanker, D., Deden-Binder, L., Herisse, R., Garcia Pallares, E., Gopinath, K., Gazula, H., Mount, C., Kozanno, L.N., Marshall, M.S., Connors, T.R., Frosch, M.P., Montine, M., Oakley, D.H., Mac Donald, C.L., Keene, C.D., Hyman, B.T., Fischl, B. and Iglesias, J.E., 2026. A public generalizable AI tool for automated segmentation of coronal brain tissue slabs for 3D neuropathology. PLOS One, 21(8), p.e0355740. doi

• Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J. and Maier-Hein, K.H., 2021. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature methods, 18(2), pp.203-211.

Click here for a video with an overview of the method

Purpose

This manual provides an end-to-end guide for using the tools available in FreeSurfer to reconstruct dissection photographs of brain slabs into a 3D reconstruction for quantitative analysis. Reconstructions are done by registering photographs to a 3D reference such as an MRI, a post-mortem surface scan, or an atlas. The guide covers acquiring surface scans prior to slabbing, preparing tissue slabs for gross photography, and preprocessing photographs using the dissection_photo graphical user interface.

Requirements

• The latest FreeSurfer development version sourced in your terminal.

• Model files (downloaded once):

  • the reconstruction model, downloaded into $FREESURFER_HOME/models

• A CUDA-enabled graphics card is optional: segmentation and reconstruction are much faster with one.

Overview

1. Acquire a 3D reference: either a surface scan or an MRI. Without one, slabs are registered to an atlas.

2. Photograph the tissue slabs

3. Preprocess the photos in the dissection photo GUI

4. Reconstruct the 3D volume

5. Downstream analysis of photographs

Part 1: Acquisition

Surface scanning

A surface scan of the specimen is taken after the cerebellum has been sectioned off and before the specimen is slabbed. The full scanning protocol is available on its own page: Surface Scanning Protocol.

Summary of the key requirements of a reference mesh:

• The cerebellum removed: it is not on the slabs, so including it in the scan leads to an incorrect reconstruction.
• The sides of the specimen captured.

• A watertight mesh, saved as .stl. Small holes in the initial point cloud due to deep sulci are fine; these will be covered by the meshing process.

Preparing Slabs

The following guidelines should be followed when preparing slab images for optimal results.

Guidelines for Brain Slab Image Acquisition

There are some important requirements for the photographic setup of the brain slabs to ensure the best results.

Use a high-contrast background that stands out in uniform contrast to the brain slabs, such as a solid black background. This will make the slabs easier to segment downstream. Using a complex background with a color similar to your brain slabs will lead to more manual intervention, such as having to manually trace contours, which is not only time-consuming but also non-reproducible.

Additionally, all tissue block faces must face the same direction: all slices should be photographed either from the anterior or from the posterior view. Note that if slabs are photographed from the posterior view, the extra flag --photos_of_posterior_side must be used in the final reconstruction command.

When a single slice produces multiple tissue blocks, they should be placed as close to the original anatomy as possible. For example, after acquiring slices from the frontal pole, the initial blocks corresponding to the temporal lobe will often be detached from the main block. The following figure shows a) proper anatomical placement of tissue, and b) tissue placed too far apart, which is not anatomically accurate.

Tissue slab placement: a) properly placed tissue; b) tissue placed in a non-anatomical manner.

Another example of difficult tissue placement is the corpus callosum. The following figure shows a good setup, with a high-contrast background and good lighting. However, the placement of the corpus callosum on the rightmost slab is incorrect, which will negatively affect volumetric estimates of the third ventricle.

Markers for pixel and perspective corrections

To calibrate pixel size, the user needs to provide landmarks with known distances between them. The minimum requirement is a ruler, which provides two discrete landmarks. This enables rough pixel size correction. This method is preferable for legacy images that only have a ruler for reference.

Tip: It is better to use landmarks that are as far apart as possible, as the relative error when manually setting the reference in pixels per millimeter decreases with greater distances.

A more effective setup uses four landmarks placed at the corners of a rectangle of known dimensions. This enables us to correct for both pixel size and perspective. In the example below, we can easily tell that the 4 points marked on the grid define a rectangle of 170 x 200 mm.

The optimal scenario is to have 4 points defined by fiducial markers with highly salient features. This enables our software to recognize the fiducials automatically, which minimizes human interaction, reducing time and maximizing reproducibility. We strongly recommend that you print this set of fiducials and place them at the corners of the board.

Tip: For best results, affix the fiducials to a small piece of rubber, wood or plastic that lifts them X mm above the board, where X is the thickness of your typical brain slice. Lifting the fiducials minimizes the error caused by the surface of the brain slice not being in the same plane as the board, but a few millimeters above it.

The fiducials form the four corners of the corrected image; hence, all tissue should be within the rectangle formed by the four fiducials. The following image shows slab photos where the fiducials were not placed far enough apart.

Drawing a rectangle through the four fiducial markers makes it clear that much of the tissue lies outside of the pixel-corrected region.

In summary, here is an example of an optimal photographic setup: a uniform, high-contrast background, the provided fiducials located at known distances, and consistent lighting on all slabs.

Tip: It does not matter if the slabs are not placed on a grid, as long as the distance between the fiducials is known. The software will draw digital rulers on your images during pixel correction.

Part 2: Preprocessing Photographs with the Dissection Photo App

The dissection photo app has five principal windows for each subject: Add photos, Correction, Segmentation, Connected components and Reconstruction. The step bar at the top of the window shows which step is being done. Users can backtrack by clicking any of the previous steps at any time, or click the house icon to return to the Projects screen with a summary of all reconstructions done or in process. For each project, work is saved as you go, so the app can be closed at any time and you can pick up later where you left off.

Before continuing, make sure the development version of FreeSurfer is properly sourced, you have downloaded the model weights, and have set NNUNET_MODEL_DIR as an environment variable pointing to the segmentation model. To start the application, use the command dissection_photo_2.

Projects Page

Each specimen corresponds to a project: a folder which will be populated with the outputs from all steps. The project home contains a summary of all projects and their progress.

Projects Home screen.

• To start processing a new batch of dissection photographs click on New Project.

• Open a project to continue where you left off or backtrack to any completed steps.

Tip: For advanced users, the gear icon can be used to change which resources are used by the automatic segmentation model, covered in automatic segmentation settings.

After naming a project you will be prompted to save a folder of the same name. It is in this folder that the outputs of all steps will be stored.

Directory Structure

For each project the following directory structure will be populated with all outputs.

Step 1. Add photographs

1. Drag and drop a folder containing dissection photographs, or click Upload Images.

2. Select the photographs that should be used for the reconstruction. This can be used to omit repeat photographs, dissection photographs of cerebellar tissue, or gross photographs done prior to slabbing. Double click on any image in the grid for a larger preview.

3. If you have or would like to create a calibration file to automate pixel correction, click on + Add a calibration file (optional) and follow the steps in Making a Calibration File and then Prospective Correction.

4. Click Get started.

Optional: Making a calibration file

This step only needs to be done once, as long as the markers are not moved. For more information on proper usage refer to the Markers for pixel and perspective corrections section in this wiki page.

1. On the Add photos menu, click + Add a calibration file (optional) to expand the calibration file box, and click on Create one under the Upload Calibration button.

2. On the next menu, if you have not printed out the fiducial markers, they can be downloaded for printing by clicking the Download Marker Printout button.

3. Next, place markers at a known distance in a rectangle formation on the photography table. Be sure that opposite sides of the rectangle created by the fiducial markers are parallel; inaccurate marker placement can lead to incorrect perspective correction.
4. Upload the photograph of the empty dissection table with just the markers present, and enter the width and height of the fiducial box.
5. For each of the markers, click its centre and outer circle. Options to zoom, pan, and redo/undo can help fix mistakes and make the best calibration possible.

6. Click Save Calibration and save your calibration file in an easy-to-access location.

Tip: Because of slight lens distortion, the overlaid labels for the outer circle of the markers might not always align throughout the whole circumference. This is not an issue.

The photograph of the empty table with the four markers, uploaded.

Step 2: Pixel Correction

This step corrects all photographs to be at the same pixel size. Additionally, it can correct for perspective when using a calibration file, or when doing manual correction in 4-point mode.

Prospective Correction (Using fiducial markers and calibration file)

• Every photo is corrected automatically as soon as Correction opens. The markers are found in each photo separately, so it does not matter if the camera moved between photos.

• Check each photo: Original shows the photo as taken, with the marker centres that were found; Corrected shows the result. A green tick in the strip at the bottom marks each corrected photo.

• If the markers cannot be found in a photo, the app says which ones. On each of them, click the centres of the four markers on the Original view and click Correct this photo.

Retrospective Correction (Using a spatial reference within the photographs)

Choose how many points you can mark, enter the real distances, and place the points on each photo:

• Ctrl + left click places a point; once all are placed, Ctrl + click moves the nearest one. Clear points starts again.

• Click Correct this photo (Ctrl + W also moves on to the next photo). Place points on the Original view; the Corrected view is for checking.

When every photo is corrected, make sure to click Continue to Segmentation.

Step 3: Binary Segmentation

While you were correcting, the app was already finding the tissue in each corrected photo. Here you check each mask and fix it.

• Brush (Q) adds to the mask, Eraser (W) removes from it, and Fill (E) fills an area. Apply the tool with Ctrl + left drag; set the sizes with the sliders.

• Mask opacity changes how see-through the mask is; Show mask (Ctrl + T) hides and shows it. Undo and redo with Ctrl + Z and Ctrl + Shift + Z.

• Leave out everything that is not slab tissue: the background, labels, and fragments that do not belong to a slab.

• IMPORTANT: the cortical surface should not be part of the masks, so the first slab (if photographed from the anterior side) or the last slab (if photographed from the posterior side) should NOT be included.

• You need to look at every photo's mask before Continue to Connected Components becomes available; a note at the top says which ones are left.

Automatic Segmentation Settings

Open them from the gear on the Projects screen. Changes apply the next time photos are segmented.

Automatic segmentation settings.

Step 4: Slab Connected Components

Here you tell the app which pieces of tissue belong to which slab.

• In each photo, draw a box inside each slab (Ctrl + left drag), always in the same order: anterior to posterior, or posterior to anterior. The slab is colored as soon as its box is drawn.
• Keep boxes small and inside the tissue, and do not let them overlap. If a slab is in several pieces (e.g. a detached temporal lobe), make sure its box touches every piece.

• Click Save components (Ctrl + W also moves on to the next photo); moving to another photo saves it too. A green tick marks saved photos. If you change a saved photo, its tick goes away until you save it again.

• Undo and redo with Ctrl + Z and Ctrl + Shift + Z. Slab overlay changes how strongly the slabs are colored.

Before any boxes are drawn.

Step 5: Photo Reconstruction

Please see our separate page on the command mri_3d_photo_recon. The last step runs mri_3d_photo_recon on the images of the project. While the reconstruction is running in the background the user can continue processing images of other cases.

1. Output folder: the folder inside the project for the results (default recon).

2. Slabs: the hemisphere, the slab thickness in mm, which face of the slabs was photographed (anterior or posterior), the order the photos were taken in, and whether the tissue is fixed or fresh.

3. Reference: none (the MNI atlas), a surface mesh (.stl), an MRI scan, or both. See Which reference should I use?

4. Optionally: the number of threads, a graphics card, and the advanced settings. Empty expert weights use the command's own defaults, shown in grey.

5. Click Run Reconstruction.

The command being run is shown on the right (Copy command copies it, for example to run it on another computer), and its output appears in the log below it. The reconstruction keeps running if you switch to another project; the Projects screen shows its progress.

A reconstruction running, with the command and its log.

Part 3: Downstream Tasks

Photo-Synthseg Segmentation

The 3D reconstructed stacks can be segmented with SynthSeg using the flag --photo. Please see the wiki page for SynthSeg for further information.

Example for running SynthSeg on a Left Hemisphere

• mri_synthseg \

• --i ./recon/photo_recon.mgz \

• --o ./recon/photo_recon_synthseg.mgz \

• --photo left \

• --parc