Differences between revisions 22 and 37 (spanning 15 versions)
⇤ ← Revision 22 as of 2016-10-18 15:56:36 →
Size: 11819
Editor: ZekeKaufman
Comment:
← Revision 37 as of 2017-03-16 18:28:33 → ⇥
Size: 8216
Editor: ZekeKaufman
Comment:
Deletions are marked like this. Additions are marked like this.
Line 8: Line 8:
This page is targeted at Martinos Center users who wish to inspect, build or develop within the !FreeSrufer code base. Non-Martinos users wishing to work with the !FreeSrufer code base should consult the read-only git repo http://surfer.nmr.mgh.harvard.edu/fswiki/freesurfer_linux_developers_page This page is targeted at Martinos Center users who wish to inspect, build or develop within the Freesurfer code base. Non-Martinos users wishing to work with the Freesurfer code base should consult the [[http://surfer.nmr.mgh.harvard.edu/fswiki/freesurfer_linux_developers_page|read-only git repo]]. Users who with to contribute to the Freesurfer code base and/or commit changes to the repo should [[https://surfer.nmr.mgh.harvard.edu/fswiki/Freesurfer_github|see the following page]] describes how to fork the Freesurfer github repository and submit pull requests .
Line 10: Line 10:
=== Git Clone === === Getting the Source Code ===
Line 12: Line 12:
First thing users at the Martinos Center will need to do is add the newer version of the {{{git}}} and {{{git-annex}}} software to their PATH environment variable. This should be added to the {{{.cshrc}}} or {{{.bashrc}}} file in the users home directory: The first thing users at the Martinos Center need to do is '''prepend''' the directory {{{/usr/pubsw/packages/git-annex/current/bin}}} to their PATH. Once that is done, the Freesurfer source code can be cloned from the official [[https://github.com/freesurfer/freesurfer|Freesurfer github page]]:
Line 15: Line 15:
## csh
$> setenv PATH /usr/pubsw/packages/git-annex/current/bin:$PATH

## bash
$> export PATH=/usr/pubsw/packages/git-annex/current/bin:$PATH		 git clone git@github.com:freesurfer/freesurfer.git
Line 22: Line 18:
Users can then clone !FreeSurfer repository as follows: ==== Get the Data Files ====
The Freesurfer repository contains a large number data files which are not included with a default {{{git clone}}} of the repo. Instead, these data files are distributed via the [[https://git-annex.branchable.com/|git-annex]] software. Users who only want the repository for the purposes of compiling binaries and/or inspecting source code, the {{{git clone}}} command from above is all you need to do. Users who want to run build time checks, or perform a full local installation, or just want all the contents of the repository, will need to add a special data store remote repository in order to retrieve these files.

To add the data store repository (this only needs to be done once):
{{{
git remote add datasrc file:///space/freesurfer/repo/annex.git
git fetch datasrc
}}}

And to retrieve data files:
Line 25: Line 30:
## Get source code files (206 MB)
$> git clone file:///space/freesurfer/repo/freesurfer		 git annex get <filename>
Line 29: Line 33:
==== Getting the Data Files ====
The !FreeSurfer repository contains a large number of data files (pdfs, test data, etc) which exist as symlinks scattered throughout the !FreeSurfer directory hierarchy. These data files are not included with a default {{{git clone}}} of the repo and thus will exist as broken symlinks. Users who wish to clone the repository for the purposes of just inspecting source code and/or build the code, the {{{git clone}}} command from above is all that is required, and the broken symlinks will not present an issue. Users who want to run build time checks, or perform a full local installation, or just want all the data files contained in the repository, will need to run the following command in order to download the required data data files:		 The data files have been broken down into categories, those being required for build time checks, those required for a local installation, and everything else. Use one of the following commands depending on your needs:
Line 33: Line 36:

## Change directories
$> cd freesurfer
Line 38: Line 37:
$> git annex get --metadata fstags=makecheck . git annex get --metadata fstags=makecheck .
Line 41: Line 40:
$> git annex get --metadata fstags=makeinstall . git annex get --metadata fstags=makeinstall .
Line 44: Line 43:
$> git annex get . git annex get .
Line 46: Line 45:
Line 54: Line 54:
$> ./setup_configure ./setup_configure
Line 63: Line 63:
$> ./configure ./configure
Line 66: Line 66:
$> ./configure --prefix=~/freesurfer_install_dir ./configure --prefix=~/freesurfer_install_dir
Line 69: Line 69:
$> ./configure --help ./configure --help
Line 73: Line 73:
!FreeSurfer builds against the following set of open-sourced libraries, which are installed under the {{{/usr/pubsw/packages}}} directory on all NMR computers: Freesurfer builds against the following set of open-sourced libraries, which are installed under the {{{/usr/pubsw/packages}}} directory on all NMR computers:
Line 88: Line 88:
$> ./configure --with-qt=/usr/pubsw/packages/qt/4.8.5 ./configure --with-qt=/usr/pubsw/packages/qt/4.8.5
Line 96: Line 96:
$> make -j 4 make -j 4
Line 102: Line 102:
$> cd mri_info
$> make		 cd mri_info
make
Line 109: Line 109:
To install all binaries and support files into your private !FreeSurfer installation, type 'make install' from the toplevel fsdev/ directory, like this: To initial a local installation, type 'make install' from the top level directory:
Line 112: Line 112:
$> make install make install
Line 115: Line 115:
This will create a local !FreeSurfer installation in the directory specified by the {{{--prefix}}} option to {{{configure}}} scipt (see above). Note that if you do not specify this location, it will try to install to /usr/local, which will probably require root access. This will create a local Freesurfer installation in the directory as specified by the {{{--prefix}}} option to {{{configure}}} script (see above). Note that if you do not specify this location, it will try to install to /usr/local, which will probably require root access. The first time you run 'make install', it will take a while to copy all the big data files to the new installation. Subsequent 'make installs' will only copy the changed files.
Line 117: Line 117:
Note that you can also run 'make release'. 'make install' makes and installs the NMR internal version of FreeSurfer, while 'make release' makes the public version which omits some stuff.

The first time you run 'make install', it will take a while to copy all the big data files to the new installation. Subsequent 'make installs' will only copy the changed files.

If you only want to install a single binary, run 'make install' from a subdirectory. For example, running 'make install' from the scuba/ directory will copy the scuba binary and its support script files to the proper locations. Running 'make install' from scripts/ will copy all the necessary scripts to the right location.		 If you only want to install a single binary, run 'make install' from a subdirectory. For example, running 'make install' from the {{{mri_convert}}} directory will copy the {{{mri_convert}}} binary to the proper locations. Running 'make install' from scripts/ will copy all the necessary scripts to the right location.
Line 131: Line 127:
$> mkdir MYPROG
$> cd MYPROG		 mkdir MYPROG
cd MYPROG
Line 169: Line 165:
Once these 4 steps are complete MYPROG should automatically be built with the rest of !FreeSurfer. Once these 4 steps are complete MYPROG should automatically be built with the rest of !FreeSurfer. Try following the [[DevelopersGuide_git#Building|building steps from above]] to verify your binary compiles and builds successfully.
Line 171: Line 167:
=== Using Git === === Contributing Changes ===
Users who with to contribute to the Freesurfer code base and/or commit changes to the repo should see the following page which describe how to fork the freesurfer repository and submit pull requests:
Line 173: Line 170:
!FreeSurfer developers should run the following commands to get their git configuration settings set up properly. These commands only need to be run once:

{{{
## User settings.
$> git config --global user.name "John Doe"
$> git config --global user.email johndoe@nmr.mgh.harvard.edu

## Some recommended configuration settings:
$> git config --global push.default current
$> git config --global color.ui true
}}}

==== Examples ====

In order to facilitate the transfer from CVS to Git, the following examples use terms like "checkout" and "branch". Generally these terms mean two different things when talking about CVS vs. Git, but for the following examples they can be thought of as similar concepts.

"Checkout" the main branch:

{{{
$> git clone file:///space/freesurfer/repo/freesurfer <optional_directory_name>
}}}

"Checkout" the stable6 branch:
{{{
$> git clone -b stable6 file:///space/freesurfer/repo/freesurfer <optional_directory_name>
}}}

Bring my local repository up to date with what's on the server:
{{{
$> git pull
}}}

View the status of your checkout (see what files are new, deleted, or modified):
{{{
$> git status .
}}}
My local repository is corrupted and junk. Just blow it up and replace it with whats on the server:
{{{
$> git reset --hard origin/master
}}}

View the commit history of a file:
{{{
$> git log <file_name>
}}}

"Commit" a modified source code file:
{{{
$> git commit -m "Added new capabilities." <file_name>
$> git push
}}}

Adding new source code file:
{{{
$> git add <file_name>
$> git commit -m "Initial add of new file." <file_name>
$> git push
}}}

From the example above [[DevelopersGuide_git#Addinganewbinarytothetree]], if your program successfully builds/installs/runs and is thus ready to be checked into the repository, you can commit the files as follows:

{{{
$> git add MYPROG/MYPROG.c MYPROG/Makefile.am
$> git commit -m "Initial add of MRPROG." MYPROG/MYPROG.c MYPROG/Makefile.am configure.in Makefile.am
$> git push
}}}

Additional information on how to use Git can be found on the Atlassian websites which has decent tutorials and examples to help users getting started with Git:
https://www.atlassian.com/git/tutorials/saving-changes

==== Data Files and git-annex ====

Data files represent a special use-case, and they should not go into the normal Git repository. Instead {{{git-annex}}} is used to store all data files that are part of the !FreeSurfer repository. Generally speaking, data files are anything not source code related - pdfs, compressed tarballs, test data, precompiled binaries, and image files are all examples of data files that should be stored in {{{git-annex}}}. Simple text files like Look-up tables and configuration files, while not exactly source code, are NOT considered data files. Those files CAN go in the normal repository.

Below are a few examples on how one would work with the git-annex:

Get any data file which exists as a broken symlink:
{{{
$> git annex get <file_name>
}}}

Get all data files in a directory (and sub-directories):
{{{
$> git annex get .
}}}

Modify and then commit a data file named ''foo.dat'':
{{{
$> git annex unlock foo.dat
## Make modifications to foo.dat
$> git annex add foo.dat
$> git commit -m "Modified foo.dat to reflect new format." foo.dat
$> git push
$> git annex copy --to origin foo.dat
}}}

Seems like a lot of work to change modify a data file right??? It is. But this is a proper/better way of dealing with binary files in a source code repository as freesurfer has had for nearly two decades. How about another example, lets add a new data file name ''testdata.tar.gz'':

{{{
$> git annex add testdata.tar.gz
$> git commit -m "Initial add of test data file." testdata.tar.gz
$> git push
$> git annex copy --to origin testdata.tar.gz
}}}

Additional information on how to use git-annex can be found on the git-annex websites website which has numerous examples:
https://git-annex.branchable.com/walkthrough/		   https://surfer.nmr.mgh.harvard.edu/fswiki/Freesurfer_github

Index

Contents

  1. Getting the Source Code
    1. Get the Data Files
  2. Building
    1. Setup Configure
    2. Configure
    3. Compile
    4. Install
  3. Adding a new binary to the tree
  4. Contributing Changes

This page is targeted at Martinos Center users who wish to inspect, build or develop within the Freesurfer code base. Non-Martinos users wishing to work with the Freesurfer code base should consult the read-only git repo. Users who with to contribute to the Freesurfer code base and/or commit changes to the repo should see the following page describes how to fork the Freesurfer github repository and submit pull requests .

1. Getting the Source Code

The first thing users at the Martinos Center need to do is prepend the directory /usr/pubsw/packages/git-annex/current/bin to their PATH. Once that is done, the Freesurfer source code can be cloned from the official Freesurfer github page: 

git clone git@github.com:freesurfer/freesurfer.git

1.1. Get the Data Files

The Freesurfer repository contains a large number data files which are not included with a default git clone of the repo. Instead, these data files are distributed via the git-annex software. Users who only want the repository for the purposes of compiling binaries and/or inspecting source code, the git clone command from above is all you need to do. Users who want to run build time checks, or perform a full local installation, or just want all the contents of the repository, will need to add a special data store remote repository in order to retrieve these files.

To add the data store repository (this only needs to be done once): 

git remote add datasrc file:///space/freesurfer/repo/annex.git
git fetch datasrc

And to retrieve data files: 

git annex get <filename>

The data files have been broken down into categories, those being required for build time checks, those required for a local installation, and everything else. Use one of the following commands depending on your needs: 

## Get only the data files required for build time checks (1.9 GB)
git annex get --metadata fstags=makecheck .

## Get only the data files required for local installation (4.3 GB)
git annex get --metadata fstags=makeinstall .

## Just give me everything! Not Recommended (6.8 GB)
git annex get .

2. Building

2.1. Setup Configure

It is necessary to run a pre-configure script, to create the platform specific tools required by configure (execute in the freesurfer directory created by git clone). This script runs a set of commands (aclocal, libtoolize, automake v1.9.6, autoconf v2.59) that creates the platform specific files for configure and puts them in the 'fsdev/config' directory. 

./setup_configure

2.2. Configure

Now you need to configure your building parameters for your machine by running the configure script. Users at the Martinos Center should for the most part be fine with the default settings, but the configure script does accept many options for pointing to specific libraries and other build specific parameters. One exception is if a user wants to perform a local installation of FreeSurfer, he/she should use the --prefix flag. Type ./configure --help for a full list of options. For example: 

## Default configuration
./configure

## Specify an installation location
./configure --prefix=~/freesurfer_install_dir

## See all possible options
./configure --help

Freesurfer builds against the following set of open-sourced libraries, which are installed under the /usr/pubsw/packages directory on all NMR computers:

  • Package

    Version

    Package

    Version

    CUDA

    v5.0.35-rh5

    ANN

    v1.1

    tiffjpegglut

    v3.6, v6b, v3.7

    itk

    v3.16

    VTK

    v5.6

    VXL

    v1.14

    MNI

    v1.5

    tcltktixblt

    v8.4, v8.4, v8.1, v2.4z

    KWWWidgets

    CVS checkout

    Qt

    v4.7

    wxWidgets

    v2.8

    cppunit

    v1.10

    xawplus

    v3.1

    petsc

    v2.3

All these packages will be found by default by the ./configure script. But there are options to specify where certain packages exists if a user wishes to build against a different version of one of the open-source libraries. For example: 

## Specify a specific version of qt
./configure --with-qt=/usr/pubsw/packages/qt/4.8.5

2.3. Compile

You can now run 'make' to build the all individual programs in the FreeSurfer source tree. Binaries will automatically be placed in their individual subdirectories. 

make -j 4

Handy hint: the -j 4 option to make tells it to run four simultaneous make processes, which, if building on a multi-processor machine, can speed-up the build.

If you want to compile just one binary at a time, for example, if you are developing an app, than cd to the directory of the program you want and use 'make' to compile it: 

cd mri_info
make

This creates mri_info in the mri_info/ directory. However, be aware the many program depends on the existence of libraries having already been build like libutils. Therefore users will need to build a few of the library directories first (e.g. utils, fsgdf, xml2, etc).

2.4. Install

To initial a local installation, type 'make install' from the top level directory: 

make install

This will create a local Freesurfer installation in the directory as specified by the --prefix option to configure script (see above). Note that if you do not specify this location, it will try to install to /usr/local, which will probably require root access. The first time you run 'make install', it will take a while to copy all the big data files to the new installation. Subsequent 'make installs' will only copy the changed files.

If you only want to install a single binary, run 'make install' from a subdirectory. For example, running 'make install' from the mri_convert directory will copy the mri_convert binary to the proper locations. Running 'make install' from scripts/ will copy all the necessary scripts to the right location.

3. Adding a new binary to the tree

For this example we will assume you want to create a program called 'MYPROG' and want to add it to the FreeSurfer tree:

1) Make a directory called MYPROG under the freesurfer directory, and put your source code there. In the simplest case you will have a single source code file called MYPROG.c. 

## Create the MYPROG directory and 'cd' into it
mkdir MYPROG
cd MYPROG
## The MYPROG.c file goes here

2) Copy freesurfer/dummy/Makefile.am into MYPROG/ and customize it, replacing 'dummy' with 'MYPROG'. Also delete the notes that are there. Be sure to change: 

bin_PROGRAMS = MYPROG

3) Modify configure.in to add MYPROG/Makefile to the list of files in the definition of AC_OUTPUT (these are in roughly alphabetical order). 

## configure.in ##

AC_OUTPUT(
... <list of files> ...
MYPROG/Makefile
... <list of files> ...
)

4) Modify freesurfer/Makefile.am to add MYPROG to the MRISUBDIRS or MRISSUBDIRS definition. (You can also alternatively add it to the end of any of the *SUBDIRS categories.) 

## Makefile.am ##

MRISUBDIRS= \
... <list of files> ...
MYPROG \
... <list of files> ...

Once these 4 steps are complete MYPROG should automatically be built with the rest of FreeSurfer. Try following the building steps from above to verify your binary compiles and builds successfully.

4. Contributing Changes

Users who with to contribute to the Freesurfer code base and/or commit changes to the repo should see the following page which describe how to fork the freesurfer repository and submit pull requests: