Differences between revisions 21 and 39 (spanning 18 versions)
Revision 21 as of 2016-10-18 15:55:35
Size: 11822
Editor: ZekeKaufman
Comment:
Revision 39 as of 2019-02-03 13:21:15
Size: 0
Editor: AndrewHoopes
Comment: not needed anymore
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
#pragma section-numbers on
#acl WikiUserGroup:read,write,delete,revert All:read

'''Index'''

<<TableOfContents(3)>>

This page is targeted at Martinos Center users who wish to inspect, build or develop within the !FreeSrufer code base. Non-Martinos users wishing to work with the !FreeSrufer code base should consult the read-only git repo http://surfer.nmr.mgh.harvard.edu/fswiki/freesurfer_linux_developers_page

=== Git Clone ===

First thing users at the Martinos Center will need to do is add the newer version of the {{{git}}} and {{{git-annex}}} software to their PATH environment variable. This should be added to the {{{.cshrc}}} or {{{.bashrc}}} file in the users home directory:

{{{
## csh
$> setenv PATH /usr/pubsw/packages/git-annex/current/bin:$PATH

## bash
$> export PATH=/usr/pubsw/packages/git-annex/current/bin:$PATH
}}}

Users can then clone !FreeSurfer repository as follows:

{{{
## Get source code files (206 MB)
$> git clone file:///space/freesurfer/repo/freesurfer
}}}

==== Getting the Data Files ====
The !FreeSurfer repository contains a large number of data files (pdfs, test data, etc) which exist as symlinks scattered throughout the !FreeSurfer directory hierarchy. These data files are not included with a default {{{git clone}}} of the repo and thus will exist as broken symlinks. Users who wish to clone the repository for the purposes of just inspecting source code and/or build the code, the {{{git clone}}} command from above is all that is required, and the broken symlinks will not present an issue. Users who want to run build time checks, or perform a full local installation, or just want all the data files contained in the repository, will need to run the following command in order to download the required data data files:

{{{

## Change directories
$> cd freesurfer

## Get only the data files required for build time checks (1.9 GB)
$> git annex get --metadata fstags=makecheck .

## Get only the data files required for local installation (4.3 GB)
$> git annex get --metadata fstags=makeinstall .

## Just give me everything! Not Recommended (6.8 GB)
$> git annex get .
}}}

=== Building ===

==== Setup Configure ====

It is necessary to run a pre-configure script, to create the platform specific tools required by configure (execute in the {{{freesurfer}}} directory created by {{{git clone}}}). This script runs a set of commands (aclocal, libtoolize, automake v1.9.6, autoconf v2.59) that creates the platform specific files for configure and puts them in the 'fsdev/config' directory.

{{{
$> ./setup_configure
}}}

==== Configure ====

Now you need to configure your building parameters for your machine by running the {{{configure}}} script. Users at the Martinos Center should for the most part be fine with the default settings, but the {{{configure}}} script does accept many options for pointing to specific libraries and other build specific parameters. One exception is if a user wants to perform a local installation of !FreeSurfer, he/she should use the {{{--prefix}}} flag. Type {{{./configure --help}}} for a full list of options. For example:

{{{
## Default configuration
$> ./configure

## Specify an installation location
$> ./configure --prefix=~/freesurfer_install_dir

## See all possible options
$> ./configure --help
}}}


!FreeSurfer builds against the following set of open-sourced libraries, which are installed under the {{{/usr/pubsw/packages}}} directory on all NMR computers:

 || Package || Version || || Package || Version ||
 || CUDA || v5.0.35-rh5 || || ANN || v1.1 ||
 || tiffjpegglut || v3.6, v6b, v3.7 || || itk || v3.16 ||
 || VTK || v5.6 || || VXL || v1.14 ||
 || MNI || v1.5 || || tcltktixblt || v8.4, v8.4, v8.1, v2.4z ||
 || KWWWidgets || CVS checkout || || Qt || v4.7 ||
 || wxWidgets || v2.8 || || cppunit || v1.10 ||
 || xawplus || v3.1 || || petsc || v2.3 ||

All these packages will be found by default by the {{{./configure}}} script. But there are options to specify where certain packages exists if a user wishes to build against a different version of one of the open-source libraries. For example:

{{{
## Specify a specific version of qt
$> ./configure --with-qt=/usr/pubsw/packages/qt/4.8.5
}}}

==== Compile ====

You can now run 'make' to build the all individual programs in the !FreeSurfer source tree. Binaries will automatically be placed in their individual subdirectories.

{{{
$> make -j 4
}}}
''Handy hint: the -j 4 option to make tells it to run four simultaneous make processes, which, if building on a multi-processor machine, can speed-up the build.''

If you want to compile just one binary at a time, for example, if you are developing an app, than {{{cd}}} to the directory of the program you want and use 'make' to compile it:
{{{
$> cd mri_info
$> make
}}}
This creates mri_info in the mri_info/ directory. However, be aware the many program depends on the existence of libraries having already been build like libutils. Therefore users will need to build a few of the library directories first (e.g. utils, fsgdf, xml2, etc).

==== Install ====

To install all binaries and support files into your private !FreeSurfer installation, type 'make install' from the toplevel fsdev/ directory, like this:

{{{
$> make install
}}}

This will create a local !FreeSurfer installation in the directory specified by the {{{--prefix}}} option to {{{configure}}} scipt (see above). Note that if you do not specify this location, it will try to install to /usr/local, which will probably require root access.

Note that you can also run 'make release'. 'make install' makes and installs the NMR internal version of FreeSurfer, while 'make release' makes the public version which omits some stuff.

The first time you run 'make install', it will take a while to copy all the big data files to the new installation. Subsequent 'make installs' will only copy the changed files.

If you only want to install a single binary, run 'make install' from a subdirectory. For example, running 'make install' from the scuba/ directory will copy the scuba binary and its support script files to the proper locations. Running 'make install' from scripts/ will copy all the necessary scripts to the right location.

=== Adding a new binary to the tree ===

For this example we will assume you want to create a program called 'MYPROG' and want to add it to the !FreeSurfer tree:

1) Make a directory called {{{MYPROG}}} under the {{{freesurfer}}} directory, and put your source code there. In the simplest case you will have a single source code file called {{{MYPROG.c}}}.

{{{
## Create the MYPROG directory and 'cd' into it
$> mkdir MYPROG
$> cd MYPROG
## The MYPROG.c file goes here
}}}

2) Copy {{{freesurfer/dummy/Makefile.am}}} into {{{MYPROG/}}} and customize it, replacing 'dummy' with 'MYPROG'. Also delete the notes that are there. Be sure to change:

{{{
bin_PROGRAMS = MYPROG
}}}


3) Modify {{{configure.in}}} to add {{{MYPROG/Makefile}}} to the list of files in the definition of {{{AC_OUTPUT}}} (these are in roughly alphabetical order).


{{{

## configure.in ##

AC_OUTPUT(
... <list of files> ...
MYPROG/Makefile
... <list of files> ...
)
}}}

4) Modify {{{freesurfer/Makefile.am}}} to add {{{MYPROG}}} to the {{{MRISUBDIRS}}} or {{{MRISSUBDIRS}}} definition. (You can also alternatively add it to the end of any of the *SUBDIRS categories.)

{{{

## Makefile.am ##

MRISUBDIRS= \
... <list of files> ...
MYPROG \
... <list of files> ...
}}}

Once these 4 steps are complete MYPROG should automatically be built with the rest of !FreeSurfer.

=== Using Git ===

!FreeSurfer developers should run the following commands to get their git configuration settings set up properly. These commands only need to be run once:

{{{
## User settings.
$> git config --global user.name "John Doe"
$> git config --global user.email johndoe@nmr.mgh.harvard.edu

## Some recommended configuration settings:
$> git config --global push.default current
$> git config --global color.ui true
}}}

==== Examples ====

In order to facilitate the transfer from CVS to Git, the following examples use terms like "checkout" and "branch". Generally these terms mean two different things when talking about CVS vs. Git, but for the following examples they can be thought of as similar concepts.

"Checkout" the main branch:

{{{
$> git clone file:///space/freesurfer/repo/freesurfer <optional_directory_name>
}}}

"Checkout" the stable6 branch:
{{{
$> git clone -b stable6 file:///space/freesurfer/repo/freesurfer <optional_directory_name>
}}}

Bring my local repository up to date with what's on the server:
{{{
$> git pull
}}}

View the status of your checkout (see what files are new, deleted, or modified):
{{{
$> git status .
}}}
My local repository is corrupted and junk. Just blow it up and replace it with whats on the server:
{{{
$> git reset --hard origin/master
}}}

View the commit history of a file:
{{{
$> git log <file_name>
}}}

"Commit" a modified source code file:
{{{
$> git commit -m "Added new capabilities." <file_name>
$> git push
}}}

Adding new source code file:
{{{
$> git add <file_name>
$> git commit -m "Initial add of new file." <file_name>
$> git push
}}}

From the example above [[DevelopersGuide_git#Addinganewbinarytothetree]], if your program successfully builds/installs/runs and is thus ready to be checked into the repository, you can commit the files as follows:

{{{
$> git add MYPROG/MYPROG.c MYPROG/Makefile.am
$> git commit -m "Initial add of MRPROG." MYPROG/MYPROG.c MYPROG/Makefile.am configure.in Makefile.am
$> git push
}}}

Additional information on how to use Git can be found on the Atlassian websites which has decent tutorials and examples to help users getting started with Git:
https://www.atlassian.com/git/tutorials/saving-changes

==== Data Files and git-annex ====

Data files represent a special use-case, and they should not go into the normal Git repository. Instead {{{git-annex}}} is used to store all data files that are part of the !FreeSurfer repository. Generally speaking, data files are anything not source code related - pdfs, compressed tarballs, test data, precompiled binaries, and image files are all examples of data files that should be stored in {{{git-annex}}}. Simple text files like Look-up tables and configuration files, while not exactly source code, are NOT considered data files. Those files CAN go in the normal repository.

Below are a few examples on how one would work with the git-annex:

Get any data file which exists as a broken symlink:
{{{
$> git annex get <file_name>
}}}

Get all data files in a directory (and sub-directories):
{{{
$> git annex get .
}}}

Modify and then commit a data file named ''foo.dat'':
{{{
$> git annex unlock foo.dat
## Make modifications to foo.dat
$> git annex add foo.dat
$> git commit -m "Modified foo.dat to reflect new format." foo.dat
$> git push
$> git annex copy --to origin foo.dat
}}}

Seems like a lot of work to change modify a data file right??? It is. But this is a proper/better way of dealing with binary files in a source code repository as freesurfer has had for nearly two decades. How about another example, lets add a new data file name ''testdata.tar.gz'':

{{{
$> git annex add testdata.tar.gz
$> git commit -m "Initial add of test data file." testdata.tar.gz
$> git push
$> git annex copy --to origin testdata.tar.gz
}}}

Additional information on how to use git-annex can be found on the git-annex websites website which has numerous examples:
https://git-annex.branchable.com/walkthrough/