Differences between revisions 6 and 37 (spanning 31 versions)
Revision 6 as of 2018-07-23 23:35:06
Size: 4636
Editor: AndrewHoopes
Comment:
Revision 37 as of 2021-03-03 01:25:26
Size: 7113
Editor: buildqa
Comment:
Deletions are marked like this. Additions are marked like this.
Line 3: Line 3:
<<BR>> == Initial Setup ==
Line 5: Line 5:
== Configuring a Simple Build == The freesurfer cmake scripts require cmake 3.5 or greater. Unfortunately, /usr/bin/cmake is quite old, so you'll have to use /usr/pubsw/bin/cmake. If it's not in your path already, add the following to your .cshrc:
Line 7: Line 7:
CMake replaces the `setup_configure` and `configure` build steps. A build can be initialized in the working directory by running `cmake </path/to/repo>`. So, assuming you have a freesurfer repository stored in `~/dev/freesurfer`, you could set up a simple in-source build with {{{
setenv PATH /usr/pubsw/bin:$PATH
}}}

=== Installing cmake 3.10 ===
If you don't have /usr/pubsw/bin, you can follow https://askubuntu.com/questions/355565/how-do-i-install-the-latest-version-of-cmake-from-the-command-line/865294

This tells you to...
{{{
sudo -i
version=3.10
build=2
cd /tmp
mkdir cmake
cd cmake
wget https://cmake.org/files/v$version/cmake-$version.$build.tar.gz
tar -xzf cmake-$version.$build.tar.gz
cd cmake-$version.$build/
./bootstrap
make -j4
make install
}}}

== Complete Cmake Build with makefile wrapper ==

You can try using this makefile to run a cmake command, build and install freesurfer on CentOS and Ubuntu linux flavors as well as MacOS. It has preset options for the cmake command for the different OS's, with targets to checkout/update the freesurfer source tree and run build/install targets. Type "make" or "make help" after downloading the file [[https://drive.google.com/file/d/1IMEWQty_7zjsglbErQOlu_KpXb8Lmgmv/view?usp=sharing|makefile.wiki]] and renaming it from makefile.wiki to makefile. (Alternately just use "make -f makefile.wiki"). This makefile will try to download a prebuilt set of third party packages so as not to reference /usr/pubsw from the Martinos Center in-house network, i.e., "build on an airplane" where everything is built from your local disks. '''The last update to makefile.wiki was on 03/03/2021 building the freesurfer dev branch on: MacOS 10.15, CentOS 7; Ubuntu 18.''' Note that the Mac build uses the native clang compilers again (instead of gcc from homebrew).

The subdirectories ./freesurfer (checked out tree from git), ./packages (location of downloaded 3rd party packages), and ./install (install location for freesurfer after build) will all be created in the subdirectory where makefile.wiki is. For the Mac, if you do not have Homebrew installed under the standard path /usr/local, it will try to create a minimal install for you under ./homebrew. See makefile.wiki for more details.

The makefile also contains some notes on what packages to install for the different OS's in order to build freesurfer, though what you may need to install will vary depending upon how your machine has already been setup.


== Configuring a Simple Cmake Build ==

CMake replaces the `setup_configure` and `configure` build steps in the automake build. A build can be configured in the working directory by running `cmake </path/to/repo>`. So, assuming you have a freesurfer repository stored in `~/dev/freesurfer` (and you're developing on the Martinos filesystem), you could set up a simple in-source build with:

'''note:''' if this is your first time transitioning from an automake configuration, you might need to run `make clean` before building with cmake. Some of the previously generated freeview moc files might interfere with the configuration.
Line 14: Line 50:
or an out-of-source build with Aside from this initial cmake step, which really only needs to be run once, and fact that `make check` is now `make test`, the rest of the build process is exactly the same. Previously, if a new freesurfer binary was added to the autotools config, you'd have to run `./configure` again. With cmake, there is no need to run the initialization command more than once (unless to modify build options), since `make` now automatically checks for modifications to the build scripts.
Line 16: Line 52:
{{{
cd ~/dev/fsbuild
cmake ~/dev/freesurfer
}}}
Alternatively, `ccmake` can be used instead of `cmake` to configure/edit cached variables in a terminal GUI. Out-of-source builds are also possible.
Line 21: Line 54:
Alternatively, `ccmake` can be used instead of `cmake` to configure/edit cached variables in a terminal GUI. '''note:''' the `make` output is more condensed now, but `make VERBOSE=1` will output everything.
Line 23: Line 56:
CMake variables are set on the command line with the `-D` flag. For example, the standard way to configure a build with an install path is ==== Configuration Options ====

CMake variables are set on the command line with the `-D` flag. For example, the standard way to configure a build with an install prefix is
Line 29: Line 64:
You can define any variable on the command line this way, even if the variable is never used in the CMakeLists.txt scripts. Boolean variables added with the `option()` function can be turned on/off on the command line as well. For example, freesurfer GUI builds can be disabled by running You can define any variable on the command line this way, even if the variable is never used in the CMakeLists.txt scripts. Boolean variables added with the `option()` function can be turned on/off on the command line as well. See, for instance, BUILD_GUIS in the Making the non-GUI section below.

Note: Do not use "~/" in these specifications, because it may not get correctly expanded causing arcane errors.

==== Making the third party packages ====

To make the rest of Freesurfer, you may have to make and install the third party packages.
'vtk' and 'kwwidgets' might not be buildable on systems that don't build the GUI.
Line 32: Line 74:
cmake . -DBUILD_GUIS=OFF cd /path/to/your/repository/packages
find . -type l -exec git annex get {} \;
./build_packages.py --no-vtk --no-kwwidgets /path/to/install/freesurfer/packages
Line 35: Line 79:
Aside from this initial configuration step (which really only needs to be run once) and fact that `make check` is now `make test`, the rest of the build process is pretty much the same. ==== Making the non-GUI ====
Line 37: Line 81:
'''note:''' the `make` output is more condensed now, but `make VERBOSE=1` will output everything.

<<BR>>

== Overview of the CMake Framework ==

The top-level CMakeLists.txt file is the main cmake configuration script (which replaces setup_configure, configure.in, and Makefile.am), and all subdirectories are added from here with the `add_subdirectory()` function. This main script is split into three parts: locating third-party packages, configuring compilation settings, and configuring freesurfer static libs and programs.

=== Third-Party Packages ===

Most of the packages required by freesurfer are located via custom "find-modules" stored in the `cmake` subdirectory. These find-modules expect each package to be installed under a common path defined by `FS_PACKAGES_DIR`. On Martinos machines, this variable automatically defaults to `/usr/pubsw/packages`, but external developers must provide this path manually:
Once this is done...
Line 50: Line 84:
cmake . -DFS_PACKAGES_DIR="/path/to/packages" cmake . \
    -DBUILD_GUIS=OFF \
    -DCMAKE_INSTALL_PREFIX="/path/to/install/destination" \
    -DFS_PACKAGES_DIR="/path/to/installed/freesurfer/packages" \
    -DGFORTRAN_LIBRARIES="/usr/lib/gcc/x86_64-linux-gnu/5/libgfortran.so" \
    -DGLIB_CONFIG_INCLUDE_DIR="/usr/lib/x86_64-linux-gnu/glib-2.0/include/"
Line 53: Line 92:
If a package is not found under `FS_PACKAGES_DIR`, cmake will continue to look through the default search paths. Alternative paths to package installs can also be specified with the `<PACKAGE>_DIR` variables. For example, to use a non-default ITK version:
=== Doing Debug and Release builds, and others ===
cmake supports building into a directory other than the source directory.
It is not clear yet which of our scripts will support the result.

To do it, first (for convenience) define with BUILD_TYPE you want
{{{
        export BUILD_TYPE=Debug
}}}
or
{{{
     export BUILD_TYPE=Release
}}}
or other cmake BUILD_TYPE.

Make a clean directory tree
{{{
        rm -rf $BUILD_TYPE
        mkdir $BUILD_TYPE
}}}

Within that tree, do a cmake and a make
{{{
        cd $BUILD_TYPE
        cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE \
            <other options> \
            ..
        make
}}}


== Adding a new binary to the Cmake Build ==

All freesurfer binaries should contain a CMakeLists.txt file (replacing Makefile.am) in their corresponding subdirectory, and the name of the subdirectory should be added to the large list at the end of the top-level CMakeLists.txt. As an example, the CMakeLists.txt for a standard freesurfer executable (called my_program) that only links to the utils library would look something like this:
Line 56: Line 128:
cmake . -DITK_DIR="/usr/pubsw/packages/itk/4.9.1" project(my_program)

include_directories(${FS_INCLUDE_DIRS})

add_executable(my_program my_program.c)
target_link_libraries(my_program utils)

install(TARGETS my_program DESTINATION bin)
Line 59: Line 138:
==== Find Modules ==== A more complex CMakeLists.txt might be required for binaries that depend on other libraries. For example, [[https://github.com/freesurfer/freesurfer/blob/dev/mris_mesh_subdivide/CMakeLists.txt|here's the configuration file]] for mris_mesh_subdivide, which requires VTK.
Line 61: Line 140:
In CMakeLists.txt, packages are located by using the `find_package()` function. Some larger, modern projects, like Qt, VTK, and ITK, distribute cmake config files within their installations, so locating the package's include directory and libraries is a fairly straightfoward process: == FAQ ==
Line 63: Line 142:
{{{
find_package(ITK HINTS ${ITK_DIR} REQUIRED)
}}}
'''Q.''' I've messed up my build and I'm getting weird cmake messages... how do I 'reset' cmake?
Line 67: Line 144:
In this example, if ITK is found by cmake, then `ITK_FOUND` is set to true, and `ITK_INCLUDE_DIR` and `ITK_LIBRARIES` are set accordingly.

However, most freesurfer dependencies don't ship with cmake configuration files, so we have to create our own find-modules. Fortunately, this isn't too difficult,

==== External Developers ====

In general, I think the goal should be to distance ourselves from distributing the pre-built package tarballs since they are difficult to maintain across multiple platforms. The `packages/build_packages.py` script is a potential alternative to the pre-built archives - it's a utility script to help external developers build freesurfer dependencies on their own.

The packages configured within `build_packages.py` are built using the tarballs and buildscripts stored in the `packages/source` dir and will get installed to a destination directory specified on the command-line:

{{{
build_packages.py "/path/to/install/destination"
}}}

This script loops through each package in the `pkgs` list, extracts the associated tarball to `destination/package-name/version/src`, and runs the package build script. The individual build scripts (like [this ITK one](??)) always expect a single input argument that points to the desired install directory. After each successful package build, the checksum of the tarball + build script is saved to the src dir. This way, if this script is rerun, the package is only rebuilt when the source code has been modified

Once the dependencies are built, developers can then point `FS_PACKAGES_DIR` to their local install directory.
'''A.''' rm -Rf CMakeFiles CMakeCache.txt

FreeSurfer CMake

Initial Setup

The freesurfer cmake scripts require cmake 3.5 or greater. Unfortunately, /usr/bin/cmake is quite old, so you'll have to use /usr/pubsw/bin/cmake. If it's not in your path already, add the following to your .cshrc:

setenv PATH /usr/pubsw/bin:$PATH

Installing cmake 3.10

If you don't have /usr/pubsw/bin, you can follow https://askubuntu.com/questions/355565/how-do-i-install-the-latest-version-of-cmake-from-the-command-line/865294

This tells you to...

sudo -i
version=3.10
build=2
cd /tmp
mkdir cmake
cd cmake
wget https://cmake.org/files/v$version/cmake-$version.$build.tar.gz
tar -xzf cmake-$version.$build.tar.gz
cd cmake-$version.$build/
./bootstrap
make -j4
make install

Complete Cmake Build with makefile wrapper

You can try using this makefile to run a cmake command, build and install freesurfer on CentOS and Ubuntu linux flavors as well as MacOS. It has preset options for the cmake command for the different OS's, with targets to checkout/update the freesurfer source tree and run build/install targets. Type "make" or "make help" after downloading the file makefile.wiki and renaming it from makefile.wiki to makefile. (Alternately just use "make -f makefile.wiki"). This makefile will try to download a prebuilt set of third party packages so as not to reference /usr/pubsw from the Martinos Center in-house network, i.e., "build on an airplane" where everything is built from your local disks. The last update to makefile.wiki was on 03/03/2021 building the freesurfer dev branch on: MacOS 10.15, CentOS 7; Ubuntu 18. Note that the Mac build uses the native clang compilers again (instead of gcc from homebrew).

The subdirectories ./freesurfer (checked out tree from git), ./packages (location of downloaded 3rd party packages), and ./install (install location for freesurfer after build) will all be created in the subdirectory where makefile.wiki is. For the Mac, if you do not have Homebrew installed under the standard path /usr/local, it will try to create a minimal install for you under ./homebrew. See makefile.wiki for more details.

The makefile also contains some notes on what packages to install for the different OS's in order to build freesurfer, though what you may need to install will vary depending upon how your machine has already been setup.

Configuring a Simple Cmake Build

CMake replaces the setup_configure and configure build steps in the automake build. A build can be configured in the working directory by running cmake </path/to/repo>. So, assuming you have a freesurfer repository stored in ~/dev/freesurfer (and you're developing on the Martinos filesystem), you could set up a simple in-source build with:

note: if this is your first time transitioning from an automake configuration, you might need to run make clean before building with cmake. Some of the previously generated freeview moc files might interfere with the configuration.

cd ~/dev/freesurfer
cmake .

Aside from this initial cmake step, which really only needs to be run once, and fact that make check is now make test, the rest of the build process is exactly the same. Previously, if a new freesurfer binary was added to the autotools config, you'd have to run ./configure again. With cmake, there is no need to run the initialization command more than once (unless to modify build options), since make now automatically checks for modifications to the build scripts.

Alternatively, ccmake can be used instead of cmake to configure/edit cached variables in a terminal GUI. Out-of-source builds are also possible.

note: the make output is more condensed now, but make VERBOSE=1 will output everything.

Configuration Options

CMake variables are set on the command line with the -D flag. For example, the standard way to configure a build with an install prefix is

cmake . -DCMAKE_INSTALL_PREFIX="/path/to/install/destination"

You can define any variable on the command line this way, even if the variable is never used in the CMakeLists.txt scripts. Boolean variables added with the option() function can be turned on/off on the command line as well. See, for instance, BUILD_GUIS in the Making the non-GUI section below.

Note: Do not use "~/" in these specifications, because it may not get correctly expanded causing arcane errors.

Making the third party packages

To make the rest of Freesurfer, you may have to make and install the third party packages. 'vtk' and 'kwwidgets' might not be buildable on systems that don't build the GUI.

cd /path/to/your/repository/packages
find . -type l -exec git annex get {} \;
./build_packages.py --no-vtk --no-kwwidgets  /path/to/install/freesurfer/packages

Making the non-GUI

Once this is done...

cmake . \
    -DBUILD_GUIS=OFF \
    -DCMAKE_INSTALL_PREFIX="/path/to/install/destination" \
    -DFS_PACKAGES_DIR="/path/to/installed/freesurfer/packages" \
    -DGFORTRAN_LIBRARIES="/usr/lib/gcc/x86_64-linux-gnu/5/libgfortran.so" \
    -DGLIB_CONFIG_INCLUDE_DIR="/usr/lib/x86_64-linux-gnu/glib-2.0/include/"

Doing Debug and Release builds, and others

cmake supports building into a directory other than the source directory. It is not clear yet which of our scripts will support the result.

To do it, first (for convenience) define with BUILD_TYPE you want

        export BUILD_TYPE=Debug

or

     export BUILD_TYPE=Release

or other cmake BUILD_TYPE.

Make a clean directory tree

        rm -rf $BUILD_TYPE
        mkdir $BUILD_TYPE

Within that tree, do a cmake and a make

        cd $BUILD_TYPE
        cmake -DCMAKE_BUILD_TYPE=$BUILD_TYPE \
            <other options> \
            ..
        make

Adding a new binary to the Cmake Build

All freesurfer binaries should contain a CMakeLists.txt file (replacing Makefile.am) in their corresponding subdirectory, and the name of the subdirectory should be added to the large list at the end of the top-level CMakeLists.txt. As an example, the CMakeLists.txt for a standard freesurfer executable (called my_program) that only links to the utils library would look something like this:

project(my_program)

include_directories(${FS_INCLUDE_DIRS})

add_executable(my_program my_program.c)
target_link_libraries(my_program utils)

install(TARGETS my_program DESTINATION bin)

A more complex CMakeLists.txt might be required for binaries that depend on other libraries. For example, here's the configuration file for mris_mesh_subdivide, which requires VTK.

FAQ

Q. I've messed up my build and I'm getting weird cmake messages... how do I 'reset' cmake?

A. rm -Rf CMakeFiles CMakeCache.txt

CMake (last edited 2021-03-03 01:25:26 by buildqa)