General

I have written a class for OpenMS. What should I do?

Follow the Coding Conventions.

To summarize:

The coding style (brackets, variable names, etc.) must conform to the conventions.
The class and all the members should be properly documented.
Check your code with the tool tools/checker.php. Call php tools/checker.php for detailed instructions.

Please open a pull request and follow the pull request guidelines.

Can the `START_SECTION-macro` for class tests not handle template methods that have two or more arguments?

Insert round brackets around the method declaration.

Where are the binary installers created?

View the binary installers at the build archive.

Please verify the creation date of the individual installers, as there may have been an error while creating the installer.

Build system

The following questions are related to the build system.

View the cmake website for more information.

What are user definable `CMake` cache variables?

User definable CMake cache variables allow the user to pass options to CMake which will influence the build system. The most important option that should be given when calling CMake is:

CMAKE_PREFIX_PATH, which is where CMake will search for additional libraries if they are not found in the default system paths. By default we add OpenMS/contrib.

If you have installed all libraries on your system already, there is no need to change CMAKE_PREFIX_PATH. For contrib libraries, set the variable CMAKE_PREFIX_PATH.

On Windows, the contrib folder is required, as there are no system developer packages. To pass this variable to CMake use the -D switch e.g. cmake -D CMAKE_PREFIX_PATH:PATH="D:\\somepath\\contrib".

Everything else can be edited using ccmake afterwards.

See Building OpenMS on GNU/Linux, Building OpenMS on Mac OS X, Building OpenMS on Windows for more flags.

View the description for each option by calling ccmake.

Can I use another solver other than GLPK or COINOR?

No. Additionally, if COINOR is found during the CMake step, it is preferred over GLPK. Refer to the documentation of the LPWrapper class, on how to abstract your code.

I changed the `contrib` path, but re-running `CMake` won't change the library paths?

Once a library is found and its location is stored in a cache variable, it will only be searched again if the corresponding entry in the cache file is set to false.

Warning: If you delete the CMakeCache.txt, all other custom settings will be lost.

The most useful targets will be shown to you by calling the targets target, i.e. make targets.

`CMake` can't seem to find a Qt library (usually `QtCore`).

CMake finds QT by looking for qmake in your PATH or for the Environment Variable QTDIR. Set these accordingly.

Make sure there is no second installation of Qt (especially the MinGW version) in your local environment.

Warning: This might lead CMake to the wrong path (it's searching for the Qt*.lib files). You should only move or delete the offending Qt version if you know what you are doing.

A save workaround is to edit the CMakeCache file (e.g. via ccmake) and set all paths relating to QT (e.g. QT_LIBRARY_DIR) manually.

(Windows) What version of Visual Studio should I use?

It is recommended to use the latest version. Get the latest CMake, as its generator needs to support your VS. If your VS is too new and there is no CMake for that yet, you're gonna be faced with a lot of conversion issues. This happens whenever the Build-System calls CMake (which can be quite often, e.g., after changes to CMakeLists.txt).

How to speed up the compile process

To speed up the compile process of OpenMS, use several threads. In general, the ninja generator (as opposed to make) uses all CPU cores by default, which may be more convenient to you. With make-based generators, supply the number of parallel jobs manually, e.g. make -j20.

On Windows, Visual Studio solution files are automatically build with the /MP flag, such that Visual Studio uses all available cores of the machine.

Continuous integration (CI)

We currently employ Github Actions (GHA), for the majority of our CI. See <OpenMS>/.github/workflows/openms_ci_matrix_full.yml for the full CI build script which gets run for every PR.

We are currently phasing out Travis.

How does `travis` work?

Travis is an automated system for continuous integration and each new commit and pull request is automatically run through the travis build system. This is controlled by a .travis.yaml file in the source tree.

Working in IDEs

Why are not all Files/Tests etc visible in my IDE, e.g. there is no `source/TEST` and `source/APPLICATIONS/TOPP` folder?

All source files added to an IDE are associated with their targets. Find the source files for each test within its own subproject. The same is true for the TOPP classes.

Thus, open the correct project (e.g. for Visual Studio find the correct .sln file) in the build tree.

I'm getting the error "Error C2471: cannot update program database"

Warning: This is a bug in Visual Studio and there is a bug fix. Only apply it if you encounter the error. The bug fix might have unwanted side effects!

Visual Studio can't read the clang-format file.

Depending on the Visual Studio version it might get an error like Error while formating with ClangFormat. This is because Visual Studio is using an outdated version of clang-format. Unfortunately there is no easy way to update this using Visual Studio itself. There is a plugin provided by LLVM designed to fix this problem, but the plugin doesn't work with every Visual Studio version. In that case, update clang-format manually using the pre-build clang-format binary. Both the binary and a link to the plugin can be found here. To update clang-format download the binary and exchange it with the clang-format binary in your Visual Studio folder. For Visual Studio 19 it should be located at: C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/Llvm/bin.

What is the difference between class tests and tool tests?

Class tests are unit tests that typically test the functionality of a class. They get built as standalone "additional" executables that include the class to be tested and the testing utility classes to test outcomes of single functions of the class. Unless you've added functions that are intended to be used outside of your new additional mode, you don't need to add anything.

Tool tests are using the tool executable that the user would also receive. We use those executables to run the full algorithm on a small test dataset, to ensure that from version to version, the results stay the same.

How to add a new TOPP test

Each TOPP test consists of:

An executable call on a test dataset (by using either fixed command line parameters or a .ini file).
A FuzzyDiff call that compares the temporary output file of the last call and a reference test output that you have to provide.
A line to add a dependency of the FuzzyDiff call on the actual executable call (so they get executed after each other). Use e.g., ctest -V -R IDMapper to only test tests that include the regex IDMapper (-V is just verbose). Make sure to build the IDMapper and IDMapper_test executable after it is edited. ctest does not have any automatic dependency on the timestamps of the executables.

See <OpenMS>/src/tests/topp/CMakeLists.txt for examples.

How to add a new class test

Write a test to every new class added to OpenMS.

To add a test:

Add the class test to src/tests/class_tests/<sub-project>/ (e.g., src/tests/class_tests/openms/source/NewFileFormat_test.cpp).
Add the test to the executables.cmake file in the test folder.
Put new files under version control git add

A test template for your specific class can be generated using the create_test.php script found in tools/.

To generate a test template:

Make sure you generate XML files containing the class information make xml.
Call:
php tools/create_test.php /BUILD_DIRECTORY/ /PATH_TO_HEADER/MyClass.h \

"FIRSTNAME LASTNAME" > ./src/tests/class_tests/openms/source/MyClass_test.cpp

How to add a new GUI test for a class

Suppose you want to create a GUI class for the class called MyClass. To add a new GUI test:

Create the MyClass_test.cpp in src/tests/class_tests/openms_gui/source.
Add it to src/tests/class_tests/openms_gui/CMakeLists.txt in the GUI section.
Have a look at existing GUI tests, as they use the QT TestLib framework and not the OpenMS macros.

All tests fail when executing `make_test`

Check the LD_LIBRARY_PATH environment variable:

Print the LD_LIBRARY_PATH with echo $LD_LIBRARY_PATH. If your /lib/ folder is included, check that libOpenMS.so is present. With the ldd command, you can show the libraries used by an executable, e.g. ldd /bin/ClassTest_test.

Debugging

The following section provides information on how to debug your code.

How to find out which shared libraries are used by an application

Linux: Use ldd.

Windows (Visual studio console): See Dependency Walker. Use the x64 version for 64 bit builds. Using the wrong version of depends.exe will give the wrong results. Or dumpbin /DEPENDENTS OpenMS.dll.

How to get a list of the symbols defined in a (shared) library or object file?

Linux: Use nm <library>.

Use nm -C to switch on demangling of low-level symbols into their C++-equivalent names. nm also accepts .a and .o files.

Windows (Visual studio console): Use dumpbin /ALL <library>.

Use dumpbin on object files (.o) or (shared) library files (.lib) or the DLL itself e.g. dumpbin /EXPORTS OpenMS.dll.

Why does my TOPP tool fail to start

Starting a (GUI) TOPP executable (like TOPPView or FeatureFinderCentroided) gives "The application was unable to start correctly (0xc0000005). Click OK to close the application"
When you run the tool in Debug mode and look where it crashes, you might actually find it to be a very weird place - e.g. when creating a perfectly legal String from a QString:
QFileInfo fi(file.toQString());

return fi.path()

The reason this happens is usually: you've mixed DLL's from multiple runtimes (e.g. compiled with VS10 and VS9). This can easily happen, if you use your Qt (build with VS9) to link against your OpenMS (build with VS10). The loader will load VS9 and VS10 runtimes and will NOT tell you that they conflict. Instead, very weird things are going to happen. You can identify if you are affected by looking at the DLL's that are loaded via daisy-chaining - see either DependencyWalker's output or the 'output' window in VS when running the app from inside VS. It might read:
'MSSimulator.exe': Loaded 'C:\Windows\winsxs\amd64_microsoft.vc90.debugcrt_1fc8b3b9a1e18e3b_9.0.21022.8_none_4ec74c6b3093419c\msvcp90d.dll', Symbols loaded.

'MSSimulator.exe': Loaded 'C:\Windows\winsxs\amd64_microsoft.vc90.debugcrt_1fc8b3b9a1e18e3b_9.0.21022.8_none_4ec74c6b3093419c\msvcr90d.dll', Symbols loaded.

'MSSimulator.exe': Loaded 'C:\dev\qt-everywhere-opensource-src-4.7.1\bin\QtSqld4.dll', Symbols loaded.

As this was a VS10 build, the vc90 DLL's have no place here, but got loaded from the Qt4 DLL's.
Solution: recompile Qt (in this case) using VS10 and link against the new Qt.
Starting a (GUI) TOPP executable (like TOPPView or FeatureFinderCentroided) gives "Entry point not found"
This usually happens when your PATH contains multiple versions of DLL's on which TOPP/OpenMS depends. Candidates are all QT (QtGui4.dll, QtCore4.dll, ...) or Xerces-C (xerces-c_3_0) libraries. Usually other external programs (Miktex, Mendeley etc) put their own binary directory (which contains these incompatible DLL's) before the contrib directory in your PATH%.
Solution: put the contrib directory at the very beginning of your PATH%. The other tools should be unaffected as DLL's are first searched in the path of the executable (where their DLL's should be as well). We do that for OpenMS as well, but only in the binary installer packages, not for the developer version.

How to profile code

To just obtain overall runtime and peak RAM usage of a TOPP tool, the easiest is probably /usr/bin/time (or https://github.com/cbielow/wintime for Windows). However, each TOPP tool will report those numbers at the end of its run anyways.

Windows: this is directly supported by Visual Studio (Depending on the edition: Team and above). Follow their documentation.

Linux: For profiling, perf is usually a very good starting point (see https://perf.wiki.kernel.org/index.php/Main_Page). Make sure to build OpenMS with debug symbols in Release mode, e.g. cmake -DMY_CXX_FLAGS="-g;-fno-omit-frame-pointer" -DCMAKE_BUILD_TYPE=Release <more flags here>. Then compile OpenMS and run perf stat (overview) or record (detailed statistics down to function level):

perf stat -e cache-misses <yourTOPPtool>

perf record -e cpu-cycles,faults -g --call-graph dwarf ./my_tool

We recommend hotspot (https://github.com/KDAB/hotspot) to view inspect the resulting perf.data (or use perf report). Further reading http://sandsoftwaresound.net/perf/perf-tutorial-hot-spots/ .

Alternatively, try Valgrind (warning: very long runtimes, up to factor 100x):

Build OpenMS in debug mode (set CMAKE_BUILD_TYPE to Debug).
Call the executable with valgrind: valgrind –tool=callgrind.
Warning
Other processes running on the same machine can influence the profiling. Make sure your application gets enough resources (memory, CPU time).
Start and stop the profiling while the executable is running e.g. to skip initialization steps.
Start valgrind with the option –instr-atstart=no.
Call callgrind -i [on|off] to start/stop the profiling.
The output can be viewed with kcachegrind callgrind.out.

Another option is IBM's profiler, available for all platforms (and free for academic use): Purify(Plus) and/or Quantify.

How to check code for memory leaks

Use the AddressSanitizer, which comes with g++/clang these days. See Building OpenMS on GNU/Linux for details on how to compile it into OpenMS.

Build OpenMS in debug mode (set CMAKE_BUILD_TYPE to Debug).
Call the executable with valgrind: valgrind –suppressions=OpenMS/tools/valgrind/openms_external.supp –leak-check=full <executable> <parameters>.

Common errors are:

Invalid write/read ... - Violation of container boundaries.
... depends on uninitialized variable - Uninitialized variables.
... definitely lost - Memory leak that has to be fixed.
... possibly lost - Possible memory leak, so have a look at the code.

For more information see the valgrind documentation.

Release

View preparation of a new OpenMS release to learn more about contributing to releases.

Table of Contents