QuantLib-SWIG and a Thread-Safe Observer Pattern in C++

Update 23.11.2015: A modified version of the patch is now part of the official QuantLib Release 1.7.

Update 28.02.2015: Please find the latest version of the patch for QuantLib 1.5 here.

Update 11.05.2014: Please find the latest version of the patch for QuantLib 1.4 here.

The QuantLib library is not really thread-safe. Successful usage in multi-core and parallel environments is often achieved by message passing between multiple processes rather than shared memory and multi-threading. If different threads are acting on distinct objects then QuantLib can also be used in the multi-threading environment (define QL_ENABLE_SESSIONS to make the singletons thread dependent.).

If you are using QuantLib in Java or Scala via SWIG [1] then the QuantLib routines are automatically executed in a multi-threading environment even if your main routine is single threaded. The garbage collector of the JVM is usually running in different thread. In conjunction with QuantLib’s implementation of the Observer pattern this can lead to serious problems, e.g. the following single threaded Scala code crashes on a multi-core computer after a short period of time.

import org.quantlib.{Array => QArray, _}
object ObserverTest {
    def main(args: Array[String]) : Unit = {
        System.loadLibrary("QuantLibJNI");
        val aSimpleQuote = new SimpleQuote(0)

        while (true) {
            (0 until 10).foreach(_ => {
                new QuoteHandle(aSimpleQuote)
                aSimpleQuote.setValue(aSimpleQuote.value + 1)
            })
            System.gc
        }
    }
}

The garbage collector might call the destructor of an observer (QuoteHandle) at the same point in time when the update method on the same object is invoked by aSimpleQuote.setValue. (Find here the original postings on the QuantLib mailing list regarding this problem.). For a greenfield project the author in [2] has a nice solution for this problem in C++. Unfortunately this solution can not be applied to the QuantLib because the solution involves a change of signature of the Observer interface and thereafter would lead to a lot of change in the QuantLib library.

The main problem here is that we have to disable the Observer before the destructor starts to work. This could in theory be achieved by adding a special “Deleter” to every new instance of a boost::shared_ptr. But again this would lead to a lot of changes in the library.

When compiled with the preprocessor directive BOOST_SP_ENABLE_DEBUG_HOOKS the boost library adds a callback hook before a destructor is called by any boost smart pointer. This hook can be utilized to disable an Observer before the actual destructor is executed. In addition the boost::signals2 library [3] provides a simple and thread-safe notification mechanism.

The corresponding thread-safe implementation of the original QuantLib Observer/Observable interface can be found here. Set the preprocessor directive BOOST_SP_ENABLE_DEBUG_HOOKS for every source file. Replace/Add the files observable.hpp and observable.cpp and recompile the QuantLib library and the QuantLib-SWIG module.

Note: if you want to use the patch with QL 1.2 and not for the trunk, please see the comment by gk below.

[1] Simplified Wrapper and Interface Generator, SWIG

[2] Shuo Chen, Where Destructors meet Threads

[3] Douglas G., Mori Hess F., Boost.Signals2

Advertisements

12 thoughts on “QuantLib-SWIG and a Thread-Safe Observer Pattern in C++

  1. Thanks a lot for the post. I was wondering if you could clarify what you mean by: “Set the preprocessor directive BOOST_SP_ENABLE_DEBUG_HOOKS for every source file”. I’m not the most proficient of programmers… at all…

  2. In VisualStudio add BOOST_SP_ENABLE_DEBUG_HOOKS to
    Project Property Page -> Configuration Properties -> C/C++ -> Preprocessor -> Preprocessor Definitions
    in all projects. Under Linux/Unix
    export CXXFLAGS=”-g -O2 -Wall -DBOOST_SP_ENABLE_DEBUG_HOOKS=1″
    ./configure
    make clean
    make all

    Other possible solution should be adding
    #define BOOST_SP_ENABLE_DEBUG_HOOKS 1
    to qldefines.hpp

  3. This could be a real life saver (been fighting with QuantLib about it for a couple of days now) but I cannot build it on linux with QuantLib 1.2, see error below, any clues? Thanks.

    libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../../../ql -I../../.. -I../../.. -DBOOST_SP_ENABLE_DEBUG_HOOKS=1 -g -O2 -MT fdblackscholesasiane
    ngine.lo -MD -MP -MF .deps/fdblackscholesasianengine.Tpo -c fdblackscholesasianengine.cpp -fPIC -DPIC -o .libs/fdblackscholesasianengine.o
    ../../../ql/patterns/observable.hpp: In member function ‘virtual void QuantLib::FdBlackScholesAsianEngine::calculate() const’:
    ../../../ql/patterns/observable.hpp:66:15: error: ‘static void* QuantLib::Observer::operator new(std::size_t)’ is inaccessible
    fdblackscholesasianengine.cpp:111:72: error: within this context
    ../../../ql/patterns/observable.hpp:67:14: error: ‘static void QuantLib::Observer::operator delete(void*)’ is inaccessible
    fdblackscholesasianengine.cpp:111:72: error: within this context
    In file included from /usr/include/boost/smart_ptr/shared_ptr.hpp:30:0,
    from /usr/include/boost/shared_ptr.hpp:17,
    from ../../../ql/errors.hpp:31,
    from ../../../ql/patterns/observable.hpp:31,
    from ../../../ql/stochasticprocess.hpp:30,
    from ../../../ql/processes/blackscholesprocess.hpp:29,
    from fdblackscholesasianengine.cpp:21:

    • Hi

      thanks for the hint! I’m always working in the trunk and therefore I haven’t recognized that the patch does not compile with QL 1.2. There is a very small change you have to apply in 1.2 to make it working. Please change in ql/methods/finitedifferences/solvers/fdmsimple2dbssolver.hpp line 38 from

      class FdmSimple2dBSSolver : LazyObject {

      to

      class FdmSimple2dBSSolver : public LazyObject {

      Please let me know if it still doesn’t work for you.

      • That did it, it builds and seems to run just fine, so far no jvm crashes on gc after 1M+ calls (and counting) *happy dance*!

        Again, thanks a lot, this was probably the single biggest problem I had with QuantLib.

  4. Could you please tell me whcih version of boost you used and which version of quantlib ?
    If i try to compile quantlib-1.2 with boost 1.51 and preprocessor-directive BOOST_SP_ENABLE_DEBUG_HOOKS i get this warnings:
    1>c:\build_ql_1_2_0\QuantLib\ql/patterns/observable.hpp(66) : warning C4290: C++ exception specification ignored except to indicate a function is not __declspec(nothrow)

    Thanks, Stephan

  5. […] Using QuantLib in Java/Scala/C# or F# applications via the SWIG layer violates the multi-threading requirement because the garbage collector runs in a different thread and therefore QuantLib objects are shared among different threads. This creates problems with QuantLib’s implementation of the observer pattern and is discussed in detail here. […]

  6. a good artical. I took a different approach on this matter, I simply disable the observer pattern in quantlib and now no crash when gc.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s