QuantLib-1.5 SWIG Patch for JVM/.NET

Update 23.11.2015: A modified version of the patch is now part of the official QuantLib Release 1.7.

Update 22.09.2015: Please find the latest and improved version of the patch for QuantLib 1.6.2 here.

The usage of QuantLib in JVM and .NET languages (e.g. Java/Scala and C#/F#) via the SWIG interface has a known shortcoming. The implementation of QuantLib’s observer pattern does not tolerate a parallel garbage collector running in a different thread. As a result programs are randomly crashing or producing “pure virtual function calls”. A detailed description of this problem can be found e.g. here and within the references.

Please find here a patch for QuantLib 1.5 to fix this issue.

Advertisement

QuantLib-SWIG Patch for JVM/.NET Languages

Update 23.11.2015: The latest version is now part of the official QuantLib Release 1.7.

Update 22.09.2015: Please find the latest and improved version of the patch for QuantLib 1.6.2 here.

The usage of QuantLib in JVM and .NET languages (e.g. Java/Scala and C#/F#) via the SWIG interface has a known shortcoming. The implementation of QuantLib’s observer pattern does not tolerate a parallel garbage collector running in a different thread. As a result programs are randomly crashing or producing “pure virtual function calls”. A detailed description of this problem can be found e.g. here and within the references.

Please find here a patch for QuantLib 1.4 to fix this issue. It contains

Installation instructions are included in the readme.txt file.

Multi-Threading and QuantLib

Update 23.11.2015: The latest version is now part of the official QuantLib Release 1.7.

Update 22.09.2015: Please find the latest version of the patch for QuantLib 1.6.2 here.

Update 28.02.2015: Please find the latest version of the patch for QuantLib 1.5 here.

Update 11.05.2014: Please find the latest version of the patch for QuantLib 1.4 here.

QuantLib is per se not thread-safe. The standard way to utilise more than one core is to spawn several independent processes. Riccardo’s thread-safe singleton patch allows to use QuantLib within multi-threading applications as long as objects aren’t explicitly shared between different threads. In fact this patch turns the singleton pattern into a thread local singleton pattern. One possible use case of this patch is to run the test-suite with a multi-threading test runner to speed it up, e.g. on a i7@3.2Ghz with four cores plus four HT cores the multi-threading test-suite runs in approx. two minutes whereas the single threaded version takes around eight minutes.

Using QuantLib in Java/Scala/C# or F# applications via the SWIG layer violates the multi-threading requirement because the garbage collector runs in a different thread and therefore QuantLib objects are shared among different threads. This creates problems with QuantLib’s implementation of the observer pattern and is discussed in detail here.

An improved implementation of the observer pattern based on boost::signals2 together with Riccardo’s thread-safe singleton patch and the multi-threading test runner can obtained from github. Under Linux/MacOS use

./configure --enable-thread-safe-observer-pattern

to enable the thread-safe singleton and the thread-safe observer pattern. Under Windows the corresponding preprocessor directives

#define QL_ENABLE_THREAD_SAFE_OBSERVER_PATTERN

are already set in the file userconfig.hpp.  In order to enable the boost shared_ptr hook change the preprocessor variable BOOST_SP_ENABLE_DEBUG_HOOKS towards BOOST_SP_ENABLE_DEBUG_HOOKS_2 in the file

boost/smart_ptr/detail/sp_counted_impl.hpp

Background: the original preprocessor variable BOOST_SP_ENABLE_DEBUG_HOOKS changes the memory layout of the class boost::shared_ptr which might lead to problems with other pre-compiled libraries which also use boost::shared_ptr.

The reward for this work is a stable SWIG interface for Java/Scala/C#/F# and a thread local singleton implementation, which allows to use QuantLib within multi-threading applications as long as SWIG/QuantLib objects aren’t explicitly shared among different threads.

QuantLib-SWIG and a Thread-Safe Observer Pattern in C++

Update 23.11.2015: A modified version of the patch is now part of the official QuantLib Release 1.7.

Update 28.02.2015: Please find the latest version of the patch for QuantLib 1.5 here.

Update 11.05.2014: Please find the latest version of the patch for QuantLib 1.4 here.

The QuantLib library is not really thread-safe. Successful usage in multi-core and parallel environments is often achieved by message passing between multiple processes rather than shared memory and multi-threading. If different threads are acting on distinct objects then QuantLib can also be used in the multi-threading environment (define QL_ENABLE_SESSIONS to make the singletons thread dependent.).

If you are using QuantLib in Java or Scala via SWIG [1] then the QuantLib routines are automatically executed in a multi-threading environment even if your main routine is single threaded. The garbage collector of the JVM is usually running in different thread. In conjunction with QuantLib’s implementation of the Observer pattern this can lead to serious problems, e.g. the following single threaded Scala code crashes on a multi-core computer after a short period of time.

import org.quantlib.{Array => QArray, _}
object ObserverTest {
    def main(args: Array[String]) : Unit = {
        System.loadLibrary("QuantLibJNI");
        val aSimpleQuote = new SimpleQuote(0)

        while (true) {
            (0 until 10).foreach(_ => {
                new QuoteHandle(aSimpleQuote)
                aSimpleQuote.setValue(aSimpleQuote.value + 1)
            })
            System.gc
        }
    }
}

The garbage collector might call the destructor of an observer (QuoteHandle) at the same point in time when the update method on the same object is invoked by aSimpleQuote.setValue. (Find here the original postings on the QuantLib mailing list regarding this problem.). For a greenfield project the author in [2] has a nice solution for this problem in C++. Unfortunately this solution can not be applied to the QuantLib because the solution involves a change of signature of the Observer interface and thereafter would lead to a lot of change in the QuantLib library.

The main problem here is that we have to disable the Observer before the destructor starts to work. This could in theory be achieved by adding a special “Deleter” to every new instance of a boost::shared_ptr. But again this would lead to a lot of changes in the library.

When compiled with the preprocessor directive BOOST_SP_ENABLE_DEBUG_HOOKS the boost library adds a callback hook before a destructor is called by any boost smart pointer. This hook can be utilized to disable an Observer before the actual destructor is executed. In addition the boost::signals2 library [3] provides a simple and thread-safe notification mechanism.

The corresponding thread-safe implementation of the original QuantLib Observer/Observable interface can be found here. Set the preprocessor directive BOOST_SP_ENABLE_DEBUG_HOOKS for every source file. Replace/Add the files observable.hpp and observable.cpp and recompile the QuantLib library and the QuantLib-SWIG module.

Note: if you want to use the patch with QL 1.2 and not for the trunk, please see the comment by gk below.

[1] Simplified Wrapper and Interface Generator, SWIG

[2] Shuo Chen, Where Destructors meet Threads

[3] Douglas G., Mori Hess F., Boost.Signals2

Using Scala for Payoff Scripting

The advantages of payoff scripting based on a build-in interpreter or “on-the-fly compiler” instead of implementing the payoffs in C++ are obvious. Faster time-to-market because there is no need to recompile and deploy the C++ pricing library and people without deep C++ knowledge are able to develop and test new structured products. One disadvantage is often the execution speed of the chosen scripting language. Examples of languages I have seen/used for payoff scripting are Python  (C++ interface boost::python), Lua, tinycc and CINT. When it comes to execution speed none of these are suited to build high performance solutions, see. e.g. [1].  This is especially true if the Monte-Carlo scenario generator is running on a GPU. The payoff scripting on the CPU can then easily become the bottleneck of your pricing library.

Scala is a modern programming language that integrates object-oriented and functional language features. The Scala compiler generates byte code for the Java VM. Therefore the execution speed of a Scala script is comparable with Java and roughly a factor of two slower than C++ [1].

The Scala compiler itself is a Scala object and can be used at runtime to compile and link new scripts or classes. In addition using JNI it is fairly easy to attach a Java VM to a C++ process and to exchange data between C++ and Scala. Also Scala offers a lot of features to design an “internal”, user-friendly domain specific language (DSL) for payoff scripting.

The code for a small QuantLib/Scala Monte-Carlo simulation in action is available here. It depends on QuantLib 1.0 or higher, a Java 1.6 VM and Scala 2.8/9. Overwrite the PayoffImpl.scala class to implement different payoffs without recompiling the C++ code.

[1] Computer Language Benchmark Game