Crazy debugging stories - Quota

This story is from about three years ago, at a previous job. I was involved in doing a lift-and-shift migration from on-prem to Google Cloud of a bespoke system we had built for a large insurance company.

The application was a very complicated and interesting distributed system, designed by some former colleagues, that I maintained at the time. It was used to implement part of a custom model for Solvency II compliance, i.e. the EU rules that govern how much money insurances need to keep aside as a liquidity buffer, created after the 2008 financial crisis by the European regulators.

The system was basically a big Montecarlo simulator - it would estimate the value of financial portfolios against a large number of scenarios (hundreds of thousands), do a lot of statistics and very complex aggregation rules, and produce a ton of reports. It processed a pretty big amount of data - a typical run would use some 200 CPUs and 1.5 TB of RAM for roughly twenty minutes to generate dozens of gigabytes of reports if I remember correctly. Most of the CPU and memory would be spent on the aggregation phase, though, which contained a lot of business logic.

One of the key steps was to evaluate the performances of the various portfolios in the stochastic scenarios, i.e. generate the Montecarlo data. This was implemented in a Java microservices that would execute the following steps:

Leave a Comment