ETL

ETL for Daily Liquidity Monitoring

Daily monitoring of liquidity has become a crucial job inside any bank. We implemented the Extract-Load-Transform (ETL) operations of liquidity tools for different trading desks for the modelling and risk-reporting departments in a large Dutch bank, which allows the bank to run its liquidity-monitoring tools daily.

Liquidity input data come from various sources and all have different formats. Some are Excel files, while some others are comma-separated text files. Moreover, date conventions are not standard and depends on external factors, such as Excel settings. In addition, all the different pieces of information have to be adjusted before they can be used. Such adjustments include specific selection and join operations.

Our input tool has been developed in C#. We have created in-memory databases and used LINQ to perform DB operations. The code has been unit-tested. Log text files and Excel output files are created daily. The tool has been accompanied by a self-contained user manual explaining the business logic, configuration setting and command-line arguments, exceptions that may arise during the execution.