About project

The client’s data platform’s solution stack is on Kubernetes and consists of:
— Apache Kafka (collecting order and deal events, data from metering points, etc. with Kafka);
— Apache NiFi (for integration with external systems);
— Apache Flink (Flink jobs for tagging, calculating positions, weighted average prices, etc. out of allocation data, load forecasts, order and deal event streams; using Apache Beam pipelines using Python, designed for parallel, distributed data processing);
— Apache Druid (OLAP, supporting real-time dashboards, optimized for time series aggregations);
— Apache Superset (for real-time dashboards as it integrates nicely with Druid);
— Kubeflow for forecasting pipelines, machine learning, Jupyter notebooks.

Main programming language: Python.

Three main pillars:
— (Near) real-time hedge book/sales book dashboards for portfolio managers;
— Forecasting (short, mid, and long term; for initial pricing, before delivery, etc.);
— Asset monitoring & control (wind turbines, etc., grid balancing).

IT-service provider is active in the Dutch/European Energy Sector. The client`s goal is to help energy companies maximize control over their energy portfolio and increase value across the entire energy supply chain.
To achieve this, the client delivers an energy management platform. Flexible and scalable process automation and big data processing software application that covers the entire energy supply chain and can be fully integrated with the company’s digital infrastructure for optimal business-IT alignment.