Summary

RL Carbon is based entirely on Rosetta’s deep reinforcement learning (DRL) model. Deep reinforcement learning is a type of machine learning that produces actions, not predictions, that are most likely to maximize (or minimize) a metric over an infinite time horizon. Beginning simply with data (the model is not programmed), the model undertakes a series of random actions. Through a process of trial and error, it receives feedback from its environment, and over time it discovers sequences of actions that provide the best results. This iterative process makes DRL particularly good at solving dynamic optimization problems like allocating risk capital, which is why we chose to use DRL as the investment process for our RL Carbon strategy. In our case, our DRL model seeks to maximize the cumulative risk-adjusted return on capital invested EUA emission futures. Starting only with data and accounting for transaction costs, the model continually adjusts its behavior to learn the optimal daily allocation of risk capital to ICE EUA futures contracts to best achieve this return. These actions, i.e., investment decisions, are expressed as a single value (or signal) ranging from 100% long to 100% short. For example, a signal of 56% would require a 56% allocation of risk capital to long exposure to the EUAs and a 44% allocation to cash. We implement these actions by taking long or short positions in the front-year December EUA futures contracts.