Description

In this notebook, I create a Deep Reinforcement Learning algorithm that operates on the inputs provided by the competition. This notebook is designed to be simple enough for users without RL experience to follow and create their own Deep RL agents. I handle the complicated logic behind the scenes, while you only need to choose parameters and, optionally, pre-process the datasets so the agent receives inputs in a better form. For more experienced users, I also explain how to create custom policies (including neural network architectures) to improve upon the simple default architecture.

High-level overview: RL bypasses the need for forecasting entirely by using policies that directly map a state to an ordering action. The state includes all information necessary for decision-making (inventory levels, past demand, side information like product category, etc.). In the current implementation, we use a neural network (hence "Deep") as the policy class. To train the neural network weights, we simulate the current policy using the dataset and update the weights using gradient-based optimization.

Since the codebase is quite large, I have put everything in a repo instead of an isolated notebook. The repo can be found at https://github.com/MatiasAlvo/vn2

Getting started:

• To clone the repo and install dependencies, follow the instructions in the README.md file
• Open main_run.ipynb for a simplified tutorial on creating and training your own Deep RL agent
• To create custom datasets (e.g., pre-processed versions of the sales data), use vn2_data_analysis.ipynb, where I also explain the input data format for the neural network policies
• For more detailed usage information, see the README.md file

I believe there is significant room for improvement through better data preprocessing and more powerful neural network architectures, so feel free to experiment!

If you find any bugs in my code, please leave a comment and I will try to address it!