Automated Medical Coding

🚀 Project Overview

This project is mainly used for a series of experimental operations, including data preparation, model training, result generation, etc. It uses Hydra for configuration management and conducts experiments through Weights and Biases Sweeps.

🚀 Quick Start

Prerequisites

• You need a server configured with 128 GB of memory.
• Ensure that the relevant dependencies of Hydra and Weights and Biases are installed.

Run Experiments

The code for running experiments is located in the src directory. You can run the corresponding experimental code according to the configuration in the configs/experiments directory.

⚠️ Important Note

• The training of LAAT and PLM-ICD may be unstable. The loss function sometimes diverges during training. This problem seems to stem from the overflow of the softmax function in label-level attention. Adding batch normalization or layer normalization before the softmax function may solve this problem. Since we didn't want to change the original method in the reproducibility study, we didn't attempt to fix this issue.
• The code has only been tested on a server configured with 128 GB of memory. One user reported that they couldn't load the MIMIC-IV dataset into memory when using 32 GB of memory.
• There is an error in the collate function in the Huggingface dataset. The attention mask is padded with 1 instead of 0. Since I want people to be able to reproduce the results in the paper, I haven't fixed this problem yet.

✨ Features

Configuration Management

We use Hydra for configuration management. Each experiment's configuration is located in the configs/experiments directory. Additionally, configuration files for hyperparameter search can be found in configs/sweeps. We mainly use Weights and Biases Sweeps for experiments.

Directory Structure

• files: This directory stores images and data.
• notebooks: This directory contains only one notebook for code analysis. Although it's not designed for others, it contains content for others to verify the data analysis results.
• prepare_data: This directory contains all the data processing code for preparing datasets and generating splits.
• reports: Here is the code for generating tables and charts in the paper. This code uses the Weights and Biases API to obtain experimental results. It's included for others to verify our graphs and tables, although it can't be directly used by other users.
• src: This directory stores the code for running experiments.
• tests: This directory contains unit test files.

📚 Documentation

My Configuration

I ran these experiments on a machine configured with 128 GB of memory and an RTX 2080 Ti (11GB per GPU card).