Clinical NLP Transformers (cnlp_transformers)

Transformers for Clinical NLP

This library was created to add abstractions on top of the Huggingface Transformers library for many clinical NLP research use cases. Primary use cases include

1. simplifying multiple tasks related to fine-tuning of transformers for building models for clinical NLP research, and
2. creating inference APIs that will allow downstream researchers easier access to clinical NLP outputs.

This library is not intended to serve as a place for clinical NLP applications to live. If you build something cool that uses transformer models that take advantage of our model definitions, the best practice is probably to rely on it as a library rather than treating it as your workspace. This library is also not intended as a deployment-ready tool for scalable clinical NLP. There is a lot of interest in developing methods and tools that are smaller and can process millions of records, and this library can potentially be used for research along those line. But it will probably never be extremely optimized or shrink-wrapped for applications. However, there should be plenty of examples and useful code for people who are interested in that type of deployment.

Install

Note: due to some dependency issues, this package does not officially support macOS on Apple Silicon. If you want to install it on Apple Silicon, you are on your own; we unofficially recommend trying it with Python 3.10, or using the docker CPU image

Note: When installing the library's dependencies, pip will probably install PyTorch with CUDA 10.2 support by default. If you would like to run the library in CPU-only mode or with a newer version of CUDA, install PyTorch to your desired specifications in your virtual environment first before installing cnlp-transformers.

Static installation

If you are installing just to fine-tune or run the REST APIs, you can install without cloning:

$ # Note: if needed, install PyTorch first (see above)
$ pip install cnlp-transformers

If you prefer, prebuilt Docker images are also available to run the REST APIs in a network. An example Docker Compose configuration is also available for reference.

Editable installation

If you want to modify code (e.g., for developing new models), then install locally:

1. Clone this repository:

   # Either the HTTPS method...
   $ git clone https://github.com/Machine-Learning-for-Medical-Language/cnlp_transformers.git
   # ...or the SSH method
   $ git clone git@github.com:Machine-Learning-for-Medical-Language/cnlp_transformers.git
   

2. Enter the repo: cd cnlp_transformers

3. Install the development dependencies:

   $ pip install -r dev-requirements.txt
   

4. See above for the note about PyTorch; if needed, manually install it now.

5. Install cnlp-transformers in editable mode:

   $ pip install -e .
   

Fine-tuning

The main entry point for fine-tuning is the cnlp_transformers/src/cnlpt/train_system.py script. Run with no arguments to show an extensive list of options that are allowed, inheriting from and extending the Huggingface training options.

Workflow

To use the library for fine-tuning, you'll need to take the following steps:

1. Write your dataset to one of the following formats in a folder with train, dev, and test files:
2. csv or tsv: The first row should have column names separated by comma or tab. The name text has special meaning as the input string. Likewise if there are columns named text_a and text_b it will be interpreted as two parts of a transformer input string separated by a -token equivalent. All other columns are treated as potential targets -- their names can be passed to the train_system.py script as --task_name arguments. For tagging targets, the field must consist of space-delimited labels, one per space-delimited token in the text field. For relation extraction targets, the field must be a , delimited list of relation tuples, where each relation tuple is (, ,label), where offset 1 and 2 are token indices into the space-delimited tokens in the text field.
3. json: The file format must be the following:

  { 'data': [
      { 'text':