Foldcomp

Foldcomp compresses protein structures with torsion angles effectively. It compresses the backbone atoms to 8 bytes and the side chain to additionally 4-5 byes per residue, thus an averaged-sized protein of 350 residues requires ~6kb.

Foldcomp efficient compressed format stores protein structures requiring only 13 bytes per residue, which reduces the required storage space by an order of magnitude compared to saving 3D coordinates directly. We achieve this reduction by encoding the torsion angles of the backbone as well as the side-chain angles in a compact binary file format (FCZ).

> Foldcomp currently only supports compression of single chain PDB files

Publications

Hyunbin Kim, Milot Mirdita, Martin Steinegger, Foldcomp: a library and format for compressing and indexing large protein structure sets, Bioinformatics, 2023;, btad153,

Usage

Installing Foldcomp

# Install Foldcomp Python package
pip install foldcomp

# Download static binaries for Linux
wget https://mmseqs.com/foldcomp/foldcomp-linux-x86_64.tar.gz

# Download static binaries for Linux (ARM64)
wget https://mmseqs.com/foldcomp/foldcomp-linux-arm64.tar.gz

# Download binary for macOS
wget https://mmseqs.com/foldcomp/foldcomp-macos-universal.tar.gz

# Download binary for Windows (x64)
wget https://mmseqs.com/foldcomp/foldcomp-windows-x64.zip

Executable

# Compression
foldcomp compress  []
foldcomp compress [-t number]  []

# Decompression
foldcomp decompress  []
foldcomp decompress [-t number]  []

# Decompressing a subset of Foldcomp database
foldcomp decompress [-t number] --id-list   []

# Extraction of sequence or pLDDT
foldcomp extract [--plddt|--amino-acid]  []
foldcomp extract [--plddt|--amino-acid] [-t number]  []

# Check
foldcomp check 
foldcomp check [-t number] 

# RMSD
foldcomp rmsd  

# Options
 -h, --help           print this help message
 -v, --version        print version
 -t, --threads        threads for (de)compression of folders/tar files [default=1]
 -r, --recursive      recursively look for files in directory [default=0]
 -f, --file           input is a list of files [default=0]
 -a, --alt            use alternative atom order [default=false]
 -b, --break          interval size to save absolute atom coordinates [default=25]
 -z, --tar            save as tar file [default=false]
 -d, --db             save as database [default=false]
 -y, --overwrite      overwrite existing files [default=false]
 -l, --id-list        a file of id list to be processed (only for database input)
 --skip-discontinuous skip PDB with with discontinuous residues (only batch compression)
 --check              check FCZ before and skip entries with error (only for batch decompression)
 --plddt              extract pLDDT score (only for extraction mode)
 --fasta              extract amino acid sequence (only for extraction mode)
 --no-merge           do not merge output files (only for extraction mode)
 --time               measure time for compression/decompression

Downloading Databases

We offer prebuilt databases for multiple large sets of predicted protein structures and a Python helper to download the database files.

You can download the AlphaFoldDB Swiss-Prot with the following command:

python -c "import foldcomp; foldcomp.setup('afdb_swissprot_v4');

Currently we offer the following databases:

• ESMAtlas full (v0 + v2023_02): foldcomp.setup('esmatlas')

• ESMAtlas v2023_02: foldcomp.setup('esmatlas_v2023_02')

• ESMAtlas high-quality: foldcomp.setup('highquality_clust30')

  Note: We skipped all structures with discontinous residues or other issues. Here is a list with the affected predictions; full (~21M), high-quality (~100k), v2023_02 (~10k)

• AlphaFoldDB Uniprot: foldcomp.setup('afdb_uniprot_v4')

• AlphaFoldDB Swiss-Prot: foldcomp.setup('afdb_swissprot_v4')

• AlphaFoldDB Model Organisms: foldcomp.setup('h_sapiens')

  • a_thaliana, c_albicans, c_elegans, d_discoideum, d_melanogaster, d_rerio, e_coli, g_max, h_sapiens, m_jannaschii, m_musculus, o_sativa, r_norvegicus, s_cerevisiae, s_pombe, z_mays

• AlphaFoldDB Cluster Representatives: foldcomp.setup('afdb_rep_v4')

• AlphaFoldDB Cluster Representatives (Dark Clusters): foldcomp.setup('afdb_rep_dark_v4')

If you want other prebuilt datasets, please get in touch with us through our GitHub issues.

If you have issues downloading the databases you can navigate directly to our download server and download the required files. E.g. afdb_uniprot_v4, afdb_uniprot_v4.index, afdb_uniprot_v4.dbtype, afdb_uniprot_v4.lookup, and optionally afdb_uniprot_v4.source.

Python API

You can find more in-depth examples of using Foldcomp's Python interface in the example notebook:

import foldcomp
# 01. Handling a FCZ file
# Open a fcz file
with open("test/compressed.fcz", "rb") as fcz:
  fcz_binary = fcz.read()

  # Decompress
  (name, pdb) = foldcomp.decompress(fcz_binary) # pdb_out[0]: file name, pdb_out[1]: pdb binary string

  # Save to a pdb file
  with open(name, "w") as pdb_file:
    pdb_file.write(pdb)

  # Get data as dictionary
  data_dict = foldcomp.get_data(fcz_binary) # foldcomp.get_data(pdb) also works
  # Keys: phi, psi, omega, torsion_angles, residues, bond_angles, coordinates
  data_dict["phi"] # phi angles (C-N-CA-C)
  data_dict["psi"] # psi angles (N-CA-C-N)
  data_dict["omega"] # omega angles (CA-C-N-CA)
  data_dict["torsion_angles"] # torsion angles of the backbone as list (phi + psi + omega)
  data_dict["bond_angles"] # bond angles of the backbone as list
  data_dict["residues"] # amino acid residues as string
  data_dict["coordinates"] # coordinates of the backbone as list

# 02. Iterate over a database of FCZ files
# Open a foldcomp database
ids = ["d1asha_", "d1it2a_"]
with foldcomp.open("test/example_db", ids=ids) as db:
  # Iterate through database
  for (name, pdb) in db:
      # save entries as seperate pdb files
      with open(name + ".pdb", "w") as pdb_file:
        pdb_file.write(pdb)

Subsetting Databases

If you are dealing with millions of entries, we recommend using createsubdb command of mmseqs2 to subset databases. The following commands can be used to subset the AlphaFold Uniprot DB with given IDs.

# mmseqs createsubdb --subdb-mode 0 --id-mode 1 id_list.txt input_foldcomp_db output_foldcomp_db
mmseqs createsubdb --subdb-mode 0 --id-mode 1 id_list.txt afdb_uniprot_v4 afdb_subset

Please note that the IDs in afdb_uniprot_v4 are in the format AF-A0A5S3Y9Q7-F1-model_v4 .

Community Contributions

• PyMOL Plugin for reading Foldcomp files by @yakomaxa

Contributor