Skip to content

seq2loc protein embeddings dataset

Do you need protein language model embeddings for human proteins, and would rather not spend a week of GPU time producing them yourself? They are already computed and sitting in our lab storage. This document explains what is here, which of the two flavours you probably want, and the handful of traps that will silently ruin your analysis if you don't know about them.

Where this comes from

These embeddings were produced for the seq2loc line of work: predicting the subcellular localization of a human protein from its amino acid sequence alone. The published benchmark is:

Zoe Wefers, Ankit Gupta, Noorsher Ahmed, Xikun Zhang, Emma Lundberg. A comprehensive benchmark of sequence-based subcellular localization predictors for human proteins. Nature Methods 23(7):1458-1469, 2026. doi:10.1038/s41592-026-03142-6 (PMCID PMC13345906, open access) Code: [ https://github.com/CellProfiling/seq2loc_benchmark ]

That paper integrates localization annotations from UniProt/SwissProt, the Human Protein Atlas and OpenCell into a three-level label hierarchy (level 1 being the most fine grained), builds a high confidence test set of 3,814 human proteins where two of the three databases independently agree, and then retrains and re-evaluates the published predictors on it. Train/test splits were made sequence-similarity aware with mmseqs2: proteins sharing at least 40% identity over at least 80% of the longer sequence are kept on the same side of the split.

Note that the models benchmarked in the paper are ESM2, ESM3-small-open, ProtT5 and ProtBert — not ESM-C. Everything in this dataset comes from ESM-C (ESMC-6B and ESMC-600M), so it is an extension of the published work rather than the exact material behind the paper's numbers. Don't cite the paper as the source of these particular embeddings.

The dataset holds two fundamentally different things, and the distinction matters more than anything else in this document:

  • ESMC_mean_pooled: one vector per protein per layer. 44 GB. Start here.
  • ESMC_per_residue: one vector per amino acid per layer. 5.1 TB. Only if you really need residue resolution.

You can always derive the mean pooled version from the per residue one. You can never go the other way, which is why both are kept.

ESMC_mean_pooled

The file you want is ESMC_mean_pooled/embeddings/esmc_6b_all_layers.h5 (8.2 GB): ESMC-6B embeddings for 21,600 human proteins across all 80 transformer layers, mean pooled over residues.

  • One HDF5 dataset per protein, each of shape (80, 2560), dtype float16.
  • Pooling is the mean over residue tokens only; the <cls> and <eos> special tokens are stripped. Sequences were truncated at 2,046 residues.
  • Keys come from two sources: 20,162 HPA proteins keyed by Ensembl gene ID (ENSG00000141510) and 1,438 UniProt-only proteins keyed by UniProt accession (Q8IZP2). Filter on the ENSG prefix if you only want the HPA-linked set.
  • ensembl_uniprot_id_mapping.tsv in the same folder converts between the two ID spaces and tells you which proteins have a real embedding. Use it instead of re-deriving the mapping yourself.
  • The folder has its own documentation, written by Konstantin Kahnert, and it is more detailed than this file: README.md is the practical guide, README_KK.md is the full derivation and validation write-up (including the precision study summarised below and the provenance of every recovered sequence), and esmc_6b_all_layers_meta.json is the same information machine readable. Read README.md before using the embeddings.
import h5py, numpy as np

with h5py.File("esmc_6b_all_layers.h5", "r") as f:
    emb = f["ENSG00000141510"][:].astype(np.float32)   # (80, 2560)
    layer_60 = emb[59]                                 # the paper's "layer 60" -- see below

Four things to get right

  1. Layer indexing is off by one from the papers. The stored layers are transformer block outputs only; the token embedding layer was dropped. So stored index 59 is what a paper would call "layer 60".
  2. 168 of the 21,600 proteins are all-zero placeholders, where the sequence could not be resolved or failed validation. They have the same shape and dtype as real entries, so nothing will error — they will just quietly poison any average or clustering you compute. Filter first: valid = [k for k in f if np.any(f[k][:])]. (Of the 795 HPA genes originally missing a sequence, 627 were recovered through a SwissProt / TrEMBL / Ensembl cascade; the per gene provenance is in recovered_provenance.CORRECTED.json.)
  3. Cast to float32 before doing any maths. Storage is float16 to save space, and most numeric libraries handle fp16 badly on CPU. Cast per protein or per batch, not the whole file at once.
  4. Layer 79 is not like the other 79. Layers 0-78 are raw block outputs; layer 79 is the output after the model's final LayerNorm, which is a different kind of quantity (cosine ~0.915 against the raw value, norm ratio ~176x). If you sweep across layers looking for the best one, treat 79 as a special case — a surprising result there is probably this, not biology.

On precision: inference ran in bfloat16 (the pooling reduction itself in float32). This was cross-checked against an independent float32 run over 20 proteins x 30 layers — layers 0-78 agree at cosine >= 0.99997, with mean relative L2 error growing gently from 0.16% to 0.58% with depth and no correlation with protein length. That noise is roughly 200 times smaller than the natural between-protein cosine spread (0.13-0.93), so it is negligible for similarity and clustering work. It could matter if you are separating near-duplicate proteins or mixing precisions between training and evaluation.

The rest of ESMC_mean_pooled

These are working materials from the same project rather than general purpose embeddings:

  • features (858 MB): derived feature matrices (human_features*.h5, bench_features*.h5).
  • ppi (2.9 GB, ~1,000 parquet shards): protein-protein interaction priors and high confidence edge lists.
  • external (245 MB): third party PPI sources — STRING v12, BioPlex 3, hu.MAP 2.
  • npmi (85 MB): normalized pointwise mutual information tables, parquet shards.
  • released (699 MB): release artifacts — the Zhang benchmark, mmseqs search databases and hits, PPI3D interface data (ppi3d_fields.md documents the fields) and SAE normalization statistics for layer 60.
  • model_weights/esmc_6b (24 GB): the ESMC-6B checkpoint itself, so runs are reproducible without re-downloading it.
  • embeddings/validation: the sequence validation audit trail behind the 168/627 numbers above.

ESMC_per_residue

Full per residue embeddings: one vector per amino acid, for 31,887 proteins, keyed by UniProt accession (note: not Ensembl, unlike the pooled file). 5.1 TB in total.

Only a subset of layers was kept — the deep ones, which are the useful ones for localization:

folder model layers dim dtype size
by_layer/ESMC6B-layer<L>/ ESMC-6B 60-79 (20 layers) 2560 float32 3.5 TB (~177 GB per layer)
esmc600m/ ESMC-600M 16-35 (20 layers) 1152 float32 1.6 TB

Layers 0-59 of the 6B model were deleted in September 2026 on the author's instruction. If you need a shallow layer at residue resolution, it no longer exists here and would have to be recomputed — but remember the pooled file still covers all 80 layers.

Layout and how to read it

Each layer is a self-contained folder: a small master .h5 plus the 64 shards that actually hold the data. The master is an HDF5 file of external links pointing at the shards by bare relative filename, so you can copy or rename the folder and it still resolves. Keep the .part*.h5 files next to the master — the master holds no data itself.

import h5py
with h5py.File("by_layer/ESMC6B-layer74/ESMC6B-layer74-4k.h5", "r") as f:
    emb = f["P04637"][:]          # (seq_len, 2560) float32

For ESMC-600M the masters sit directly in esmc600m/ with the shards under esmc600m/esmc600m_shards/, same idea:

with h5py.File("esmc600m/ESMC600M-layer35-4k.h5", "r") as f:
    emb = f["P04637"][:]          # (seq_len, 1152) float32

To copy a single layer elsewhere, take the whole folder: rsync -aP by_layer/ESMC6B-layer74/ /destination/ESMC6B-layer74/

by_layer/README.txt documents this in more detail, including the verification that was done before the duplicate copy was removed.

Joining the two flavours

They are keyed differently and they are not the same protein set, so joins need care:

  • pooled: 21,600 keys, Ensembl gene IDs for HPA proteins, UniProt accessions for the other 1,438.
  • per residue: 31,887 keys, UniProt accessions throughout.

Use ESMC_mean_pooled/embeddings/ensembl_uniprot_id_mapping.tsv to bridge them. The layer numbering is consistent between the two (both sit one below a paper's "layer N"), so layer74 per residue corresponds to stored index 74 in the pooled file — but remember caveat 4 above: pooled layer 79 is post-LayerNorm while per residue layer 79 is the raw block output. They are not the same quantity.

State of the data

Checked 2026-09-23:

  • ESMC-6B per residue: all 20 layer folders (60-79) present, each with its master and all 64 shards. 31,887 keys per layer; masters open and resolve their external links.
  • ESMC-600M per residue: all 20 layers (16-35) present, 1,280 shards (20 x 64), no layer short of its 64. Masters open and resolve.
  • Pooled: 21,600 keys as documented, 20,162 Ensembl-keyed and 1,438 UniProt-keyed, shape (80, 2560) float16.

Two things to be aware of:

  • Some of the older in-folder documentation still refers to the previous location of this data (/deep/datasets/HPA/Subcellular/newEmbeddings and newESMC-embeddings), which is where it lived before being moved into this dataset. Affected: ESMC_mean_pooled/embeddings/esmc_6b_all_layers_meta.json, ESMC_mean_pooled/embeddings/README_KK.md and ESMC_per_residue/by_layer/README.txt. The data is fine; only the paths are stale.
  • by_layer/README.txt documents the repo's EmbeddingDataset loader with label levels of 22, 10 and 8 classes, whereas the published paper reports 21, 9 and 7. The difference is probably an extra "other"/unannotated bucket in the code, but it has not been confirmed — check against the repo before you rely on either number.

Additional resources

  • seq2loc_benchmark [ https://github.com/CellProfiling/seq2loc_benchmark ]: the code for building the datasets and training/evaluating the localization models.
  • ESM-C — the protein language model family these embeddings come from; the ESMC-6B checkpoint used here is kept under ESMC_mean_pooled/model_weights/esmc_6b.
  • ESMC_mean_pooled/embeddings/README.md: the practical guide to the pooled embeddings, by their author. The first thing to read if you are going to use them.
  • ESMC_mean_pooled/embeddings/README_KK.md: the deep dive on the same — derivation, sequence recovery cascade and the full precision validation.
  • ESMC_mean_pooled/released/ppi3d_fields.md: field definitions for the PPI3D interface data.
  • ESMC_per_residue/by_layer/README.txt: the deep dive on the per residue layout.