ProteinBERT: a universal deep-learning model of protein sequence and function

🤖 Plain-English Summary

SUMMARY: Self-supervised deep language modeling has shown unprecedented success across natural language tasks, and has recently been repurposed to biological sequences. AVAILABILITY AND IMPLEMENTATION: Code and pretrained model weights are available at https://github.com/nadavbra/protein_bert.

🔑 Key Findings

• However, existing models and pretraining methods are designed and optimized for text analysis.
• We introduce ProteinBERT, a deep language model specifically designed for proteins.
• Our pretraining scheme combines language modeling with a novel task of Gene Ontology (GO) annotation prediction.

🔗 Related Resources