Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling

🤖 Plain-English Summary

We present Point-BERT, a new paradigm for learning Transformers to generalize the concept of BERT to 3D point cloud. We also demonstrate that the representations learned by Point-BERT transfer well to new tasks and domains, where our models largely advance the advanced of few-shot point cloud classification task.

🔑 Key Findings

• Inspired by BERT, we devise a Masked Point Modeling (MPM) task to pre-train point cloud Transformers.
• Specifically, we first divide a point cloud into several local point patches, and a point cloud Tokenizer with a discrete Variational AutoEncoder (dVAE) is designed to generate discrete point tokens containing meaningful local information.
• Then, we randomly mask out some patches of input point clouds and feed them into the backbone Transformers.

🔗 Related Resources