Frozen in time: A joint video and image encoder for end-to-end retrieval

Our objective in this work is video-text retrieval – in particular a joint embedding that enables efficient text-to-video retrieval. We also provide a new video-text pretraining dataset WebVid-2M, comprised of over two million videos with weak captions scraped from the internet.

⚡ This is an original paraphrased summary — not copied from the abstract. Full paper available at the source link below.

• 1 The challenges in this area include the design of the visual architecture and the nature of the training data, in that the available large scale video-text training datasets, such as HowTo100M, are noisy and hence competitive performance is achieved only at scale through large amounts of compute.We address both these challenges in this paper.
• 2 We propose an end-to-end trainable model that is designed to take advantage of both large-scale image and video captioning datasets.
• 3 Our model is an adaptation and extension of the recent ViT and Timesformer architectures, and consists of attention in both space and time.

This research advances how AI systems learn, reason, and solve problems — with direct implications for automation and scientific discovery.

This summary is based on publicly available metadata and abstract. For the full research paper, visit the original source: