Image BERT Pre-training with Online Tokenizer

By Jinghao Zhou, Chen Wei, Huiyu Wang, Wei Shen, Cihang Xie, Alan L. Yuille, Tao Kong. International Conference on Learning Representations, 2022

296 citations recorded by the source corpus.

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

A foundation model for clinical-grade computational pathology and rare cancers detection (2024, unknown): “Phikon was trained using the iBOT algorithm 23 , and both Virchow and UNI were trained using the DINO v.2 algorithm 33 with similar hyperparameters. iBOT and DINO v.2 are related approaches as the latter builds on the masked image modeling proposal of the former.”
Should You Mask 15% in Masked Language Modeling? (2022, background): “Recently, a number of works extend MLM training to images and videos and demonstrate strong pre-training results (He et al., 2022; Zhou et al., 2022; Feichtenhofer et al., 2022; Tong et al., 2022) .”
BEiT v2: Masked Image Modeling with Vector-Quantized Visual Tokenizers (2022, background): “, 2021), iBoT (Zhou et al., 2022), MaskFeat (Wei et al.”