SCENIC: A JAX Library for Computer Vision Research and Beyond

By Mostafa Dehghani, Alexey A. Gritsenko, Anurag Arnab, Matthias Minderer, Yi Tay. 2021

78 citations recorded by the source corpus.

Scenic is an open-source11https://github.com/google-research/scenic JAX library with a focus on transformer-based models for computer vision research and beyond. The goal of this toolkit is to facilitate rapid experimentation, prototyping, and research of new architectures and models. Scenic supports a diverse range of tasks (e.g., classification, segmentation, detection) and facilitates working on multi-modal problems, along with GPU/TPU support for large-scale, multi-host and multi-device training. Scenic also offers optimized implementations of state-of-the-art research models spanning a wide range of modalities. Scenic has been successfully used for numerous projects and published papers and continues serving as the library of choice for rapid prototyping and publication of new research ideas.

Subfields: Computer Vision and Pattern Recognition, ML Systems & Hardware

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Computer Vision and Pattern Recognition

In ML Systems & Hardware

Citations in the corpus

The Efficiency Misnomer (2021, background): “Model (Dehghani et al., 2021a) when running on 64 TPU-V3. with bigger patch sizes have less FLOPs and are faster, while having more parameters, due to the larger patch embedding module.”
ViViT: A Video Vision Transformer (2021, methodology): “We implement our method using the Scenic library [15] and have released our code and models.”
Simple Open-Vocabulary Object Detection with Vision Transformers (2022, methodology): “We implement our model using JAX [5] and the Scenic library [8].”
Scaling Open-Vocabulary Object Detection (2023, unknown): “'themselves', 'then', 'there', 'these', 'they', 'this', 'those', 'through',”
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning (2023, methodology): “[19] Mostafa Dehghani, Alexey Gritsenko, Anurag Arnab, Matthias Minderer, and Yi Tay.”
TokenLearner: What Can 8 Learned Tokens Do for Images and Videos? (2021, methodology): “We use the Vision Transformer architecture [12], following its detailed settings and implementation [18].”
Scaling Vision Transformers to 22 Billion Parameters (2023, methodology): “Software: JAX (Bradbury et al., 2018), Flax (Heek et al., 2020), Scenic (Dehghani et al., 2022).”
Attention Bottlenecks for Multimodal Fusion (2021, methodology): “5 epochs on TPU accelerators using the Scenic library [15].”