TouchCut: Fast image and video segmentation using single-touch interaction

By Tinghuai Wang, Bo Han, John Philip Collomosse. 2014

94 citations recorded by the source corpus.

We present TouchCut; a robust and efficient algorithm for segmenting image and video sequences with minimal user interaction. Our algorithm requires only a single finger touch to identify the object of interest in the image or first frame of video. Our approach is based on a level set framework, with an appearance model fusing edge, region texture and geometric information sampled local to the touched point. We first present our image segmentation solution, then extend this framework to progressive (per-frame) video segmentation, encouraging temporal coherence by incorporating motion estimation and a shape prior learned from previous frames. This new approach to visual object cut-out provides a practical solution for image and video segmentation on compact touch screen devices, facilitating spatially localized media manipulation. We describe such a case study, enabling users to selectively stylize video objects to create a hand-painted effect. We demonstrate the advantages of TouchCut by quantitatively comparing against the state of the art both in terms of accuracy, and run-time performance.

Subfields: Cognitive Neuroscience, Computer Vision and Pattern Recognition

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Cognitive Neuroscience

In Computer Vision and Pattern Recognition

Citations in the corpus

COCO-Stuff: Thing and Stuff Classes in Context (2016, methodology): “Several works use interactive segmentation methods [42, 57, 10] to speedup annotation; others annotate superpixels [61, 22, 40].”
Extreme Clicking for Efficient Object Annotation (2017, methodology): “To reduce annotation time many interactive segmentation techniques have been proposed, which require the user to input either a bounding box around the object [42, 51, 71], or scribbles [3, 16, 24, 26, 27, 42, 50, 64, 65, 72], or clicks [31, 69].”
What's the Point: Semantic Segmentation with Point Supervision (2015, background): “These annotations can be collected as points [39] or free-form squiggles [30].”
Learning Video Object Segmentation from Static Images (2016, background): “Thus several works have also considered video segmentation with variable annotation effort, enabling human interaction using clicks [22, 47, 51] or strokes [1, 16, 57].”
Kubric: A scalable dataset generator (2022, methodology): “Classical methods involve active-contour [47, 69] or graphcut [20, 104] techniques, but there also exist techniques with human-in-the-loop [72, 81, 100], and more recently deep learning variants [30, 73, 74, 105, 117].”
Interactive Full Image Segmentation by Considering All Regions Jointly (2018, background): “Many works address semantic segmentation by training from weakly labeled data, such as image-level labels [28, 44, 58], point-clicks [5, 6, 13, 57], boxes [26, 37, 41] and scribbles [33, 59].”
A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation (2016, methodology): “Supervised approaches assume manual annotation to be repeatedly added during the segmentation process, with a human correcting the algorithm results in an iterative fashion [2, 14, 49, 53].”
Lucid Data Dreaming for Object Tracking (2017, background): “Interactive segmentation [34] [22] [48] [59] uses more flexible user inputs (e.g. strokes), and requires interactive processing speed rather than providing maximal quality.”
Y Qualification Main task Y Return N N Y User qualified ? N Feedback Pass ? Pass ? Annotator training Annotating images (2017, methodology): “To reduce annotation time many interactive segmentation techniques have been proposed, which require the user to input either a bounding box around the object [42, 51, 71], or scribbles [3, 16, 24, 26, 27, 42, 50, 64, 65, 72], or clicks [31, 69].”
Click Carving: Segmenting Objects in Video with Point Clicks (2016, methodology): “It outperforms all similarly fast methods, and is competitive or better than those requiring 2 to 12 times the effort.”
Counting in the Wild (2016, methodology): “Dots have also been used in the context of interactive segmentation [18, 20] with cues such as background annotations, which are easy to provide in an interactive context.”