Chenliang Xu

pic

Supervoxels: Toward an Efficient
Hierarchical Representation for Video Analysis


Overview of Goals and Challenges:

Images have many pixels; videos have more. It has thus become standard practice to first preprocess images and videos into more tractable sets by either extraction of salient points or oversegmentation into superpixels. Preprocessing these outputs is more perceptually meaningful than raw pixels, which are merely a consequence of digital sampling. However, the same practice does not entirely exist in video analysis. Although many methods do indeed initially extract salient points or dense trajectories, few methods rely on a supervoxel segmentation, which is the video analog to a superpixel segmentation. Here, the focuses of our research are on exploring the potential of supervoxel segmentations in advancing various video analysis tasks and on designing efficient supervoxel algorithms to better model space-time of video. We have created a supervoxel library and benchmark, i.e., LIBSVX, that is widely used in the early video analysis community. We have promoted the idea of streaming processing in video segmentation and developed the first streaming hierarchical supervoxel algorithm. The supervoxels generated by our algorithm are being used in various video analysis tasks, such as object tracking, video object segmentation, and action localization.

Related Publications:

  1. LIBSVX: A Supervoxel Library and Benchmark for Early Video Processing. C. Xu and J. J. Corso. International Journal of Computer Vision (IJCV), 2016.
  2. Flattening Supervoxel Hierarchies by the Uniform Entropy Slice. C. Xu, S. Whitt and J. J. Corso. IEEE/CVF International Conference on Computer Vision (ICCV), 2013.
  3. Streaming Hierarchical Video Segmentation. C. Xu, C. Xiong and J. J. Corso. European Conference on Computer Vision (ECCV), 2012.
  4. Evaluation of Super-Voxel Methods for Early Video Processing. C. Xu and J. J. Corso. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012.