Chenliang Xu
LIBSVX: A Supervoxel Library and Benchmark
Primary Contributors: Chenliang Xu and Jason J. Corso
Overview: Images have many pixels; videos have more. Despite the strong potential of supervoxels to enhance video analysis and the successful usage of superpixel in many aspects of image understanding, supervoxels have yet to become mainstream in video understanding research. Two likely causes for this are (1) the lack of an available implementation for many supervoxels methods and (2) the lack of solid evaluation results or a benchmark on which to base the choice of one supervoxel method over another. In this project, we overcome both of these limitations: LIBSVX is a library of supervoxel and video segmentation methods coupled with a principled evaluation benchmark based on quantitative 3D criteria for good supervoxels.
What's new in LIBSVX 4.0?
This release corresponds to our IJCV paper with new evaluation metrics.
What's new in LIBSVX 3.0?
We include the method in our ICCV 2013 paper for flattening supervoxel hierarchies by the uniform entropy slice (UES). UES flattens a supervoxel hierarchy into a single segmentation such that it overcomes the limitations of trivially selecting an arbitrary level. It selects supervoxels in the hierarchy that balances the relative level of information in the final segmentation based on various post hoc feature criteria, such as motion, object-ness or human-ness. Three challenging video examples (with two actors in each) used in our ICCV 2013 paper are also included in this release. We also include bug-fixes in gbh, gbh_stream and swa.
What's new in LIBSVX 2.0?
The use of video segmentation as an early processing step in video analysis lags behind the use of image segmentation for image analysis, despite many available video segmentation methods. A major reason for this lag is simply that videos are an order of magnitude bigger than images; yet most methods require all voxels in the video to be loaded into memory, which is clearly prohibitive for even medium length videos. We address this limitation by proposing an approximation framework for streaming hierarchical video segmentation motivated by data stream algorithms: each video frame is processed only once and does not change the segmentation of previous frames. We implement the graph-based hierarchical segmentation method within our streaming framework; our method is the first streaming hierarchical video segmentation method proposed. This is the code we used in support of our ECCV 2012 paper.
Download code: LIBSVX v4.0 Beta
Download datasets in our format:
BuffaloXiph.tar.bz2 (290MB);
SegTrackv2 (519MB);
BVDS (3.2GB);
CamVid (3.1GB);
CeLiuFlow (1.2GB);
MiddleburyFlow (27MB).
Download our results:
BuffaloXiph.tar.bz2 (1.5GB);
SegTrackv2 (3.4GB);
BVDS (19GB);
CamVid (20GB);
CeLiuFlow (462MB);
MiddleburyFlow (65MB).
Archive Versions: