AudeoSynth: Music-Driven Video Montage

to appear at SIGGRAPH 2015

Zicheng Liao
 

Yizhou Yu*
 

Bingchen Gong
 

Lechao Cheng
 
Zhejiang University

*The University of Hong Kong


Accompanying videos of the paper. Copyrighted contents. Do not use for commercial purpose.


Abstract

We propose music-driven video montage, a media format that offers a pleasant way to browse or summarize video clips collected from various occasions, including gatherings and adventures. In musicdriven video montage, the music drives the composition of the video content. According to musical movement and beats, video clips are organized to form a montage that "visually reflects the experiential properties of the music". Nonetheless, it takes enormous manual work and artistic expertise to create it. In this paper, we develop a framework for automatically generating music-driven video montages. The input is a set of video clips and a piece of background music. By analyzing the music and video content, our system extracts carefully designed temporal features from the input, and casts the synthesis problem as an optimization and solves the parameters through Markov Chain Monte Carlo sampling. The output is a video montage whose visual activities are cut and synchronized with the rhythm of the music, rendering a symphony of audio-visual resonance.
    






Paper


Citation
@article{liao:montage2015,
    title={audeosynth: Music-Driven Video Montage},
    author={Liao, Zicheng and Yu, Yizhou and Gong, Bingchen and Cheng, Lechao},
    journal = {ACM Trans. Graph. (SIGGRAPH) },
    volume = {34},
    number = {4},
    year = {2015},
}