Sub-URMP for Cross-Modal Audio-Visual Generation Authors: Lele Chen (lchen63@cs.rochester.edu, primary contact for the dataset), Sudhanshu Srivastava (ssrivas6@cs.rochester.edu), Zhiyao Duan (zhiyao.duan@rochester.edu) and Chenliang Xu (chenliang.xu@rochester.edu) ##################################### DESCRIPTION This dataset is a subset of the University of Rochester Music Performance (URMP) dataset. Here is a link to the original URMP dataset: http://www.ece.rochester.edu/projects/air/projects/datasetproject.html. The Sub-URMP contains images and audios cut from the URMP. We use a sliding window method to obtain the samples. The size of the sliding window is 0.5 seconds and the stride is 0.1 seconds. We use the first frame of each video chunk to represent the visual content of the sliding window. The audio files are in WAVE format with a sampling rate of 44 KHz and bit depth of 16 bits, stereo channel. The image files are 1080P (1080x1920). We list the file numbers for both training and validation data. Training data (In total: 71230 paired samples): {'viola': 6530, 'oboe': 4505, 'bassoon': 1735, 'flute': 5690, 'tuba': 3285, 'horn': 5540, 'sax': 7615, 'double_bass': 1270, 'cello': 9800, 'trombone': 8690, 'violin': 7430, 'clarinet': 8125, 'trumpet': 1015} validation data (In total: 9575 paired samples): {'viola': 485, 'oboe': 390, 'bassoon': 390, 'flute': 925, 'tuba': 525, 'horn': 525, 'sax': 910, 'double_bass': 1180, 'cello': 1030, 'trombone': 805, 'violin': 945, 'clarinet': 945, 'trumpet': 520} The filename consists by three parts. First part is instrument name, second part is the index of songs, and the third part is the time information. Example: tuba04_34100.wav: 'tuba' is the instrument category, '04' means it belongs to No.4 songs play in tuba. '34100' means that the time duration of this small chunk is 34.1 seconds to 34.6 seconds. The corrresponding image file (tuba04_34100.jpg) should be sampled around time 34.1 seconds of the original video. ###################################### CITATION If you use this dataset, please cite the following papers: 1) L. Chen, S. Srivastava, Z. Duan and C. Xu. Deep Cross-Modal Audio-Visual Generation. In Proc. of ACM International Conference on Multimedia Thematic Workshops, 2017. 2) B. Li, X. Liu, K. Dinesh, Z. Duan and G. Sharma. Creating A Musical Performance Dataset for Multimodal Music Analysis: Challenges, Insights, and Applications. arXiv:1612.08727, 2016. (The original URMP dataset paper.)