SAMPLE DATA AND SAMPLE CODE FOR THE ONE-SHOT-LEARNING CHALEARN GESTURE CHALLENGE November 1, 2011 Copyright (c) CHALEARN, 2011 http://gesture.chalearn.org -- events@chalearn.org -o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o- DISCLAIMER: ALL INFORMATION, SOFTWARE, DOCUMENTATION, AND DATA ARE PROVIDED "AS-IS" BY CHALEARN, A CALIFORNIA NON-FOR-PROFIT. THE CONTRIBUTORS DISCLAIM ANY EXPRESSED OR IMPLIED WARRANTIES INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR ANY PARTICULAR PURPOSE, AND THE WARRANTY OF NON-INFRIGEMENT OF ANY THIRD PARTY'S INTELLECTUAL PROPERTY RIGHTS. IN NO EVENT SHALL ISABELLE GUYON AND/OR OTHER CONTRIBUTORS BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF DATA, SOFTWARE, DOCUMENTS,MATERIALS, PUBLICATIONS, OR INFORMATION MADE AVAILABLE FOR THE CHALLENGE. -o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o- Permission is granted to the challenge participants to use the data for the one-shot-learning gesture challenge. Redistribution of the data in its original or modified form must be accompanied by this notice. The data may be use for research purpose only. Permission to use the data for commercial purpose must be requested at events@chalearn.org. If you want to cite the data, please use: "ChaLearn Gesture Dataset (CGD2011), ChaLearn, California, 2011" -o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o- DATA: This directory should contain a subdirectory Data. If it is empty, see the README.txt file to get the data. The data are organized in subdirectories "typeXX", called "baches", where type is either "devel" of "valid" and XX is a number from 01 to 20. The prefix "devel" means that the data may be used for development purpose: we provide all truth labels for such data. The prefix "valid" means that the data may be used for validation: we provide only the training labels. The prediction results may be submitted to the website of the challenge to get feed-back on performance. Each data "batch" includes the recordings of 100 gestures as RGB videos (M_x.avi) and depth videos (K_x.avi) recorded with a Kinect (TM) camera.There are 47 pairs of {M_x.avi, K_x.avi} movies, including sequences of 1 to 5 gestures. The gestures in one batch are performed by a single user. They are instances of N unique gestures from a vocabulary of 8 to 15 gestures. The training labels are in a comma separated file called typeXX_train.csv and the test labels in typeXX_test.csv, with the row ID in the first column and the labels separated by spaces in the scond column. There is a single labeled training example of each gestures of the vocabulary of a given batch. The goal of the challenge is, for each batch, to train a system on the training examples, and to make predictions of the labels for the test examples. VIDEO FORMAT: The videos are formatted is a quasi-lossless compressed AVI format. The compression was performed with FFMPEG http://ffmpeg.org/, using: ffmpeg -i original.avi -sameq compressed.avi Compressed file can be read with one of many readers available for free, including SMplayer http://smplayer.org/. From Matlab, you may use mmreader or mmread. The Kinect depth images were produced after a normalization f(x)=(x-mini)/(maxi-mini), where mini is the minimum distance to the camera and maxi the maximum distance to the camera for an entire batch, The table of normalization constants is given at the end of this document, SUBMISSION FORMAT: The predictions of the missing test labels must be returned in the same csv format as the labels are provided, all concatenated in a single file. For final testing, new batches will be provided to registered participants shortly before the end of the challenge. ADDITIONAL DEVELOPMENT DATA: Additional development data are available from the website of the challenge http://gesture.chalearn.org. Hard copies can be requested at events@chalearn.org. -o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o- SAMPLE MATLAB (R) CODE: The directory Sample_code contains Matlab code to browse through the data, run experiments, and format the results for submission. The main point of entry is main.m. You must edit the top to make sure the code finds the functions and data needed. From the Sample_code directory, type at the Matlab prompt: > main; To simply browse through the data, use: > browse; Some of the most useful objects and functions: @databatch: An object to load a data batch. Usage: D=databatch(dirname); browser('D'); @recog_template: A very simple example of recognizer based on template matching. Usage: [tr_resu, mymodel] = train(recog_template, Dtr); te_resu = test(mymodel, Dte); show(mymodel); tr_resu and te_resu are @result objects. @result: An object to store results. Usage: save(te_resu, dirname); -o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o-|-o- TABLE OF NORMALIZATION CONSTANTS mini: Minimum distance to the camera maxi: Maximum distance to the camera resol: Number of unique depth values acc: Accuracy with which the depth is known: (1) Good. (2) Very good. Depending on the version of our recording software. set num mini maxi resol acc devel 1 801 1964 76 2 devel 2 801 2056 74 1 devel 3 801 1568 96 1 devel 4 801 1922 101 1 devel 5 824 1964 91 2 devel 6 832 1931 96 2 devel 7 928 3115 145 2 devel 8 801 1483 100 2 devel 9 801 3115 117 2 devel 10 895 1458 147 2 devel 11 914 1975 60 2 devel 12 801 1848 110 2 devel 13 801 1607 131 2 devel 14 801 1466 65 1 devel 15 921 1645 119 2 devel 16 834 1728 94 2 devel 17 801 1549 145 2 devel 18 801 2531 112 1 devel 19 801 1835 94 1 devel 20 809 2505 155 2 valid 1 801 2174 166 2 valid 2 801 3059 116 2 valid 3 801 1998 85 2 valid 4 879 2259 77 2 valid 5 888 2580 152 2 valid 6 1809 3975 59 2 valid 7 801 1998 88 2 valid 8 801 2147 192 2 valid 9 805 1711 77 2 valid 10 870 1745 54 2 valid 11 855 2033 149 2 valid 12 801 1563 157 2 valid 13 801 2174 153 2 valid 14 803 1868 115 2 valid 15 822 1719 83 2 valid 16 836 2082 77 2 valid 17 959 1931 89 2 valid 18 916 1711 79 2 valid 19 801 1502 116 2 valid 20 813 1909 61 2