[ file av16.3_v6/EXAMPLES/3D-RECONSTRUCTION/README ]

This directory gives the Matlab code necessary to achieve
3D reconstruction of the mouth location, starting from,
for a given person in a given sequence:

- "continuous" 2D ball location annotation on all three cameras: 
  "continuous" means that most frames were annotated, for example
  using the Ball Annotation Interface (BAI) and its
  integrated tracker.

- "sparse" 2D mouth location annotation on all three cameras:
 "sparse"  means that some frames were annotated manually, 
  for example using the Mouth Annotation Interface (MAI).
  Half a second (12 video frames) is a reasonable interval.

IMPORTANT NOTE: in the case of mouth location annotation, care must be
taken to annotate frames with same TIMECODE on all three cameras
(frame index is likely to be different).

Guillaume Lathoud ( lathoud@idiap.ch )

--------------------------------------------------
Below are MATLAB lines detailing how the various steps are implemented.
Note that all functions need the "seq_av163.m" functions,
available in the base directory of this corpus: 
http://mmm.idiap.ch/Lathoud/av16.3_v6/seq_av163.m.

% Start with for example these six files sitting in the "DATA" directory:
%
% DATA/seq11-1p-0100-cam1-person1.ballgt
% DATA/seq11-1p-0100-cam2-person1.ballgt
% DATA/seq11-1p-0100-cam3-person1.ballgt
%
% DATA/seq11-1p-0100-cam1-person1.mouthgt
% DATA/seq11-1p-0100-cam2-person1.mouthgt
% DATA/seq11-1p-0100-cam3-person1.mouthgt




% ( 1 ) 3D reconstruction for the continuous ball measurements
% Output files:
%
% DATA/seq11-1p-0100-person1.3dballgt

do_3dballgt( 'DATA' );




% ( 2 ) 3D reconstruction for the sparse mouth measurements
% Output file:
%
% DATA/seq11-1p-0100-person1.3dmouthgt

do_3dmouthgt( 'DATA' );




% ( 3 ) 3D interpolation of the mouth location,
% using the 3D ball location as origin of the referent.
% Output files:
%
% DATA/seq11-1p-0100-person1-interpolated.3dmouthgt
% DATA/seq11-1p-0100-cam1-person1-interpolated-reprojected.mouthgt
% DATA/seq11-1p-0100-cam2-person1-interpolated-reprojected.mouthgt
% DATA/seq11-1p-0100-cam3-person1-interpolated-reprojected.mouthgt
%
% You can look at the last three files to see where additional
% manual measurements of the 2D mouth location are necessary.
% If the reprojection is too much off the mouth,
% add some more measurements to the original
% "*.mouthgt" files and go to ( 2 ).

interpolate_3dmouthgt( 'DATA' );