Multi-Sensor Visual Tracking System for At-Risk Children


The purpose of this project was to develop a system to monitor a classroom of per-kindergarten children and track behaviors that are indicative of developmental disorders such as autism.  To do this, multiple Microsoft Kinect sensors were set up around an open floor plan, pre-kindergarten classroom.  There was a lot of overlap in the views, which did cause interference in the sensors, but children moving through the scene could be extracted quite cleanly in each of the sensors.

I worked with grad students Josh Fasching and Nick Walczak and primarily helped determine the cause and severity of depth inaccuracies with the Kinect sensors.  I also helped improve the calibration process used when setting up the system.

The overall project worked out pretty well, with collaborators from Computer Science to Psychology to Childhood Development getting a lot out of it.


  • R. Sivalingam, A. Cherian, J. Fasching, N. Walczak, N. Bird, V. Morellas, B. Murphy, K. Cullen, K. Lim, G. Sapiro, and N. Papanikolopoulos, “A Multi-Sensor Visual Tracking System for Behavior Monitoring of At-Risk Children,” Proceedings of the IEEE International Conference on Robotics and Automation, May 2012.
  • N. Walczak, J. Fasching, W. Toczyski, R. Sivalingam, N. Bird, K. Cullen, V. Morellas, B. Murphy, G. Sapiro, and N. Papanikolopoulos, “A Nonintrusive System for Behavioral Analysis of Children Using Multiple RGB+Depth Sensors,” Proceedings of the Workshop on Applications in Computer Vision, January 2012.