Web-based Object Category Learning using Human-Robot Interaction Cues

We present our method for learning object categories from the internet using cues obtained through human-robot interaction. Such cues include an object model acquired by observation and the name of the object. Our learning approach emulates the natural learning process of children when they observe their environment, encounter unknown objects and ask adults the name of the object. Using this learning approach, our robot is able to discover objects in a domestic environment by observing when humans naturally move objects as part of their daily activities. Using speech interface, the robot directly asks humans the name of the object by showing an example of the acquired model. The name in text format and the previously learnt model serve as input parameters to retrieve object category images from a search engine, select similar object images, and build a classifi er.

 

    

Demo Videos

Related Publications

Christian I. Penaloza, Yasushi Mae, Kenichi Ohara, and Tatsuo Arai: "Web-Enhanced Object Category Learning from an Object Model", ACM/IEEE International Conference on Human-Robot Interaction (HRI) , pp.223, Laussanne Switzerland. March 6-9, 2011.

Generic Object Classifiers based on Real Image Selection from the Web

We present our semi-supervised technique for building object category classifiers using real image data from the Internet. Our technique not only reduces the overhead of manual training by humans, but also achieves robust classifiers that can be evaluated in real time. Given a sample object and its name (keyword), we collect a large amount of objectrelated images from two main image sources: Google Images and the LabelMe website. We deal with the problem of separating good training samples from noisy images by performing two steps: similar image selection and non-real image filtering. We use a variant of Gaussian Discriminant Analysis (GDA) to filter out non-real images (drawings, cartoons, etc.) that tentatively affect classifier performance in real environments. In order to select true object-related training samples, we introduce a Simile Selector Classifier (SSC) that is constructed from a small set of images taken from the sample object. The SSC not only is able to select similar samples from the large unordered set of images, but also it can separate desired object category images from other categories that have the same name (polysemes), i.e. ”apple” as a fruit, or as a company logo. Finally, the experiments which we performed in real environments demonstrate the performance of our object classifiers.

 

 

Demo Videos

Related Publications

Christian I. Penaloza, Yasushi Mae, Kenichi Ohara, Tomohito Takubo and Tatsuo Arai: "Web-Enhanced Object Category Learning from an Object Model", Proceedings of the 17th Robotics Symposium, Yamaguchi, Japan, March 14-15, 2012.

Christian I. Penaloza, Yasushi Mae, Kenichi Ohara, Tomohito Takubo and Tatsuo Arai: "Generic Object Classifiers based on Real Image Selection from the Web", The First Asian Conference on Pattern Recognition, pp.239-243, Beijing, China, Nov 28-30, 2011.