What's next?

Once that's done I am going to create a converter in order to facilitate the retrieval phase and I probably will use CGAL library [1] besides the OpenSG [2] (due to LTouchIT project) to achieve this challenge. Note that in the future LTouchIT will use Unity3D [3] game engine, one of the most relevant tools on today's market.

I'll also start writing the first phase of the thesis next week that consists of both related work (research) and a definition of what is the problem that I'll solve.

[1] http://www.cgal.org/
[2] http://www.opensg.org/
[3] http://unity3d.com/

Reconstruction

To better understand the whole reconstruction process and particularly the capture phase I have read these articles:

Integrated High-Quality Acquisition of Geometry and Appearance for Cultural Heritage
C. Schwartz, M. Weinmann, R. Ruiters and R. Klein
University of Bonn, Germany
The Eurographics Association 2011.

Procedural 3D Building Reconstruction using Shape Grammars and Detectors
Mathias, Markus and Martinovic, Andelo and Weissenberg, Julien and Gool, Luc J. Van
3DIMPVT IEEE (2011) , p. 304-311

The Digital Michelangelo Project: 3D Scanning of Large Statues
Levoy, M., K. Pulli, B. Curless, S. Rusinkiewics, D. Koller, L. Pereira, M. Ginzton, S. Anderson, J. Davis, J. Gensberg, J. Shade, and D. Fulk. (2000).
SIGGRAPH 2000. pp. 131-144.

Protected interactive 3D graphics via remote rendering
David Koller, Michael Turitzin, Marc Levoy, Marco Tarini, Giuseppe Croccia, Paolo Cignoni, Roberto Scopigno
ACM Trans. Graph., Vol. 23, No. 3. (Aug 2004), pp. 695-703

In collaboration with Diogo Henriques (my collegue responsible for the LTouchIT OR project), I explored a part of the LTouchIT project and created a list of 200 important pieces that we will use in the retrieving phase. Using LDraw library we corrected all of these bricks (new .dat files are generated).

About the retrieval phase I also searched MDS-CM-BOF ("Non-rigid 3d shape retrieval using multidimensional scaling and bag-of-features") and some papers related related to the following one.

Non-rigid 3d Shape Retrieval using multidimensional scaling and Bag-of-Features

Z.Lian,A.Godil,X.Sun,H.Zhang,,

International Conference on Image Processing (ICIP2010),2010,pp.3181–3184.

3D Segmentation Algorithms

This paper is about segmentation and compares seven algorithms (K-means, random walks, fitting primitives, Normalized Cuts, Randomized Cuts, Core Extraction and Shape Diameter Function) with human segmentation. Some important metrics are also presented to evaluate these algorithms.

A Benchmark for 3D Mesh Segmentation
Xiaobai Chen, Aleksey Golovinskiy, Thomas Funkhouser
ACM Transactions on Graphics (Proc. SIGGRAPH), August 2009.

What follows is learning how to use LDraw library and initiate the retrieval process, using some LEGO bricks. Although the algorithms presented doesn't consider color information, I'm going to use it in my work.

Features & Segmentation

I am reading some articles about segmentation (how to combine both color and depth information) and feature extraction (knowing more about SIFT, SURF, NARF, etc). Although the last reference is out of scope, it gives some ideas about the use of Microsoft Kinect, Features and how to do feature matching.

Color image segmentation in RGB using vector angle and absolute difference measures
Sanmati S. Kamath and Joel R. Jackson
[14th European Signal Processing Conference (EUSIPCO 2006)]

Sparse Distance Learning for Object Recognition Combining RGB and Depth Information
Kevin Lai, Liefeng Bo, Xiaofeng Ren, and Dieter Fox

Building a 3D map from RGB-D sensors
Virgile Hogman

More Research

I have read some articles about descriptors and how to use depth and color segmentation together, which are going to be an important phase of my work.

Shape Distributions on Voxel Surfaces for 3D Object Classification From Depth Images 

Walter Wohlkinger, Markus Vincze 
Vision4Robotics Group, Automation and Control Institute, Vienna University of Technology, Austria
[2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA2011)]

3DNet: Large-Scale Object Class Recognition from CAD Models 
Walter Wohlkinger and Aitor Aldoma and Radu B. Rusu and Markus Vincze
[2012 IEEE International Conference on Robotics and Automation]

Object Detection using the Kinect
Jason Owens,Vehicle Technology Directorate, ARL

Due to configuration process, I still have problems in using PCL. Probably I will spend more time on achieving this next week.

Latest Research

Recognition and Pose Estimation of Rigid Transparent Objects with a Kinect Sensor
Ilya Lysenkov, Victor Eruhimov and Gary Bradski ,Robotics: Science and Systems Conference (RSS), 2012.

Humanising GrabCut: Learning to segment humans using the Kinect
V. Gulshan, V. Lempitsky and A. Zisserman at the Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV), ICCV 2011.

Benchmarking CAD Search Techniques
Dmitriy Bespalov, Cheuk Yiu Ip, William C. Regli, _ Joshua Shaffer, Geometric and Intelligent Computing Laboratory, 2005

Exploiting Segmentation for Robust 3D object Matching
Michael Krainin, Kurt Konolige, Dieter Fox, University of Washington (2011?)

Recovering Missing Depth Information from Microsoft’s Kinect
Abdul Dakkak, Ammar Husain (>2010)

3D is here: Point Cloud Library (PCL)
Radu Bogdan Rusu and Steve Cousins
- some tutoriais are being performed

One related film