Enhanced Planar Pattern Tracking for an Outdoor Augmented Reality System

Authors: L. Yu, W. K. Li, S. K. Ong, A. Y. C. Nee

Abstract:

In this paper, a scalable augmented reality framework for handheld devices is presented. The presented framework is enabled by using a server-client data communication structure, in which the search for tracking targets among a database of images is performed on the server-side while pixel-wise 3D tracking is performed on the client-side, which, in this case, is a handheld mobile device. Image search on the server-side adopts a residual-enhanced image descriptors representation that gives the framework a scalability property. The tracking algorithm on the client-side is based on a gravity-aligned feature descriptor which takes the advantage of a sensor-equipped mobile device and an optimized intensity-based image alignment approach that ensures the accuracy of 3D tracking. Automatic content streaming is achieved by using a key-frame selection algorithm, client working phase monitoring and standardized rules for content communication between the server and client. The recognition accuracy test performed on a standard dataset shows that the method adopted in the presented framework outperforms the Bag-of-Words (BoW) method that has been used in some of the previous systems. Experimental test conducted on a set of video sequences indicated the real-time performance of the tracking system with a frame rate at 15-30 frames per second. The presented framework is exposed to be functional in practical situations with a demonstration application on a campus walk-around.

Keywords: Augmented reality framework, server-client model, vision-based tracking, image search.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1339832

References:

[1] S. Feiner, B. Macintype, T. Hollerer and T. Webster “A Touring Machine: Prototyping 3D Mobile Augmented Reality Systems for Exploring the Urban Environment,” In Proc. International Symposium on Wearable Computers, Cambridge, Massachusetts, 13-14 Oct 1997.
[2] T. Guan, Y. He, L. Duan, J. Gao, J. Yang and J. Yu, “Efficient Bag-of-Features Generation and Compression for On-Device Mobile Visual Location Recognition, ” IEEE MultiMedia, 21(2):32-41, 2013.
[3] D. Wagner, G. Reitmayr, A. Mulloni, T. Drummond and D. Schmalstieg, “Real-time detection and tracking for augmented reality on mobile phone,” IEEE Transaction on Visualization and Computer Graphics 16(3):355-368, 2010.
[4] D. Schmalstieg, T. Langlotz and M. Billinghurst, “Augmented Reality 2.0. Virtual Realities,” Vienna: Springer-Verlag/Wien, pp 13-37, 2011.
[5] D. Nister, H. Stewenius, “Scalable Recognition with a Vocabulary Tree,” In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 2, New York, USA, 17-22 June, 2006, pp. 2161-2168.
[6] V. Lepetit, P. Fua, “Keypoint recognition using randomized trees,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(9):1465-1479, 2006.
[7] S. Gammeter, A. Gassmann, L. Bossard, T. Quack and L. Van Gool “Server-side object recognition and client-side object tracking for mobile augmented reality,” In Proc. of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, San Francisco, CA, 13-18 June, 2010, pp. 1-8.
[8] G. Takacs, V. Chandrasekhar, N. Gelfand, Y. Xiong, W. Chen, T. Bismpigiannis, R. Grzeszczuk, K. Pulli and B. Girod, “Outdoors augmented reality on mobile phone using Loxel-based visual feature organization,” In Proc. of 1st International ACM Conference on Multimedia Information Retrieval, Vancouver, BC, Canada, 30-31 August, 2008, pp. 427-434.
[9] H. Jaewon, C. Kyusung, F. A. Rojas and H. S. Yang, “Real-time scalable recognition and tracking based on the server-client model for mobile Augmented Reality,” In Proc. of 2011 IEEE International Symposium on VR Innovation, Singapore, 19-20 March, 2011, pp. 267-272.
[10] H. Jegou, M. Douze, C. Schmid and P. Perez, “Aggregating local descriptors into a compact image representation,” In Proc. of IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, USA, 13-18 June, 2010, pp. 3304-3311.
[11] M. Muja and D. G. Lowe, “Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration,” In Proc. of International Conference on Computer Vision Theory and Application, Lisboa, Portugal, 5-8 Feb, 2009, pp. 331-340.
[12] W. T. Fong, L. Yu, S. K. Ong and A. Y. C. Nee, “Marker-less Computer Vision Tracking for Augmented Reality,” In Proc. of 25th Annual Conference on Computer Animation and Social Agents, Singapore, 9-11 May, 2012, pp. 46-49.
[13] L. Yu, S. K. Ong and A. Y. C. Nee, “Inertial Sensor-aided Feature Detection and Tracking for Outdoor Augmented Reality Applications on Mobile Handheld Devices,” In Proc. of Computer Graphics International, Hannover, Germany, 11-14 June, 2013, pp. 459-462.
[14] T. Langlotz, S. Mooslechner, S. Zollmann, C. Degendorfer, G. Reitmayr and D. Schmalstieg, “Sketching up the world: in situ authoring for mobile augmented reality.” Personal and Ubiquitous Computing, 2012, 16(6):623-630.
[15] B. MacIntyre, A. Hill, H. Rouzati, M. Gandy and B. Davidson, “The Argon AR Web Browser and standards-based AR application environment,” In Proc. of 10th IEEE International Symposium on Mixed and Augmented Reality, Basel, Switzerland, 26-29 Oct, 2011, pp. 65-74.
[16] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, 2004, 60(2):91-110.
[17] M. A. Fischler, R. C. Bolles, “Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,” 1981 Communication of the ACM 24(6):381-395.
[18] M. V. Calonder, V. Lepetit, C. Strecha and P. Fua, “BRIEF: Binary robust independent elementary features,” In Proc. of 11th European Conference on Computer Vision, Heraklion, Greece, 5-11 Sep, 2010, pp. 778-792.
[19] E. Rosten, T. Drummond, “Machine learning for high-speed corner detection,” In Proc. of European Conference on Computer Vision, Graz, Austria, 7-13 May, 2006, pp 430-443.
[20] S. Benhimane, E. Malis, “Homography-based 2D Visual Tracking and Servoing,” International Journal of Robotic Research, 2007, 26(7):661-676.
[21] V. Chandrasekhar, D. Chen, S. Tsai, N. M. Cheung, H. Chen, G. Takacs, Y. Reznik, R. Vedantham, R. Grzezczuk, J. Bach and B. Girod, “The Stanford mobile visual search dataset,” In Proc. of the second annual ACM conference on Multimedia Systems, San Jose, CA, 23-25 February, 2011, pp. 117-122.
[22] http://www.robots.ox.ac.uk/~vgg/research/affine. Accessed 9 Jan, 2015.