Mobile Augmented Reality

Most of today's mobile internet devices contain facilities to search the user's immediate environment. The search results are then displayed on a map. Several research projects have aimed to provide more intuitive displays by using augmented reality to overlay points of interest on a video image of the real world.

Although this display is more intuitive, it introduces new challenges; how to display points of interest that are: 1) outside of the user's field of view, or 2) occluded by real-world objects? Below, you can see three examples of how we address these challenges. XrayVision and MeltVision enable users to explore occluded points of interest. DistortVision enables users to explore points of interest that are outside their field of view.

 XrayVision
  MeltVision
  DistortVision
Longer videos
Schematics
Our Augmented Reality XrayVision system creates an image-based reconstruction of a remote scene (b) based on captured images and models. The reconstruction is overlaid on the user's view of the environment (a) to provide x-ray vision (c). Note that the foreground edges are highlighted to support correct depth perception. Schematics for MeltVision before melting (d) and after melting (e). The user is interested to see object C. (e) illustrates the effects of melting. Schematics for DistortVision before distortion (f) and after distortion (g). The user is interested to see object D. (g) illustrates the effects of the distortion, where the model of (f) has been distorted to reveal D in the viewport.
X-Ray 1   (a)
Distort (a) (d) Distort (a)(f)
X-Ray 2(b) melt concept(e) distort scheme b(g)
xray 3(c)
Creating a textured 3D model of Adelaide

People

Publications

Copyright Disclaimer: This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each authors copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.

The colors next to the publications denote the lab in which they were created:
  Magic Vision Lab. University of South Australia. Adelaide, Australia.
  Wearable Computer Laboratory. University of South Australia. Adelaide, Australia.
  Canon's Leading-Edge Technology Research Headquarters. Human Machine Perception Laboratory. Tokyo, Japan.
  Fachgebiet Augmented Reality. Munich University of Technology. Munich, Germany.
  Fachgebiet Computer Aided Medical Procedures. Munich University of Technology. Munich, Germany.

Conferences

2014
Global Illumination for Augmented Reality on Mobile Phones Csongei, M., Hoang, L., Sandor, C., and Lee, Y.
Global Illumination for Augmented Reality on Mobile Phones
Accepted as Poster in Proceedings of the IEEE Virtual Reality Conference, pages 1--2, Minneapolis, MN, USA, March, 2014.
PDF | video | bib

2013
Depth Perception in Tablet-Based Augmented Reality at Medium- and Far-Field Distances Kuparinen, L., Swan, E., Rapson, S., and Sandor, C.
Depth Perception in Tablet-Based Augmented Reality at Medium- and Far-Field Distances
Poster in Proceedings of the ACM Symposium on Applied Perception, page 121, Dublin, Ireland, August, 2013.
PDF | bib

2012
Tablet versus Phone: Depth Perception in Handheld Augmented Reality Dey, A., Jarvis, G., Sandor, C., and Reitmayr, G.
Tablet versus Phone: Depth Perception in Handheld Augmented Reality
In Proceedings of IEEE International Symposium on Mixed and Augmented Reality, pages 187--196, Atlanta, USA, November, 2012. Best Paper Award Nominee
PDF | bib

Dynamic Eye Convergence for Head-mounted Displays Improves User Performance in Virtual Environments Sherstyuk, A., Dey, A., Sandor, C., and State, A.
Dynamic Eye Convergence for Head-mounted Displays Improves User Performance in Virtual Environments
In Proceedings of ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, pages 23--30, California, USA, March, 2012.
PDF | bib

Head-turning Approach to Eye-tracking in Immersive Virtual Environments Sherstyuk, A., Dey, A., and Sandor, C.
Head-turning Approach to Eye-tracking in Immersive Virtual Environments
In Proceedings of IEEE Virtual Reality Conference, pages 137--138, California, USA, March, 2012.
PDF | bib

2011
An Evaluation of Augmented Reality X-Ray Vision for Outdoor Navigation Dey, A., Jarvis, G., Sandor, C., Wibowo, A., and Mattila, V.
An Evaluation of Augmented Reality X-Ray Vision for Outdoor Navigation
In Proceedings of International Conference on Artificial Reality and Telexistence, pages 28--32, Osaka, Japan, November, 2011.
PDF | bib

2010
PTAMM-Plus: Refactoring and Extending PTAMM Nguyen, T., Sandor, C., and Park, J.
PTAMM-Plus: Refactoring and Extending PTAMM
In Proceedings of International Conference on Artificial Reality and Telexistence, pages 84--88, Adelaide, SA, Australia, December, 2010.
PDF | video | bib

Evaluating Depth Perception of Photorealistic Mixed Reality Visualizations for Occluded Objects in Outdoor Environments Dey, A., Cunningham, A., and Sandor, C.
Evaluating Depth Perception of Photorealistic Mixed Reality Visualizations for Occluded Objects in Outdoor Environments
In Proceedings of ACM Symposium on Virtual Reality Software and Technology, pages 211--218, Hong Kong, China, November, 2010.
PDF | video | bib

An Augmented Reality X-Ray System Based on Visual Saliency Sandor, C., Cunningham, A., Dey, A., and Mattila, V.
An Augmented Reality X-Ray System Based on Visual Saliency
In Proceedings of IEEE International Symposium on Mixed and Augmented Reality, pages 27--36, Seoul, Korea, October, 2010.
PDF | video | bib

Evaluating Depth Perception of Photorealistic Mixed Reality Visualizations for Occluded Objects in Outdoor Environments Dey, A., Cunningham, A., and Sandor, C.
Evaluating Depth Perception of Photorealistic Mixed Reality Visualizations for Occluded Objects in Outdoor Environments
Proceedings of the Fifth IEEE Symposium on 3D User Interfaces, pages 127--128, Waltham, MA, USA, March, 2010.
PDF | bib

TINT: Towards a Pure Python Augmented Reality Framework Eck, U., and Sandor, C.
TINT: Towards a Pure Python Augmented Reality Framework
Proceedings of the Third Workshop on Software Engineering and Architectures for Realtime Interactive Systems, pages 41--46, Waltham, MA, USA, March, 2010.
PDF | bib

Egocentric Space-Distorting Visualizations for Rapid Environment Exploration in Mobile Mixed Reality Sandor, C., Cunningham, A., Eck, U., Urquhart, D., Jarvis, G., Dey, A., Barbier, S., Marner, M., and Rhee, S.
Egocentric Space-Distorting Visualizations for Rapid Environment Exploration in Mobile Mixed Reality
Proceedings of the IEEE Conference on Virtual Reality, pages 47--50, Waltham, MA, USA, March, 2010.
PDF | video | bib

2009
Egocentric Space-Distorting Visualizations for Rapid Environment Exploration in Mobile Mixed Reality Sandor, C., Cunningham, A., Eck, U., Urquhart, D., Jarvis, G., Dey, A., Barbier, S., Marner, M., and Rhee, S.
Egocentric Space-Distorting Visualizations for Rapid Environment Exploration in Mobile Mixed Reality
Proceedings of IEEE and ACM International Symposium on Mixed and Augmented Reality, pages 211--212, Orlando, FL, USA, October, 2009.
PDF | video | bib

Improving Spatial Perception for Augmented Reality X-Ray Vision Avery, B., Sandor, C., and Thomas, B.
Improving Spatial Perception for Augmented Reality X-Ray Vision
VR '09: Proceedings of the IEEE Conference on Virtual Reality, pages 79--82, Lafayette, Louisiana, USA, March, 2009.
PDF | video | bib

2008
An Augmented Reality Weather System Heinrich, M., Thomas, B., Mueller, S., and Sandor, C.
An Augmented Reality Weather System
ACE '08: Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, pages 170--173, New York, NY, USA, ACM, 2008.
PDF | video | bib

2005
Experimental Evaluation of an Augmented Reality Visualization for Directing a Car Driver's Attention Tonnis, M., Sandor, C., Lange, C., Bubb, H., and Klinker, G.
Experimental Evaluation of an Augmented Reality Visualization for Directing a Car Driver's Attention
ISMAR '05: Proceedings of the 4th IEEE/ACM International Symposium on Mixed and Augmented Reality, pages 56--59, Vienna, Austria, IEEE Computer Society, 2005.
PDF | presentation | video | bib

2002
An Architecture Concept for Ubiqutous Computing Aware Wearable Computers Bauer, M., Bruegge, B., Klinker, G., MacWilliams, A., Reicher, T., Sandor, C., and Wagner, M.
An Architecture Concept for Ubiqutous Computing Aware Wearable Computers
International Workshop on Smart Appliances and Wearable Computing (IWSAWC), pages 785--790, July, 2002.
PDF | bib

2001
Design of a Component-Based Augmented Reality Framework Bauer, M., Bruegge, B., Klinker, G., MacWilliams, A., Reicher, T., Riss, S., Sandor, C., and Wagner, M.
Design of a Component-Based Augmented Reality Framework
Proceedings of the International Symposium on Augmented Reality (ISAR), pages 45--54, October, 2001.
PDF | presentation | bib

Journals

2009
What Wearable Augmented Reality Can Do for You Thomas, B., and Sandor, C.
What Wearable Augmented Reality Can Do for You
Pervasive Computing, IEEE, pages 8-11, April-June, 2009.
PDF | bib

2005
A Rapid Prototyping Software Infrastructure for User Interfaces in Ubiquitous Augmented Reality Sandor, C., and Klinker, G.
A Rapid Prototyping Software Infrastructure for User Interfaces in Ubiquitous Augmented Reality
Personal Ubiquitous Computing, pages 169--185, London, UK, Springer-Verlag, 2005.
PDF | bib

Theses

2009
Animated Direct Manipulation Interface for Interactive Environment Exploration in Handheld Augmented Reality Moyne, R.
Animated Direct Manipulation Interface for Interactive Environment Exploration in Handheld Augmented Reality
October, 2009.
bib

2005
A Software Toolkit and Authoring Tools for User Interfaces in Ubiquitous Augmented Reality Sandor, C.
A Software Toolkit and Authoring Tools for User Interfaces in Ubiquitous Augmented Reality
2005.
PDF | bib

Book Chapters

2013
Pursuit of X-ray Vision for Augmented Reality Livingston, M., Dey, A., Sandor, C., and Thomas, B.
Pursuit of X-ray Vision for Augmented Reality
In Tony, L., and Livingston, M., editors, In Human Factors in Augmented Reality Environments, pages 67--107, Springer, USA, 2013.
PDF | bib

2011
Augmented Reality Visualization Techniques Kalkofen, D., Sandor, C., White, S., and Schmalstieg, D.
Augmented Reality Visualization Techniques
In Furht, B., editors, Handbook of Augmented Reality Technologies and Applications, pages 65--99, Springer, 2011.
PDF | bib

2006
Lessons Learned in Designing Ubiquitous Augmented Reality User Interfaces Sandor, C., and Klinker, G.
Lessons Learned in Designing Ubiquitous Augmented Reality User Interfaces
In Haller, M., Billinghurst, M., and Thomas, B., editors, Emerging Technologies of Augmented Reality: Interfaces {\&} Design, Idea group inc., 2006.
bib

Patents

2010
Sandor, C., Cunningham, A., and Ville-Veikko, M.
Method and Apparatus for an Augmented Reality X-Ray. US Patent application 12/785,170 (Filed 21 May 2010). Available online at google patent search.
May, 2010.
bib

2009
Sandor, C., and Cunningham, A.
Augmented and Virtual Reality Methods and Means. Australian Patent application AU2009905083 (Filed 16 Oct 2009)
October, 2009.
bib

2007
Sandor, C., and Nishimura., N.
Information Processing Method and Apparatus for Specifying Point in Three-Dimensional Space. Japanese patent 2007139373 (Filed 5/2007). US patent application 12/125,773 (Filed 22 May 2008). Available online at google patent search.
May, 2007.
bib

Demos

2008
See-Through Vision for Mobile Outdoor Augmented Reality Avery, B., Thomas, B., Piekarski, W., and Sandor, C.
See-Through Vision for Mobile Outdoor Augmented Reality
Demo at IEEE and ACM International Symposium on Mixed and Augmented Reality, Cambridge, England, September, 2008.
PDF | video | bib

An Augmented Reality Weather System Heinrich, M., Thomas, B., Mueller, S., and Sandor, C.
An Augmented Reality Weather System
Demo at IEEE and ACM International Symposium on Mixed and Augmented Reality, Cambridge, England, September, 2008.
PDF | video | bib