AR Glasses in Education

Harry Underwood
NMAC 4483
Senior Project Introduction
3/28/2012

Abstract

Augmented reality (AR) inserts virtual objects onto and into our own reality, bringing the benefits of virtual reality within the grasp of human beings rather than virtual avatars. The benefits of AR have been realized in a growing number of mobile handheld applications, ranging from the cultural to the commercial to the political. However, such applications are hindered from their greater potential as viable modes of presentation by the lack of a marketable device which presents AR content without a reliance upon handheld devices. For students of the educational disciplines which will benefit most from AR applications, the perfect AR-presenting device will be wearable as eyewear. Mobile AR eyewear will provide more immersive visualizations to students and enhance their online educational experiences.

1. Introduction

1.1 Goals

This paper argues for the potential utility and benefits of augmented reality (AR) glasses in the online, remote education of students. It argues that, if and when augmented reality applications are to be applied through lightweight pairs of glasses connected to mobile computers, such applications can be effectively purposed toward the elucidation of the surrounding reality environment with three-dimensional information for various, beneficial purposes. A major benefit of such an immersive medium of communication for the consumer, and also the main focus of this paper, would be the practice of education.

Educational pursuits would easily serve as a niche use for such glasses, expanding upon the offerings and utility of desktop computer interfaces to the educational market. Just as within full-on stationary virtual reality environments (Pantelidis), AR glasses could also harness and expand upon the mobility of currently-marketable mobile devices such as smartphones and tablet computers in order to easily deliver educational content to users, no matter their current physical location.

To understand the intersection of AR and mobile computing with distance education, it is necessary to ascertain several core understandings of the roots of these approaches:

• One must first have a prior understand of the histories and evolutions of such technologies or practices up to the present.

• One must also understand how AR glasses would be best constructed to enable perception of, and interaction with, a virtually-augmented real world environment.

• Finally, one must be aware of the best possible user interfaces for the access of an AR classroom through AR glasses.

1.2 Definition

Augmented reality refers to any computer interface which overlays the user’s perception of reality with a virtual “layer” of computer-generated properties. An extension of the virtual reality interface, an AR software program relies upon computer recognition of a camera capture of the real world (Azuma 356). Rather than completely overlaying the perception of the real world with a completely-alternative virtual space, the AR program inserts select virtual objects both onto and into camera capture of the real world. In other words, AR extends and elucidates the real-world environment rather than replace it altogether (Kaufmann).

Research into augmented reality and related approaches has revealed a wealth of information on how AR can be best applied. AR has been utilized in defense, healthcare, entertainment, journalism, architecture, commerce and other fields, with many uses being further extended by way of differing hardware approaches.

The development and consumerization of AR for desktop and mobile computers has opened up opportunities for a variety of user interfaces and applications which often may not be achievable through two-dimensional user interfaces. Indeed, Liarokapis calls it the “ultimate immersive system where participants cannot become more immersed in the real environment (Liarokapis 23).” However, as of the 2010s, AR remains a niche, largely-unexploited usage or interface for personal computing. The application of consumer-grade AR glasses to purposes such as mobile education will expand the possibilities and potential of such glasses in countless related fields, especially those taught by professional teachers.

1.3 Motive

The primary motive for this paper is an interest in the democratization of further aspects of education beyond not only the borders of any physical campus, but also beyond the confines of computer displays and user interfaces which rely upon interaction in front of, or on top of, the display surface. The disadvantages of attending physical campuses are well-documented, and were in fact the reasons for why distance learning was developed as an alternative process for millions of students (Oblinger). However, the lack of immersion of non-eye-based interfaces such as those used for desktop computers, laptops, tablets and smartphones may pose a problem for those who desire to not only easily collaborate with other online students off-campus, but also grasp an understanding of educational material beyond two-dimensional presentations.

In keeping with the aforemetioned motive, the following sections present a detailed overview of the history, architecture and challenges behind such an intersection of technological and presentational approaches. It is intended that such motives are fulfilled by the explanation of the utility of AR glasses in mobile education.

2. History

2.1 Distance Learning

Distance learning, or the performance of education through remote, often long-distance education, evolved from the earliest advertisements for mail-based educational courses in the 18th century in the United Kingdom (Holmberg). Distance learning by mail-order courses would be augmented throughout the 20th century with the purposing of broadcast radio and, later, television toward distance education (Gooch).

Beginning in the mid-1990s, the modern era of Internet-based, rapid-fire delivery of content between teacher and student took off, marking a further shift from geographic locations and hard or broadcast media to Internet-connected computers as channels of education delivery. The World Wide Web became a primary portal for the sharing and publishing of educational documents among students with each other and with teachers, increasingly supplanting amateur radio and televised courses. It also became an easier means by which students could communicate to their teachers and vice versa.

In the 2000s, the increase in wireless communication and the shrinking of computer form factors allowed students an ever broader range of physical movement outside of the campus without necessarily causing breakdowns in education. The growth in diversity of “apps”, or service-specific software made primarily for mobile devices, offered newer options for the display of educational material, as well as the ability to respond to such material. Educational media were also made available for playback on mobile devices such as iPod, iPhone (Apple Inc. 2012) and iPad (Albanesius 2012).

From the beginning, the greatest beneficiaries of distance-learning were those lower-income or less-geographically-accessible students who could not afford to live on or near an educational campus. Today, it remains not only the least-expensive means of education for those demographics, but also the most time-effective means for those who provide incomes for themselves and others.

2.2 Augmented Reality

AR, as a branch of virtual reality, was developed rather early in the computing revolution, with the head-mounted display, a head-worn device for the display of three-dimensional digital graphics, being invented by Ivan Sutherland in 1968 in the form of “The Sword of Damocles”. The device itself included a partially translucent display, designed so that users would be presented computer-generated visual content without being visually cut off from their surroundings. This feature, while making the “Sword” one of the pioneering implementations of virtual reality, was also the first demonstration of a “mixing” of reality with elements of virtuality (Science Clarified 2012).

The hardware for virtual reality was developed in capability and shrunk in size in the decades afterward. While the Sword of Damocles of the 1960s was so heavy that it had to be suspended from a ceiling at MIT, the HMDs of the 1980s were comparatively light, with the eyePhone and DataGlove of VPL Research being produced by Jaron Lanier’s VPL Research during this time (“Virtual Reality”). By the 1990s, the first consumer-marketable eye-focused display devices were released, including the short-lived Virtual Boy from Nintendo.

The development of webcams and smartphones in the 2000s afforded consumers a novel glimpse of augmented reality as applied for such uses as geotracking of surrounding areas. Application software such as ARToolkit allowed average computer users to create and display virtual objects onto real-world backgrounds as captured by webcams. Mobile software such as Layar extended this capability to smartphones, taking advantage of accelerometers, GPS navigation and an integrated camera prebuilt into smartphones specifically for the purpose of overlaying visual information onto real-world environs.

3. AR in Science Fiction

As is the case for many other advances in science and engineering, the concept of augmented reality has many roots in works of science fiction. Several literary and filmed works have featured AR in action, ranging from Vernor Vinge’s Rainbow’s End to John Favreau’s Iron Man (2008). However, few other filmed works in the history of augmented reality have had as much impact upon public awareness or perception of AR as Dennō Coil.

Dennō Coil, a 2006 Japanese animated series created by Mitsuo Iso, is perhaps one of the most modern explorations of a hypothetical ubiquitous AR experience. Centered around the lives and mysteries of elementary-school students as they explore the ins and outs of the AR layer over their town as well as their own pasts, the series provides to the viewer a believable sample of what life could be like for school children in a future not quite distant from the present (Carroll 2012).

The series is described by Rice as one of the “best examples of L3 (Level 3, or ‘Augmented Vision’) AR”, and those who are interested in the capability of AR are advised to “pay attention to” both this series and Rainbow’s End, “if you don’t bother with anything else (Rice 2009).” The inspiration for the research documented in this paper as evidence of the efficacy of AR glasses is partly derived from Dennō Coil, including the envisioning of the glasses as being as similar to work goggles or prescription glasses as possible.

4. Constitution of AR

4.1 The construction of AR glasses

The glasses, often called “head-mounted displays” or “headsets”, are a key component in the architecture of a more pedagogically-friendly AR for students. Compared to the miniscule cameras integrated into mobile smartphones, desktop or notebook devices, and barring any further advances with other approaches, glasses would provide the most immediate, least-intrusive form factor for AR presentation to the user. This construction is a simplification of that identified by Azuma as an “optical see-through HMD”, to contrast it to other HMD constructions (Azuma 365).

Ideally, AR glasses would provide real-time visual overlays over and into real-world surroundings. In other words, not only would graphical elements be overlaid on top of the viewer’s perspective of one’s own surroundings, but such graphical elements could easily weave “through” and “behind” real-world objects.

Besides the internal computer, two main components are necessary to the function of the glasses: the tracker and the lenses. The real-time tracking of natural surroundings must include depth perception of the surrounding area. The 3D scanning capabilities of a device such as the Microsoft Kinect provide a means by which such occlusion can be achieved from the perspective of the wearer; at the least, the Kinect device can serve as a landmark of depth perception capabilities to be met by the AR glasses (Hinck et. al 25).

The lenses would be constructed so as to allow a transparent view of the outside world while being able to superimpose virtual graphics onto and into the real world from the perspective of the wearer. To date, the best material for the construction of such lenses would be some form of transparent organic light-emitting diode in order to allow the best possible blending of lenses.

4.2 The AR-driven user interface

The AR-driven user interface, at its most ideal, would be one which allowed the user to interact with, manipulate and produce virtual objects or properties with more range of movement than allowed by a desktop or mobile computing device. In this interface, most virtual utilities – keyboards, canvasses, windows and buttons of all types – would be depicted as floating in front of the user’s eyes, similar to the AR interface featured in the eponymous suit in the 2006 film Iron Man (Downey, Jr.).

Perhaps the linchpin of the growth in popularity of touchscreen computing devices in the 2000s and 2010s is the virtual keyboard as the default means of data entry. This can be carried into the AR-based user interface as a floating 3D virtual keyboard, as depicted in various instances in Dennō Coil (“Kids With Glasses”). This keyboard would fully integrate with other elements and widgets which can appear and disappear at the whim of the user.

Another benefit of this interface, one which supersedes the utility of two-dimensional presentational surfaces, is the ability to encounter three-dimensional models and tools transposed into the natural real-world environment for both personal and collaborative use. Starner opines that a ubiquitous network and interface for AR will enable synchronous collaboration between users by way of visualized “file systems, design tools, and information searches (Starner 65).” Such collaboration happens to already be a core factor in the process of education, so such a tool as AR glasses

5. Layout of AR-based Distance Learning

5.1 Overview

Just as the World Wide Web-based online portals for access to class materials has come to define modern concepts of correspondence education on the desktop and laptop, so would dedicated rooms within three-dimensional virtual space be the repository for three-dimensional educational materials. These classrooms would exist within the layer of augmented reality, but would also be remotely accessible from any location in the world. Furthermore, the repertoire of digital class materials would expand in all disciplines to include the production and application of interactive graphical works.

For educators, the opportunity to apply an educational environment in augmented reality for both on- and off-campus students becomes ever present with each new advancement in AR-enabling technologies. Kimberley Ostberg wrote in 1993 that “The technology is moving ahead, regardless of what we as educators may wish. So we can either become a part of the research and development effort, adding the cognitive component to the mix, or we can sit back and let technology take the educational process by storm (Ostberg 1993).”

5.2 Use in education

One such subject which benefits from AR-driven distance learning is medical science, among other life sciences. A number of applications of AR in this scenario, ranging from “switching out” body parts, overlaying virtual “inside” images of internal organs, to other means of training students in the science of health and surgery, have been demonstrated in HMD research. Danciu et. al point to how “patient-specific procedure rehearsal” can be accomplished through augmented reality as a means of preparation for surgeons upon 3D models of body parts prior to medical intervention upon physical subjects (Danciu et. al 19; Gorini et. al).

AR glasses can also be utilized in math and geometry education. This application has been explored by various researchers through use of the Construct3D software framework. Construct3D, based on the earlier Studierstube framework, was utilized by Kaufmann et. al to build interactive AR environments with 3D models of mathematical and geometric problems (Kaufmann et. al 263). Making use of a head-mounted display, 3D models are superimposed on the perspective of wearers who also manipulate these models using utensils such as styluses.

In addition, one of the most important venues of education to logically benefit from AR glasses is the museum. For as long as museums have existed for various sectors of the public to explore their interiors, they have been most purposed toward enabling museum-goers to view artifacts, replicas or physical models in ways which have historically been inaccessible or perceived as inaccessible by the public. Museums also serve the purpose of establishing and spreading the institutional hegemony of human (and non-human) experiences (Gaither 1989). AR glasses will allow lay persons to interact with a much more animated, three-dimensional presentation of exhibits, be it remotely or inside the museum complex.

Finally, in addition to the ubiquitization of educational and museological content to those of reduced income or geographical access to the campus, AR can also spread an educational and therapeutic influence to those who are physically disabled or cognitively troubled. Liarokapis et. al posit that, by remotely visiting virtual museums in augmented reality, patients can “simulate a visit to a ‘real’ museum environment, develop skills and recover knowledge that may be partially lost (Liarokapis 2004).” Such an approach allows any patient access to educational and museological experiences (as is the patient’s legal right in many countries, such as the United Kingdom under its Equality Act 2010) at the patient’s own pace.

5.3 Impact and Potential problems

An issue which may pose a problem for augmented reality glasses when used on a regular basis is the interference of surrounding ambient lighting or lack thereof. MacIntyre cautions that integrated transparent displays would provide poor visibility to wearers when too much or too little ambient lighting is available: in an area with little ambient lighting, the overlay would be overemphasized against the real background, while in an area with too much ambient lighting, the view of the overlay would be negligible in perception (Baldwin). This may interfere with the perception of educational content.

From the 1980s onward, the perception of such devices has suffered through a reputation as a cludgy, weighty, hobbyist-oriented device. The short shelf life and limited audience of the few and the mass media depiction of headsets and computerized glasses up until the 21st century likely had a negative impact upon the public perception of the aesthetics for such glasses.

The potential for this approach, however, is something that cannot be ignored or passed off as unworthy of consideration. For educators of disciplines which have not translated effectively to distance learning environments on the Web, AR for the common student brings the possibility of holding more students responsible for both the learning and experiencing of the curriculum. AR, just like VR and virtual worlds, also brings forth the likelihood of spreading standard educational disciplines to students who are disabled and medically benefit from learning and experiencing the curriculum within their own personal range of competency.

Our perception of telecommunications will also be changed by a ubiquitous AR, let alone our perception of educational environments. Rice predicts that everything that we know about the World Wide Web, “virtual worlds, interface design, client/server, [and] internet domains” will be dramatically affected by AR, to the point where the contemporary means of communication and communication maintenance which we identify as part of “Web 2.0” will not necessarily translate to a ubiquitous mixed reality (Rice). Such a change may be disruptive to contemporary institutions of communication and communication regulation when AR becomes relatively inexpensive to utilize for the lay human being.

6. Conclusion

In conclusion, AR-based distance learning through lightweight glasses is a means of education which can be of immense value to students and teachers at all levels and topics of education. It continues to build upon the importance of telecommunications to the expansion of education to all possible learners, and also somewhat reduces the importance of physical presence on the physical campus to students. As a result, it increases the variety of disciplines which can have an impact upon online students just as much as it can upon on-campus students.

While the histories of both distance education and augmented reality offer opportunities to look at the increase in capability both media have achieved in presenting content to students, the intersection of such media through AR eyewear will offer newer opportunities and challenges to the developers of platforms for both devices. Electronics will have to be simultaneously reduced in size and increased in both computational power and presentational capability to engage individual students with sensory appeal. The software applications and platforms of collaboration must also be prepared to host the broader range of student experience of personalized augmented reality. Finally, the design of such eyewear matters to the acceptability of such devices to consumer students, as does the capability to fluidly and gracefully present content to the user over a networked platform.

Such a means of presenting educational material to students, however, allows for the further expansion of the spatial and interactive benefits of on-campus classes to off-campus students. Beginning with mail-order courses in the 19th century and increasing in viability through improvements in telecommunications in the 20th century, distance learning has expanded the benefits of education to millions of students who may otherwise have lacked for the knowledge to fulfill their life goals. Just as the World Wide Web has allowed for students to become better participants in distance classes, so can AR glasses allow students to become visually and interactively verbose in those same classes.

Works cited

Albanesius, Chloe. “Apple Targets Educators Via iBooks 2, iBooks Author, iTunes U App.” PCMag.com. 19 Jan 2012. Web.

Apple Inc. “Apple Announces iTunes U on the iTunes Store.” Apple.com. 30 May 2007. Web.

Azuma, Ronald T. “A Survey of Augmented Reality.” PRESENCE: Teleoperators and Virtual Environments, Vol. 6, No. 4, pp. 355-385, 1997. Web.

Baldwin, Roberto. “Google Glasses Face Serious Hurdles, Augmented-Reality Experts Say.” Wired. 5 Apr, 2012. Web.

Carroll, Luke. “Review: Dennōu Coil – Part 1 Sub.DVD.” Anime News Network. 20 Apr 2012. Web.

Danciu, Marius, Mihaela Gordan, Aurel Vlaicu and Alexandru Antone. “A Survey of Augmented Reality in Health Care.” Acta Technica Napocensis – Electronics and Telecommunications. Vol. 52, No. 1, pp. 13-22. Cluj-Napoca: Technical University of Cluj-Napoca, 2011. Web.

Downey, Jr., Robert, perf. Iron Man. Paramount Pictures, 2008. Film.

Druin, Allison. “Mobile Technology for Children: Designing for Interaction and Learning.” Boston: Morgan Kaufmann, 2009. Print.

Gaither, Edmund Barry. “Voicing Varied Opinions.” Museum News 68, No. 2, March-April 1989. 52. Print.

Gorini, Alessandra, Andrea Gaggioli, Cinzia Vigna, and Giuseppe Riva. “A Second Life for eHealth: Prospects for the Use of 3-D Virtual Worlds in Clinical Psychology.” Journal of Medical and Internet Research. 2008 Jul-Sep; 10(3): e21. National Center for Biotechnology Information. Web.

Hinck, Debbie, Jake Stout, John Solit, Mark Hobson and Monique Priestley. “The History of the Xbox Kinect Exhibition Proposal.” pp. 1-34. MEPriestly.com. n.p., 9 Nov 2011. Web.

Holmberg, Börje. The evolution, principles and practices of distance education. Studien und Berichte der Arbeitsstelle Fernstudienforschung der Carl von Ossietzky Universität Oldenburg. Oldenburg: Bibliotheks-und Informationssystem der Universitat Oldenburg, 2005. p. 13. Print.

“Kids with Glasses.” Dennō Coil. Writ. and Dir. Mitsuo Iso. Madhouse, 2007. Web.

Kaufmann, Hannes. “Collaborative Augmented Reality in Education.” 3. February 2003. Vienna University of Technology. Web.

—, Dieter Schmaltsteig and Michael Wagner. “Construct3D: A Virtual Reality Application for Mathematics and Geometry Education”. Education and Information Technologies 5:4 (2000): 263-276. Kluwer Academic Publishers. Web.

Liarokapis, Fotis. “An Augmented Reality Interface for Visualizing and Interacting with Virtual Content.” Virtual Reality 11.1 (2007): 23-43. ProQuest Computing; ProQuest Research Library. Web. 17 Mar. 2012.

Liarokapis, Fotis, S. Sylaiou, A. Basu, N. Mourkoussis, M. White and P.F. Lister. “An Interactive Visualisation Interface for Virtual Museums.” Proceeding of the 5th International Symposium on Virtual Reality, Archaeology and Cultural Heritage. Ed. Y. Chrysanthou, K. Cain, N. Silberman, F. Niccolucci. Brussels: VAST, 2004.  Print.

Oblinger, Diana G. “The Nature and Purpose of Distance Education”. The Technology Source. Michigan: Michigan Virtual University, March/April 2000. Web.

Osberg, K.. “Virtual Reality and Education: A Look at Both Sides of the Sword.” Technical R-93-7. Seattle: Human Interface Technology Lab, 1993. Web.

Pantelidis, V. S. “Reasons to Use Virtual Reality in Education.” VR in the Schools 1(1), 1995. Web.

Rice, Robert. “Augmented Vision and the Decade of Ubiquity.” Curious Raven. 30 Mar 2009. Web.

Roussos, M., Johnson, A., Moher, T., Leigh, J., Vasilakis, C., and Barnes, C. “Learning and Building Together in an Immersive Virtual World.” PRESENCE 8(3), pp. 247-263, MIT Press, June 1999. Print.

Schmalstieg, D., Fuhrmann, A., Hesina, G., Szalavari, Z., Encarnação, M., Gervautz, M., and Purgathofer, W. “The Studierstube AR Project.” PRESENCE: Teleoperators and Virtual Environments 11(1), pp. 32-54, MIT Press, 2002. Web.

Science Clarified. “Airplanes to Arcades: The Development of Virtual Reality.” Science Clarified. 2012. Web.

Starner, Thad. “The Challenges of Wearable Computing – Part 2.” IEEE Micro Magazine. July 2001: Volume 21 Issue 4. Print.

United States Department of Defense. National Security Agency. “Virtual Reality.” Cryptologic Quarterly. Vol. 12, Nos. 3-4 (Fall/Winter 1993): 21-50. Washington: GPO. Print.