03

may 2013

AUGMENTED REALITY, ART AND TECHNOLOGY

Re+Public: Re+Imagining Public Space
BC “Heavy” Biermann

Pre-Digital Augmented Reality
Maarten H. Lamers Vestibular Stimulation

as an Augmented Reality Layer?
Antal Ruhl

AR[t]
Magazine about Augmented Reality, art and technology

MAY 2013
2

HOW & NOSm MURAL AUGmEnT, ThE HEAVY PROJECTS, ARTICLE On pAGE 8

3

Colophon
ISSN
2213-2481

COnTACT
The Augmented Reality Lab (AR Lab) Royal Academy of Art, The Hague (Koninklijke Academie van Beeldende Kunsten) Prinsessegracht 4 2514 AN The Hague The Netherlands +31 (0)70 3154795 www.arlab.nl info@arlab.nl

EDITORIAL TEAm
Hanna Schraffenberger, Mariana Kniveton, Yolande Kolstee, Jouke Verlinden

COnTRIBUTORS
AR LAB & PARTnERS: Wim van Eck, Edwin van der Heide, Pieter Jonker, Maarten Lamers, Maaike Roozenburg GUEST COnTRIBUTORS: Alejandro Veliz Reyes, Antal Ruhl, Lotte de Reus, Matt Ramirez, Oliver Percivall, Robin de Lange, BC “Heavy” Biermann

GRAphIC DESIGn
Esmé Vahrmeijer

pRInTInG
Klomp Reproka, Amersfoort

COVER
Our AR[t]y cover is a work by Royal Academy of Art student Donna van West who participated in the Smart Replica’s project, see: www.donnavanwest.nl

www.arlab.nl

4

Lamers AUGmEnTED EDUCATIOn Robin de Lange VESTIBULAR STImULATIOn AS An AR LAYER? Antal Ruhl HOW DID WE DO IT Wim van Eck AR[t] PICK: ImmATERIALS Editors' Choice BELIEVABILITY Edwin van der Heide NOnLInEAR STORIES TOLD BY CUpS AnD SAUCERS Lotte de Reus 5 . ThE nEW AFFORDABLE hEADSET — PART 2 Pieter Jonker and Yolande Kolstee A STUDY In SCARLET Matt Ramirez SUBJECT: InTERVIEW From: Hanna Schraffenberger To: Lev Manovich InTRODUCInG AR TO FIRST YEAR GRAphIC DESIGn STUDEnTS Yolande Kolstee PRE-DIGITAL AR Maarten H.Table of contents 30 WELCOmE to AR[t] 50 WhO OWnS ThE SpACE 2 Yolande Kolstee 06 08 16 18 24 26 30 34 40 44 50 56 62 64 72 76 RE+PUBLIC: RE+ImAGInInG PUBLIC SpACE BC “Heavy” Biermann — PhD ThE MISADVEnTURES In AR Oliver Percivall AUGmEnTED pEDAGOGIES Alejandro Veliz Reyes MARTY.

FINE ART STUDENTS SHOW THEIR WORK ON THE POP-UP GALLERY TWO GRAPHIC DESIGN STUDENTS (DAVE POPPING AND GABOR KEREKES) MADE A MOIRÉ-PATTERN AS A DECODING KEY 6 .

. A recurring topic is set out by Yolande Kolstee: the legal ramifications of AR initiated in AR[t] #2. These drawings & 3D model will be downloadable from our website. We are very pleased to introduce to you the improved version of Marty: the video see-through AR head-up display. We are confidant you will enjoy this issue. Please feel free to contact us to tell us what caught your eye. in the attempt to leverage AR to allow artists to make incursions into public spaces. on the edge of art and technology. Crucial to AR experiences is the concept of believability. which will provide you with all the information on our artistic and technical research — but also that of fellow researchers.nl. scientists. Furthermore. In our new section AR[t] Pick.WELCOME. well-known among many people since the publication of his book ‘The Language of New Media’ in 2001. in the part News Picks we post short news items on AR artists. events and developments. We feature both articles with a philosophical perspective and articles with a more technical point of view. which might lead to an item there. Lotte de Reus discusses a spatial audio intervention to enhance museum exhibits. The results are a wide variety of implementations and reflections in different creative contexts. On this website. to the third edition of AR[t]. art and technology! In this issue we present articles by contributors from all over the world who are involved in stretching the borders of augmented reality.. we feature a short science fiction sequel by Oliver Percivall. Robin de Lange considers the long-term ramifications of extending both the mind and cognition itself with AR. We invite you to visit our new website. describing some AR software programmes which are widely available. to print out at your local 3D print facility! Hanna Schraffenberger sets about interviewing Lev Manovich. we share artworks that caught our eye. the magazine about Augmented Reality. In this issue. In Re+Public’s article we can read about blurring private properties’ boundaries. Yolande Kolstee.arlab. look out for our call for contributions. and in ways they were previously physically unable to do. via the same URL www. which will be posted on our website soon! . Matt Ramirez and Yolande Kolstee. Maarten Lamers takes us back to pre-digital AR with his story on Pepper’s Ghost and at the other end of the spectrum Antal Ruhl explores the potential of using vestibular stimulation in order to create new AR experiences. special attention is given to education: Experiences of bringing AR into the classroom are presented in separate articles by Alejandro Veliz Reyes. explored by Edwin van der Heide. Head of AR Lab 7 In the second part of this issue. Wim van Eck continues his AR tutorials in the series ‘How did we do it’. and information about our experiments in the cultural domain. we have chosen 'Immaterials' — the result of a collaboration between the onformative design studio and Christopher Warnow. Should you like to contribute to issue #4.

2011).OUTDOOR ADVERTISING: AR | AD TAKEOVER (NYC. RON ENGLISH. photo by WILL SHERMAN BC “HEAVY” BIERMANN — PHD RE+PUBLIC: RE+IMAGINING PUBLIC SPACE 8 .

and augmented reality (AR) in particular. Re+Public seeks to leverage AR in an effort to allow artists to make incursions into public spaces in ways they were previously physically unable to do.RE+PUBLIC A creative collaboration between The Heavy Projects (Los Angeles) and the Public Ad Campaign (New York City). As such. Seeking to understand a profit-driven governmentality and its affect on 9 . Re+Public has developed an experimental mobile device application that digitally resurfaces three specific areas of public space: outdoor advertising. the ability to pay for its usage. or a willingness to break the law. OUTDOOR ADVERTISING: AR | AD TAKEOVER (NYC. Blurring private property boundaries. and buildings. to alter current expectations of our public media environment generally dictated by property ownership. this article focuses on these domains and demonstrates how Re+Public has used AR to transcend current private property boundaries. With this goal in mind. Re+Public is dedicated to using emerging media technology. murals. which lies at the heart of our endeavor to re+imagine public space as a more open visual commons. 2011) AR presents unique opportunities to creatively problematize the political and economic systems that shape the awareness of both individuals and the public writ large.

real estate developer and arts supporter Tony Goldman started the Bowery Mural with Jeffery Deitch and Deitch Projects. Viewing previous murals online. In 2012. removes the viewer from the space by placing them squarely in the absent. PosterBoy. users could trigger web-based information related to the showcased artists whose work has historically addressed commercial advertising in public space such as: Ron English. To this end. We foresee AR mobile device technology as a first step in the transformation of public space into an arena shaped by user created content. During Art Basel 2012. OUtdoor MUrals: Bowery Wall (NYC. The Bowery Mural Wall is an outdoor mural exhibition space in Manhattan. The AR | AD Takeover used street level ads and billboards to trigger a curated digital art installation that displayed on mobile devices. we augmented ads in Times Square with artistic content. in the space. Our digital infiltration into public space and takeover of commercial ads created a place of dialogic interaction rather than a monologic con- sumptive message. artists who have contributed to the Wynwood Walls include Os Gemeos. Kenny Scharf. John Fekner. as Drawing from an international pool of talent. It is precisely this kind of spatial “aura” that distinguishes AR from other types of emerging media technology. we used AR to rupture public space with a new kind of artistic interactivity. In addiIn 2009. D. they might start demanding a better version of public messaging than the billboard default. Goldman endeavored to create a center that developed the area’s pedestrian potential. and Swoon among many others. which commercial ads do not necessarily have to dominate. Barry McGee. 2012) In contradistinction to the use of AR to problematize the consumptive monologue of outdoor advertising. Invader. and Shepard Fairey. they move in perspectival relation to the viewer. in situ. AR permits users to see these murals as if they were actually back on the wall. we envision AR as the first step in the evolution of better tools of expression that can democratize public media production.public space. In 2008. In other words. Goldman Properties and Tony Goldman who was looking to transform the industrial warehouse district of Miami also conceived the Wynwood Walls. Specifically. In other words. by pointing a mobile device at the present mural. 10 . that the current artist has painted over. Owned by Goldman Properties since 1984. for example. at both the Bowery and Wynwood Walls sites. Nunca. Dr. While users could certainly view the previous murals online. digital world. the mural series commenced with a recreation of Keith Haring’s famous 1982 mural followed by work by such recognized artists as Os Gemeos. Saner & Sego. Re+Public used AR to resurrect murals that once existed on the wall. Faile. Aiko. 2012) and Wynwood Walls (Miami. and OX. Retna. whereas other technologies tend to disconnect the viewer from their immediate physical surroundings. Once individuals experience this AR version of reality. Re+Public was commissioned by the Wynwood Walls to create an AR experience. Re+Public investigated how civic authorities allow certain private parties to profit while preventing or discouraging other forms of public media production. users are able to see the former murals. In New York City. AR connects the digital with the physical in an intimately “present” way. Beginning with the 25th and 26th Street complex of six separate buildings. AR is an incremental step towards showing the public an alternate view of their landscape.

stream. and flowing waterfall. rather abstract 3D environment and permitted users to both pull apart and reconstruct the mural elements. In considering our deployment at the Wynwood Walls in particular. In the case of each traditional mural. In addition to an immersive garden with a bridge. and Retna. 11 . we created a 3D Kabuki theater that allowed users to walk into a digitized version of the AR is an incremental step towards showing the public an alternate view of their landscape. we created another immersive. RESURRECTED KEITH HARING tion to resurrecting a Shepard Fairey mural that he recently painted over with a new mural in tribute to the recently deceased Tony Goldman. Aiko. interactive environments for four other murals by How and Nosm. the AR overlay used the 2D paintings as feature tracking markers and source material to produce an original expres- sion of creative content. mural where all of the elements were separated in Z-space.OUTDOOR MURALS: BOWERY WALL (NYC. with the McGinness. 2012). we wrestled with this new type of work and wondered if it constituted a new mode of art. which commercial ads do not necessarily have to dominate. Additionally. we made it appear as if the paint colors were draining out of the mural. we worked directly with MOMO and collaborated on an AR version of his indoor mural at the Nicelook Gallery on site. Ryan McGinness. it is arguable that the AR assets represent original works in that they contain a sufficiently new visual expression of ideas. Re+Public created 3D. With the Retna. we built the mural shapes in 3D and animated them to extend out of the wall and placed them both on the ground and above the wall. Finally. With the How and Nosm. Without pretending to discern any immediate resolution. over and above those embodied in the 2D mural.

screenshot 12 . fUll view Retna MUral AUgment.Retna MUral AUgment.

” Specifically. using AR. 2012) In addition to using both outdoor ads and murals as the markers that trigger our AR deployments.AUGMENTED ARCHITECTURE: PEARL PAINT. we converted the Bradbury Building. In other words. Converting MOMO’s 2D designs into digital 3D models optimized for mobile. in some meaningful way. made it possible for MOMO to put his art on buildings that he could not have accessed in his traditional 2D format. murals. we used AR to provide an artistic rendering of the re+imagined building by projecting it into a Blade Runner style future. Ultimately. This city vision type of deployment potentially provides more practical architectural and urban planning uses and maintains our notion that the AR experience should be spatially relevant in order to maintain the physi- cal aura that may have drawn the viewer to the building in the first place. Re+Public seeks to continue to deploy AR in an effort to democratize access to our shared visual environment and alter the current expectations of urban media in accomplishing our core mission of re+imagining public space. With the coming advancements in wearable computing. in our attempt to use AR to re+imagine public space. we placed his art on both the Pearl Paint and the Williamsburg Art & Historical Center buildings in New York City. Working with muralist MOMO. 2012) BRADBURY BUILDING (LA. Both structures have a long “art history” in the city and AR allowed us to blur the lines between private and public space. In Los Angeles. It is vital to the health of any city that its inhabitants are able to participate. or “skinned” physical buildings in urban centers by overlaying 3D content onto the physical environment. and their unique architecture much in the same way. in the visual urban messaging systems that surround them. the digital overlay will become a much more seamless and natural part of our daily existence. site of many interior shots in the film Blade Runner (1982) into a futuristic version of itself. or entire buildings. rather than the current commercial hegemony. To this end. FINAL REMARKS Whether it is outdoor ads. we chose three buildings that had a particular cultural significance and. WILLIAMSBURG ART & HISTORICAL CENTER BUILDING AUGMENTED WITH MOMO URBAN ART 13 . instead of placing converted art on the structure. WILLIAMSBURG ART & DESIGN BUILDINGS (NYC. we created the first example of what we refer to as “city visions. we have also experimented with using buildings. we really see the city as a canvas that allows for a multiplicity of voices to enter into our visual landscape. we digitally resurfaced. In this way. the AR assets should maintain some logical connection to the building or space upon which we have attached them. It is our hope that these early entrants will help create experiences that consider art and design as an important part of the way the public adopts this technology.

RESURRECTED SHEPARD FAIREY. With a PhD in Humanities [Intermedia Analysis] from the Universiteit van Amsterdam. BC has worked as both a university professor and a tech developer in Anaheim. and the arts. design. After finally giving up his painfully amateur skateboarding career due to a bum right knee… BC plans to use his extra time continuing to examine meaningful ways to fuse tech + creativity. BC founded The Heavy Projects [and its collaborative spin-off Re+Public] to investigate how the fusion of creativity and technology can uncover new modes of relaying ideas. and interaction. and Saint Louis. BC “Heavy” Biermann possesses an interdisciplinary background that comprises technology. Building upon existing technological and theoretical frameworks. art and semiotics in public space. Prague.BC “Heavy” Biermann Deriving his pseudonym from his penchant for philosophical discussion. photo by JORDAN SEILER 14 . Heavy creates innovative interfaces between digital design and physical worlds in ways that that provoke the imagination and challenge existing styles of art. BC has internationally presented his academic work. OUTDOOR MURALS: WYNWOOD WALLS (MIAMI. which explores augmented reality. Since 2007. As a kind of synthesis between scholarly inquiry and emerging media. 2012). academia.

15 .

Jonker. however. If they are valuable we will publish about it in our next AR[t] magazine. natural features like dominant corners or lines in the scene are tracked. partner of the AR Lab. Smulders. with the aim to do research on co-operative AR. Segers.MARTY. It is. We are also aware of Google glasses and other equipment and we welcome those developments! However. AR-toolkit like markers can do with one camera as the size of the marker is known and REFEREnCES ¢¢ Bridging the multiple reality gap: Application of Augmented Reality in new product development. Jouke C. Piet Van den Bossche. “why can laying on bed or in an armchair not the ceiling or the white wall. Man and Cybernetics. In markerless systems. Due to a lack of manpower and funds this might. cramped shoulders from long computer days and the sighing. Verlinden In proceedings of the IEEE International Conference on Systems. however. It was designed by Niels Mulder from studio RAAR on assignment of the TU-Delft. the geek that walked around with an AR display. artists and designers in the area of AR and stimulate industry to come up with affordable AR equipment. no assumptions on scale can be made and two cameras that see that same feature are necessary. Pieter P. Chatzimichali. so anyone with access to rapid prototyping facilities can reproduce Marty for his/her own scientific or artistic research. We are aiming at adapting Marty to version 2. Anchorage. 2011 16 . we will post the design files as well as a photo series how to assemble the Marty. In the mean time be our guest and contribute to AR to come up with your own solutions. One of the aims of the AR Lab is to stimulate AR in the world by putting our developments in public domain. Keep an eye on our website as we will also link to the software to do 3D pose tracking based on natural feature tracking. take a while. Alaska. R. Hetty van Emmerik.nl. meaning that no large scale industrial production can be based on this design without the consent of the copyright owner. Anna P. which has another face mask.0 that is based on the HMZ T2. still many companies do not understand that you need two camera’s to see virtual objects in 3D and two camera’s to track (salient) keypoints in 3D in order to track your head pose while walking around with your headset in any unknown environment. subject to copyright. hence there is extra knowledge on scale. We are well aware that the Sony HMTZ T2 is on the market. Mien S. But also lower backpain. USA. ThE nEW AFFORDABLE hEADSET — PART 2 Pieter Jonker AND Yolande Kolstee In AR[t] 2 we announced the new MARTY videosee-through headset as a design based on the Sony HMZ T1. When the new AR Lab website will be in the air at about the same time as this AR[t] 3 magazine is published. For more information: info@arlab. Frido E. October 9-12. Wim Gijselaers. the air. And finally: Yes we were already decades ago inspired by fighter pilots with heads-up display and of course Steve Mann. or anything at any time be my display?” DISCLAImER The industrial design of Marty is meant to do experiments by researchers.

NL) WE WILL SOON POST A PHOTO SERIES ON HOW TO ASSEMBLE THE MARTY HEADSET. Image by FERENC MOLNAR ON OUR NEW WEBSITE (WWW.ARLAB. 17 . THE NEW AFFORDABLE HEADSET.MARTY.

I realized that I would like to interview you about Augmented Reality for the AR[t] magazine. ThE LAnGUAGE OF nEW mEDIA. .WORDpRESS. As a consequence.JEWIShphILOSOphYpLACE. P. 2001. ThE MIT pRESS. So I’ll challenge myself to challenge you. Hanna P. my supervisor (Edwin van der Heide) told me that I could/should be more critical towards my interview partners. A short time ago. I work at the Augmented Reality Lab in The Hague and I am one of the editors of the AR[t] magazine. When I read your article The Poetics of Augmented Space. Maybe we can print my questions in issue 3 and your answers in issue 4? 18 MAnOVICh.S.COm Dear Lev. After my last few interviews.P. So I hope you’ll agree to an interview for the magazine? Best regards. I finally also read The Language of New Media. LEV. I’d like to interview you even more.nL> DATE: 2013/4/3 15:53 SUBJECT: InTERVIEW AR[t] mAGAZInE TO: LEV MAnOVICh Maybe you remember me from Facebook.S. ImAGE SOURCE: WWW.FORWARDED mESSAGE ---------FROm: HAnnA SChRAFFEnBERGER <HAnnA@ARLAB.---------.

And secondly. Personally. the user is presented with auditory information that relates to the immediate surrounding space. I would call this Augmented Reality. as well as spoken narrative elements and instructions such as where to go and what to look at. one gets to listen to a mix of edited sounds that blend in with the sounds of the surroundings. It would be great if you’d address the topic once more. because I think that this point of view unnecessarily limits AR to the visual sense. 19 . for example. Firstly. ubiquitous computing. Augmented Space refers to all those physical spaces that are overlaid with dynamic information such as shopping malls and entertainment centers that are filled with electronic screens and all those places where one can access information wirelessly on phones. you mention Janet Cardiff’s audio walks as great examples of laying information over physical space. Wouldn’t you? Augmented Space What is special about AR compared to other forms of Augmented Space? In your article The Poetics of Augmented Space you discuss the concept of Augmented Space. you mention several other technological developments in the context of Augmented Space. monitoring. tablets or laptops. because our readers might not have read your article. I would like to ask you what you consider Augmented Reality (AR) to be. among which.Augmented Reality What is Augmented Reality? To begin with. While navigating the environment. In The Poetics of Augmented Space. In The Poetics of Augmented Space you describe AR as ‘the laying of dynamic and context-specific information over the visual field of a user’. Besides AR. In contrast to ‘typical’ visual AR. These walks are designed for specific walking routes.

I don’t think augmentation is limited to a space or an environment. independent layer of content. an event could sound so real and so nearby. but something else. For example. relates to one individual in the space (me). Do you think that all forms of augmentation bring along an augmentation of space or influence our experience of the immediate surrounding space? What else can be augmented? Something I really like about your article is that you see augmentation as an idea and a practice rather than a collection of technologies. there are more possibilities. information and the surrounding space can be perceived as one single mixed thing as well as independently. just to find. but likewise. but nevertheless. focus of attention or interests of their user. I was wondering whether you have considered other manifestations of augmentation as well. something virtual augments something real. This information Information and space — one coherent gestalt? In The Poetics of Augmented Space you raise a question that intrigues me a lot. the newsreader might tell me about a traffic jam and thereby inform me about my immediate physical space. For example. The voice of the newsreader doesn’t mix with the voice of my colleague. that nothing is happening there. that I’d turn around. In our view. nor does it relate to my environment. the virtual augments that to which it relates. Hence. augmented content and augmented activities. Do the real space and the dynamically presented information add up to one single coherent phenomenological gestalt or are they processed as separate layers? I am a bit of a sound-person and it has always fascinated me that sometimes the sounds of a radio seem to mix in with environmental sounds. I might consider it an augmentation of the activity (not moving. But of course. However. Here. most often. something virtual augments something real. I’d even say that often it’s not the space that is augmented. More specifically.tangible interfaces and smart objects. we have considered things like augmented objects. augmented humans. so far. you mention software that performs tasks according to the mood. and has nothing to do with the surrounding space. I am curious whether you’d agree. sitting still) or an augmentation of the user (me). the ticking of a red streetlight might perfectly mix in with the rhythm of the song that is currently playing. But besides these two options. the sound of the radio just exists as a separate. there is a relationship between both. augmented perception. However. Let’s imagine that my phone registered that I have been sitting still for a long time and reminds me to take a short break to stretch my legs. Listening to a radio play. to the activity the person is performing (sitting still). Most of the time. So judging from my experience of listening to the radio. In AR. but I don’t think it has anything to do with the surrounding space. For example. space is one of the possibilities. a song is just a song. Is AR just one of many related recent phenomena that play a role in overlaying the physical space with information? What’s special about AR compared to other forms of Augmented Space? that in AR. you have only discussed the augmentation of space. I am doubtful whether our experience of a space is affected by this kind of information. Edwin (my supervisor) and I have recently given this topic a lot of thought and we were fascinated by the questions: “What is actually augmented in Augmented Reality? What else can (we imagine to) be augmented?” We came up with the answer. 20 . What is augmented depends on what the additional content relates to. pattern of work. but I don’t consider it an augmentation of space. the information and my spatial surroundings aren’t perceived as a single gestalt.

Is this a fundamental change in visual culture? AR & the quest for realism The quest for realism in computer graphics New Media One of the main questions I want to ask you is: What makes Augmented Reality special? I have posed that question with respect to other forms of augmented space. you relate different forms of new media — e. You note that new technological developments illustrate how unrealistic the previous existing images were. It’s the same physical space. a virtual bird is sitting on a real tree) there is no second space. AR technologies and new media works that work with AR. we find examples that date back centuries. It uses a second room. I’d like to ask it again with respect to the history of new media. what’s special about it and what isn’t? is something that has always bored me. But even if the concept isn’t new. information and space might be related. I understood that throughout media history. when we consider the concept of AR.g. Virtual Reality. On the other hand. there are more and more so-called AR applications. At the same time they remind us that current images will also be superseded. Will photorealism be traded in for a form of realism that encompasses all senses? Do you think new media will develop towards a more multimodal form? AR & the second space From The Language of New Media. taste like them and behave like them. websites and CD-ROMs — to cinema. Often. An example of ancient AR is the Pepper’s Ghost trick (which is discussed by Maarten Lamers on page 24). disappear or morph into each other in an otherwise real. the virtual leaves the realm of virtual space and enters our real physical environments — with respect to that the images might be experienced as more realistic than ever… Will AR take the quest for realism to a new level? I can imagine. this function of the screen applies AR & cinema In The Language of New Media. However. glass and special lighting in order to let objects seem to appear. I was wondering: How does AR fit in the widespread aspiration towards realism? On the one hand. For example. even when they don’t add up to one phenomenological gestalt. When we imagine a typical AR scenario in which virtual objects are integrated into a real scene (e. physical environment. Of course. the virtual things that appear to exist in our physical space should not only look like real things — ideally they also feel like them. How about the relation between AR and cinema? 21 . which contains both virtual and real elements. So some questions I’d like you to answer with respect to Augmented Space are: When are information and space perceived independently from each other — would you still call these occurrences Augmented Space? When are information and space perceived as separate but related layers? And when and why do they add up to one single gestalt? equally to renaissance paintings and to modern computer displays.g.I think the same is true for Augmented Space. smell like them. current manifestations of AR might still bring something new and special to the table. The 3D models that are usually integrated in real space don't come close to the level of photorealism we know from cinema. the screen was used to separate two absolutely different spaces. Personally. when striving for realism. I don’t think of AR as a recent phenomenon. If we look at contemporary AR and compare that with other forms of new media. visual AR could be considered a huge step back.

The MIT press.. Do we have montage in AR as well? (You give the example of montage within a shot. Are there certain aspects of Augmented Reality you consider especially interesting for future experiments and explorations? AR & future research References 22 .) What is the interface to access the database? Is it my phone? What does the artist create? I think it is usually the virtual content and its re¢¢ ¢¢ Manovich. We can see this as a database filled with virtual objects.) Does visual AR use similar concepts as cinema? Does cinema use other techniques to create fictional realities that are not (yet) used in AR? Does AR use techniques that might be adapted by cinema in the future? lationship to something real. Manovich. all new media works are databases. artists and designers create a database for an existing interface? I have one more question about databases.I’m certainly not a cinema expert. AR research is very concerned with registering virtual objects in real space.g. where an image of a dream appears over a man’s sleeping head. The database might hold a virtual chair.) AR as spatialized databases With The Language of New Media. 219-240. provide a theory of new media. distributing the elements in space. a virtual pen and a virtual painting. we could say the real world serves as a database index for those virtual elements. montage aims to create visual. semantic. Visual Communication. (Technically speaking. The poetics of augmented space. L. they will be inevitably ordered. when creating a new media work. e. for storytelling? As you point out. Do you agree? You oppose compositing to montage: while compositing aims to blend different elements into a single gestalt. You argue that what artists or designers do. (2001). (2006). In The Language of New Media you write about the elements of a database: “If the elements exist in one dimension (time of a film. L. Could we say that when working with AR. 5(2). virtual elements are distributed in real space. So the only way to create a pure database is to spatialise it. but I guess most of what we see in visual AR has been present in cinema for a long time.. and emotional dissonance between them.” In AR. we cannot assume that elements will form a narrative when they are accessed in an arbitrary order. For example. Can we understand this as a pure database? What are the consequences of working with spatialized elements? What are the inherent limitations and possibilities when working with this form? (I can imagine it has consequences. Let’s apply this database theory to a typical AR scenario in which virtual objects (seem to) appear in a real environment. stylistic. you did not only One of the main claims in The Language of New Media is that at their basis. The same could easily be done in AR. is constructing an interface to such a database. this can be seen as an analogy to compositing in films: an attempt to blend the virtual and the real into a seamless whole ‘augmented’ reality. you also pointed your readers towards aspects of new media that were still relatively unexplored at that time and you suggested directions for practical experimentation. The language of new media. As far as I understand it. list on a page). These virtual objects are displayed as part of a real room when a user views the augmented environment with a smartphone.

distributing the elements in space.] the only way to create a pure database is to spatialise it...“[.” Lev Manovich. The Language of New Media 23 Image COURTESY OF LEV MAnOVICh .

I was most impressed by the augmented reality of Pepper’s Ghost... 24 .. by 26 years! In 1978 my parents took me and my brother to Disneyland. Besides meeting Mickey Mouse. I lied. I mentioned how the “Mozzies” mobile game of 2004 was my earliest encounter with augmented reality.Disneyla nd. LAMERS In the first issue of this magazine.. 1978 n Valentij rten a a M PRE-DIGITAL AUGMENTED REALITY MAARTEN H. With no computer required.

in current head-up see-through displays Pepper’s Ghost technology is less apparent but nonetheless used in the same fashion. facing a large mirror.net lboUrke rke.com Sitting beside my brother. He explained something about glass. digital technology lets us define. But good old-fashioned Pepper’s Ghost projection is what augments our reality with that content. 25 Image of coUrtesy . and help me put AR in perspective. It is still used in amuse- ment parks and museums today. I asked my dad how the illusion worked. Disney augmented the reality that we hold our mirrored image to be: pre-digital augmented reality. facing each other. To me it is interesting that we still rely on John Pepper’s idea to add digital content to our optical reality.microsoft. the ghost wasn’t really between us. Naturally. Actually. Scary stuff. if you know other pre-digital augmented reality techniques. Pepper’s Ghost technique was first described in the 16th century and later refined by John Pepper around 1860. Thank God. Who would have guessed that such basic illusionary tricks are crucial to what we now consider cutting-edge technology? In fact.Image coU rtesy of D avid http Llewellyn ://ravenmanor . rotated 90 degrees to the right.nl. most realistically. The mirror showed us some weird illusion. Combining this with a large mirror. its use is apparent. of which I can remember only one thing: the car stopped. but also as part of modern optical see-through AR technology. In effect. render and interact with virtual content. darkness and reflecting light. send a short description to lamers@liacs.com/projects/holodesk/) project. In Microsoft Research’s HoloDesk (see research. paU PaUl BoU . a mechanical funhouse car drove us through Disney’s “Haunted Mansion” ride. and on the seat between us appeared the scariest moving ghost ever! Instantly our heads turned. However. he explained what is known as Pepper’s Ghost: a simple but clever technique that creates holographic scenes in 2 or 3 dimensions. .

Galvanic Vestibular Stimulation (GVS) stimulates the human vestibular system (balance organ) by means of electric stimulation.ImAGE BY KRISTIn SmITh VESTIBULAR STImULATIOn AS An AUGmEnTED REALITY LAYER? Antal RUhl Over the past few years the field of Augmented Reality has started to include non-visual forms of augmentation to their scope. I constructed a device and developed a series of experiments. but never has it been used as an extra sensory input. In order to use GVS as a form of augmented reality. are placed on the mastoids (the bones behind the ear). usually an anode and a cathode. Two electrodes. This technique has been around for a long time and many studies have been done in order to learn more about the human balance system. I needed a connection to the physical world. Applying a specific current to those mastoids will result in an altered balance perception related to both to the intensity and the direction of the current. 26 . To find out whether GVS can be used as a new form of AR. I used my body orientation to alter my balance. Since I also wanted to investigate what the effect would be to our balance system. When I became interested in galvanic vestibular stimulation I realized that this could become a whole new form of AR.

it would push me back to the center. The control unit calculates the appropriate intensity and direction of the current in real-time and provides feedback to the electrodes. The headphones are in fact merely rings around the ears. it had the opposite effect. In other words. There were two possible modi for the control unit. I realized that. it felt like I was moving through a liquid or a thick syrup-like medium. In modus B. This provided me with the setup as shown in Figure 1. Is this a form of augmented reality or is it an alter- Conditions The first step of my research project was measuring the effect of an altered balance. and to make sure this wouldn’t influence the results. when put in modus A. Testing this simple setup immediately revealed some potential AR applications of vestibular stimulation. it felt like my resistance was really low since the device backed up every movement I made. Without even using external data to augment an extra layer to reality I created an altered world based on data from the physical world.GALVANIC VESTIBULAR STIMULATION CAN BE USED TO ALTER BALANCE PERCEPTION Setup To do so. based on my 27 . The data from the accelerometer is sent to the control unit. Modus A could counteract my balance. own orientation. if I was tilted to the left in modus A. so it took more effort to move around. While testing. it would push me further to the left. The electrodes are incorporated in a pair of headphones to make sure they are pressed against the mastoids properly. in modus B. The middle part is left uncovered to retain the full use of users’ auditive orientation. I mounted an accelerometer on top of my head-mounted GVS device in order to measure its orientation. The GVS device counteracted all my movements. while modus B could amplify it.

But a more practical solution must be found. Can this be used as an interface? I demonstrated a system in which the real world environment. This creates a broad range of possibilities. there are some practical considerations to keep in mind. I experienced some burn marks of the skin around the mastoids. Another. modified and used as an input to alter the actuators (in this case the electrodes which in turn alter my own balance). playing Wii Balance Board games and many other examples. The skin’s resistance is an important factor in getting the electric pulses down to the desired area. Using my self-constructed device. and nausea. for now. Or simulate the G-forces in a racing game’s sharp corners. before we can use this. I was wondering whether I can diminish or amplify motion sickness. it does fall within the definition of mediated reality. cumstances my balance actually improved while using this device. I have used this device for over two weeks to see if there was any progress. For example. could we use this system to add computer generated data to an extra sensory layer of our environment? The easy answer is: Yes! It’s not hard to imagine that you can enrich game play with wave motions for example. walking to targets on the street. This might be of use for people with an impaired balance system or in those situations when accurate balance is crucial. we can already create an enriched environment. But can we also use it in a more traditional AR environment? In other words. During the testing phase. This system could enhance a simulated environment by distorting the balance organ based on the simulators’ virtual orientation. which is the broader field of manipulating users’ sensory perception through a wearable device. improve my balancing skills etc. which could be used to train divers or astronauts who work in other environments. (in this case body orientation. both are not very userfriendly. perhaps more Figure 1: Galvanic VestibUlar StimUlation device troublesome. But you can also think about fighter pilots who fly in simulators that don’t alter pilots’ balance when flying upside down. Using electrode gel and covering the electrodes with sponges soaked in a salt-water solution did the trick. experience I had was the adapta- 28 . Tests involved bus rides. Using only these two simple orientationbased modi. However. Although we have to be very careful about drawing conclusions from these results (given the self-experimentation and singlesubject constraints). Other issues were the bright light flashes I experienced while testing.The tests that I have done focused on enhanced performances in everyday life situations. and letting them experience their surroundings in a modified fashion. I found that in certain cirnate reality? Be that as it may.) is measured. but for what use? The examples point towards simulations of the physical world. walking over a balance beam. But these issues might all be overcome with the right execution.

festivals and event. After removing the goggles. which are very important design-considerations if we want to use a GVS device as an interface. when I was wearing the device while it was set in modus A. You might remember that study from the 1960’s where a test subject was wearing reversal goggles.pdf Antal Ruhl Antal Ruhl is a media artist with a background in design. but they always share the goal to intrigue people. giving off the impression of moving through a low friction environment.antalruhl. After removing the GVS device I experienced the exact opposite. But when I used an alternating pulse. When I used a constant pulse on the electrodes (a constant current in the same direction) the effect diminished quite soon. Sometimes they are purely conceptual or formal and sometimes they serve a more commercial purpose. the subject saw the world upside down because the body had adapted to this new situation. This article is based on research presented at Chi Sparks 2011 (Chi Nederland 15th conference).com 29 . Antal has also started a company to develop interactive installation for brands. While using GVS for augmented reality purposes is still in a research stage. Creative in Motion is a creative brand activation studio: www. Antal studied Industrial Design at the The Hague University of Applied Sciences. After a few days the brain adapted to the newly displayed environment and reversed the image back to normal. it felt like I had switched it to modus B. The presented paper. Your body is extremely good at adapting to new situations and environments. So when I turned it off. June 23. Media Technology at Leiden University and Design and Media Arts at UCLA.antalruhl. I saw that.nl For an overview of his work or to contact Antal please visit: www. Secondly. when I was wearing the device for about fifteen minutes. it did so quite convincingly. Los Angeles. These objects vary from kinetic sculptures to interactive installations. The same effect occurred with my balance organ. the effect was constant. Antal worked at distinguished design companies in Amsterdam and Barcelona. travelled the world and currently works a freelance artist/ designer. While my body needed some time to get used to my ‘new and improved’ balance. can be found here: www. but at a much quicker pace. the possibilities are endless if you use your imagination. it gave me the experience as if I was moving through a liquid. Apparently I was much more sensitive to a change in stimulation than to the actual stimulation itself.creativeinmotion. Arnhem.com/media_tech/paper. And it did so in two ways. The Netherlands. science and art. has had several (national and international) performances and exhibition. published a scientific paper. thus. Antal creates objects that let us rethink our environment. Interactive and playful objects are much better in holding someone´s attention while they convey their message”. 2011. and only changed the pulse-width to affect my balance. and removed it after that time. my body had to adapt to this ‘new’ non-stimulated situation. His work can be described as visually and technically attractive with a focus on natural and physical phenomena.tion effect. Experiments with Galvanic Vestibular Stimulation in Daily Activities: An Informal Look into its Suitability for HCI. “Using the technological possibilities at hand we can enrich our work and create an engaging object.

and we enjoy it.BELIEVABILITY Edwin van der Heide Believability is something we deal with on a continuous basis. The story is believable. while not real. it pretends to be real. independent from something being real or ‘virtual’. When we receive an e-mail that claims to be from our bank with the request to enter our account information on ‘their’ website in order to upgrade its security. We’ve learned to get suspicious and need to verify the credibility of the story. What we also learn from 30 . An important factor for a story to be believable is that we can relate ourselves to it (or the story relates to us). we enter a situation in which the content appears to be believable. The examples from the bank’s e-mail as well as the novel show us that believability is actually independent from something being real or fake. The e-mail is believable. the story can be entirely fictional and it nevertheless draws us in without questioning its realness or truthfulness. but is in fact fake. When we read a novel. appears to be authentic. or in other words.

I realized that the believability (of the behavior) of an artwork was not (yet) seen as a fundamental topic and might deserve its own study and experimentation. The reason that it fascinated me is that it made me not only think about believability in the context of robots. more specifically. In a novel we can meet creatures from Mars. We can ‘simply’ imagine these things and believe them.Spatial SoUnds (100dB at 100km/h) at Wood Street Galeries. but it also triggered me to think about the believability of an artwork and. 31 . the believability of the behavior of an interactive artwork (as opposed to the believability of a robot’s behavior). USA.Image coUrtesy StUdio Edwin van der Heide the example of the novel is that things that happen in the story don’t have to be possible in reality. or never go to bed. When I was writing my paper about the interactive art installation Spatial Sounds (100dB at 100km/h) for the Third International conference on HumanRobot Personal Relationships (HRPR) I was introduced to a topic addressed by Kerstin Dautenhahn (2007) that fascinated me: The believability of a Robot. PittsbUrgh. 2009 . time-travel.

for example.e. and if so.” I tried to imagine an example and came up with the idea of a robot that makes and brings you a cup of coffee. i. it has to be useful and perform tasks around the house. but it also has to ‘do the things right’. Does this then mean that the representation is a requirement for believability? No. Robots are often made to look and behave like humans (the humanoid) and the coffee machine isn’t. it allows us to somehow identify ourselves with it.e. In the HRPR paper about Spatial Sounds (100dB at 100km/h) I put it like this: “The installation can be seen as a non-verbal abstract robot and does not imitate an animal or human-like look or behavior. We imagine that the robot has a certain amount of intelligence and. Soon after I had that idea I had to think about a full automatic coffee machine that grinds the beans and steams fresh milk for each individual cup of coffee that it makes. It can be an abstract work that is believable on its own. welcome and/ or thank the user for using it. It’s by imagining that we can change what we believe and thereby turn a machine into a believable affective robot. a robot (or interactive installation) doesn’t have to represent something (else). imagine that the coffee machine made that coffee especially for us things start to change. i. we don’t imagine any intelligence and we simply think of it as a machine without affection for us. It is interesting to question whether these machines indeed make us believe they show affection. Spatial Sounds (100dB at 100km/h) is an example of a believable robot in the sense that the visitors believe they 32 . if it lasts or wears off.Kerstin Dautenhahn has an interest in socially intelligent robots: “A robot companion in a home environment needs to ‘do the right things’. even ‘feels’ affection for us. There is another possible difference between the coffee machine and the robot making coffee. Nevertheless. In the case of the coffee machine. in a manner that is believable and acceptable to humans. This might be the reason that in Japan a lot of machines talk or include animations to. How do the two differ from each other? With the robot we can imagine that he read our mind and therefore made a coffee for us. If we. perhaps. however. It is a machine-like object but does not resemble existing machines.

Verbeek F. once we discover they are fake. 2011). Socially intelligent robots: dimensions of human–robot interaction. We have also learned that something doesn't have to be possible in order to be believable. A. In: Lamers M. Japan. 59. 362(1480).. while we keep on believing other things that we know are fake. The aspect of believability is so strong that people accept the installation as a real being and want to interact with it over and over” (van der Heide.understand the behavior of the installation and find it worthwhile to interact with. ¢¢ de Rooij. 27-33. J. Philosophical Transactions of the Royal Society B: Biological Sciences. Abstract Affective Robotics. I’m curious how the abovementioned thoughts apply to augmented realities. E. (2010). What aspects make combinations of.edu/ research/theses/abstract-affective-robotics Spatial SoUnds (100dB at 100km/h) at DAF Tokyo. H. Human-Robot Personal Relationships (HRPR 2010). (2007). ¢¢ van der Heide.. Besides that. (2011). the real and the virtual in augmented reality believable? We have learned that something doesn’t have to be true to be believable. we’ve learned that something doesn’t have to represent something. LNICST Vol. Spatial Sounds (100dB at 100km/h) in the Context of Human Robot Personal Relationships. and interactions between. The former makes me believe that believability forms an interesting perspective to think about what we can imagine in augmented realities! There are certain things we believe in that. completely lose their believability. http://mediatechnology. 679-704. Interesting to read in this context is Alwin de Rooij’s (2010) graduation research project for the Media Technology Master program on ‘Abstract Affective Robotics’.leiden. References ¢¢ Dautenhahn. K. it can be abstract and nevertheless be believable. 2006 — Image coUrtesy StUdio Edwin van der Heide 33 .

the so-called ‘smart replica’ was proposed in the previous issue of Ar[t] (Roozenburg. These are currently on display in a new exhibition on design and pre-industrial design. that stretches the boundaries of the replica’s concept as an autonomous object based on a historical artefact. let alone use them. and ending with the small scale production of porcelain in the Netherlands. something that needs reconsideration in our present time where the ‘experience’ is essential. To provide a way around this situation. communicate and enhance those qualities of the historical artefact that are the most meaningful to us. the goal is not to make the most realistic copy of the original. We hypothesize that by using these 3D imaging techniques the value of our cultural heritage can be increased. Starting with the first import of porcelain from China in the seventeenth century by the Dutch East Indies Company. As a result. 2012): a new kind of reproduction. 34 . in the shape of a 3D print. Maaike RoozenbUrg Introduction In museum exhibitions historical objects are usually shown by visual display. Here we present the design project of Lotte de Reus in connection with this paradigm shift. in a showcase with extra textual information added to it. Completed as a graduation project. now. Depicted in figure 1. The objects that are central in this project are seven teacups and saucers that are part of the collection of Museum Boijmans Van Beuningen. but to analyse. New methods of access and new digitization strategies based on the study of the relationship between the ‘bits’ and ‘atoms’ are being developed.NOnLInEAR STORIES TOLD BY CUpS AnD SAUCERS Smart Replic as with responsive 3D aUdio Lotte de ReUs. it presents an auditory environment to augment the artefact in an unobtrusive and non-linear way. followed by the invention of Delft’s blue as an attempt to copy Chinese porcelain. visitors ‘scan’ the displayed objects from a distance. JoUke Verlinden. Museum visitors can never touch the objects. each of these teacup and saucer sets represents a milestone in the Dutch history of porcelain. In other words.

while the information passively waits for the visitor act upon it. These types of encounters could correspond with respectively a museum or a theme park. The participants walked into a room. looked at an object briefly. 35 . This storytelling model categorises the information in three layers: engagement. thereby increasing the opportunity to reflect and learn even after the visit. In the project discussed here. a focus group that visited the museum exactly proved this problem case in hand. the exhibitions of Boijmans Van Beuningen can be characterised as enriching experiences: an encounter in which the visitor is conscious of the artefact and the (hi)story it represents. –Visitors’ overall interaction with Boijmans’ exhibitions can mainly be described as scanning. it takes effort on the part of the visitors to maintain concentration. restaurants and so forth. and integrity. This means that a museum requires the visitor to reach out for information. Concept of tangible interaction with responsive audio For the seven teacups and saucers that have been chosen to represent the history of the Dutch quest for porcelain. Knowledge and culture become more accessible because of the actual presence of the objects. THE SEVEN CUPS AND SAUCERS picture by HUGO BES In the spectrum of recreational activities. The artefacts shown and experienced in an exhibition construct a powerful medium which instils visitors with thoughts they perhaps have not yet fostered. leaving the exhibition behind. These qualities feed the resulting research questions: How can museums anticipate and facili- Figure 1. went on to the next. 2012). In the vision of the curator. For this design project our aim was to convey the following qualities of interaction: intrigue. context and reference. The engagement of visitors is limited as they are not experts on the particular subject of the exhibition. rituals. Augmented matter — the mixture between bits and atoms — allows novel interaction techniques to embody these networks of information. locations. paying a visit to an exhibition enables the visitor to have an intimate encounter with the artefacts on display (van Dongen.Active enrichment but not entertainment? Museums and their collections house a specific group of objects with their corresponding historical data. and tate the active assemblage of old and new stories – and how do these stories refer back to the replica’s original? How can digital databases be employed in linking smart replicas to their collections? On a philosophical level: does the original still attract interest? In the case of pre-industrial utensils such as the teacups and saucers this question is very relevant. understanding. Because of the passive character of the objects and their corresponding information. consisting of stories. An important part of the collection are the nonlinear narratives. started at a random showcase. including museum shops. 2012). as shown in Table 1. the ECR-model is used to classify the valuable information (Wong. the collection’s past and the like. They draw audiences in through exhibitions and related facilities. networks of information associated with the object. full of untold stories. Our hypothesis is that more active means will lead to a more comprehensive museum experience. satisfaction. materials.

When the visitor returns home. e. Interaction characteristics Spatialised audio. Information plaque. Triggered by handling an object. In this stage the visitor is encouraged to turn and explore the object. Identification data (text. Information a. Proposed dialogUe between visitor and exhibition. d. The app contains more background information on porcelain and the objects. Enabling spatial audio As discussed in Table 1. the moment when a visitor has some immediate sensory. Triggered by proximity to object. c. When the visitor is in the proximity of an object. every teacup and saucer holds a series of three audio clips. Exhibition app. Background information (text and images). Together with an app that is made available.ECR_Phase Engagement: Draw the visitors’ attention. Table 1. Context: Draw the visitors’ attention. a 3D ambient soundscape that fits in with the history of the specific teacup and saucer will appear — once the first soundscape has been heard. The narrator tells about the role the specific objects (“audio story”). Close proximity to object. Reference: Gives the visitor the opportunity to draw conclusions and connect to related resources. b. When the visitor picks up a replica. Ambient soundscape (audio). The audio clips are played when the visitor shows interest through their actions. 2012). the visitor is intrigued and triggered to participate in the exhibition.  Narration 2 — information on specifics that can directly be related to the object (audio). emotional or intellectual response to the artefact. Narration 1. These audio clips give the impression that they are experiencing more: it is as if their auditory attention has doubled their visual attention (Erens. emotional or intellectual response to the artefact. depending on user location / viewing angle. visitors are attracted to participate in the experience by other visitors who are listening to the 3D audio clips. it attracts the visitor to go to other objects accompanied by a spatial soundscape. to browse the additional information for example. It provides more detailed and interpretive information about the work. The visitor can put down an object and pick up another object. while retaining the soundscape. available after the exhibition. the narra- tor focuses on the features of the teacup or saucer (“audio specifics”). connecting the narrative with what is visualised about the object. visual). information on the role (audio). the moment when a visitor has some immediate sensory. proximity to an object and the han- 36 . In the first phase. Arriving in close proximity of the teacup and saucer. the spatial audio of a narrator starts. she can consult the app at her convenience.

Presenting the information via responsive audio facilitated the visitors to consciously turn the object to find the image that the narrator referred to. whilst the visitor is holding the object. non-visual immersive content. 140 people interacted with a selection of the configurations presented in pairs. Presenting the information via audio. 2012). SPATIAL AUDIO INTERFACE OF AUDIOSTAGE. even those visitors who preferred text to audio clips. With the use of tangible replicas and 3D audio clips. cf. An example of a high quality 3D audio clip is “In your head” by Big Orange. When listening through headphones. see Burgess and Verlinden (1993) for example. The soundscape is the auditory equivalent of an ambient image. Participants appeared to be pleased with the responsive auditory system. The character of 3D audio relates to the idea that the sounds seem to come from sound sources placed anywhere in a space (a surround sound effect). Information about the teacups and saucers was presented via text or audio clips. audio clips allow temporal cues.dling of an object. 37 . Furthermore. Preliminary evaluation The core of the concept was tested during the Object design fair (Rotterdam. does not clutter the visual sense. Figure 2. To connect this aspect to the exhibition visitor. Ambient soundscapes were on or off. 7-10 February 2013). Here optical or magnetic tracking principles also make sense. The essential observations include: Participants seem to enjoy the ambient soundscape: it triggers the imagination and the recognition that the objects used to be utensils and not art object as they are now. typically with so-called earcons which represent specific objects or events. scanning behaviour can be transformed into an immersive encounter. The technology to spatialise audio was developed two decades ago. We used AudioStage to produce our audio clips with a visual interface. and speaks directly to the visitor (Erens. yielding four permutations. Auditory feedback enriches the visitors’ experience actively. they were asked to choose between the two displays and support their reasoning. the brain places the sources of the audio clip in your head. as they are fit for indoor use. Figure 2. it works more associatively. position and orientation tracking of the human head as well as the objects at hand is required. without visual clutter and conserves traditional values on how art should be experienced.

Impression of the preliminary evalUation. one-sided encounter with the objects now becomes an active two-sided encounter. texture. it is more or less compatible with existing gear already employed by many museums. moulded physical replicas afford what Dutch historian Johan Huizinga has called a “historical sensation”. weight. Through the app. Combining ambient soundscapes. understanding is nurtured because the encounter is intuitive and the information presented by the artefact responds to the visitor’s body language. triggers an immersive encounter in which this sensibility is restored: the passive. the experience is saved. Future work includes experimenting and researching the effect of the design in the environment of a museum: Figure 3. this poses an intriguing quality that pulls the visitor in by using 3D audio clips and ambient soundscapes. the feeling as though you are somehow “in touch” with the past (Ankersmit. by the layered structure of the narratives.  Experimentation with augmenting untouchable artwork with ambient soundscapes. as well as the people who used them. a satisfactorily quality is propelled. 38 . colour. the experience fits in with the integrity values of the museum Boijmans Van Beuningen – the 3D scanned and printed. it rather extends the existing strengths with emerging technologies such as indoor tracking and spatialised soundscapes. Secondly.Conclusion Once the teacups and saucers were objects of dailylife and their form. and can be connected to various forms of social networking websites and location-based services. Lastly. will lead to a more meaningful experience. this 1. The proposed system is by no means the first auditory guide for exhibits. In essence this proposal presents a new type of relationship between visitor and object that has interaction qualities equal to a human conversation. substance. Firstly. Because the visitor has a direct intellectual sensory dialogue with the object. Furthermore. which can be browsed in a non-linear mode by the visitor. decoration make sense primarily in the context of their functions and relations to other objects. Thirdly. tangible interaction with physical replicas and the connection between information and corresponding visuals. creating curiosity and making visitors want to engage with the artefacts.  Implementation of responsive audio tours in the current exhibition context. 2005). 2.

¢¢ Ministerie van Onderwijs. Lotte de Reus Lotte de Reus recently received her Master’s degree in Design for Interaction at the Faculty of Industrial Design Engineering at Delft University of Technology. ¢¢ The Futures Channel. (1993). Dr. In July 2012 she started her graduation project for the Smart Replicas project. we would like to acknowledge the valuable advice and support by Cilia Erens. curator at the Museum Boijmans Van Beuningen for her constructive collaboration. Rotterdam: Erasmus Centrum voor Kunst. fr/web/en/prods/audiostage ¢¢ Interactive installations regarding preindustrial utensils: http://new. Proceedings of Virtual Reality Systems Conference (Fall ‘93). ¢¢ Erens. M. Interviewed by Lotte de Reus. J. Burgess. 4. Bewaren om teweeg te brengen. Interview on soundscapes. More information ¢¢ Video presentation of this project (headphones required): http://www. Themaparken. An architecture for spatial audio servers. Cultuur en Wetenschap (2005). Wim Verwaal. 21 November 2012.php ¢¢ Van Dongen. dierentuinen en musea.  Creating guidelines on how the concept could be implemented to suit different kinds of objects in the museum (or maybe even utensils in our everyday context). (2012).com/ watch?v=enR1Ggbuf_8 ¢¢ Software to render binaural output by visually placing audioclips in 3D: http://www.C. F. Wolf Song and Ir. Amsterdam. Interviewed by Lotte de Reus.lottedereus.nl Official Smart Replica blog: http://smartreplicas. A. In the future.R.youtube. immersive soundscapes and 3D audio now hold a new. Woerden: Drukkerij Zuidam. Sublime historical experience. she was driven by a fascination for porcelain and the wish to create effective storytelling experiences. Furthermore. [online] Available at: http:// www.3. Lotte would again like to work in the domains that combine art and technology.en Cultuurwetenschappen. (1995). References ¢¢ ¢¢ Ankersmit. Her portfolio can be found at www. Professor Joris Dik. C. Museale strategie. D. special interest for her. Rotterdam. something she had not yet encountered in her studies. Interview on the museum context. with special attention to smartphone infrastructure. com/2008/05/new-work-detroit-institute-of-1 ¢¢ ¢¢ In your head. a high quality 3D audio example: http://www. ¢¢ Halbertsma. New technologies such as augmented audio. (2005). Many thanks to DOEN foundation for funding the Smart Replicas project (including the graduation project of Lotte de Reus) and many thanks to Mareco prototyping for their contribution in the form of 3D micro prints.big-orange. Stanford University Press.longcat.nl Acknowledgements We would like to thank Alexandra van Dongen. Conversation with Curtis Wong. 14 December.com/conversations_ archive/wong_conversation.blogspot..pentagram. Verlinden.nl.  Prototyping indoor tracking and interaction sensing possibilities. 39 . (2012). A.thefutureschannel. (1999).. During the project she was pleasantly surprised by the world of audio.

GUIDO LAGERWEIJ 40 40 .Image: REVEALInG IDEnTITY.

everybody has the same access to virtual content. The only thing you need to do 41 . For some this might be a bit frustrating. Although.678 virtual cubes whose colours can be changed by simply tapping on a mobile phone. this is not the case. made a lot of people aware that AR could be everywhere. seeing his art work Biggar and playing with it for the first time.463. the blog Wassom. de facto. helps them to realize that this AR art piece encapsulates the entire earth with its 7. giving off the impression as if people are gossiping about them in a different language or at a very low tone.and artistic purposes. unsure whether or not they are they are talking about you… Sander Veenhof.com. we don’t have the means with which to recognize the virtual content floating in the air around us. using information from. amongst other sources. In AR[t] 2.WhO OWnS ThE SpACE 2 discUssing legal issUes of AR Yolande Kolstee WHAT IS PRESENT IN AR SPACE? As mentioned in ‘Who owns the space’ in AR[t] 2. we keep a close look on the developments concerning the invasion of our physical space with virtual information. I contended that we as passers-by might not be aware of any virtual content at all and. Dutch AR artist. and actually is everywhere and that this is a special characteristic of AR for as a new medium for artists. For many people.185. and discuss some legal aspects thereof. when we don’t have the right app (software) and/or mobile (hardware). in theory.

Natural feature tracking systems are in rapid development. without any prior screening or approval from an examination committee.com was developed and released (under a Creative Commons license) by a new London-based group called “Stop the Cyborgs.000 billboards with Augmented Reality markers throughout many cities in the Netherlands. into the virtual world. what happens when this information about our eye-movements is stored in databases. again. Real immersive AR depends on knowing exactly what our eyes are looking at. Seattle’s 5 Point Cafe became the  first known establishment to publicly ban Google Glass. to AR patents. detect the salient points. let our camera survey the space. predicted five issues related to AR Law for 2012. inserting a lot of information from the physical world. Since most of the public are not familiar with AR marketing activities like this. but for him it was more a matter of time than a matter of substantial incorrect prediction. analysed and even sold to marketing companies. it floats above holy places in Rome or in the Himalayas. entering information about the physical world (‘burglary information’ as we called this in the last issue of AR[t]) into the virtual world. TWO WAY TRAFFIC For 2013. on the other hand. it will have a significant informative impact. and. the group raises a significant concern: namely. However. The “No Glass” logo that the café published on its website http://the5pointcafe. Using the eye-movements of the person wearing AR glasses gives extra accuracy. and the girls will come to life allowing you to watch them in a video right there on the street. What is at stake is.to actually see Biggar. This virtual sculpture has true omnipresence. although it never manages to become the Actual Current Big Thing”. On the Stop the Cyborgs site.” The group is composed of three young Londoners who decided to make a public case against Google Glass and other similar devices.2013. this information is used to position the virtual content there where our eyes are looking at.5 out of 5. we see a concern regarding the possibility to record without the public knowing this. above warzones and above peaceful places… Brian Wassom. and negligence. 42 42 PREVALEnCE OF AR John Moe (the host of Marketplace Tech Report and handles web content for the program). Concerns raised range from on the one hand. the app distributed via Layar works by pointing your smartphone at one of these posters. some predictions were made and trends were spotted. When downloaded. we can enter a room. and make a virtual grid on which we position our virtual objects or scenes. is to download this app via Layar. blogger and specialist in AR Law. WhAT IS RECORDED ? With the announced introduction of Google glass. On March 11. “Stop the Cyborgs!” Mr Wassom recognized he was not right in all his predictions. ranging from public resistance to ‘adult’ AR (AR porn that is) via licensing. the highly anticipated augmented reality device set to be released later this year. This adult content will certainly appeal to a lot of people and this will definitely result in another large group of people coming into contact with AR. EYE-TRACKING Another type of warning from Mr Wassom is related to the information derived from tracking our eye-movements to detect were we are looking at. maybe voluntarily via players of AR games. that there’s no obvious way to know when the device is on or what it’s actually doing (recording or not). wrote in 2011 that “Augmented Reality has been the Next Big Thing for a while now. . scoring 1. in February 2013 an adult magazine has distributed around 1. as well as above industrial places.

we have privacy laws to handle this.org FALSE STAYS FALSE Recording without the recorded person knowing this. We might come to the preliminary insight that as for now.com  Critical site on wearable technology: http://stopthecyborgs. regardless whether it is posted on Twitter. An incorrect accusation or incorrect information remains incorrect. To add correct or incorrect virtual information to a virtual object or a space. our current laws might be sufficient. “But if the Google glass will become prevalent” as one the people from ‘Stop the Cyborg’ argues in a conversation with a journalist of arstechnica. which has been working in the augmented reality field since 2006.snapsapp. Facebook or in a space around us. For the time being. we have to change those laws.Images coUrtesy of GoldRUn. see www. object or space in our physical world. our actual privacy laws are sufficient enough to deal with privacy concerns in augmented reality. it’s about the social culture around it”. 43 43 . The intertwinement of AR in our day-to-day lives is actually quite slow — especially compared to the revolutionary predictions. and will be considered as such. However. we don’t need special AR criminal legislation.com and www. It’s not about the tech. FURThER READInG  Critical blog on law and social media: http://wassom.goldrUnner2013.com This quote gives us a direction for discussing the abovementioned AR related legal issues. even closely related people within our Academy. As an example. is possible with our smartphones and is widely spread among youtube videos. We are getting used to it at a nice pace. the concept of AR is completely new.and if those laws are not adequate enough. com “and suddenly everyone is wearing it and this becomes as prevalent as smartphones—you can see it becomes very intrusive very quickly. I can take our AR Lab. is as simple as assigning correct or incorrect information to any subject. via an AR app. and still for some.

OLIVER PERCIVALL. ThE AUGmEnTED STAR. ILLUSTRATOR: PAUL HADCOCK 44 . BOOK COVER.

I DISCOVERED ITS MAIN USES WERE AS A NAVIGATION TOOL USED IN CITY CENTERS. PART ONE FOLLOWS OUR ACCIDENTAL EXPLORER LYNDEN AS HE POSTS HIS ADVENTURES TO AN ONLINE FORUM. POINTING PEOPLE FOR EXAMPLE TO PLACES OF INTEREST OR NEAREST TUBE STATIONS. 45 . WHAT IF THERE WAS SOME AUGMENTED REALITY TEXT MOST OF US WERE NOT SUPPOSED TO SEE AND YOU HAD A DEVICE THAT WAS ABLE TO READ IT… WOULD YOU FOLLOW ITS INSTRUCTIONS? THIS IDEA FASCINATED ME INTO WRITING A SERIES OF STORIES TITLED THE MISADVENTURES IN AUGMENTED REALITY. THE THOUGHT OF CONCEALED TEXTUAL MARKERS STRUCK A STORY IDEA IN MY MIND.The Misadventures in Augmented Reality OLIVER PERCIVALL I FIRST LEARNED ABOUT AUGMENTED REALITY AROUND TWO YEARS AGO.

goldrUnner2013. Now there is probably nothing I don’t know about the integrated software algorithms used in Augmented Reality. But be warned: Augmented Reality text as we know it was not developed within The Augmented Star so it makes it pretty hard to find your way out. But nearly nine years ago now he disappeared from his hometown and has never been seen since.snapsapp. Images coUrtesy of GoldRUn. But once you start exploring all the hidden signs and directions it’s impossible to stop. You’re immediately plunged into its rich diverse landscapes and to find your way around or locate anything at all you must be in possession of the special device — The VAARR or Very Advanced Augmented Reality Reader. the noise levels. And because of this I have a story to tell. shows unseen Augmented Reality text none of which is detected by any other device. Using his ingenious engineering skills he somehow created a VAARR (Very Advanced Augmented Reality Reader) It works just the same as other regular Augmented Reality apps I am led to believe but with one unique difference… It has a setting that once activated. the temperature. the shy inventor type. He could probably be best described as a recluse. 46 . I don’t think he’s lost. One was entrusted to me. My uncle made it possible to post directly online from this device. the beings.com Life in this newly discovered domain is a far cry from home. So my blog entries below are concise. It’s everything.Part One My uncle was one clever soul.com and www. It’s not easy to write long paragraphs on this VAARR. The smell. A few years ago I wasn’t exactly a technology fanatic although admittedly I had more than a passing interest in new gadgets. everything is entirely unfamiliar when you first arrive. He created two actually. There’s no one thing that makes it uniquely different. not wanting to be found is my guess and I think I know why. But that was nearly three years ago now before I set foot within the Augmented Star. see www.

I’ve come here with a strict agenda. To find my uncle. it’s about seeing the present through a unique lens. There’s just so much to get distracted by.’ That’s the first message you see when you arrive in The Augmented Star. The first post I made earlier was date stamped Apr 12 2011 but the post I made just a couple of hours after that is stamped July 12 2011. Blog Entry #2 Lynden Jul 12 2011 OK it took me a couple of hours to get acquainted here this time. From learning a new food recipe to breathtaking advertising campaigns. On the downside crossing the street can be something of a perilous activity.The Ultimate Frequency Forum This forum was created for anyone to post to that feels lost. Blog Entry #3 Lynden Oct 12 2011 Just noticed a glitch with this online forum that I would like to point out to the moderators. those that do not wish to be found or find themselves beyond the confines of earthly boundaries. It puts things in perspective when vehicles move from 0 to 600 MPH in less time than it takes to chalk a pool cue. Three months later? I think not! 47 . Every time. And you never arrive at the same place twice. Blog Entry #1 Lynden Apr 12 2011 ‘It’s not about gazing into the future. the Augmented Reality here really is something else.

I set myself the complex task of plotting my science fiction novel and created an alternative fictional world that became The Augmented Star. jsut follow the QR code on the left! 48 . I just learned that a thimbleful of a neutron star would weigh over 100 million tons. which eventually led to a career in various I. The Augmented Star is now available on The http://tiny. L. The mysterious side of technology especially intrigued me including the possibilities of where it could take us in ‘What If’ scenarios. I feel I’m ready for that science degree now! O. So equipped with a reasonable understanding of Augmented Reality and enjoying a challenge. Although I feel like I have been.T support roles leading up to Project Management. The whole place is an encyclopedia of knowledge. They promote learning a Science degree in Artificial Intelligence can be completed in three months here. After leaving school I took a basic Computer studies diploma.Blog Entry #4 Lynden Jan 12 2012 It’s funny because since I’ve been wandering the streets here the last few hours I’ve regretted never going to Cambridge University. Knowledge is fluidly brought to astonishing life everywhere by Augmented Reality.cc/augmented-star Amazon Kindle Store. It appears there is nothing you can’t learn here by waving the VAARR around. Percivall I have always had an interest in technology and gadgets. Must be something in the air here as my clothes are becoming dirty and ragged really quickly.

It feels like I’m on the run myself. I’ve gotta move quickly. I’ve come to realize now that I may never be able to leave the Augmented Star.SNAPSAPP.Blog Entry #5 Lynden Apr 12 2012 The risk of being struck by a falling meteorite for a human is one occurrence every 9300 years.COM AND WWW. SEE WWW. 49 . Wow that’s a curious fact. I wonder what the odds are for walking into an Augmented Universe like this one. The AR campaigns here really resonate with customers in a way that most other ad platforms fall miserably short. Someone is coming.GOLDRUNNER2013. I’ve met a lot of people here within The Augmented Star and most of them know my uncle but apparently he’s on the run from some bad people. Using my VAARR triggered a full Augmented Reality breakdown of its features from performance figures to finance options and then it invited you to take a virtual test drive in a car simulation game.COM Turns out my VAARR is a pretty valuable commodity here. After an exhausting and bloody battle earlier today. I saw an advert on a billboard for an automobile just now. It can read over 150 types of Augmented Reality text and other types too that alternative devices can’t. End of part one Images COURTESY OF GOLDRUN. I have learned some creatures here would even kill to own one. Blog Entry #6 Lynden Jul 12 2012 Blog Entry #7 Lynden Oct 12 2012 Just received some news that was pretty hard to comprehend.

hence taking advantage of the benefits of physical modelling methods (dexterity skills. This design-technology synchronic co-evolution has been historically aligned with the zeitgeist of the architectural practice. in which new technological developments shape the way we express ourselves. among others). unavoidably. modifiability. As a result. One of the main grips in which this topic is grounded is that the growing academic concern on the dialogs between physical and digital realms in architecture is becoming blurry.AUGmEnTED mODELLInG In ThE DESIGn STUDIO: AUGmEnTED pEDAGOGIES Alejandro Veliz Reyes University of Salford Digital physicality / Physical digitality From very early stages of the design process (sketches and diagrams) up to detailing and construction. Augmented models are those which blend a physical and a digital counterpart in a resulting synchronic manner. so far. Likewise. models are built by using several different — yet interrelated techniques. the biggest amount of published articles in the history of the eCAADe Association Conference. This recent conference had. scanning and reverse engineering methods allow a fluent interaction between models which embeds information and knowledge as the design process proceeds. this section is titled with the topic of the latest conference of the Association of Education and Research on Computer Aided Architectural Design in Europe (eCAADe) held at the Czech Technical University in Prague (September 2012): 50 . tools and techniques merge into tailored modelling flow works according to each project’s requirements. . augmented models.augmented reality now gives us the chance to build hybrid. and further evidence of this trend is the growing participation of architects and designers in professional events such as Ars Electronica. augmented reality now gives us the chance to build hybrid. such as ubiquitous computing systems or environmentally responsive technologies.. Indeed. a digital counterpart. augmented models. it also Digital physicality. since the distinction between both cannot be fully depicted in a world where digital information is continuously embedded into ‘real world’ situations. the concept of augmentation has a major role in this work. in addition to the technical definition of augmented reality. developers.. SIGGRAPH or the ISMAR community. and define new design forms and organizational/social schemes. This work claims that beyond physical or digital modelling techniques. a complex orchestration of users. manipulability) as well as digital (accuracy. Even if the semantics of augmentation suggests an ‘increment on size or amount’. Physical digitality. Also. The construction of a physical model by computercontrolled machinery requires.

TRAINING SESSION ON AUGMENTED REALITY AUTHORING AND AUGMENTED MODELLING. Actually. analysis. the architecture studio and its interactions as a subject matter is a quite complex challenge. this approach to the augmentation of reality fits with the major aim of educational research which is to enhance and improve educational processes and methods. UNIVERSITY OF SALFORD. MODELS IN THE DESIGN STUDIO “It looks better like this”. students. In the same design crit session. says an architectural design instructor while chopping a piece of cardboard off a building model. synthesis) are mainly developed in architectural education programs around the world. suggests some enhancement. As stated by Allen Cunnigham in 2005. thus naming this work as ‘Augmented Pedagogies’. 51 . As a result. diagrams and sketches. representing) and high-order cognitive skills (critical thinking. rendered views of a project. instructors and observers engage in a dialog based on the use of plans and sections. planning. physical mock-ups. since the studio teaching scheme has been largely acknowledged as the core practice-based module in which both design (composition. after a centennial adaptation and evolution the studio teaching scheme and “project-based education around architecture employing the studio system is the most advanced method of teaching complex problem solving that exists”. This scenario might not be unknown for any architecture student. SCHOOL OF THE BUILT ENVIRONMENT.

The usefulness of models within the design studio is clear. Beyond the fact that its construction itself entails the development of technical skills, models also embed design information and knowledge, affecting organizational dynamics (the design critique or peer to peer collaborative work), the creation of students’ ‘toolkits’ or the final presentation of the design solutions, among other benefits. During this research being conducted at the University of Salford (UK), the extent of the impact of augmented models in this complex studio-system is yet to be depicted. Augmented models will be used, therefore, as a way to understand how new technologies impact design education and how can we describe that impact

from a scientific research perspective, that is following the guiding principles of generalizability, communicability and transferability of that resulting knowledge. The deep impact of new digital tools in design pedagogy has been explored recently by design theorists, such as Dr Rivka Oxman. The particularity of the design studio as a research setting is spiced by theoretical underpins that can potentially lead the path to depict this impact. For example, it has been stated that the studio teaching is usually an unstructured process, in which perceptions and interpretations of information and models play a major role in the students’

DRAFT VERSION OF AN AUGMENTED MODEL FOR INTERIOR DESIGN. FURNITURE CAN BE RE-ARRANGED IN A PHYSICAL MODEL OF A HOUSE ROOM (STUDENTS’ WORK).

52

progression in the courses, mostly based on ‘design dialogs’ between students, and students and instructors. Also, digital tools have the potential to not only re-shape the toolkits being used for design, but also mediate in the way design methods are structured, offers new ‘digital materials’ to work with or changes the very nature of the design problems to be faced in different courses. It is not clear, however, how this impacts occurs. The interactions within the studio that make use of representations and models to design are well established ‘rituals’ such as peer-to-peer collaborative activities or the design critique, but the nature of each studio differs from each other. Variables such as the experience of the instructors, the background of the students, the nature of the design problems to be faced or the institutional standpoint turn the studio into highly context-dependant modules. As a result those variables are usually highly controlled and the study of the impact of different technologies is commonly constrained to the description of technical challenges to be solved, the development of new systems/software or metrics of student satisfaction, rather than on the provision of a theoretical account of their impact into this complex teaching/learning process. The lack of a theory that describes how technology re-shapes the studio results in very limited knowledge re-usability and in turn, into very caged and localized pedagogical frameworks that do not allow cross-institutional or cross-disciplinary collaboration, to evaluate the constant infusion and evolution of new digital tools for educational purposes or to re-use a pedagogical approach and its associated knowledge.
IMAGES OF THE TRAINING SESSIONS ON AUGMENTED REALITY AUTHORING AND AUGMENTED MODELLING. STUDENTS OF THE PROGRAMS MSC DIGITAL ARCHITECTURAL DESIGN AND MSC BUILDING INFORMATION MODELLING AND INTEGRATED DESIGN, SCHOOL OF THE BUILT ENVIRONMENT, UNIVERSITY OF SALFORD.

A THEORETICAL APPROACH TOWARDS AUGMENTED PEDAGOGIES
Indeed, there is not fixed methodology to study the impact of digital technologies in design education. Rather, each study requires an own

53

standpoint in terms of validity and fitness to the research problem and the subject matter. As Wanda Orlikowski and Suzanne Iacono (2001) state on their work on information systems theory research, this corresponds to the fact that the use of technologies depend on the context and hence, “there is no single, one-size-fits-all conceptualization of technology that will work for all studies. As a result, IS researchers need to develop the theoretical apparatus that is appropriate for their particular types of investigations, given their questions, focus, methodology, and units of analysis.” In order to overcome this challenge, this ongoing research proposes a theoretical approach to depict the impact of augmented models in design education. By following a grounded theory methodology, observations and recordings are being collected in diverse settings on an attempt to de-

scribe the resulting studio dynamics by using augmented models. Several trainings on augmented reality and augmented modelling have been made at the University of Salford (MSc Digital Architectural Design, MSc in Building Information Modelling and Integrated Design), and two more experimental settings are now being arranged in different European countries. These multiple settings are not only intended to provide a wide view of the subject being studied, but also fits with the current recommendations for theory construction methodologies, since the manipulation and observation of data in many divergent ways and the juxtaposition of different conflicting realities and sources counteracts the tendency of reaching false or incomplete results, or informationprocessing biases of the investigator. This work is expected to be finished by end-2014.

VIEW OF THE STUDENTS’ TOOLKIT DURING THE FIRST AR WORKSHOP AT THE UNIVERSITY OF SALFORD.

54

Related literatUre and soUrces
¢¢ Achten, H., Pavlicek, J., Hulin, J., Matejdan, D. (Eds.). (2012). Digital Physicality / Physical Digitality - Proceedings of the 30th eCAADe Conference - Volume 1 / 2. Czech Technical University in Prague, Faculty of Architecture (Czech Republic) 12-14 September 2012. ¢¢ Buechner, J. (2011). Fictional Entities and Augmented Reality: A Metaphysical Impossibility Result. Journal of Evolution & Technology, 22(Minds and Machines Special Issue), 53-72. ¢¢ Cunningham, A. (2005). Notes on education and research around architecture. The Journal of Architecture, 10(4), 415-441. ¢¢ Eisenhardt, K. M. (1989). Building theories from case study research. Academy of management review, 14(4), 532-550. ¢¢ Orlikowski, W. J., & Iacono, C. S. (2001). Research commentary: Desperately seeking the "IT" in IT research—a call to theorizing the it artifact. Information systems research, 12(2), 121-134. ¢¢ Oxman, R. (2008). Digital architecture as a challenge for design pedagogy: theory, knowledge, models and medium. Design Studies, 29(2), 99-120. I finished my Architecture degree in 2007, in Chile. Today I am a teaching assistant and PhD student on digital architectural design at the University of Salford (United Kingdom) and my current research interests are design pedagogy, augmented reality, and collaborative technologies for design. I have been involved in research activities since 2007, but my background is quite mixed and has involved the domains of sustainable design, buildings in extreme/cold environments, mass customization and digital design. My published work has led me to present on conferences and invited lectures in Chile, United Kingdom, Italy, Belgium, Slovenia, Czech Republic, Canada, among others.

Alejandro Veliz Reyes

55

56 SCARLET PROJECT © UnIVERSITY OF MAnChESTER .

UNIVERSITY OF MANCHESTER Jean Vacher Curator. After all. If the focus for the student is the technology. The report provided the catalyst for the SCARLET (Special Collections using Augmented Reality to Enhance Learning and Teaching) project. this can inevitably lead to dissatisfaction and resistance. Many observers have already buried AR as a fleeting fad in education due to its lack of use cases and documented impact studies. on deep rooted pedagogical benefits rather than short-term gimmickry. educators are understandably cautious electing to concentrate. The benefits to student learning should always be central to the introduction of any new technology and AR is no different.A STUDY IN SCARLET MATT RAMIREZ Laura Skilton Learning and Teaching Co-ordinator Mimas. Some of the most persuasive arguments are described below: established learning methods and styles. The benefits of using Augmented Reality in Education 57 . in education technology should be transparent and not an overpowering driver. new technology should integrate seamlessly into Students can view and touch real manuscripts/ editions in conjunction with guided support from trusted sources. In addition. supporting independent learning. Crafts Study Centre University for the Creative Arts Marie-Therese Gramstadt Educational Technologist. users do not want to spend time adapting to a new way of learning. the learning experience can be diluted. Crafts Study Centre/ Research Associate. The University of Manchester Rose Lock Special Collections Supervisor University of Sussex MIMAS . It is always useful when dealing with new methods of delivery. perhaps rightly. Whenever the words “technical innovation” is spoken in education circles. particularly where the emphasis is on the teaching material. to be armed with a long list of tangible gains for adopting the technology. Visual Arts Data Service University for the Creative Arts Introduction Augmented Reality (AR) was identified in the 2011 Horizon Report1 as a key technology trend with potential impact on education. as a way of leveraging innovative technology with pedagogical processes.

 The learning curve for new users engaging  with mobile AR through browsers is relatively shallow enabling the learning/pedagogy to be the driver. not the technology. linking 3D models. learning technologists and students. and information on related objects. academics. concentrated on delivering the benefits to student learning without being a flag bearer for the technology. Layering AR on texts/images can encourage  interaction (e. and a slightly larger percentage of what is shown to them. images. learners will remember and retain the majority of the information presented to them. Junaio was the only AR browser to harness optical tracking functionality.g. 58 . isolated from much of the secondary supporting materials and a growing mass of related digital assets. augmented 3D models that overlay the physical image and require user touch gestures to proceed) and spark enthusiasm. 3D models. so this experience can be foreign and a barrier to their use of special collections. maximizing  the opportunity for interaction. whilst enhancing the learning experience by surrounding the book with digitised content.May 2012) The main challenge with the use of special collections in learning and teaching is that students are required to consult archives. The SCARLET project. By implementing an object based AR experience. but when we become actively involved in an experience. while embracing the potential of AR. a national centre of expertise at The University of Manchester. SCARLET was led by the Learning and Teaching team at Mimas2. such as an early printed book in the library. Students are used to having access to electronic information on demand. encouraging critical response and the adoption of new perspectives and positions. students could simultaneously experience SCARLET (June 2011. A mixed team was pulled together dedicated to enhancing the student experience through the application of technology including librarians.  AR can harness both asynchronous (emailing tutor questions) and synchronous (discussion with peers) e-learning methods. Users retain a very small amount of the information that is delivered. manuscripts and the magic of original primary materials. videos and information to images in the form of “GLUE” based channels. rare books within the controlled conditions of reading rooms. This is in opposition to traditional didactic methods that are predominantly teacher led. preparing them for solo research.  AR promotes ‘active’ teaching.  Abstract concepts or ideas that might otherwise be difficult for students to comprehend can be presented through an enhanced learning environment offering access to source historical artefacts and online research in situ. This coupled with an open API and compatibility on Android. Sources for primary content were ten key editions of ‘The Divine Comedy’ by Dante (between 1472 and 1555) particularly important in terms of publishing and intellectual history and the world-renowned oldest fragment of the Gospel of John. At the start of the project in 2011. translations. iOS and Nokia devices would prove decisive in the decision to use Junaio.

AR was most successful layered over the printed marker instead of signposting to other web based resources already accessible using traditional teaching scaffolds (e. planning for essay or presentation). contextual and less generic.g. student feedback noted that AR experiences should be best used as part of a learning activity (either independent or group based) acting as enabler to achieving a key course objective (e. UNIVERSITY OF MANCHESTER. making them unique and packaged in digestible chunks produces more positive feedback and value. Emphasising the need to align technology to teaching and learning objectives was paramount from the outset to maximise student benefit and impact. DANTE AR CONTENT © MIMAS. Simply adding existing web assets to an object is insufficient. providing a foundation for further investigation and research. CMS). It would also liaise with staff at these universities to transfer the skills learnt and hone the toolkit that was used as a high level framework for replicating similar AR practices in education. student evaluation was critical. UNIVERSITY OF MANCHESTER. further funding was made available to work with two other UK institutions to facilitate and guide them in the development of AR content. In addition.Evaluation A dominant theme that became evident in the evaluation was that the two academics found differing responses dependent on student user groups. 59 .g. directly feeding into the application using an iterative development approach. The learning experience was most enhanced by AR when information delivered was FRAGMENT OF THE GOSPEL OF JOHN AR CONTENT © MIMAS. Further funding was made available through the SCARLET+ project whose primary focus was to apply the process and framework to other institutions special collections embedding the methodology using a toolkit5 (University of Sussex and the Craft Study Centre at the University for the Creative Arts). The team went on to win the 2nd prize for the ALT learning technologist of the year team award3 and won the Innovation in HE award at Education Innovation 20134. SCARLET+ (June 2012 – April 2013) Following the success of the SCARLET project. Students who had little prior subject knowledge found the app most useful. Throughout the project lifespan.

a crucial part in decision making processes. In conjunction with this. collaborated with colleagues from the Landmap9 service to create an AR experience around the UKMap dataset. Working with other subject areas In parallel with SCARLET and SCARLET+. In the area of geo-spatial mapping. bringing their static objects to life. building heights and ground usage to name a few. The partner universities involved were the Special Collections at the University of Sussex and 6 which will allow students to access surrounding resources. using AR to identify abnormalities. This project has both developed Mimas’ understanding of implementing AR in education and embedding best practice and methodology to other institutions. the two projects have presented important findings in the impact of AR in education. early development has started using 3D object recognition in the clinical skills laboratories. a multi-disciplinary team approach was adopted. There is vast potential in this field enabling students to examine x-rays/biopsies. as with SCARLET. focusing on content from 1980s’ mass observation and 20th century crafts. Mimas has the Crafts Study Centre at the University for the Creative Arts7. delivering a suite of rich materials especially given the small amount of funding that was available. Sharing was a key element at the heart 60 . It demonstrates a visual representation of raw materials that are often extremely large in size and difficult to comprehend. multi-layered information accurately locating building types.AR VISUALISATION OF UKMAP DATA © MIMAS. Both succeeded in providing a showcase for the Special Collections held at their respective institutions using AR. UNIVERSITY OF MANCHESTER. Conclusion To conclude. The challenge was to incorporate this tabular data in a visual 3D model that a handheld device could render efficiently. Mimas have worked with the Manchester Medical School8 helping students to understand pattern recognition. This provides a wealth of rich. cannula application). This ensured that the content developed made an impact on learning and teaching as well as enabling AR skills to spread across the institutions.g. Crucial to the success was ensuring that. layering anatomical information and reinforcing learning with instructional demonstrations (e.

Blender. iBook/mobile/multimedia development projects and haptics) with the Manchester Medical and Dental Schools. References 1.uk/library/ specialcollections 7.uk 61 .uk/cms SCARLET Toolkit: http://scarlet. Further information: http://teamscarlet. Unity to name a few. section on augmented reality: http://wp. While it may not be suitable for all students and situations.of SCARLET and SCARLET+. 9.ac. 3. this coupled with a strong team ethic enabling stakeholders to buy into the long-term vision.uk/mediawiki/index.org/ horizon2011/sections/augmentedreality 2.co.mimas. 8. this follows on from his involvement in the award winning SCARLET project. Matt's role is also concerned with the research and development of new technologies (e. special collections and business. 2011 HORIZON report.ucreative.ac. Read more about Matt’s work at http://teamscarlet.com Matt Ramirez Matt Ramirez is currently working on the technical development and support of the JISC funded SCARLET+ Augmented Reality project.manchester.nmc.com and @team_scarlet.ac. AR opens up huge possibilities for creating immersive learning activities. IT. It is hoped that the legacy from these small projects will be to inspire others to undertake similar work and display the student led benefits AR can offer.uk Manchester Medical School: http://www. It is particularly effective in explaining abstract concepts visually.uk SCARLET team are joint second in learning and teaching awards.ac. ac. Flash. wordpress. wordpress. Education Innovation 2013: http://educationinnovation.g.php/Main_Page Special Collections at the University of Sussex: http://www.csc. when employed well it can capture the attention like few other technological mediums.sussex.com/2012/09/13/scarletteam-are-joint-runner-up-in-learning-technology-awards 4. These have used multimedia content authoring tools such as Flex. allowing active based learners to better absorb the transfer of knowledge. Mimas: http://mimas.mms.ac. This aims to improve the student experience by embracing innovative learning methods and providing cutting edge support materials. Crafts Study Centre at the University for the Creative Arts: http://www. 6. 13th September 2012: http://teamscarlet. science.wordpress. He has over 15 years experience in web design and e-learning content development for a variety of subject areas including medicine. 5.uk Landmap: http://landmap.

we made a head start with the introduction of AR via projects differing in length and with different student groups. felt pens or a collage of other material is not the most important issue at that moment. one-day exhibition of the results. both young and digitally savvy. With Augmented Reality new.INTRODUCING AR TO FIRST YEAR GRAPHIC DESIGN STUDENTS YOLANDE KOLSTEE Graphic Design students learn how to work with the latest Adobe suit. in cooperation with the Studium Generale programme. by the way — new elements should be considered in the concept and design phase. because it will change aspects of their discipline as well as their ‘metier’. At the beginning of the Royal Academy of Arts’ current study year. started to work with their classes of 27 students each (!) together with some third year students as well. very new. experience augmented reality. Most important. At various other art academies we see similar projects. meaning a short. at the University for the Creative Arts. The idea was to work towards a special PopUp Gallery. The search to get to know what AR is and how it could be a 62 . and the department involved. Photoshop and website development tools. however. freshmen and -women. while also taking co-authorship and performance into account. in the UK. However. Two Graphic Design teachers. We. one of which is the Scarlet (Special Collections using Augmented Reality to Enhance Learning and Teaching) project. Whether a concept is devised with colour pencils. they often have a (underlying) preference for pen and ink based handwork. while. When working with Augmented Reality — a very suitable medium for graphic designers. in effect. What is important for the Academy. is that they want the students to become acquainted with creative or artistic research. aspects are added to the world of graphics regarding the Internet and social media. the majority of mainstream teachers place analogue and digital techniques at the same level when developing concepts. as AR Lab are thoroughly convinced that it is important to let first-year students. are the courses in which concept development forms the main part. a programme covering lectures throughout the year.

meaningful technique for graphic designers began comprised two steps.” In completing the first step of the two step project. We provided technical help when students were really eager to get their ideas to work as a fully functioning app. phones and tablets. In the words of the students’ teachers : “Spectacles. Not one. This is obvious because the assumption in AR is that reality must be supplemented with digital information. The first was a very active YouTube-Google search. a publication of all the projects was made with a spread for each student in which the description and intention of their project were incorporated. The rationale underlying the research project therefore lies especially in the use of digital techniques to make this extra layer onto reality visible. Words emerged by moving strips of black paper over the seemingly incoherent rows of letters. These are some tools one associates with Augmented Reality (AR). turned out to be very exciting. These types of projects show the broad range of ways in which an extra layer on top of the physical world poses questions about what we see. step two. Or it would be just reality. Without digital technology. The variety of final projects was staggering and impressive. for the students themselves as well as for us. Aurasma — a simple and elegant tool for beginners in the AR app world. They were part of the jury that awarded the two ‘best’ projects at the end of the first Pop-Up Gallery. The end-results would be displayed during our first Pop-Up Gallery on AR. Is Augmented Reality only achievable by the use of digital means? Can hidden information be augmented. headsets. Jente won with her project 'Had je dat gedicht?' (roughly translated as 'How does that rhyme with you?'). Finally. What type of problem would they want to solve using AR? With what classification should it work – analogue and/or digital? With this exercise we hoped to widen the concept. A special recommendation was given for the dyslexia project of Janne van Hooff who visualised how people suffering from dyslexia see letters and numbers. no addition to reality and certainly no interaction would be possible. the idea was to provide an enriching experience for the students and to give them the tools with which they could incorporate AR in their work. What all the students’ projects have in common. but two winners were awarded: Jente Hoogeveen en Jorick de Quaatsteniet. The same goes for those students who. so to say. Jorick won with his project 'Get your shit together' where animated QR-codes formed text. 63 .g. We were lucky to have two lecturers who gave their talk on AR as part of the Studium Generale programme preceding the Pop-Up Gallery: AR artist Sander Veenhof and data-visualisation expert Niels Schrader. It is exactly this assumption that we want to investigate. is that they appear to invite spectators to look at the world from another perspective. while also seeking to come up with new applications and reasons for the use of Augmented Reality. We asked them to think about the kind of information they wanted to disclose in specific situations. Ultimately. Error messages we are confronted with on a daily basis are transformed into poems in order to take some of the zing out of stressful situations. The concept developing part. regardless whether they are analogue or digital. the students gave us a huge input of examples they encountered: they found all sorts of material via social media or as search results that they consequently discussed. and the second step was to learn to play with the idea of adding extra information — both analogue and/or digital — a concept development trajectory of sorts. we invited freshmen and -women from the Graphic Design department to get started with Augmented Reality. without the aid digital techniques and devices? Following that. at first. We showed them our AR projects and introduced them to e. were not completely convinced about using AR and how they could incorporate this technique in their projects.

64 .

COM ample is the 4D Anatomy project by daqri (2012). which uses the latest text recognition algorithms. with which developers are trying to create new. just as in many other fields. On the screen you see a 3D model of the human body. These examples show some of the new forms of interaction and presenting information. When the device recognizes the word. interesting. and memorable learning experiences. or switch on the layer showing the nervous system for example. The (young) users of this upcoming app are asked to look for certain words in their home and aim their device at it. WWW. An interesting exImage by MARCELO GERPE. Another interesting project in development is the Sesame Street app Big Bird’s Words (Qualcomm. With sliders and buttons you can set the transparency of the skin. 65 . This way the user is asked to involve their environment in the process of learning words.CREATIVASTOCK.AUGMENTED EDUCATION HOW AR TECHNOLOGIES EXTEND OUR MINDS ROBIN DE LANGE In the field of education. 2013). researchers and developers are experimenting with potential applications of Augmented Reality technologies. Many of these experiments draw on of the possibilities to explore virtual information spatially. where you can explore the physiology of a human being by moving the display device along a piece of paper with markers. it gives points to the user.

By showing how beliefs are not bound by the borders of the body. these kinds of cases are actually not as abstract as they might seem: they describe a very well-known real-life situation. Clark and Chalmers present a thought experiment involving the fictional characters Otto and Inga. Otto has Alzheimer’s disease and uses a notebook to serve the function of his memory. Furthermore. Because of this change of perspective regarding external information sources. were it done in the head. but were also trying to determine whether the shape fits in the slot. (c) The same situation in a possible future.In this article I will argue that the developments in AR technologies will make digital information sources much more transparently available to us. Clark and Chalmers argue. In their paper about cognitive performances while playing Tetris. while Inga’s biological memory is functioning properly. Clark and Chalmers show that true mental events can extend in the environment as well. who are remembering how to get to the museum. AR technologies could not only lead to new learning methods. (Kirsh and Maglio. To backup this claim. thereby simplifying the task. where the person can choose between using his neural implant that does the rotation operation as fast as the computer does in (b). Otto can be said to have a belief about the location of the museum before he actually consults his notebook. under the right circumstances. which Clark and Chalmers employ to introduce the Parity Principle: “if a part of the world functions as a process EXTEnDED MInD AnD ThE PARITY PRInCIpLE Andy Clark and David Chalmers start their renowned paper ‘The Extended Mind’ with a thought experiment. the notebook can be seen as an extension of Otto’s memory. In certain cases. but could. (b) The same situation as in (a). In the same manner. this information may even be seen as part of our cognitive process. the human mind is not bound by the borders of skin and skull. I will first introduce the concept of the Extended Mind. or using ‘old-fashioned’ mental rotation. then that part of the world is (so we claim) part of the cognitive process. In this experiment the reader is asked to consider three cases and to think about how much cognition is present in each case: (a) A person sits in front of a computer screen which displays geometrical shapes and is asked to fit the shapes in the displayed ‘sockets’. by rotating the shapes in his mind. Now. All the examples of cognitive extension that Clark1 gives in his books and papers are not the typical futuristic technologies that come to mind when thinking about humans merging with tech- 66 . Inga is thought to have a belief about the location of the museum. and in my opinion should. cal rotation in (b) is actually much faster than the mental rotation. namely playing the video game Tetris.” (Clark and Chalmers. To make this claim plausible. 1994) It is this example of the human capacity to manipulate the environment to solve problems. also trigger debates about the very goals of education itself. the latter having some speed advantage. Kirsh and Maglio found that the physi- which. players were not only physically rotating the shapes to fit the slot. 1998) According to the Parity Principle. Thereby. but now the person can choose either to mentally rotate the shape or physically rotate it by pressing a button. we would have no hesitation in recognizing as part of the cognitive process. before she recalls this from her internal memory.

When somebody asks us on the street whether we know what time it is. writing utensils even for centuries. An interesting question is whether more modern external information sources could obtain the same status as these age-old technologies and play a similar. then wait for the downloading of the page 67 . the information access costs when retrieving information from the Web is way too high. The user has to put physical and cognitive effort into navigating the browser to the right page. More precisely: the information access costs. we often answer “yes”. You actually do know what time it is. we can refer to the three criteria suggested by Clark: 1. this direct interaction with brains is by no means necessary to become part of the cognitive process (nor are they sufficient for cognitive extension: communicating with technology through a Brain Machine Interface usually still takes too much cognitive effort. for example parts of the Web. according to Clark this is not simply loosely formulated informal language. and we are wearing our watch. look at our watch and see what time it is. Subsequently we raise our arm. Furthermore. In fact. that Clark considers us to be “natural born cyborgs” (Clark. 2003). we use an algorithm that divides the process of multiplying arbitrary large numbers into very simple steps. Now. we use the pen to manipulate the external memory source. The external resource should be directly available without difficulty. wrist watches have been around for many decades. 2003).) Now. 2008). due to the open character of the Web. Could digital information sources. During this time these technologies have become ubiquitous. active role in cognitive processes. actually become parts of our minds? Could digital information sources actually become parts of our minds? To begin to answer this question and decide whether a part of the world should (temporarily) be seen as part of the cognitive process. 3. The technological aid should always be available when needed. a measure of the combination of time and physical and cognitive effort. probably a wise attitude. actually part of this process.) Our brains incorporate the world and some of the technologies therein in their cognitive processes in such an intimate way. One example he mentions is the use of pen and paper when doing long multiplications. From our critical stance we do not immediately believe most information we encounter on the Web something which is. it is quite clear that the Web with which we interact on a daily basis cannot be seen as part of our cognition. according to the Parity Principle. Another example shows that it has become common to talk about the information that is in some of our technologies as if part of our own knowledge. They have become socially accepted and actually shaped culture itself. the technologies which Clark considers as cognitive extensions of our cyborg minds are hardly identified as technology anymore. 2. Although the possibilities of Brain-Machine Interfaces and neural implants such as in case (c) offer very exciting new ways of communicating with technology. By writing down figures in certain locations. should be as low as with an equivalent function of the brain (Smart et al.nology. ‘you’ is only “the hybrid biotechnological system that now includes the wristwatch as a proper part” (Clark. To calculate the product of two numbers. the paper. The information retrieved from the external source should be directly endorsed and trusted. The writing utensils play a crucial role in this cognitive process and are therefore. From this. (This proven transparency of the wrist watch is what makes the development of smart watches interesting.

information can be presented to the user at all LOWERInG InFORmATIOn ACCESS COSTS However. I think you might start to feel like you really do know it. know the meaning of a certain word that is not in your biological memory. and a short. would you say that you know the meaning of this word? I can imagine that you — after you get more and more used to the device and have experienced this situation a few times already — might say ‘yes’. Let us consider the following scenario to get a better idea: For a few years now you have your own AR glasses.and scan through the text to find the information he needs. The technological developments that are associated with Augmented Reality especially have the potential to lower the information access costs for digital sources considerably. An application running in the background does speech and text recognition and keeps track of words you (the biological ‘you’) don’t know the meaning of. clear description of the word pops up immediately in the corner of your field of view. But what is it with AR technologies. when a friend (who is not really into new technology and rather asks a friend to help him) asks you whether you 68 . quicker and more intuitive ways. Processors in smartphones are becoming faster. by eliminating the physical effort of getting your smartphone from your pocket and having to hold it in within your view. When it does indeed become ordinary to wear HUDs. the information access costs of retrieving information from the Web are way too high to be considered as part of the cognitive process. batteries are getting better.2 More so. Despite of what many critics expected. In the widespread current way of interacting. etc. Now. that they could lower information access cost so significantly? Of course Head-Up Displays (HUDs) play a great part in this. user interfaces are becoming more intuitive. it has become socially quite acceptable to be wearing a Head-Up Display at all times. similar to the situation with the wrist watch. technology is ever-evolving and many of the present advancements will make information available to us in more reliable. tion knows when you’re busy driving for example and doesn’t bother you then. telecommunication service providers keep improving their networks to decrease download times. You’ve been using this application for a few months now and it’s starting to have quite a good sense of when to present you with the meaning of a word you encounter. in a very real sense. Of course. the applica- The main goal of education should be to train the technologically extended cognitive system.

By doing so.times. By accepting digital external information sources as a proper part of memory. One could argue from this perspective that the main goal of education should be to train the technologically extended cognitive system. at the exact moment when it is needed. you can interact with digital information in similar ways to how you interact with the physical world. Another important aspect of AR is the use of information from different sensors and smart algorithms doing image and speech recognition. personalized cloud of knowledge. These are the characteristics of AR that create the potential of making digital information much more transparently available to us. well designed. According to the Parity Principle. By combining these. possibilities are created to present information in context-sensitive ways responding both to the environment and the user. I suggest that under certain conditions. it could be decided to store certain information that we want to remember in an easily accessible. intuitive interface.COm 69 . which is of course the main idea of AR3. Adopting this philosophical perspective can have huge consequences for a field like education. Furthermore. creating a very natural. digital information can be placed over the world. these digital information sources could then be seen as proper parts of our hybrid minds. SEE hTTp://DAqRI. EXTEnDED COGnITIOn AnD EDUCATIOn In the previous I have introduced the concept of extended cognition and explored the possibilities of digital technology for cognitive extension. personalized information sources are able to compete with mental resources in terms of costs of information access. Instead of trying to store all information in biological memory by endless repetition. Image COURTESY OF GAIA DEmpSEY AnD DAqRI.

Oxford: Oxford Univ. Retrieved from http://www. Supersizing the Mind. a piece of paper and maybe a dictionary or an outdated (graphi4 de Lange.. Braines. J.this task of storing information could be off-loaded to an external source which is constantly available to us at low information access costs. For the most part of their education. Press. In R. Het Semantic Web en netwerktechnologische cognitieve uitbreidingen. can offer a valuable perspective in this discussion. 513-549. (2012). Smart. (2008). ¢¢ ¢¢ ¢¢ Dror. Analysis . Kirsh. Education as Socialization and as Individualization. Technologies.com/ media/videos/big-birds-words-demo-reelvideo ¢¢ ¢¢ Clark.. P.Realising the Vision for UK Education. (2012). P. R.. Hendler. (1996). P. E. D. R. (1999). A. London: Penguin Books. and the future of human intelligence. D. On distinguishing epistemic from pragmatic action.daqri.. ¢¢ Clark. D. Natural-Born Cyborgs. D. & Shadbolt. Cognitive science . in which external resources have an active role in the cognitive process. Digital technology is developing very rapidly. (1998). Offloading Cognition onto Cognitive Technology. The 70 . & Chalmers. 1-23). Retrieved from http://site. cal) calculator5. Embodiment. System Upgrade . Do socio-technical systems cognise? 2nd Symposium on Computing and Philosophy . E. robindelange. Kinds of Minds: Toward an Understanding of Consciousness. Network-Enabled Cognition: The Contribution of Social and Technological Networks to Human Cognition . there should be an active discussion on the goals of education and what technologies students can use to reach these goals..ac. N.qualcomm. Press. P.com/wikisofie ¢¢ Dennett. ¢¢ daqri. (2011). & Maglio. To come to these decisions. (2013). Retrieved from http://www. ¢¢ ¢¢ Clark. 4D ANATOMY. (2009). Philosophy and Social Hope (pp. (1994). 114-126). ¢¢ Rorty. ESRC/EPSRC Technology Enhanced Learning Programme. (2008). A. & S. Cognition Distributed: How Cognitive Technology Extends Our Minds (pp.. Rorty. An extended view of the mind. S. (2010 (In Press)).uk.com/labs/4d-anatomy ¢¢ The general view on the use of technology in education is quite different from the view expressed in this article though. a lot of research on the use of technology in the learning process is needed. O. This critical attitude towards the use of technology is very understandable. (2008). Cognition and the Web. Harnad. REFEREnCES ¢¢ Blomberg. I. careful decisions have to be made about how to use it in education. The Extended Mind. Engelbrecht. Big Birds words. In I. London: tel. Dror.. & Harnad. A. Action and Cognitive Extension. New York USA: Basic Books. (2003). Minds. Furthermore. Oxford: Oxford Univ. 10-23. students still only get to use some basic technologies: a pen. Smart. ¢¢ Qualcomm.

Moore’s law seems to be failing here. The information access costs of looking up a word in the dictionary go through the roof. If the word would be jargon of a field you are not familiar with for example.com 71 . from University of Southampton . Besides his studies. I will refer to Clark further on. Because Clark has written many other papers and books on this subject. M. 2011. Retrieved June 24. 3. For more information: www.Extended Mind and Network-Enabled Cognition. However. one might argue that the example does not really show AR. The initial paper ‘The Extended Mind’ was written by Andy Clark and David Chalmers. 4. 5. even remains the same price! Robin de Lange Robin de Lange has a bachelor’s degree in Physics and Philosophy and has followed courses on Artificial Intelligence. For his graduation research project he is developing an Augmented Reality application that supports the graphical solving of mathematical equations.robindelange. from the Standpoint of Extended Cognition. you would probably not understand the meaning directly and need to look up more information. He is now a student at the Media Technology MSc program at Leiden University and is particularly interested in technologically extended cognition.ac. At the moment. soton.pdf ¢¢ Wheeler. 2. Computational Culture . November). Robin has taken part in several entrepreneurial projects. it does use certain AR technologies intensively to provide context-sensitive information to the user who interacts with the world. He is a freelance video producer and science communicator.uk/16649/1/Network-Enabled_Cognitionv17. The hardware in these devices stays roughly the same. Most notably.School of Electronics and Computer Science: http://eprints. For this reason. This characteristic of AR of overlay ing the physical world with virtual objects is not really present in this scenario. Thinking Beyond the Brain: Educating and Building. thereby showing educational challenges and possibilities. he is looking for funding to do a PhD within his field of interest. EnDnOTES 1. thereby increasing the costs of information access.ecs. (2011. he was the co-owner of a company that specialized in homework guidance and tutoring.

HOW DID WE DO IT: WhICh AUGmEnTED REALITY SOFTWARE DOES ThE AR LAB USE? Wim van Eck 72 .

there are so many programs available it is difficult to actually have tried them all. NETHERLANDS) ON MAY 17TH. we merely aim to give an insight into our daily workflow. and updates and new programs pop up all the time. At the AR Lab we use a variety of augmented reality programs to realize our projects. Easy-to-use for mobile devices Since the AR Lab is based at the Royal Academy of Art The Hague most of our projects are developed together with our students. THE APPLICATION WILL PREMIER AT MUSEUM CATHARIJNECOVENT (UTRECHT. Often we are asked “What is the best augmented reality software?” which is a difficult question to answer.THIS APPLICATION. I will give an overview of the software we use and why we use it. ‘augmented reality software for interactions’ and ‘head-mounted augmented reality’. COMMISSIONED BY THE RCE (CULTURAL HERITAGE AGENCY OF THE NETHERLANDS). This doesn’t mean we claim that this is the best software and that there are no alternatives. Secondly. choosing which software to use really depends on what you want to achieve in the end. or we supervise 73 . ‘easy-to-use for desktop augmented reality’. For this overview we created four categories: ‘easy-to-use for mobile devices’. THE PAINTING 'ISAAC BLESSING JACOB' BY THE DUTCH PAINTER GOVERT FLINCK (16151660) HAS BEEN AUGMENTED TO GIVE INSIGHT INTO THE PAINTERS PRACTICE AND OFFERS SCIENTIFIC DATA IN A PLAYFUL WAY. First of all. IS DEVELOPED BY THE AR LAB USING VUFORIA AND UNITY 3D.

removing any technical barriers (Figure 1).aurasma.nz) is the software we use most often. so easy-to-use software is preferable. The ‘BuildAR Pro 2 Trial Download’ is the newest version for Windows and features both marker tracking and image based tracking. and BuildAR (http://www. BuildAR is based on the world-renowned ARToolKit. but students who want to develop their own application.The app provides clear information for every step you have to take.com/partners). Markers can be generated on the spot by taking pictures with the device. there are quite some versions which can be a bit confusing. still prefer to present their creations on Fig. The tracking quality of Aurasma is Fig. however.aurasma. co. and photographs and videos which are stored on the device can be used to augment the chosen scene. you only need to register to be able to access it. Some of our students. 2 a large screen. desktop augmented reality software is getting scarcer.buildar. Cinema 4d. for example. Easy-to-use for desktop augmented reality Nowadays most software companies seem to focus on mobile devices. but there is a ‘demo’ watermark on your video-background and you won’t be able to save your scene (Figure 2). they often have their own smartphone or tablet and are very keen to develop for it. It runs on both Windows and OSX.com) quickly became our favourite choice for this group of students. fast camera movements will result in loss of tracking. 1 not the best around. It is very easy though to use video with transparency. It is a pity that Aurasma only seems to give good support for 3ds Max and Maya. and also to import images and videos which are not on your mobile device. a not very common option which can give great results. By downloading the free Aurasma app from the iOS App Store or Google Play store students can immediately begin creating their first project. When downloading BuildAR. Many of these students don’t have a technical background. but it is definitely good enough for most projects. as well as 3D models. though the OSX version lacks some options and seems to be less stable. Aurasma (www. Furthermore. You can use this version for free. all they need is their mobile device. ‘BuildAR Pro Mac 74 . When more possibilities are needed there is also a free online application (www. adds an easy-to-use graphical interface. This well documented application makes it easier to precisely position your virtual objects. it’s more difficult to import 3D models from.

but many of our students don’t mind this logo so much though. Also. In issue 2 of AR[t] we introduced our latest augmented reality headset named Marty and its tracking software. it very easy to import 3d-models including animations from almost all 3d-packages.com). The only limitation is a ‘HITLabNZ’ logo.unity3d. You can natively develop for iOS (Xcode). video. you will need to buy the Unity iOS or Android add-on to be able to export to a mobile device. this issue of AR[t] features more information about this system. the demo-limitations are the same as in the Windows version. However.com) allowing you to easily export to iOS and Android. Android (Eclipse) or use it as a plugin for Unity (www. Adding interactions through scripting is relatively easy. there are many tutorials around and the Unity forum is a great source for information. BuildAR is easy-to-use. Unity (Figure 3) is a popular game engine offering excellent visual quality on mobile devices. which use a pattern of black and white cubes positioned around the image. there is the ‘BuildAR Free version Download (2008 version)’. 3 75 .Trial Download’ is the OSX version and only features marker tracking. audio and 3D models. Fig. AR software for interactions A program we often use at the AR Lab ourselves is Vuforia (https://www. the tracking quality stays good even if the tracking image is partly occluded or when there is little light available. which is only available for Windows and is free of charge. Lastly. Semiconductor company Qualcomm offers Vuforia for free and Unity has a free version as well.vuforia. the new version can augment your scene with images. Besides image markers there are also frame markers available. Unity makes Head-mounted augmented reality For head mounted based augmented reality we have solely used software and hardware developed by TU Delft. Vuforia’s image based tracking is extremely stable.

2011 76 .AR[t] PICK ImmATERIALS BY DESIGn STUDIO OnFORmATIVE AnD ChRISTOphER WARnOW.

DATA IS pLACED In A ROOm. USInG ThE LIGhT pAInTInG TEChnIqUE. ThE RESULTInG FORmS DEpICT pOSSIBLE DATA SETS AnD EXAmInE ThE DESIGn pOSSIBILITIES BETWEEn TEChnOID hOLOGRAmS AnD pERSOnAL nOTES.OnFORmATIVE.com/work/immaterials/ 77 . ThE pROJECT EXpLORES hOW InFORmATIOn CAn BE InTEGRATED InTO phYSICAL SpACE.IMMATERiALS IS ThE RESULT OF A COLLABORATIOn BETWEEn ThE DESIGn STUDIO OnFORmATIVE (WWW.com/portfolio/?p=244 http://www.COm). Further information: ¢¢ ¢¢ http://christopherwarnow.COm) AnD ThE COmpUTATIOnAL DESIGnER ChRISTOphER WARnOW (ChRISTOphERWARnOW.onformative.

nl ESmÉ VAhRmEIJER Royal Academy of Art (KABK) e.vaneck@kabk.Kolstee@kabk. scientific creativity. EDWIn VAn DER HEIDE Leiden University evdheide@liacs. She holds the post of Lector (Dutch for researcher in professional universities) in the ­ field of ‘Innovative Visualisation Techniques in higher Art Education’ for the Royal Academy of Art. Her research interests include interaction in interactive art and (non-visual) Augmented Reality. web design. game design and creative research. YOLAnDE KOLSTEE  Royal Academy of Art (KABK) Y. MAARTEn LAmERS Leiden University lamers@liacs.nl Yolande Kolstee is head of the AR Lab since 2006. Maritime and Materials Engineering (3ME).nl MARIAnA KnIVETOn Royal Academy of Art (KABK) m. parallel image processing architectures.nl HAnnA SChRAFFEnBERGER Leiden University hkschraf@liacs. Esmé Vahrmeijer is the graphic designer and webmaster of the AR Lab. studying New Media and Digital Culture. photography and education. 78 . bio-hybrid computer games. supporting and supervising students and creating 3d content. Her interests are in graphic design. The Hague. Mariana took up editing duties for this current issue.nl Hanna Schraffenberger works as a researcher and PhD student at the Leiden Institute of Advanced Computer Science (LIACS) and at the AR Lab in The Hague. His main interests and fields of research are: real-time embedded­ image processing. she is a part time student at the Royal Academy of Art (KABK) and runs her own graphic design studio Ooxo. Pieter Jonker is Professor at  Delft University of Technology.nl Mariana Kniveton is currently a master student at Utrecht Universtity. Specializations include social robotics. robot learning and Augmented Reality. Beside’s running his own studio he’s part-time assistant professor at Leiden University (LIACS / Media Technology MSc programme) and heading the Spatial Interaction Lab at the ArtScience Interfaculty of the Royal Conservatoire and Arts Academy in The Hague. Maarten Lamers is assistant professor at the Leiden Institute of Advanced Computer Science (LIACS) and board member of the Media Technology MSc program. real-time 3d animation.ContribUtors  WIm VAn ECK Royal Academy of Art (KABK) w.nl Wim van Eck is the 3D animation specialist of the AR Lab.Jonker@tudelft. His interests are.vahrmeijer@kabk. PIETER JOnKER Delft University of Technology P. robot vision. typography. His main tasks are developing Augmented Reality projects.kniveton@kabk. Besides her work at the AR Lab. After a brief stint as a cover model for AR[t] #2. space and interaction.P.nl Edwin van der Heide is an artist and researcher in the field of sound. and models for perceptualization. among others. Faculty Mechanical. Since september 2012 she has worked as an intern at the Research Department IVT and the AR Lab.

He creates objects that let us rethink our environment. he leads the “Augmented Matter in Context” lab that focuses on blend between bits and atoms for design and creativity.wordpress. IT. Heavy creates innovative interfaces between digital design and physical worlds in ways that that provoke the imagination and challenge existing styles of art. and interaction.com Lotte de Reus recently received her Master’s degree in Design for Interaction at the Faculty of Industrial Design Engineering at Delft University of Technology. He is looking for funding to do a PhD on technologically extended cognition. With a background in virtual reality and interaction design. 79 . LOTTE DE REUS www. and collaborative technologies for design ANTAL RUHL www. augmented reality.c. Lotte would again like to work in the domains that combine art and technology.robindelange. This interest has resulted in the science fiction novel "The Augmented Star". special collections and business. His research interests are design pedagogy.com Jouke Verlinden is assistant professor at the section of computer aided design engineering at the Faculty of Industrial Design Engineering. science and art.verlinden@tudelft. Oliver Percivall works in I. BC "Heavy" is founder of The Heavy Projects [and its collaborative spin-off Re+Public].com OLIVER PERCIVALL Antal Ruhl is a media artist with a background in design. He is interested in where technology could take us. ALEJANDRO VELIZ REYES University of Salford ROBIN DE LANGE www. design.nl MATT RAMIREZ teamscarlet. science. These objects vary from kinetic sculptures to interactive installations.T Project Management.JOUKE VERLINDEN Delft University of Technology j. In the future. Matt Ramirez has over 15 years experience in web design and e-learning content development for a variety of subject areas including medicine.antalruhl.nl GUEST CONTRIBUTORS BC “HEAVY” BIERMANN The Heavy Projects www. NEXT ISSUE The next issue of AR[t] will be out in the fourth quarter of 2013.lottedereus.theheavyprojects. Alejandro Veliz Reyes is a teaching assistant and PhD student on digital architectural design at the University of Salford (United Kingdom).com Robin de Lange is a student at the Media Technology MSc program at Leiden University and part-time entrepreneur.

Sign up to vote on this title
UsefulNot useful