03

may 2013

AUGMENTED REALITY, ART AND TECHNOLOGY

Re+Public: Re+Imagining Public Space
BC “Heavy” Biermann

Pre-Digital Augmented Reality
Maarten H. Lamers Vestibular Stimulation

as an Augmented Reality Layer?
Antal Ruhl

AR[t]
Magazine about Augmented Reality, art and technology

MAY 2013
2

HOW & NOSm MURAL AUGmEnT, ThE HEAVY PROJECTS, ARTICLE On pAGE 8

3

Colophon
ISSN
2213-2481

COnTACT
The Augmented Reality Lab (AR Lab) Royal Academy of Art, The Hague (Koninklijke Academie van Beeldende Kunsten) Prinsessegracht 4 2514 AN The Hague The Netherlands +31 (0)70 3154795 www.arlab.nl info@arlab.nl

EDITORIAL TEAm
Hanna Schraffenberger, Mariana Kniveton, Yolande Kolstee, Jouke Verlinden

COnTRIBUTORS
AR LAB & PARTnERS: Wim van Eck, Edwin van der Heide, Pieter Jonker, Maarten Lamers, Maaike Roozenburg GUEST COnTRIBUTORS: Alejandro Veliz Reyes, Antal Ruhl, Lotte de Reus, Matt Ramirez, Oliver Percivall, Robin de Lange, BC “Heavy” Biermann

GRAphIC DESIGn
Esmé Vahrmeijer

pRInTInG
Klomp Reproka, Amersfoort

COVER
Our AR[t]y cover is a work by Royal Academy of Art student Donna van West who participated in the Smart Replica’s project, see: www.donnavanwest.nl

www.arlab.nl

4

Lamers AUGmEnTED EDUCATIOn Robin de Lange VESTIBULAR STImULATIOn AS An AR LAYER? Antal Ruhl HOW DID WE DO IT Wim van Eck AR[t] PICK: ImmATERIALS Editors' Choice BELIEVABILITY Edwin van der Heide NOnLInEAR STORIES TOLD BY CUpS AnD SAUCERS Lotte de Reus 5 .Table of contents 30 WELCOmE to AR[t] 50 WhO OWnS ThE SpACE 2 Yolande Kolstee 06 08 16 18 24 26 30 34 40 44 50 56 62 64 72 76 RE+PUBLIC: RE+ImAGInInG PUBLIC SpACE BC “Heavy” Biermann — PhD ThE MISADVEnTURES In AR Oliver Percivall AUGmEnTED pEDAGOGIES Alejandro Veliz Reyes MARTY. ThE nEW AFFORDABLE hEADSET — PART 2 Pieter Jonker and Yolande Kolstee A STUDY In SCARLET Matt Ramirez SUBJECT: InTERVIEW From: Hanna Schraffenberger To: Lev Manovich InTRODUCInG AR TO FIRST YEAR GRAphIC DESIGn STUDEnTS Yolande Kolstee PRE-DIGITAL AR Maarten H.

FINE ART STUDENTS SHOW THEIR WORK ON THE POP-UP GALLERY TWO GRAPHIC DESIGN STUDENTS (DAVE POPPING AND GABOR KEREKES) MADE A MOIRÉ-PATTERN AS A DECODING KEY 6 .

in the part News Picks we post short news items on AR artists. These drawings & 3D model will be downloadable from our website. The results are a wide variety of implementations and reflections in different creative contexts. Lotte de Reus discusses a spatial audio intervention to enhance museum exhibits. we have chosen 'Immaterials' — the result of a collaboration between the onformative design studio and Christopher Warnow. and information about our experiments in the cultural domain. Please feel free to contact us to tell us what caught your eye.. look out for our call for contributions. in the attempt to leverage AR to allow artists to make incursions into public spaces. Head of AR Lab 7 In the second part of this issue. In Re+Public’s article we can read about blurring private properties’ boundaries. On this website.WELCOME.arlab. which will be posted on our website soon! . special attention is given to education: Experiences of bringing AR into the classroom are presented in separate articles by Alejandro Veliz Reyes. We invite you to visit our new website. Robin de Lange considers the long-term ramifications of extending both the mind and cognition itself with AR. In this issue. events and developments. We feature both articles with a philosophical perspective and articles with a more technical point of view. A recurring topic is set out by Yolande Kolstee: the legal ramifications of AR initiated in AR[t] #2. to the third edition of AR[t]. and in ways they were previously physically unable to do. Furthermore. We are confidant you will enjoy this issue. Crucial to AR experiences is the concept of believability. Maarten Lamers takes us back to pre-digital AR with his story on Pepper’s Ghost and at the other end of the spectrum Antal Ruhl explores the potential of using vestibular stimulation in order to create new AR experiences. we share artworks that caught our eye. explored by Edwin van der Heide. Should you like to contribute to issue #4. well-known among many people since the publication of his book ‘The Language of New Media’ in 2001. In our new section AR[t] Pick. the magazine about Augmented Reality.nl.. art and technology! In this issue we present articles by contributors from all over the world who are involved in stretching the borders of augmented reality. on the edge of art and technology. to print out at your local 3D print facility! Hanna Schraffenberger sets about interviewing Lev Manovich. we feature a short science fiction sequel by Oliver Percivall. Yolande Kolstee. which will provide you with all the information on our artistic and technical research — but also that of fellow researchers. We are very pleased to introduce to you the improved version of Marty: the video see-through AR head-up display. which might lead to an item there. describing some AR software programmes which are widely available. Wim van Eck continues his AR tutorials in the series ‘How did we do it’. scientists. Matt Ramirez and Yolande Kolstee. via the same URL www.

2011). photo by WILL SHERMAN BC “HEAVY” BIERMANN — PHD RE+PUBLIC: RE+IMAGINING PUBLIC SPACE 8 .OUTDOOR ADVERTISING: AR | AD TAKEOVER (NYC. RON ENGLISH.

As such. OUTDOOR ADVERTISING: AR | AD TAKEOVER (NYC. Re+Public has developed an experimental mobile device application that digitally resurfaces three specific areas of public space: outdoor advertising. which lies at the heart of our endeavor to re+imagine public space as a more open visual commons. the ability to pay for its usage. and buildings. With this goal in mind. murals. Seeking to understand a profit-driven governmentality and its affect on 9 . or a willingness to break the law.RE+PUBLIC A creative collaboration between The Heavy Projects (Los Angeles) and the Public Ad Campaign (New York City). this article focuses on these domains and demonstrates how Re+Public has used AR to transcend current private property boundaries. and augmented reality (AR) in particular. Re+Public seeks to leverage AR in an effort to allow artists to make incursions into public spaces in ways they were previously physically unable to do. Blurring private property boundaries. Re+Public is dedicated to using emerging media technology. to alter current expectations of our public media environment generally dictated by property ownership. 2011) AR presents unique opportunities to creatively problematize the political and economic systems that shape the awareness of both individuals and the public writ large.

AR permits users to see these murals as if they were actually back on the wall. Our digital infiltration into public space and takeover of commercial ads created a place of dialogic interaction rather than a monologic con- sumptive message. and Shepard Fairey. In addiIn 2009. To this end. The Bowery Mural Wall is an outdoor mural exhibition space in Manhattan. OUtdoor MUrals: Bowery Wall (NYC. whereas other technologies tend to disconnect the viewer from their immediate physical surroundings. Viewing previous murals online. We foresee AR mobile device technology as a first step in the transformation of public space into an arena shaped by user created content. the mural series commenced with a recreation of Keith Haring’s famous 1982 mural followed by work by such recognized artists as Os Gemeos. Owned by Goldman Properties since 1984. During Art Basel 2012. The AR | AD Takeover used street level ads and billboards to trigger a curated digital art installation that displayed on mobile devices. 2012) In contradistinction to the use of AR to problematize the consumptive monologue of outdoor advertising.public space. removes the viewer from the space by placing them squarely in the absent. and Swoon among many others. 10 . In 2012. Barry McGee. Once individuals experience this AR version of reality. 2012) and Wynwood Walls (Miami. Saner & Sego. John Fekner. artists who have contributed to the Wynwood Walls include Os Gemeos. In other words. Kenny Scharf. In other words. we used AR to rupture public space with a new kind of artistic interactivity. by pointing a mobile device at the present mural. they might start demanding a better version of public messaging than the billboard default. Nunca. real estate developer and arts supporter Tony Goldman started the Bowery Mural with Jeffery Deitch and Deitch Projects. It is precisely this kind of spatial “aura” that distinguishes AR from other types of emerging media technology. for example. Specifically. Goldman Properties and Tony Goldman who was looking to transform the industrial warehouse district of Miami also conceived the Wynwood Walls. Goldman endeavored to create a center that developed the area’s pedestrian potential. Retna. D. Beginning with the 25th and 26th Street complex of six separate buildings. Re+Public used AR to resurrect murals that once existed on the wall. as Drawing from an international pool of talent. in the space. Aiko. In New York City. While users could certainly view the previous murals online. AR is an incremental step towards showing the public an alternate view of their landscape. digital world. users could trigger web-based information related to the showcased artists whose work has historically addressed commercial advertising in public space such as: Ron English. at both the Bowery and Wynwood Walls sites. PosterBoy. and OX. that the current artist has painted over. Invader. Dr. they move in perspectival relation to the viewer. Re+Public investigated how civic authorities allow certain private parties to profit while preventing or discouraging other forms of public media production. In 2008. Faile. in situ. we envision AR as the first step in the evolution of better tools of expression that can democratize public media production. Re+Public was commissioned by the Wynwood Walls to create an AR experience. users are able to see the former murals. we augmented ads in Times Square with artistic content. AR connects the digital with the physical in an intimately “present” way. which commercial ads do not necessarily have to dominate.

Additionally. we created a 3D Kabuki theater that allowed users to walk into a digitized version of the AR is an incremental step towards showing the public an alternate view of their landscape. Finally. In addition to an immersive garden with a bridge. interactive environments for four other murals by How and Nosm. with the McGinness. Re+Public created 3D. Without pretending to discern any immediate resolution. which commercial ads do not necessarily have to dominate. 2012). the AR overlay used the 2D paintings as feature tracking markers and source material to produce an original expres- sion of creative content. we made it appear as if the paint colors were draining out of the mural. Aiko. over and above those embodied in the 2D mural. With the How and Nosm. rather abstract 3D environment and permitted users to both pull apart and reconstruct the mural elements. In the case of each traditional mural. RESURRECTED KEITH HARING tion to resurrecting a Shepard Fairey mural that he recently painted over with a new mural in tribute to the recently deceased Tony Goldman. we created another immersive. Ryan McGinness. With the Retna. mural where all of the elements were separated in Z-space. 11 . and flowing waterfall. we wrestled with this new type of work and wondered if it constituted a new mode of art. we worked directly with MOMO and collaborated on an AR version of his indoor mural at the Nicelook Gallery on site. stream. and Retna. we built the mural shapes in 3D and animated them to extend out of the wall and placed them both on the ground and above the wall. In considering our deployment at the Wynwood Walls in particular.OUTDOOR MURALS: BOWERY WALL (NYC. it is arguable that the AR assets represent original works in that they contain a sufficiently new visual expression of ideas.

Retna MUral AUgment. fUll view Retna MUral AUgment. screenshot 12 .

instead of placing converted art on the structure. In this way. WILLIAMSBURG ART & HISTORICAL CENTER BUILDING AUGMENTED WITH MOMO URBAN ART 13 . the AR assets should maintain some logical connection to the building or space upon which we have attached them. Both structures have a long “art history” in the city and AR allowed us to blur the lines between private and public space. we created the first example of what we refer to as “city visions. or entire buildings. in some meaningful way. FINAL REMARKS Whether it is outdoor ads. we placed his art on both the Pearl Paint and the Williamsburg Art & Historical Center buildings in New York City. This city vision type of deployment potentially provides more practical architectural and urban planning uses and maintains our notion that the AR experience should be spatially relevant in order to maintain the physi- cal aura that may have drawn the viewer to the building in the first place. we chose three buildings that had a particular cultural significance and. It is our hope that these early entrants will help create experiences that consider art and design as an important part of the way the public adopts this technology. we have also experimented with using buildings. we used AR to provide an artistic rendering of the re+imagined building by projecting it into a Blade Runner style future. To this end. site of many interior shots in the film Blade Runner (1982) into a futuristic version of itself. in our attempt to use AR to re+imagine public space. Ultimately. we really see the city as a canvas that allows for a multiplicity of voices to enter into our visual landscape. the digital overlay will become a much more seamless and natural part of our daily existence. rather than the current commercial hegemony. using AR. murals. in the visual urban messaging systems that surround them. we converted the Bradbury Building. In other words. or “skinned” physical buildings in urban centers by overlaying 3D content onto the physical environment. 2012) BRADBURY BUILDING (LA. Re+Public seeks to continue to deploy AR in an effort to democratize access to our shared visual environment and alter the current expectations of urban media in accomplishing our core mission of re+imagining public space. 2012) In addition to using both outdoor ads and murals as the markers that trigger our AR deployments. Converting MOMO’s 2D designs into digital 3D models optimized for mobile. made it possible for MOMO to put his art on buildings that he could not have accessed in his traditional 2D format. With the coming advancements in wearable computing.” Specifically. WILLIAMSBURG ART & DESIGN BUILDINGS (NYC. It is vital to the health of any city that its inhabitants are able to participate. In Los Angeles.AUGMENTED ARCHITECTURE: PEARL PAINT. and their unique architecture much in the same way. Working with muralist MOMO. we digitally resurfaced.

Heavy creates innovative interfaces between digital design and physical worlds in ways that that provoke the imagination and challenge existing styles of art. BC has internationally presented his academic work. RESURRECTED SHEPARD FAIREY.BC “Heavy” Biermann Deriving his pseudonym from his penchant for philosophical discussion. OUTDOOR MURALS: WYNWOOD WALLS (MIAMI. After finally giving up his painfully amateur skateboarding career due to a bum right knee… BC plans to use his extra time continuing to examine meaningful ways to fuse tech + creativity. Prague. With a PhD in Humanities [Intermedia Analysis] from the Universiteit van Amsterdam. As a kind of synthesis between scholarly inquiry and emerging media. 2012). and interaction. art and semiotics in public space. and Saint Louis. BC has worked as both a university professor and a tech developer in Anaheim. Since 2007. which explores augmented reality. photo by JORDAN SEILER 14 . BC “Heavy” Biermann possesses an interdisciplinary background that comprises technology. academia. and the arts. design. BC founded The Heavy Projects [and its collaborative spin-off Re+Public] to investigate how the fusion of creativity and technology can uncover new modes of relaying ideas. Building upon existing technological and theoretical frameworks.

15 .

or anything at any time be my display?” DISCLAImER The industrial design of Marty is meant to do experiments by researchers. 2011 16 . however. Verlinden In proceedings of the IEEE International Conference on Systems. natural features like dominant corners or lines in the scene are tracked. the air. partner of the AR Lab. Man and Cybernetics. One of the aims of the AR Lab is to stimulate AR in the world by putting our developments in public domain. meaning that no large scale industrial production can be based on this design without the consent of the copyright owner. We are also aware of Google glasses and other equipment and we welcome those developments! However. Jouke C. still many companies do not understand that you need two camera’s to see virtual objects in 3D and two camera’s to track (salient) keypoints in 3D in order to track your head pose while walking around with your headset in any unknown environment. which has another face mask. For more information: info@arlab. we will post the design files as well as a photo series how to assemble the Marty. When the new AR Lab website will be in the air at about the same time as this AR[t] 3 magazine is published. however. We are aiming at adapting Marty to version 2. Alaska.0 that is based on the HMZ T2. In the mean time be our guest and contribute to AR to come up with your own solutions. Chatzimichali. with the aim to do research on co-operative AR.MARTY. Pieter P. take a while. But also lower backpain. “why can laying on bed or in an armchair not the ceiling or the white wall. It is. cramped shoulders from long computer days and the sighing. hence there is extra knowledge on scale. Mien S. no assumptions on scale can be made and two cameras that see that same feature are necessary. If they are valuable we will publish about it in our next AR[t] magazine. In markerless systems. Anchorage. Keep an eye on our website as we will also link to the software to do 3D pose tracking based on natural feature tracking. subject to copyright. AR-toolkit like markers can do with one camera as the size of the marker is known and REFEREnCES ¢¢ Bridging the multiple reality gap: Application of Augmented Reality in new product development. Smulders. Anna P. ThE nEW AFFORDABLE hEADSET — PART 2 Pieter Jonker AND Yolande Kolstee In AR[t] 2 we announced the new MARTY videosee-through headset as a design based on the Sony HMZ T1. Frido E. USA. R. It was designed by Niels Mulder from studio RAAR on assignment of the TU-Delft. And finally: Yes we were already decades ago inspired by fighter pilots with heads-up display and of course Steve Mann. October 9-12. Due to a lack of manpower and funds this might.nl. the geek that walked around with an AR display. Piet Van den Bossche. We are well aware that the Sony HMTZ T2 is on the market. Wim Gijselaers. so anyone with access to rapid prototyping facilities can reproduce Marty for his/her own scientific or artistic research. Segers. artists and designers in the area of AR and stimulate industry to come up with affordable AR equipment. Hetty van Emmerik. Jonker.

ARLAB. 17 . THE NEW AFFORDABLE HEADSET.NL) WE WILL SOON POST A PHOTO SERIES ON HOW TO ASSEMBLE THE MARTY HEADSET.MARTY. Image by FERENC MOLNAR ON OUR NEW WEBSITE (WWW.

S. I work at the Augmented Reality Lab in The Hague and I am one of the editors of the AR[t] magazine. 2001.JEWIShphILOSOphYpLACE. So I hope you’ll agree to an interview for the magazine? Best regards.P. As a consequence. I finally also read The Language of New Media. . ImAGE SOURCE: WWW.WORDpRESS.FORWARDED mESSAGE ---------FROm: HAnnA SChRAFFEnBERGER <HAnnA@ARLAB.nL> DATE: 2013/4/3 15:53 SUBJECT: InTERVIEW AR[t] mAGAZInE TO: LEV MAnOVICh Maybe you remember me from Facebook. I realized that I would like to interview you about Augmented Reality for the AR[t] magazine. P. ThE MIT pRESS.---------. So I’ll challenge myself to challenge you. Maybe we can print my questions in issue 3 and your answers in issue 4? 18 MAnOVICh. ThE LAnGUAGE OF nEW mEDIA. LEV.COm Dear Lev. After my last few interviews. Hanna P. my supervisor (Edwin van der Heide) told me that I could/should be more critical towards my interview partners. When I read your article The Poetics of Augmented Space.S. I’d like to interview you even more. A short time ago.

In The Poetics of Augmented Space you describe AR as ‘the laying of dynamic and context-specific information over the visual field of a user’. Besides AR. you mention Janet Cardiff’s audio walks as great examples of laying information over physical space. Augmented Space refers to all those physical spaces that are overlaid with dynamic information such as shopping malls and entertainment centers that are filled with electronic screens and all those places where one can access information wirelessly on phones. Firstly. And secondly. While navigating the environment. I would like to ask you what you consider Augmented Reality (AR) to be. monitoring. 19 . In The Poetics of Augmented Space. because our readers might not have read your article.Augmented Reality What is Augmented Reality? To begin with. Wouldn’t you? Augmented Space What is special about AR compared to other forms of Augmented Space? In your article The Poetics of Augmented Space you discuss the concept of Augmented Space. among which. In contrast to ‘typical’ visual AR. because I think that this point of view unnecessarily limits AR to the visual sense. for example. ubiquitous computing. Personally. I would call this Augmented Reality. These walks are designed for specific walking routes. the user is presented with auditory information that relates to the immediate surrounding space. tablets or laptops. one gets to listen to a mix of edited sounds that blend in with the sounds of the surroundings. you mention several other technological developments in the context of Augmented Space. It would be great if you’d address the topic once more. as well as spoken narrative elements and instructions such as where to go and what to look at.

the information and my spatial surroundings aren’t perceived as a single gestalt. In AR. something virtual augments something real. but I don’t consider it an augmentation of space. So judging from my experience of listening to the radio. What is augmented depends on what the additional content relates to. augmented humans. information and the surrounding space can be perceived as one single mixed thing as well as independently. there is a relationship between both. you have only discussed the augmentation of space. For example. Listening to a radio play. that nothing is happening there. a song is just a song. 20 . pattern of work. augmented perception. Is AR just one of many related recent phenomena that play a role in overlaying the physical space with information? What’s special about AR compared to other forms of Augmented Space? that in AR. Edwin (my supervisor) and I have recently given this topic a lot of thought and we were fascinated by the questions: “What is actually augmented in Augmented Reality? What else can (we imagine to) be augmented?” We came up with the answer. the sound of the radio just exists as a separate. there are more possibilities. and has nothing to do with the surrounding space. focus of attention or interests of their user. In our view. relates to one individual in the space (me). I was wondering whether you have considered other manifestations of augmentation as well. Do the real space and the dynamically presented information add up to one single coherent phenomenological gestalt or are they processed as separate layers? I am a bit of a sound-person and it has always fascinated me that sometimes the sounds of a radio seem to mix in with environmental sounds.tangible interfaces and smart objects. space is one of the possibilities. just to find. an event could sound so real and so nearby. but nevertheless. you mention software that performs tasks according to the mood. I might consider it an augmentation of the activity (not moving. augmented content and augmented activities. most often. so far. Do you think that all forms of augmentation bring along an augmentation of space or influence our experience of the immediate surrounding space? What else can be augmented? Something I really like about your article is that you see augmentation as an idea and a practice rather than a collection of technologies. but likewise. independent layer of content. the ticking of a red streetlight might perfectly mix in with the rhythm of the song that is currently playing. This information Information and space — one coherent gestalt? In The Poetics of Augmented Space you raise a question that intrigues me a lot. sitting still) or an augmentation of the user (me). For example. However. More specifically. I am curious whether you’d agree. The voice of the newsreader doesn’t mix with the voice of my colleague. that I’d turn around. something virtual augments something real. I don’t think augmentation is limited to a space or an environment. But besides these two options. the virtual augments that to which it relates. but I don’t think it has anything to do with the surrounding space. I’d even say that often it’s not the space that is augmented. I am doubtful whether our experience of a space is affected by this kind of information. But of course. we have considered things like augmented objects. to the activity the person is performing (sitting still). For example. nor does it relate to my environment. Here. Most of the time. the newsreader might tell me about a traffic jam and thereby inform me about my immediate physical space. Let’s imagine that my phone registered that I have been sitting still for a long time and reminds me to take a short break to stretch my legs. but something else. However. Hence.

Will photorealism be traded in for a form of realism that encompasses all senses? Do you think new media will develop towards a more multimodal form? AR & the second space From The Language of New Media. On the other hand. we find examples that date back centuries. when we consider the concept of AR. the virtual things that appear to exist in our physical space should not only look like real things — ideally they also feel like them.g. Personally. which contains both virtual and real elements. smell like them. Virtual Reality. I was wondering: How does AR fit in the widespread aspiration towards realism? On the one hand. there are more and more so-called AR applications.g. If we look at contemporary AR and compare that with other forms of new media. a virtual bird is sitting on a real tree) there is no second space. However. Of course. websites and CD-ROMs — to cinema. disappear or morph into each other in an otherwise real. AR technologies and new media works that work with AR. But even if the concept isn’t new. So some questions I’d like you to answer with respect to Augmented Space are: When are information and space perceived independently from each other — would you still call these occurrences Augmented Space? When are information and space perceived as separate but related layers? And when and why do they add up to one single gestalt? equally to renaissance paintings and to modern computer displays. How about the relation between AR and cinema? 21 . taste like them and behave like them. I don’t think of AR as a recent phenomenon. you relate different forms of new media — e. When we imagine a typical AR scenario in which virtual objects are integrated into a real scene (e. information and space might be related. An example of ancient AR is the Pepper’s Ghost trick (which is discussed by Maarten Lamers on page 24). the virtual leaves the realm of virtual space and enters our real physical environments — with respect to that the images might be experienced as more realistic than ever… Will AR take the quest for realism to a new level? I can imagine. You note that new technological developments illustrate how unrealistic the previous existing images were. physical environment.I think the same is true for Augmented Space. what’s special about it and what isn’t? is something that has always bored me. glass and special lighting in order to let objects seem to appear. this function of the screen applies AR & cinema In The Language of New Media. Is this a fundamental change in visual culture? AR & the quest for realism The quest for realism in computer graphics New Media One of the main questions I want to ask you is: What makes Augmented Reality special? I have posed that question with respect to other forms of augmented space. I understood that throughout media history. visual AR could be considered a huge step back. when striving for realism. The 3D models that are usually integrated in real space don't come close to the level of photorealism we know from cinema. Often. It uses a second room. At the same time they remind us that current images will also be superseded. I’d like to ask it again with respect to the history of new media. For example. It’s the same physical space. even when they don’t add up to one phenomenological gestalt. the screen was used to separate two absolutely different spaces. current manifestations of AR might still bring something new and special to the table.

Can we understand this as a pure database? What are the consequences of working with spatialized elements? What are the inherent limitations and possibilities when working with this form? (I can imagine it has consequences. You argue that what artists or designers do. provide a theory of new media. list on a page). So the only way to create a pure database is to spatialise it. In The Language of New Media you write about the elements of a database: “If the elements exist in one dimension (time of a film. L.” In AR. a virtual pen and a virtual painting. e.) What is the interface to access the database? Is it my phone? What does the artist create? I think it is usually the virtual content and its re¢¢ ¢¢ Manovich. but I guess most of what we see in visual AR has been present in cinema for a long time. you also pointed your readers towards aspects of new media that were still relatively unexplored at that time and you suggested directions for practical experimentation. virtual elements are distributed in real space. Do you agree? You oppose compositing to montage: while compositing aims to blend different elements into a single gestalt. artists and designers create a database for an existing interface? I have one more question about databases.. The MIT press.I’m certainly not a cinema expert. they will be inevitably ordered. is constructing an interface to such a database. you did not only One of the main claims in The Language of New Media is that at their basis. we cannot assume that elements will form a narrative when they are accessed in an arbitrary order. (Technically speaking. when creating a new media work. 219-240. The database might hold a virtual chair.g. where an image of a dream appears over a man’s sleeping head. and emotional dissonance between them. Manovich. As far as I understand it. We can see this as a database filled with virtual objects. The poetics of augmented space.) Does visual AR use similar concepts as cinema? Does cinema use other techniques to create fictional realities that are not (yet) used in AR? Does AR use techniques that might be adapted by cinema in the future? lationship to something real. Are there certain aspects of Augmented Reality you consider especially interesting for future experiments and explorations? AR & future research References 22 . this can be seen as an analogy to compositing in films: an attempt to blend the virtual and the real into a seamless whole ‘augmented’ reality. 5(2). These virtual objects are displayed as part of a real room when a user views the augmented environment with a smartphone. The same could easily be done in AR. (2001). The language of new media. Visual Communication.) AR as spatialized databases With The Language of New Media. stylistic. Do we have montage in AR as well? (You give the example of montage within a shot. semantic. (2006). For example.. AR research is very concerned with registering virtual objects in real space. all new media works are databases. Let’s apply this database theory to a typical AR scenario in which virtual objects (seem to) appear in a real environment. distributing the elements in space. L. Could we say that when working with AR. for storytelling? As you point out. we could say the real world serves as a database index for those virtual elements. montage aims to create visual.

“[.. The Language of New Media 23 Image COURTESY OF LEV MAnOVICh . distributing the elements in space.” Lev Manovich.] the only way to create a pure database is to spatialise it..

With no computer required. 24 ... I was most impressed by the augmented reality of Pepper’s Ghost. Besides meeting Mickey Mouse.Disneyla nd. I mentioned how the “Mozzies” mobile game of 2004 was my earliest encounter with augmented reality. LAMERS In the first issue of this magazine. I lied. by 26 years! In 1978 my parents took me and my brother to Disneyland. 1978 n Valentij rten a a M PRE-DIGITAL AUGMENTED REALITY MAARTEN H...

But good old-fashioned Pepper’s Ghost projection is what augments our reality with that content. its use is apparent. render and interact with virtual content. He explained something about glass. but also as part of modern optical see-through AR technology. Pepper’s Ghost technique was first described in the 16th century and later refined by John Pepper around 1860. In effect. darkness and reflecting light. Thank God.com/projects/holodesk/) project. Scary stuff.net lboUrke rke. of which I can remember only one thing: the car stopped. Actually. if you know other pre-digital augmented reality techniques. Disney augmented the reality that we hold our mirrored image to be: pre-digital augmented reality.microsoft. However. digital technology lets us define. in current head-up see-through displays Pepper’s Ghost technology is less apparent but nonetheless used in the same fashion. send a short description to lamers@liacs. To me it is interesting that we still rely on John Pepper’s idea to add digital content to our optical reality. Naturally. rotated 90 degrees to the right. a mechanical funhouse car drove us through Disney’s “Haunted Mansion” ride. facing each other. and help me put AR in perspective. paU PaUl BoU . . Combining this with a large mirror. It is still used in amuse- ment parks and museums today. he explained what is known as Pepper’s Ghost: a simple but clever technique that creates holographic scenes in 2 or 3 dimensions. In Microsoft Research’s HoloDesk (see research. Who would have guessed that such basic illusionary tricks are crucial to what we now consider cutting-edge technology? In fact. facing a large mirror. and on the seat between us appeared the scariest moving ghost ever! Instantly our heads turned.com Sitting beside my brother. The mirror showed us some weird illusion. the ghost wasn’t really between us. most realistically. I asked my dad how the illusion worked. 25 Image of coUrtesy .Image coU rtesy of D avid http Llewellyn ://ravenmanor .nl.

Two electrodes.ImAGE BY KRISTIn SmITh VESTIBULAR STImULATIOn AS An AUGmEnTED REALITY LAYER? Antal RUhl Over the past few years the field of Augmented Reality has started to include non-visual forms of augmentation to their scope. I needed a connection to the physical world. 26 . To find out whether GVS can be used as a new form of AR. I constructed a device and developed a series of experiments. This technique has been around for a long time and many studies have been done in order to learn more about the human balance system. Applying a specific current to those mastoids will result in an altered balance perception related to both to the intensity and the direction of the current. usually an anode and a cathode. Galvanic Vestibular Stimulation (GVS) stimulates the human vestibular system (balance organ) by means of electric stimulation. When I became interested in galvanic vestibular stimulation I realized that this could become a whole new form of AR. are placed on the mastoids (the bones behind the ear). I used my body orientation to alter my balance. Since I also wanted to investigate what the effect would be to our balance system. but never has it been used as an extra sensory input. In order to use GVS as a form of augmented reality.

The control unit calculates the appropriate intensity and direction of the current in real-time and provides feedback to the electrodes. own orientation. Without even using external data to augment an extra layer to reality I created an altered world based on data from the physical world. While testing. it felt like my resistance was really low since the device backed up every movement I made.GALVANIC VESTIBULAR STIMULATION CAN BE USED TO ALTER BALANCE PERCEPTION Setup To do so. I mounted an accelerometer on top of my head-mounted GVS device in order to measure its orientation. if I was tilted to the left in modus A. while modus B could amplify it. The middle part is left uncovered to retain the full use of users’ auditive orientation. it had the opposite effect. The electrodes are incorporated in a pair of headphones to make sure they are pressed against the mastoids properly. it felt like I was moving through a liquid or a thick syrup-like medium. Testing this simple setup immediately revealed some potential AR applications of vestibular stimulation. The GVS device counteracted all my movements. Is this a form of augmented reality or is it an alter- Conditions The first step of my research project was measuring the effect of an altered balance. In modus B. Modus A could counteract my balance. I realized that. it would push me back to the center. based on my 27 . in modus B. when put in modus A. The data from the accelerometer is sent to the control unit. There were two possible modi for the control unit. it would push me further to the left. In other words. The headphones are in fact merely rings around the ears. and to make sure this wouldn’t influence the results. so it took more effort to move around. This provided me with the setup as shown in Figure 1.

we can already create an enriched environment. perhaps more Figure 1: Galvanic VestibUlar StimUlation device troublesome. But a more practical solution must be found. I found that in certain cirnate reality? Be that as it may.) is measured. But can we also use it in a more traditional AR environment? In other words. Although we have to be very careful about drawing conclusions from these results (given the self-experimentation and singlesubject constraints). Using my self-constructed device. for now. Other issues were the bright light flashes I experienced while testing. But you can also think about fighter pilots who fly in simulators that don’t alter pilots’ balance when flying upside down. I have used this device for over two weeks to see if there was any progress. Using electrode gel and covering the electrodes with sponges soaked in a salt-water solution did the trick. which could be used to train divers or astronauts who work in other environments. Using only these two simple orientationbased modi. experience I had was the adapta- 28 . could we use this system to add computer generated data to an extra sensory layer of our environment? The easy answer is: Yes! It’s not hard to imagine that you can enrich game play with wave motions for example. modified and used as an input to alter the actuators (in this case the electrodes which in turn alter my own balance). it does fall within the definition of mediated reality. and letting them experience their surroundings in a modified fashion. walking to targets on the street. During the testing phase. walking over a balance beam. playing Wii Balance Board games and many other examples. (in this case body orientation. The skin’s resistance is an important factor in getting the electric pulses down to the desired area. Another. cumstances my balance actually improved while using this device. I was wondering whether I can diminish or amplify motion sickness. Or simulate the G-forces in a racing game’s sharp corners. For example. but for what use? The examples point towards simulations of the physical world. and nausea. improve my balancing skills etc. which is the broader field of manipulating users’ sensory perception through a wearable device. before we can use this. Tests involved bus rides. This system could enhance a simulated environment by distorting the balance organ based on the simulators’ virtual orientation. This creates a broad range of possibilities. However. But these issues might all be overcome with the right execution.The tests that I have done focused on enhanced performances in everyday life situations. there are some practical considerations to keep in mind. both are not very userfriendly. I experienced some burn marks of the skin around the mastoids. Can this be used as an interface? I demonstrated a system in which the real world environment. This might be of use for people with an impaired balance system or in those situations when accurate balance is crucial.

Interactive and playful objects are much better in holding someone´s attention while they convey their message”. Los Angeles. science and art.tion effect. I saw that. June 23.com/media_tech/paper. the possibilities are endless if you use your imagination. Media Technology at Leiden University and Design and Media Arts at UCLA. the effect was constant. “Using the technological possibilities at hand we can enrich our work and create an engaging object. which are very important design-considerations if we want to use a GVS device as an interface. Sometimes they are purely conceptual or formal and sometimes they serve a more commercial purpose. and removed it after that time. and only changed the pulse-width to affect my balance. But when I used an alternating pulse. when I was wearing the device for about fifteen minutes. Secondly. Arnhem. it gave me the experience as if I was moving through a liquid. While using GVS for augmented reality purposes is still in a research stage. His work can be described as visually and technically attractive with a focus on natural and physical phenomena. After a few days the brain adapted to the newly displayed environment and reversed the image back to normal. While my body needed some time to get used to my ‘new and improved’ balance.creativeinmotion. When I used a constant pulse on the electrodes (a constant current in the same direction) the effect diminished quite soon. And it did so in two ways. it felt like I had switched it to modus B. giving off the impression of moving through a low friction environment. Antal worked at distinguished design companies in Amsterdam and Barcelona. These objects vary from kinetic sculptures to interactive installations. Your body is extremely good at adapting to new situations and environments. but they always share the goal to intrigue people. Creative in Motion is a creative brand activation studio: www. You might remember that study from the 1960’s where a test subject was wearing reversal goggles. The Netherlands. published a scientific paper.antalruhl. After removing the goggles.nl For an overview of his work or to contact Antal please visit: www. it did so quite convincingly. Antal creates objects that let us rethink our environment. Apparently I was much more sensitive to a change in stimulation than to the actual stimulation itself. The same effect occurred with my balance organ. This article is based on research presented at Chi Sparks 2011 (Chi Nederland 15th conference). can be found here: www. After removing the GVS device I experienced the exact opposite. my body had to adapt to this ‘new’ non-stimulated situation.com 29 . Antal studied Industrial Design at the The Hague University of Applied Sciences. festivals and event. 2011.antalruhl.pdf Antal Ruhl Antal Ruhl is a media artist with a background in design. the subject saw the world upside down because the body had adapted to this new situation. Experiments with Galvanic Vestibular Stimulation in Daily Activities: An Informal Look into its Suitability for HCI. Antal has also started a company to develop interactive installation for brands. has had several (national and international) performances and exhibition. The presented paper. So when I turned it off. travelled the world and currently works a freelance artist/ designer. thus. but at a much quicker pace. when I was wearing the device while it was set in modus A.

the story can be entirely fictional and it nevertheless draws us in without questioning its realness or truthfulness. The examples from the bank’s e-mail as well as the novel show us that believability is actually independent from something being real or fake. but is in fact fake. An important factor for a story to be believable is that we can relate ourselves to it (or the story relates to us). When we receive an e-mail that claims to be from our bank with the request to enter our account information on ‘their’ website in order to upgrade its security. it pretends to be real. The e-mail is believable. independent from something being real or ‘virtual’. while not real. or in other words. The story is believable. and we enjoy it. We’ve learned to get suspicious and need to verify the credibility of the story. appears to be authentic. What we also learn from 30 . we enter a situation in which the content appears to be believable. When we read a novel.BELIEVABILITY Edwin van der Heide Believability is something we deal with on a continuous basis.

When I was writing my paper about the interactive art installation Spatial Sounds (100dB at 100km/h) for the Third International conference on HumanRobot Personal Relationships (HRPR) I was introduced to a topic addressed by Kerstin Dautenhahn (2007) that fascinated me: The believability of a Robot.Spatial SoUnds (100dB at 100km/h) at Wood Street Galeries. USA. 2009 .Image coUrtesy StUdio Edwin van der Heide the example of the novel is that things that happen in the story don’t have to be possible in reality. In a novel we can meet creatures from Mars. I realized that the believability (of the behavior) of an artwork was not (yet) seen as a fundamental topic and might deserve its own study and experimentation. 31 . PittsbUrgh. We can ‘simply’ imagine these things and believe them. time-travel. but it also triggered me to think about the believability of an artwork and. The reason that it fascinated me is that it made me not only think about believability in the context of robots. the believability of the behavior of an interactive artwork (as opposed to the believability of a robot’s behavior). or never go to bed. more specifically.

” I tried to imagine an example and came up with the idea of a robot that makes and brings you a cup of coffee. It’s by imagining that we can change what we believe and thereby turn a machine into a believable affective robot. we don’t imagine any intelligence and we simply think of it as a machine without affection for us. It is interesting to question whether these machines indeed make us believe they show affection. Does this then mean that the representation is a requirement for believability? No. This might be the reason that in Japan a lot of machines talk or include animations to. it allows us to somehow identify ourselves with it. In the HRPR paper about Spatial Sounds (100dB at 100km/h) I put it like this: “The installation can be seen as a non-verbal abstract robot and does not imitate an animal or human-like look or behavior. but it also has to ‘do the things right’.e. a robot (or interactive installation) doesn’t have to represent something (else). for example. i.Kerstin Dautenhahn has an interest in socially intelligent robots: “A robot companion in a home environment needs to ‘do the right things’.e. even ‘feels’ affection for us. It is a machine-like object but does not resemble existing machines. Nevertheless. Robots are often made to look and behave like humans (the humanoid) and the coffee machine isn’t. If we. How do the two differ from each other? With the robot we can imagine that he read our mind and therefore made a coffee for us. it has to be useful and perform tasks around the house. There is another possible difference between the coffee machine and the robot making coffee. and if so. perhaps. It can be an abstract work that is believable on its own. if it lasts or wears off. We imagine that the robot has a certain amount of intelligence and. Soon after I had that idea I had to think about a full automatic coffee machine that grinds the beans and steams fresh milk for each individual cup of coffee that it makes. in a manner that is believable and acceptable to humans. In the case of the coffee machine. however. i. Spatial Sounds (100dB at 100km/h) is an example of a believable robot in the sense that the visitors believe they 32 . welcome and/ or thank the user for using it. imagine that the coffee machine made that coffee especially for us things start to change.

679-704. The aspect of believability is so strong that people accept the installation as a real being and want to interact with it over and over” (van der Heide. (2010). H. Japan. (2011). 2011).. What aspects make combinations of. References ¢¢ Dautenhahn. Verbeek F. 2006 — Image coUrtesy StUdio Edwin van der Heide 33 . (2007). Human-Robot Personal Relationships (HRPR 2010). http://mediatechnology. J. 59..understand the behavior of the installation and find it worthwhile to interact with. We have also learned that something doesn't have to be possible in order to be believable. The former makes me believe that believability forms an interesting perspective to think about what we can imagine in augmented realities! There are certain things we believe in that. Interesting to read in this context is Alwin de Rooij’s (2010) graduation research project for the Media Technology Master program on ‘Abstract Affective Robotics’. Socially intelligent robots: dimensions of human–robot interaction. it can be abstract and nevertheless be believable. In: Lamers M. I’m curious how the abovementioned thoughts apply to augmented realities. Spatial Sounds (100dB at 100km/h) in the Context of Human Robot Personal Relationships. Abstract Affective Robotics. ¢¢ de Rooij. 362(1480). the real and the virtual in augmented reality believable? We have learned that something doesn’t have to be true to be believable. and interactions between. E. A.leiden. ¢¢ van der Heide. LNICST Vol. 27-33. Philosophical Transactions of the Royal Society B: Biological Sciences. we’ve learned that something doesn’t have to represent something. K. while we keep on believing other things that we know are fake. completely lose their believability. Besides that.edu/ research/theses/abstract-affective-robotics Spatial SoUnds (100dB at 100km/h) at DAF Tokyo. once we discover they are fake.

In other words. communicate and enhance those qualities of the historical artefact that are the most meaningful to us. visitors ‘scan’ the displayed objects from a distance. JoUke Verlinden. followed by the invention of Delft’s blue as an attempt to copy Chinese porcelain. 34 . it presents an auditory environment to augment the artefact in an unobtrusive and non-linear way. We hypothesize that by using these 3D imaging techniques the value of our cultural heritage can be increased.NOnLInEAR STORIES TOLD BY CUpS AnD SAUCERS Smart Replic as with responsive 3D aUdio Lotte de ReUs. 2012): a new kind of reproduction. Completed as a graduation project. let alone use them. in a showcase with extra textual information added to it. As a result. Museum visitors can never touch the objects. New methods of access and new digitization strategies based on the study of the relationship between the ‘bits’ and ‘atoms’ are being developed. something that needs reconsideration in our present time where the ‘experience’ is essential. that stretches the boundaries of the replica’s concept as an autonomous object based on a historical artefact. These are currently on display in a new exhibition on design and pre-industrial design. in the shape of a 3D print. Starting with the first import of porcelain from China in the seventeenth century by the Dutch East Indies Company. Here we present the design project of Lotte de Reus in connection with this paradigm shift. the goal is not to make the most realistic copy of the original. Maaike RoozenbUrg Introduction In museum exhibitions historical objects are usually shown by visual display. each of these teacup and saucer sets represents a milestone in the Dutch history of porcelain. the so-called ‘smart replica’ was proposed in the previous issue of Ar[t] (Roozenburg. Depicted in figure 1. To provide a way around this situation. but to analyse. now. The objects that are central in this project are seven teacups and saucers that are part of the collection of Museum Boijmans Van Beuningen. and ending with the small scale production of porcelain in the Netherlands.

full of untold stories. An important part of the collection are the nonlinear narratives. while the information passively waits for the visitor act upon it. Because of the passive character of the objects and their corresponding information. Concept of tangible interaction with responsive audio For the seven teacups and saucers that have been chosen to represent the history of the Dutch quest for porcelain. THE SEVEN CUPS AND SAUCERS picture by HUGO BES In the spectrum of recreational activities. These types of encounters could correspond with respectively a museum or a theme park. In the project discussed here. These qualities feed the resulting research questions: How can museums anticipate and facili- Figure 1. Augmented matter — the mixture between bits and atoms — allows novel interaction techniques to embody these networks of information. thereby increasing the opportunity to reflect and learn even after the visit. networks of information associated with the object. leaving the exhibition behind. the exhibitions of Boijmans Van Beuningen can be characterised as enriching experiences: an encounter in which the visitor is conscious of the artefact and the (hi)story it represents. 2012). it takes effort on the part of the visitors to maintain concentration. as shown in Table 1. The participants walked into a room. Knowledge and culture become more accessible because of the actual presence of the objects. a focus group that visited the museum exactly proved this problem case in hand. rituals. understanding. paying a visit to an exhibition enables the visitor to have an intimate encounter with the artefacts on display (van Dongen. In the vision of the curator. the collection’s past and the like. context and reference. went on to the next. For this design project our aim was to convey the following qualities of interaction: intrigue. –Visitors’ overall interaction with Boijmans’ exhibitions can mainly be described as scanning. the ECR-model is used to classify the valuable information (Wong. The engagement of visitors is limited as they are not experts on the particular subject of the exhibition. looked at an object briefly. locations. This storytelling model categorises the information in three layers: engagement. The artefacts shown and experienced in an exhibition construct a powerful medium which instils visitors with thoughts they perhaps have not yet fostered. satisfaction. started at a random showcase. This means that a museum requires the visitor to reach out for information. Our hypothesis is that more active means will lead to a more comprehensive museum experience. including museum shops. 2012). 35 . and integrity. materials. and tate the active assemblage of old and new stories – and how do these stories refer back to the replica’s original? How can digital databases be employed in linking smart replicas to their collections? On a philosophical level: does the original still attract interest? In the case of pre-industrial utensils such as the teacups and saucers this question is very relevant. They draw audiences in through exhibitions and related facilities. restaurants and so forth. consisting of stories.Active enrichment but not entertainment? Museums and their collections house a specific group of objects with their corresponding historical data.

connecting the narrative with what is visualised about the object. c. Proposed dialogUe between visitor and exhibition. the moment when a visitor has some immediate sensory. e. visitors are attracted to participate in the experience by other visitors who are listening to the 3D audio clips. the spatial audio of a narrator starts. depending on user location / viewing angle. the visitor is intrigued and triggered to participate in the exhibition. 2012). b. Context: Draw the visitors’ attention. Table 1. visual). a 3D ambient soundscape that fits in with the history of the specific teacup and saucer will appear — once the first soundscape has been heard. Identification data (text. while retaining the soundscape. information on the role (audio). The app contains more background information on porcelain and the objects. available after the exhibition. Close proximity to object. These audio clips give the impression that they are experiencing more: it is as if their auditory attention has doubled their visual attention (Erens. Interaction characteristics Spatialised audio.  Narration 2 — information on specifics that can directly be related to the object (audio). Together with an app that is made available. it attracts the visitor to go to other objects accompanied by a spatial soundscape. In the first phase. Background information (text and images). proximity to an object and the han- 36 . When the visitor picks up a replica. When the visitor is in the proximity of an object. Narration 1. When the visitor returns home. Ambient soundscape (audio). The audio clips are played when the visitor shows interest through their actions. the moment when a visitor has some immediate sensory. The narrator tells about the role the specific objects (“audio story”). every teacup and saucer holds a series of three audio clips. emotional or intellectual response to the artefact. Information a. Enabling spatial audio As discussed in Table 1. to browse the additional information for example. Arriving in close proximity of the teacup and saucer. It provides more detailed and interpretive information about the work. the narra- tor focuses on the features of the teacup or saucer (“audio specifics”). Information plaque. In this stage the visitor is encouraged to turn and explore the object. d. Triggered by handling an object.ECR_Phase Engagement: Draw the visitors’ attention. she can consult the app at her convenience. Triggered by proximity to object. Reference: Gives the visitor the opportunity to draw conclusions and connect to related resources. Exhibition app. emotional or intellectual response to the artefact. The visitor can put down an object and pick up another object.

SPATIAL AUDIO INTERFACE OF AUDIOSTAGE. it works more associatively. scanning behaviour can be transformed into an immersive encounter. The essential observations include: Participants seem to enjoy the ambient soundscape: it triggers the imagination and the recognition that the objects used to be utensils and not art object as they are now. 140 people interacted with a selection of the configurations presented in pairs. The technology to spatialise audio was developed two decades ago. without visual clutter and conserves traditional values on how art should be experienced. and speaks directly to the visitor (Erens. Presenting the information via audio. 7-10 February 2013). 37 . cf. yielding four permutations. The soundscape is the auditory equivalent of an ambient image. To connect this aspect to the exhibition visitor. position and orientation tracking of the human head as well as the objects at hand is required. audio clips allow temporal cues. typically with so-called earcons which represent specific objects or events. Furthermore. Presenting the information via responsive audio facilitated the visitors to consciously turn the object to find the image that the narrator referred to. Participants appeared to be pleased with the responsive auditory system. Preliminary evaluation The core of the concept was tested during the Object design fair (Rotterdam. even those visitors who preferred text to audio clips. With the use of tangible replicas and 3D audio clips. Ambient soundscapes were on or off. does not clutter the visual sense. Figure 2. We used AudioStage to produce our audio clips with a visual interface. whilst the visitor is holding the object. the brain places the sources of the audio clip in your head.dling of an object. An example of a high quality 3D audio clip is “In your head” by Big Orange. non-visual immersive content. Here optical or magnetic tracking principles also make sense. Auditory feedback enriches the visitors’ experience actively. see Burgess and Verlinden (1993) for example. as they are fit for indoor use. Information about the teacups and saucers was presented via text or audio clips. Figure 2. The character of 3D audio relates to the idea that the sounds seem to come from sound sources placed anywhere in a space (a surround sound effect). When listening through headphones. 2012). they were asked to choose between the two displays and support their reasoning.

the feeling as though you are somehow “in touch” with the past (Ankersmit. the experience fits in with the integrity values of the museum Boijmans Van Beuningen – the 3D scanned and printed. texture. this 1. one-sided encounter with the objects now becomes an active two-sided encounter. Impression of the preliminary evalUation. a satisfactorily quality is propelled. tangible interaction with physical replicas and the connection between information and corresponding visuals. Combining ambient soundscapes. 2. Secondly. understanding is nurtured because the encounter is intuitive and the information presented by the artefact responds to the visitor’s body language. weight. Through the app.Conclusion Once the teacups and saucers were objects of dailylife and their form. as well as the people who used them. Lastly. and can be connected to various forms of social networking websites and location-based services. triggers an immersive encounter in which this sensibility is restored: the passive. Furthermore. it is more or less compatible with existing gear already employed by many museums. moulded physical replicas afford what Dutch historian Johan Huizinga has called a “historical sensation”. the experience is saved. colour. substance. Firstly. which can be browsed in a non-linear mode by the visitor.  Experimentation with augmenting untouchable artwork with ambient soundscapes.  Implementation of responsive audio tours in the current exhibition context. creating curiosity and making visitors want to engage with the artefacts. Because the visitor has a direct intellectual sensory dialogue with the object. In essence this proposal presents a new type of relationship between visitor and object that has interaction qualities equal to a human conversation. 38 . by the layered structure of the narratives. it rather extends the existing strengths with emerging technologies such as indoor tracking and spatialised soundscapes. The proposed system is by no means the first auditory guide for exhibits. Future work includes experimenting and researching the effect of the design in the environment of a museum: Figure 3. this poses an intriguing quality that pulls the visitor in by using 3D audio clips and ambient soundscapes. Thirdly. will lead to a more meaningful experience. 2005). decoration make sense primarily in the context of their functions and relations to other objects.

Rotterdam: Erasmus Centrum voor Kunst. C. Lotte de Reus Lotte de Reus recently received her Master’s degree in Design for Interaction at the Faculty of Industrial Design Engineering at Delft University of Technology. (2005).R. a high quality 3D audio example: http://www. 21 November 2012. References ¢¢ ¢¢ Ankersmit. A. Proceedings of Virtual Reality Systems Conference (Fall ‘93). curator at the Museum Boijmans Van Beuningen for her constructive collaboration. Professor Joris Dik.longcat.blogspot. ¢¢ Halbertsma.big-orange. (1995). More information ¢¢ Video presentation of this project (headphones required): http://www. Dr. (2012). com/2008/05/new-work-detroit-institute-of-1 ¢¢ ¢¢ In your head. Burgess.nl Acknowledgements We would like to thank Alexandra van Dongen. special interest for her.. ¢¢ Erens. F. immersive soundscapes and 3D audio now hold a new. Conversation with Curtis Wong. we would like to acknowledge the valuable advice and support by Cilia Erens. Many thanks to DOEN foundation for funding the Smart Replicas project (including the graduation project of Lotte de Reus) and many thanks to Mareco prototyping for their contribution in the form of 3D micro prints. Wolf Song and Ir. ¢¢ The Futures Channel. D. [online] Available at: http:// www. Wim Verwaal. Sublime historical experience. Woerden: Drukkerij Zuidam. something she had not yet encountered in her studies.nl. During the project she was pleasantly surprised by the world of audio. (1999). with special attention to smartphone infrastructure. she was driven by a fascination for porcelain and the wish to create effective storytelling experiences. An architecture for spatial audio servers. Museale strategie. Interviewed by Lotte de Reus. Her portfolio can be found at www. (1993). Verlinden. Bewaren om teweeg te brengen. M.youtube.3. dierentuinen en musea. Interview on soundscapes. Furthermore.com/ watch?v=enR1Ggbuf_8 ¢¢ Software to render binaural output by visually placing audioclips in 3D: http://www. Lotte would again like to work in the domains that combine art and technology. Interview on the museum context. 39 . In July 2012 she started her graduation project for the Smart Replicas project. Cultuur en Wetenschap (2005). Interviewed by Lotte de Reus.en Cultuurwetenschappen.. Rotterdam.php ¢¢ Van Dongen.thefutureschannel. Amsterdam. In the future. 4.  Prototyping indoor tracking and interaction sensing possibilities. fr/web/en/prods/audiostage ¢¢ Interactive installations regarding preindustrial utensils: http://new.C. (2012). J. Stanford University Press.lottedereus. New technologies such as augmented audio.nl Official Smart Replica blog: http://smartreplicas. 14 December. ¢¢ Ministerie van Onderwijs. Themaparken.  Creating guidelines on how the concept could be implemented to suit different kinds of objects in the museum (or maybe even utensils in our everyday context).pentagram. A.com/conversations_ archive/wong_conversation.

GUIDO LAGERWEIJ 40 40 .Image: REVEALInG IDEnTITY.

this is not the case.185. de facto. giving off the impression as if people are gossiping about them in a different language or at a very low tone. in theory. and actually is everywhere and that this is a special characteristic of AR for as a new medium for artists. The only thing you need to do 41 . I contended that we as passers-by might not be aware of any virtual content at all and. seeing his art work Biggar and playing with it for the first time. we don’t have the means with which to recognize the virtual content floating in the air around us. For some this might be a bit frustrating. and discuss some legal aspects thereof. Although. Dutch AR artist.WhO OWnS ThE SpACE 2 discUssing legal issUes of AR Yolande Kolstee WHAT IS PRESENT IN AR SPACE? As mentioned in ‘Who owns the space’ in AR[t] 2. In AR[t] 2. helps them to realize that this AR art piece encapsulates the entire earth with its 7.and artistic purposes. unsure whether or not they are they are talking about you… Sander Veenhof. when we don’t have the right app (software) and/or mobile (hardware). amongst other sources. everybody has the same access to virtual content. For many people.678 virtual cubes whose colours can be changed by simply tapping on a mobile phone. made a lot of people aware that AR could be everywhere. the blog Wassom. we keep a close look on the developments concerning the invasion of our physical space with virtual information.com. using information from.463.

above warzones and above peaceful places… Brian Wassom. the group raises a significant concern: namely. This adult content will certainly appeal to a lot of people and this will definitely result in another large group of people coming into contact with AR. The “No Glass” logo that the café published on its website http://the5pointcafe. ranging from public resistance to ‘adult’ AR (AR porn that is) via licensing. . to AR patents. some predictions were made and trends were spotted. it will have a significant informative impact. on the other hand.000 billboards with Augmented Reality markers throughout many cities in the Netherlands. entering information about the physical world (‘burglary information’ as we called this in the last issue of AR[t]) into the virtual world. this information is used to position the virtual content there where our eyes are looking at. TWO WAY TRAFFIC For 2013. we see a concern regarding the possibility to record without the public knowing this. and make a virtual grid on which we position our virtual objects or scenes. wrote in 2011 that “Augmented Reality has been the Next Big Thing for a while now. although it never manages to become the Actual Current Big Thing”. again. However. as well as above industrial places. blogger and specialist in AR Law. maybe voluntarily via players of AR games. that there’s no obvious way to know when the device is on or what it’s actually doing (recording or not). When downloaded. it floats above holy places in Rome or in the Himalayas. inserting a lot of information from the physical world. into the virtual world. we can enter a room. the app distributed via Layar works by pointing your smartphone at one of these posters. but for him it was more a matter of time than a matter of substantial incorrect prediction. This virtual sculpture has true omnipresence. and the girls will come to life allowing you to watch them in a video right there on the street. predicted five issues related to AR Law for 2012. in February 2013 an adult magazine has distributed around 1. is to download this app via Layar. let our camera survey the space. What is at stake is.5 out of 5. Seattle’s 5 Point Cafe became the  first known establishment to publicly ban Google Glass.” The group is composed of three young Londoners who decided to make a public case against Google Glass and other similar devices. what happens when this information about our eye-movements is stored in databases.com was developed and released (under a Creative Commons license) by a new London-based group called “Stop the Cyborgs. detect the salient points. analysed and even sold to marketing companies. and negligence. EYE-TRACKING Another type of warning from Mr Wassom is related to the information derived from tracking our eye-movements to detect were we are looking at. Since most of the public are not familiar with AR marketing activities like this. On March 11. Natural feature tracking systems are in rapid development. On the Stop the Cyborgs site. “Stop the Cyborgs!” Mr Wassom recognized he was not right in all his predictions. WhAT IS RECORDED ? With the announced introduction of Google glass. Concerns raised range from on the one hand. Real immersive AR depends on knowing exactly what our eyes are looking at. and.2013. scoring 1. Using the eye-movements of the person wearing AR glasses gives extra accuracy.to actually see Biggar. the highly anticipated augmented reality device set to be released later this year. without any prior screening or approval from an examination committee. 42 42 PREVALEnCE OF AR John Moe (the host of Marketplace Tech Report and handles web content for the program).

even closely related people within our Academy.goldrUnner2013. For the time being. our actual privacy laws are sufficient enough to deal with privacy concerns in augmented reality. we have privacy laws to handle this. 43 43 . is as simple as assigning correct or incorrect information to any subject. com “and suddenly everyone is wearing it and this becomes as prevalent as smartphones—you can see it becomes very intrusive very quickly. it’s about the social culture around it”. which has been working in the augmented reality field since 2006. It’s not about the tech.snapsapp. As an example.com and www. Facebook or in a space around us. object or space in our physical world. I can take our AR Lab. see www. the concept of AR is completely new. An incorrect accusation or incorrect information remains incorrect.org FALSE STAYS FALSE Recording without the recorded person knowing this.and if those laws are not adequate enough. We might come to the preliminary insight that as for now. and still for some. regardless whether it is posted on Twitter. via an AR app. we don’t need special AR criminal legislation. The intertwinement of AR in our day-to-day lives is actually quite slow — especially compared to the revolutionary predictions.com  Critical site on wearable technology: http://stopthecyborgs. To add correct or incorrect virtual information to a virtual object or a space. our current laws might be sufficient. “But if the Google glass will become prevalent” as one the people from ‘Stop the Cyborg’ argues in a conversation with a journalist of arstechnica. FURThER READInG  Critical blog on law and social media: http://wassom. is possible with our smartphones and is widely spread among youtube videos.Images coUrtesy of GoldRUn. We are getting used to it at a nice pace. However. we have to change those laws. and will be considered as such.com This quote gives us a direction for discussing the abovementioned AR related legal issues.

OLIVER PERCIVALL. ThE AUGmEnTED STAR. ILLUSTRATOR: PAUL HADCOCK 44 . BOOK COVER.

I DISCOVERED ITS MAIN USES WERE AS A NAVIGATION TOOL USED IN CITY CENTERS. WHAT IF THERE WAS SOME AUGMENTED REALITY TEXT MOST OF US WERE NOT SUPPOSED TO SEE AND YOU HAD A DEVICE THAT WAS ABLE TO READ IT… WOULD YOU FOLLOW ITS INSTRUCTIONS? THIS IDEA FASCINATED ME INTO WRITING A SERIES OF STORIES TITLED THE MISADVENTURES IN AUGMENTED REALITY. POINTING PEOPLE FOR EXAMPLE TO PLACES OF INTEREST OR NEAREST TUBE STATIONS. PART ONE FOLLOWS OUR ACCIDENTAL EXPLORER LYNDEN AS HE POSTS HIS ADVENTURES TO AN ONLINE FORUM.The Misadventures in Augmented Reality OLIVER PERCIVALL I FIRST LEARNED ABOUT AUGMENTED REALITY AROUND TWO YEARS AGO. 45 . THE THOUGHT OF CONCEALED TEXTUAL MARKERS STRUCK A STORY IDEA IN MY MIND.

It’s not easy to write long paragraphs on this VAARR.snapsapp. A few years ago I wasn’t exactly a technology fanatic although admittedly I had more than a passing interest in new gadgets. But once you start exploring all the hidden signs and directions it’s impossible to stop. One was entrusted to me. the shy inventor type. My uncle made it possible to post directly online from this device. the temperature. But nearly nine years ago now he disappeared from his hometown and has never been seen since.com and www. see www. everything is entirely unfamiliar when you first arrive. It’s everything. the beings.goldrUnner2013.Part One My uncle was one clever soul. He created two actually. Using his ingenious engineering skills he somehow created a VAARR (Very Advanced Augmented Reality Reader) It works just the same as other regular Augmented Reality apps I am led to believe but with one unique difference… It has a setting that once activated. Now there is probably nothing I don’t know about the integrated software algorithms used in Augmented Reality. He could probably be best described as a recluse. But be warned: Augmented Reality text as we know it was not developed within The Augmented Star so it makes it pretty hard to find your way out. You’re immediately plunged into its rich diverse landscapes and to find your way around or locate anything at all you must be in possession of the special device — The VAARR or Very Advanced Augmented Reality Reader. the noise levels. I don’t think he’s lost. 46 . Images coUrtesy of GoldRUn. But that was nearly three years ago now before I set foot within the Augmented Star. shows unseen Augmented Reality text none of which is detected by any other device. The smell. So my blog entries below are concise. And because of this I have a story to tell. There’s no one thing that makes it uniquely different.com Life in this newly discovered domain is a far cry from home. not wanting to be found is my guess and I think I know why.

’ That’s the first message you see when you arrive in The Augmented Star. From learning a new food recipe to breathtaking advertising campaigns.The Ultimate Frequency Forum This forum was created for anyone to post to that feels lost. And you never arrive at the same place twice. it’s about seeing the present through a unique lens. Three months later? I think not! 47 . Blog Entry #1 Lynden Apr 12 2011 ‘It’s not about gazing into the future. To find my uncle. On the downside crossing the street can be something of a perilous activity. Blog Entry #2 Lynden Jul 12 2011 OK it took me a couple of hours to get acquainted here this time. those that do not wish to be found or find themselves beyond the confines of earthly boundaries. I’ve come here with a strict agenda. It puts things in perspective when vehicles move from 0 to 600 MPH in less time than it takes to chalk a pool cue. The first post I made earlier was date stamped Apr 12 2011 but the post I made just a couple of hours after that is stamped July 12 2011. the Augmented Reality here really is something else. There’s just so much to get distracted by. Blog Entry #3 Lynden Oct 12 2011 Just noticed a glitch with this online forum that I would like to point out to the moderators. Every time.

Percivall I have always had an interest in technology and gadgets. It appears there is nothing you can’t learn here by waving the VAARR around. jsut follow the QR code on the left! 48 . After leaving school I took a basic Computer studies diploma. So equipped with a reasonable understanding of Augmented Reality and enjoying a challenge.cc/augmented-star Amazon Kindle Store. The mysterious side of technology especially intrigued me including the possibilities of where it could take us in ‘What If’ scenarios.T support roles leading up to Project Management. I just learned that a thimbleful of a neutron star would weigh over 100 million tons. Knowledge is fluidly brought to astonishing life everywhere by Augmented Reality.Blog Entry #4 Lynden Jan 12 2012 It’s funny because since I’ve been wandering the streets here the last few hours I’ve regretted never going to Cambridge University. Although I feel like I have been. The whole place is an encyclopedia of knowledge. They promote learning a Science degree in Artificial Intelligence can be completed in three months here. The Augmented Star is now available on The http://tiny. L. I set myself the complex task of plotting my science fiction novel and created an alternative fictional world that became The Augmented Star. I feel I’m ready for that science degree now! O. Must be something in the air here as my clothes are becoming dirty and ragged really quickly. which eventually led to a career in various I.

I’ve met a lot of people here within The Augmented Star and most of them know my uncle but apparently he’s on the run from some bad people.Blog Entry #5 Lynden Apr 12 2012 The risk of being struck by a falling meteorite for a human is one occurrence every 9300 years. Someone is coming. I wonder what the odds are for walking into an Augmented Universe like this one. It feels like I’m on the run myself. 49 . Blog Entry #6 Lynden Jul 12 2012 Blog Entry #7 Lynden Oct 12 2012 Just received some news that was pretty hard to comprehend. It can read over 150 types of Augmented Reality text and other types too that alternative devices can’t. Using my VAARR triggered a full Augmented Reality breakdown of its features from performance figures to finance options and then it invited you to take a virtual test drive in a car simulation game. The AR campaigns here really resonate with customers in a way that most other ad platforms fall miserably short. I have learned some creatures here would even kill to own one. I’ve come to realize now that I may never be able to leave the Augmented Star. SEE WWW. End of part one Images COURTESY OF GOLDRUN.COM AND WWW.GOLDRUNNER2013.COM Turns out my VAARR is a pretty valuable commodity here. I’ve gotta move quickly. After an exhausting and bloody battle earlier today. Wow that’s a curious fact.SNAPSAPP. I saw an advert on a billboard for an automobile just now.

One of the main grips in which this topic is grounded is that the growing academic concern on the dialogs between physical and digital realms in architecture is becoming blurry. this section is titled with the topic of the latest conference of the Association of Education and Research on Computer Aided Architectural Design in Europe (eCAADe) held at the Czech Technical University in Prague (September 2012): 50 .AUGmEnTED mODELLInG In ThE DESIGn STUDIO: AUGmEnTED pEDAGOGIES Alejandro Veliz Reyes University of Salford Digital physicality / Physical digitality From very early stages of the design process (sketches and diagrams) up to detailing and construction. tools and techniques merge into tailored modelling flow works according to each project’s requirements. it also Digital physicality. developers. such as ubiquitous computing systems or environmentally responsive technologies. a digital counterpart. .. This work claims that beyond physical or digital modelling techniques. manipulability) as well as digital (accuracy. in which new technological developments shape the way we express ourselves. the biggest amount of published articles in the history of the eCAADe Association Conference. The construction of a physical model by computercontrolled machinery requires. scanning and reverse engineering methods allow a fluent interaction between models which embeds information and knowledge as the design process proceeds. As a result. so far. augmented reality now gives us the chance to build hybrid. Augmented models are those which blend a physical and a digital counterpart in a resulting synchronic manner. and define new design forms and organizational/social schemes. Even if the semantics of augmentation suggests an ‘increment on size or amount’. Also. a complex orchestration of users. and further evidence of this trend is the growing participation of architects and designers in professional events such as Ars Electronica. augmented models. since the distinction between both cannot be fully depicted in a world where digital information is continuously embedded into ‘real world’ situations. among others).. Physical digitality.augmented reality now gives us the chance to build hybrid. models are built by using several different — yet interrelated techniques. the concept of augmentation has a major role in this work. augmented models. SIGGRAPH or the ISMAR community. unavoidably. hence taking advantage of the benefits of physical modelling methods (dexterity skills. This design-technology synchronic co-evolution has been historically aligned with the zeitgeist of the architectural practice. modifiability. Indeed. This recent conference had. Likewise. in addition to the technical definition of augmented reality.

suggests some enhancement. This scenario might not be unknown for any architecture student. the architecture studio and its interactions as a subject matter is a quite complex challenge. analysis. this approach to the augmentation of reality fits with the major aim of educational research which is to enhance and improve educational processes and methods. planning. since the studio teaching scheme has been largely acknowledged as the core practice-based module in which both design (composition. MODELS IN THE DESIGN STUDIO “It looks better like this”. 51 . SCHOOL OF THE BUILT ENVIRONMENT. synthesis) are mainly developed in architectural education programs around the world. diagrams and sketches. As a result. thus naming this work as ‘Augmented Pedagogies’. As stated by Allen Cunnigham in 2005. students. instructors and observers engage in a dialog based on the use of plans and sections. UNIVERSITY OF SALFORD.TRAINING SESSION ON AUGMENTED REALITY AUTHORING AND AUGMENTED MODELLING. after a centennial adaptation and evolution the studio teaching scheme and “project-based education around architecture employing the studio system is the most advanced method of teaching complex problem solving that exists”. physical mock-ups. rendered views of a project. representing) and high-order cognitive skills (critical thinking. says an architectural design instructor while chopping a piece of cardboard off a building model. In the same design crit session. Actually.

The usefulness of models within the design studio is clear. Beyond the fact that its construction itself entails the development of technical skills, models also embed design information and knowledge, affecting organizational dynamics (the design critique or peer to peer collaborative work), the creation of students’ ‘toolkits’ or the final presentation of the design solutions, among other benefits. During this research being conducted at the University of Salford (UK), the extent of the impact of augmented models in this complex studio-system is yet to be depicted. Augmented models will be used, therefore, as a way to understand how new technologies impact design education and how can we describe that impact

from a scientific research perspective, that is following the guiding principles of generalizability, communicability and transferability of that resulting knowledge. The deep impact of new digital tools in design pedagogy has been explored recently by design theorists, such as Dr Rivka Oxman. The particularity of the design studio as a research setting is spiced by theoretical underpins that can potentially lead the path to depict this impact. For example, it has been stated that the studio teaching is usually an unstructured process, in which perceptions and interpretations of information and models play a major role in the students’

DRAFT VERSION OF AN AUGMENTED MODEL FOR INTERIOR DESIGN. FURNITURE CAN BE RE-ARRANGED IN A PHYSICAL MODEL OF A HOUSE ROOM (STUDENTS’ WORK).

52

progression in the courses, mostly based on ‘design dialogs’ between students, and students and instructors. Also, digital tools have the potential to not only re-shape the toolkits being used for design, but also mediate in the way design methods are structured, offers new ‘digital materials’ to work with or changes the very nature of the design problems to be faced in different courses. It is not clear, however, how this impacts occurs. The interactions within the studio that make use of representations and models to design are well established ‘rituals’ such as peer-to-peer collaborative activities or the design critique, but the nature of each studio differs from each other. Variables such as the experience of the instructors, the background of the students, the nature of the design problems to be faced or the institutional standpoint turn the studio into highly context-dependant modules. As a result those variables are usually highly controlled and the study of the impact of different technologies is commonly constrained to the description of technical challenges to be solved, the development of new systems/software or metrics of student satisfaction, rather than on the provision of a theoretical account of their impact into this complex teaching/learning process. The lack of a theory that describes how technology re-shapes the studio results in very limited knowledge re-usability and in turn, into very caged and localized pedagogical frameworks that do not allow cross-institutional or cross-disciplinary collaboration, to evaluate the constant infusion and evolution of new digital tools for educational purposes or to re-use a pedagogical approach and its associated knowledge.
IMAGES OF THE TRAINING SESSIONS ON AUGMENTED REALITY AUTHORING AND AUGMENTED MODELLING. STUDENTS OF THE PROGRAMS MSC DIGITAL ARCHITECTURAL DESIGN AND MSC BUILDING INFORMATION MODELLING AND INTEGRATED DESIGN, SCHOOL OF THE BUILT ENVIRONMENT, UNIVERSITY OF SALFORD.

A THEORETICAL APPROACH TOWARDS AUGMENTED PEDAGOGIES
Indeed, there is not fixed methodology to study the impact of digital technologies in design education. Rather, each study requires an own

53

standpoint in terms of validity and fitness to the research problem and the subject matter. As Wanda Orlikowski and Suzanne Iacono (2001) state on their work on information systems theory research, this corresponds to the fact that the use of technologies depend on the context and hence, “there is no single, one-size-fits-all conceptualization of technology that will work for all studies. As a result, IS researchers need to develop the theoretical apparatus that is appropriate for their particular types of investigations, given their questions, focus, methodology, and units of analysis.” In order to overcome this challenge, this ongoing research proposes a theoretical approach to depict the impact of augmented models in design education. By following a grounded theory methodology, observations and recordings are being collected in diverse settings on an attempt to de-

scribe the resulting studio dynamics by using augmented models. Several trainings on augmented reality and augmented modelling have been made at the University of Salford (MSc Digital Architectural Design, MSc in Building Information Modelling and Integrated Design), and two more experimental settings are now being arranged in different European countries. These multiple settings are not only intended to provide a wide view of the subject being studied, but also fits with the current recommendations for theory construction methodologies, since the manipulation and observation of data in many divergent ways and the juxtaposition of different conflicting realities and sources counteracts the tendency of reaching false or incomplete results, or informationprocessing biases of the investigator. This work is expected to be finished by end-2014.

VIEW OF THE STUDENTS’ TOOLKIT DURING THE FIRST AR WORKSHOP AT THE UNIVERSITY OF SALFORD.

54

Related literatUre and soUrces
¢¢ Achten, H., Pavlicek, J., Hulin, J., Matejdan, D. (Eds.). (2012). Digital Physicality / Physical Digitality - Proceedings of the 30th eCAADe Conference - Volume 1 / 2. Czech Technical University in Prague, Faculty of Architecture (Czech Republic) 12-14 September 2012. ¢¢ Buechner, J. (2011). Fictional Entities and Augmented Reality: A Metaphysical Impossibility Result. Journal of Evolution & Technology, 22(Minds and Machines Special Issue), 53-72. ¢¢ Cunningham, A. (2005). Notes on education and research around architecture. The Journal of Architecture, 10(4), 415-441. ¢¢ Eisenhardt, K. M. (1989). Building theories from case study research. Academy of management review, 14(4), 532-550. ¢¢ Orlikowski, W. J., & Iacono, C. S. (2001). Research commentary: Desperately seeking the "IT" in IT research—a call to theorizing the it artifact. Information systems research, 12(2), 121-134. ¢¢ Oxman, R. (2008). Digital architecture as a challenge for design pedagogy: theory, knowledge, models and medium. Design Studies, 29(2), 99-120. I finished my Architecture degree in 2007, in Chile. Today I am a teaching assistant and PhD student on digital architectural design at the University of Salford (United Kingdom) and my current research interests are design pedagogy, augmented reality, and collaborative technologies for design. I have been involved in research activities since 2007, but my background is quite mixed and has involved the domains of sustainable design, buildings in extreme/cold environments, mass customization and digital design. My published work has led me to present on conferences and invited lectures in Chile, United Kingdom, Italy, Belgium, Slovenia, Czech Republic, Canada, among others.

Alejandro Veliz Reyes

55

56 SCARLET PROJECT © UnIVERSITY OF MAnChESTER .

Visual Arts Data Service University for the Creative Arts Introduction Augmented Reality (AR) was identified in the 2011 Horizon Report1 as a key technology trend with potential impact on education. Whenever the words “technical innovation” is spoken in education circles. The benefits to student learning should always be central to the introduction of any new technology and AR is no different. in education technology should be transparent and not an overpowering driver. users do not want to spend time adapting to a new way of learning.UNIVERSITY OF MANCHESTER Jean Vacher Curator. perhaps rightly. on deep rooted pedagogical benefits rather than short-term gimmickry. Some of the most persuasive arguments are described below: established learning methods and styles. If the focus for the student is the technology. supporting independent learning. The benefits of using Augmented Reality in Education 57 . as a way of leveraging innovative technology with pedagogical processes. After all. new technology should integrate seamlessly into Students can view and touch real manuscripts/ editions in conjunction with guided support from trusted sources. In addition. educators are understandably cautious electing to concentrate. The University of Manchester Rose Lock Special Collections Supervisor University of Sussex MIMAS . It is always useful when dealing with new methods of delivery.A STUDY IN SCARLET MATT RAMIREZ Laura Skilton Learning and Teaching Co-ordinator Mimas. Crafts Study Centre/ Research Associate. Crafts Study Centre University for the Creative Arts Marie-Therese Gramstadt Educational Technologist. this can inevitably lead to dissatisfaction and resistance. The report provided the catalyst for the SCARLET (Special Collections using Augmented Reality to Enhance Learning and Teaching) project. Many observers have already buried AR as a fleeting fad in education due to its lack of use cases and documented impact studies. to be armed with a long list of tangible gains for adopting the technology. the learning experience can be diluted. particularly where the emphasis is on the teaching material.

The SCARLET project. iOS and Nokia devices would prove decisive in the decision to use Junaio. rare books within the controlled conditions of reading rooms. manuscripts and the magic of original primary materials. and information on related objects. 3D models. linking 3D models. whilst enhancing the learning experience by surrounding the book with digitised content. learning technologists and students. not the technology. images.  AR promotes ‘active’ teaching. isolated from much of the secondary supporting materials and a growing mass of related digital assets.  The learning curve for new users engaging  with mobile AR through browsers is relatively shallow enabling the learning/pedagogy to be the driver. learners will remember and retain the majority of the information presented to them.  Abstract concepts or ideas that might otherwise be difficult for students to comprehend can be presented through an enhanced learning environment offering access to source historical artefacts and online research in situ. Junaio was the only AR browser to harness optical tracking functionality.  AR can harness both asynchronous (emailing tutor questions) and synchronous (discussion with peers) e-learning methods. a national centre of expertise at The University of Manchester. concentrated on delivering the benefits to student learning without being a flag bearer for the technology. By implementing an object based AR experience. Sources for primary content were ten key editions of ‘The Divine Comedy’ by Dante (between 1472 and 1555) particularly important in terms of publishing and intellectual history and the world-renowned oldest fragment of the Gospel of John. videos and information to images in the form of “GLUE” based channels. This is in opposition to traditional didactic methods that are predominantly teacher led. SCARLET was led by the Learning and Teaching team at Mimas2. but when we become actively involved in an experience. students could simultaneously experience SCARLET (June 2011. preparing them for solo research. Users retain a very small amount of the information that is delivered. maximizing  the opportunity for interaction. and a slightly larger percentage of what is shown to them. augmented 3D models that overlay the physical image and require user touch gestures to proceed) and spark enthusiasm. A mixed team was pulled together dedicated to enhancing the student experience through the application of technology including librarians. while embracing the potential of AR. translations. At the start of the project in 2011.g. so this experience can be foreign and a barrier to their use of special collections. such as an early printed book in the library.May 2012) The main challenge with the use of special collections in learning and teaching is that students are required to consult archives. 58 . encouraging critical response and the adoption of new perspectives and positions. This coupled with an open API and compatibility on Android. academics. Layering AR on texts/images can encourage  interaction (e. Students are used to having access to electronic information on demand.

59 .g. DANTE AR CONTENT © MIMAS. It would also liaise with staff at these universities to transfer the skills learnt and hone the toolkit that was used as a high level framework for replicating similar AR practices in education. providing a foundation for further investigation and research. directly feeding into the application using an iterative development approach. AR was most successful layered over the printed marker instead of signposting to other web based resources already accessible using traditional teaching scaffolds (e. In addition.Evaluation A dominant theme that became evident in the evaluation was that the two academics found differing responses dependent on student user groups. UNIVERSITY OF MANCHESTER.g. student evaluation was critical. Simply adding existing web assets to an object is insufficient. UNIVERSITY OF MANCHESTER. contextual and less generic. The team went on to win the 2nd prize for the ALT learning technologist of the year team award3 and won the Innovation in HE award at Education Innovation 20134. SCARLET+ (June 2012 – April 2013) Following the success of the SCARLET project. Further funding was made available through the SCARLET+ project whose primary focus was to apply the process and framework to other institutions special collections embedding the methodology using a toolkit5 (University of Sussex and the Craft Study Centre at the University for the Creative Arts). making them unique and packaged in digestible chunks produces more positive feedback and value. The learning experience was most enhanced by AR when information delivered was FRAGMENT OF THE GOSPEL OF JOHN AR CONTENT © MIMAS. Throughout the project lifespan. Students who had little prior subject knowledge found the app most useful. CMS). further funding was made available to work with two other UK institutions to facilitate and guide them in the development of AR content. Emphasising the need to align technology to teaching and learning objectives was paramount from the outset to maximise student benefit and impact. student feedback noted that AR experiences should be best used as part of a learning activity (either independent or group based) acting as enabler to achieving a key course objective (e. planning for essay or presentation).

delivering a suite of rich materials especially given the small amount of funding that was available. a crucial part in decision making processes. This project has both developed Mimas’ understanding of implementing AR in education and embedding best practice and methodology to other institutions. Mimas has the Crafts Study Centre at the University for the Creative Arts7. In conjunction with this. bringing their static objects to life. a multi-disciplinary team approach was adopted. There is vast potential in this field enabling students to examine x-rays/biopsies. The partner universities involved were the Special Collections at the University of Sussex and 6 which will allow students to access surrounding resources.g.AR VISUALISATION OF UKMAP DATA © MIMAS. This ensured that the content developed made an impact on learning and teaching as well as enabling AR skills to spread across the institutions. building heights and ground usage to name a few. Conclusion To conclude. focusing on content from 1980s’ mass observation and 20th century crafts. It demonstrates a visual representation of raw materials that are often extremely large in size and difficult to comprehend. UNIVERSITY OF MANCHESTER. Working with other subject areas In parallel with SCARLET and SCARLET+. Sharing was a key element at the heart 60 . cannula application). Both succeeded in providing a showcase for the Special Collections held at their respective institutions using AR. Crucial to the success was ensuring that. This provides a wealth of rich. as with SCARLET. multi-layered information accurately locating building types. Mimas have worked with the Manchester Medical School8 helping students to understand pattern recognition. using AR to identify abnormalities. layering anatomical information and reinforcing learning with instructional demonstrations (e. The challenge was to incorporate this tabular data in a visual 3D model that a handheld device could render efficiently. early development has started using 3D object recognition in the clinical skills laboratories. In the area of geo-spatial mapping. collaborated with colleagues from the Landmap9 service to create an AR experience around the UKMap dataset. the two projects have presented important findings in the impact of AR in education.

org/ horizon2011/sections/augmentedreality 2. Flash. Unity to name a few. wordpress.uk SCARLET team are joint second in learning and teaching awards. Mimas: http://mimas. this follows on from his involvement in the award winning SCARLET project.uk Landmap: http://landmap.uk 61 . While it may not be suitable for all students and situations. this coupled with a strong team ethic enabling stakeholders to buy into the long-term vision. ac.uk/cms SCARLET Toolkit: http://scarlet. 2011 HORIZON report.mimas.uk/mediawiki/index. Crafts Study Centre at the University for the Creative Arts: http://www. section on augmented reality: http://wp.mms.manchester. 8. These have used multimedia content authoring tools such as Flex. Matt's role is also concerned with the research and development of new technologies (e. 6. This aims to improve the student experience by embracing innovative learning methods and providing cutting edge support materials. 9.nmc. wordpress.sussex.csc. Further information: http://teamscarlet. AR opens up huge possibilities for creating immersive learning activities.php/Main_Page Special Collections at the University of Sussex: http://www. It is hoped that the legacy from these small projects will be to inspire others to undertake similar work and display the student led benefits AR can offer. allowing active based learners to better absorb the transfer of knowledge. 13th September 2012: http://teamscarlet. 3.wordpress.ac. Blender. science.co. IT.uk Manchester Medical School: http://www. when employed well it can capture the attention like few other technological mediums. Read more about Matt’s work at http://teamscarlet. References 1. He has over 15 years experience in web design and e-learning content development for a variety of subject areas including medicine.of SCARLET and SCARLET+. Education Innovation 2013: http://educationinnovation. 5.ac. It is particularly effective in explaining abstract concepts visually.ucreative.uk/library/ specialcollections 7.ac. special collections and business.g.com/2012/09/13/scarletteam-are-joint-runner-up-in-learning-technology-awards 4. iBook/mobile/multimedia development projects and haptics) with the Manchester Medical and Dental Schools.com Matt Ramirez Matt Ramirez is currently working on the technical development and support of the JISC funded SCARLET+ Augmented Reality project.ac.com and @team_scarlet.ac.

are the courses in which concept development forms the main part. felt pens or a collage of other material is not the most important issue at that moment. very new. in effect. Most important. in cooperation with the Studium Generale programme. meaning a short. The search to get to know what AR is and how it could be a 62 .INTRODUCING AR TO FIRST YEAR GRAPHIC DESIGN STUDENTS YOLANDE KOLSTEE Graphic Design students learn how to work with the latest Adobe suit. however. At various other art academies we see similar projects. because it will change aspects of their discipline as well as their ‘metier’. the majority of mainstream teachers place analogue and digital techniques at the same level when developing concepts. experience augmented reality. Photoshop and website development tools. by the way — new elements should be considered in the concept and design phase. We. Two Graphic Design teachers. What is important for the Academy. However. both young and digitally savvy. and the department involved. one-day exhibition of the results. while also taking co-authorship and performance into account. freshmen and -women. When working with Augmented Reality — a very suitable medium for graphic designers. as AR Lab are thoroughly convinced that it is important to let first-year students. they often have a (underlying) preference for pen and ink based handwork. in the UK. we made a head start with the introduction of AR via projects differing in length and with different student groups. a programme covering lectures throughout the year. one of which is the Scarlet (Special Collections using Augmented Reality to Enhance Learning and Teaching) project. The idea was to work towards a special PopUp Gallery. at the University for the Creative Arts. started to work with their classes of 27 students each (!) together with some third year students as well. aspects are added to the world of graphics regarding the Internet and social media. At the beginning of the Royal Academy of Arts’ current study year. is that they want the students to become acquainted with creative or artistic research. Whether a concept is devised with colour pencils. With Augmented Reality new. while.

while also seeking to come up with new applications and reasons for the use of Augmented Reality. Or it would be just reality. We showed them our AR projects and introduced them to e. regardless whether they are analogue or digital. were not completely convinced about using AR and how they could incorporate this technique in their projects. for the students themselves as well as for us. but two winners were awarded: Jente Hoogeveen en Jorick de Quaatsteniet. These types of projects show the broad range of ways in which an extra layer on top of the physical world poses questions about what we see. We provided technical help when students were really eager to get their ideas to work as a fully functioning app. Is Augmented Reality only achievable by the use of digital means? Can hidden information be augmented. Ultimately. A special recommendation was given for the dyslexia project of Janne van Hooff who visualised how people suffering from dyslexia see letters and numbers. no addition to reality and certainly no interaction would be possible. phones and tablets. We asked them to think about the kind of information they wanted to disclose in specific situations. turned out to be very exciting. headsets. without the aid digital techniques and devices? Following that. Jente won with her project 'Had je dat gedicht?' (roughly translated as 'How does that rhyme with you?'). Aurasma — a simple and elegant tool for beginners in the AR app world. The end-results would be displayed during our first Pop-Up Gallery on AR. Finally. so to say. at first. Without digital technology. What type of problem would they want to solve using AR? With what classification should it work – analogue and/or digital? With this exercise we hoped to widen the concept. They were part of the jury that awarded the two ‘best’ projects at the end of the first Pop-Up Gallery. the students gave us a huge input of examples they encountered: they found all sorts of material via social media or as search results that they consequently discussed.g. a publication of all the projects was made with a spread for each student in which the description and intention of their project were incorporated. The variety of final projects was staggering and impressive. It is exactly this assumption that we want to investigate. we invited freshmen and -women from the Graphic Design department to get started with Augmented Reality. The same goes for those students who. We were lucky to have two lecturers who gave their talk on AR as part of the Studium Generale programme preceding the Pop-Up Gallery: AR artist Sander Veenhof and data-visualisation expert Niels Schrader. The concept developing part. the idea was to provide an enriching experience for the students and to give them the tools with which they could incorporate AR in their work. The first was a very active YouTube-Google search. is that they appear to invite spectators to look at the world from another perspective. These are some tools one associates with Augmented Reality (AR). Jorick won with his project 'Get your shit together' where animated QR-codes formed text. The rationale underlying the research project therefore lies especially in the use of digital techniques to make this extra layer onto reality visible.” In completing the first step of the two step project. This is obvious because the assumption in AR is that reality must be supplemented with digital information. and the second step was to learn to play with the idea of adding extra information — both analogue and/or digital — a concept development trajectory of sorts. Error messages we are confronted with on a daily basis are transformed into poems in order to take some of the zing out of stressful situations.meaningful technique for graphic designers began comprised two steps. What all the students’ projects have in common. Words emerged by moving strips of black paper over the seemingly incoherent rows of letters. Not one. 63 . In the words of the students’ teachers : “Spectacles. step two.

64 .

with which developers are trying to create new. Many of these experiments draw on of the possibilities to explore virtual information spatially.CREATIVASTOCK. just as in many other fields. These examples show some of the new forms of interaction and presenting information. WWW. it gives points to the user. This way the user is asked to involve their environment in the process of learning words. researchers and developers are experimenting with potential applications of Augmented Reality technologies. On the screen you see a 3D model of the human body. 2013). and memorable learning experiences. interesting.AUGMENTED EDUCATION HOW AR TECHNOLOGIES EXTEND OUR MINDS ROBIN DE LANGE In the field of education. where you can explore the physiology of a human being by moving the display device along a piece of paper with markers. which uses the latest text recognition algorithms. The (young) users of this upcoming app are asked to look for certain words in their home and aim their device at it.COM ample is the 4D Anatomy project by daqri (2012). With sliders and buttons you can set the transparency of the skin. or switch on the layer showing the nervous system for example. Another interesting project in development is the Sesame Street app Big Bird’s Words (Qualcomm. When the device recognizes the word. 65 . An interesting exImage by MARCELO GERPE.

Inga is thought to have a belief about the location of the museum. under the right circumstances. Now. Otto has Alzheimer’s disease and uses a notebook to serve the function of his memory. In the same manner. or using ‘old-fashioned’ mental rotation. also trigger debates about the very goals of education itself. but could. All the examples of cognitive extension that Clark1 gives in his books and papers are not the typical futuristic technologies that come to mind when thinking about humans merging with tech- 66 . To backup this claim. (c) The same situation in a possible future. 1994) It is this example of the human capacity to manipulate the environment to solve problems. Furthermore. (Kirsh and Maglio. In their paper about cognitive performances while playing Tetris. and in my opinion should. we would have no hesitation in recognizing as part of the cognitive process. Thereby. who are remembering how to get to the museum. Clark and Chalmers present a thought experiment involving the fictional characters Otto and Inga. but were also trying to determine whether the shape fits in the slot. this information may even be seen as part of our cognitive process. Clark and Chalmers argue. Otto can be said to have a belief about the location of the museum before he actually consults his notebook. AR technologies could not only lead to new learning methods. players were not only physically rotating the shapes to fit the slot. then that part of the world is (so we claim) part of the cognitive process. before she recalls this from her internal memory. In certain cases. which Clark and Chalmers employ to introduce the Parity Principle: “if a part of the world functions as a process EXTEnDED MInD AnD ThE PARITY PRInCIpLE Andy Clark and David Chalmers start their renowned paper ‘The Extended Mind’ with a thought experiment. these kinds of cases are actually not as abstract as they might seem: they describe a very well-known real-life situation. the notebook can be seen as an extension of Otto’s memory. while Inga’s biological memory is functioning properly. Kirsh and Maglio found that the physi- which. cal rotation in (b) is actually much faster than the mental rotation. where the person can choose between using his neural implant that does the rotation operation as fast as the computer does in (b). the human mind is not bound by the borders of skin and skull.In this article I will argue that the developments in AR technologies will make digital information sources much more transparently available to us. In this experiment the reader is asked to consider three cases and to think about how much cognition is present in each case: (a) A person sits in front of a computer screen which displays geometrical shapes and is asked to fit the shapes in the displayed ‘sockets’. the latter having some speed advantage. I will first introduce the concept of the Extended Mind. namely playing the video game Tetris.” (Clark and Chalmers. thereby simplifying the task. were it done in the head. (b) The same situation as in (a). but now the person can choose either to mentally rotate the shape or physically rotate it by pressing a button. By showing how beliefs are not bound by the borders of the body. To make this claim plausible. Because of this change of perspective regarding external information sources. Clark and Chalmers show that true mental events can extend in the environment as well. 1998) According to the Parity Principle. by rotating the shapes in his mind.

When somebody asks us on the street whether we know what time it is. 2003). By writing down figures in certain locations. ‘you’ is only “the hybrid biotechnological system that now includes the wristwatch as a proper part” (Clark. a measure of the combination of time and physical and cognitive effort. From this. according to the Parity Principle. for example parts of the Web. the technologies which Clark considers as cognitive extensions of our cyborg minds are hardly identified as technology anymore. Another example shows that it has become common to talk about the information that is in some of our technologies as if part of our own knowledge. we use an algorithm that divides the process of multiplying arbitrary large numbers into very simple steps. An interesting question is whether more modern external information sources could obtain the same status as these age-old technologies and play a similar. The user has to put physical and cognitive effort into navigating the browser to the right page.) Now. The information retrieved from the external source should be directly endorsed and trusted. actually part of this process. 2003). One example he mentions is the use of pen and paper when doing long multiplications. probably a wise attitude. writing utensils even for centuries. More precisely: the information access costs. Now. The writing utensils play a crucial role in this cognitive process and are therefore. it is quite clear that the Web with which we interact on a daily basis cannot be seen as part of our cognition. The technological aid should always be available when needed. the information access costs when retrieving information from the Web is way too high.) Our brains incorporate the world and some of the technologies therein in their cognitive processes in such an intimate way. the paper. (This proven transparency of the wrist watch is what makes the development of smart watches interesting. that Clark considers us to be “natural born cyborgs” (Clark. 3. During this time these technologies have become ubiquitous. actually become parts of our minds? Could digital information sources actually become parts of our minds? To begin to answer this question and decide whether a part of the world should (temporarily) be seen as part of the cognitive process. Furthermore. according to Clark this is not simply loosely formulated informal language. due to the open character of the Web. The external resource should be directly available without difficulty. active role in cognitive processes. we can refer to the three criteria suggested by Clark: 1. Although the possibilities of Brain-Machine Interfaces and neural implants such as in case (c) offer very exciting new ways of communicating with technology.nology. To calculate the product of two numbers. 2. and we are wearing our watch. wrist watches have been around for many decades. should be as low as with an equivalent function of the brain (Smart et al. we use the pen to manipulate the external memory source. They have become socially accepted and actually shaped culture itself. In fact. look at our watch and see what time it is. we often answer “yes”. 2008). You actually do know what time it is. Could digital information sources. From our critical stance we do not immediately believe most information we encounter on the Web something which is. then wait for the downloading of the page 67 . Subsequently we raise our arm. this direct interaction with brains is by no means necessary to become part of the cognitive process (nor are they sufficient for cognitive extension: communicating with technology through a Brain Machine Interface usually still takes too much cognitive effort.

telecommunication service providers keep improving their networks to decrease download times. the applica- The main goal of education should be to train the technologically extended cognitive system. when a friend (who is not really into new technology and rather asks a friend to help him) asks you whether you 68 . etc. it has become socially quite acceptable to be wearing a Head-Up Display at all times. and a short. In the widespread current way of interacting. clear description of the word pops up immediately in the corner of your field of view. by eliminating the physical effort of getting your smartphone from your pocket and having to hold it in within your view. would you say that you know the meaning of this word? I can imagine that you — after you get more and more used to the device and have experienced this situation a few times already — might say ‘yes’. The technological developments that are associated with Augmented Reality especially have the potential to lower the information access costs for digital sources considerably. the information access costs of retrieving information from the Web are way too high to be considered as part of the cognitive process. tion knows when you’re busy driving for example and doesn’t bother you then.2 More so. Now. Processors in smartphones are becoming faster. know the meaning of a certain word that is not in your biological memory. Despite of what many critics expected. similar to the situation with the wrist watch. that they could lower information access cost so significantly? Of course Head-Up Displays (HUDs) play a great part in this. in a very real sense.and scan through the text to find the information he needs. But what is it with AR technologies. batteries are getting better. user interfaces are becoming more intuitive. Let us consider the following scenario to get a better idea: For a few years now you have your own AR glasses. When it does indeed become ordinary to wear HUDs. information can be presented to the user at all LOWERInG InFORmATIOn ACCESS COSTS However. Of course. I think you might start to feel like you really do know it. quicker and more intuitive ways. You’ve been using this application for a few months now and it’s starting to have quite a good sense of when to present you with the meaning of a word you encounter. An application running in the background does speech and text recognition and keeps track of words you (the biological ‘you’) don’t know the meaning of. technology is ever-evolving and many of the present advancements will make information available to us in more reliable.

personalized cloud of knowledge. you can interact with digital information in similar ways to how you interact with the physical world. I suggest that under certain conditions. These are the characteristics of AR that create the potential of making digital information much more transparently available to us.COm 69 . intuitive interface. Instead of trying to store all information in biological memory by endless repetition. digital information can be placed over the world. possibilities are created to present information in context-sensitive ways responding both to the environment and the user. By doing so. EXTEnDED COGnITIOn AnD EDUCATIOn In the previous I have introduced the concept of extended cognition and explored the possibilities of digital technology for cognitive extension. personalized information sources are able to compete with mental resources in terms of costs of information access. Adopting this philosophical perspective can have huge consequences for a field like education. at the exact moment when it is needed. these digital information sources could then be seen as proper parts of our hybrid minds. creating a very natural. it could be decided to store certain information that we want to remember in an easily accessible. One could argue from this perspective that the main goal of education should be to train the technologically extended cognitive system. well designed.times. Image COURTESY OF GAIA DEmpSEY AnD DAqRI. SEE hTTp://DAqRI. Another important aspect of AR is the use of information from different sensors and smart algorithms doing image and speech recognition. By accepting digital external information sources as a proper part of memory. Furthermore. By combining these. which is of course the main idea of AR3. According to the Parity Principle.

com/ media/videos/big-birds-words-demo-reelvideo ¢¢ ¢¢ Clark. On distinguishing epistemic from pragmatic action. ESRC/EPSRC Technology Enhanced Learning Programme. Kirsh. Philosophy and Social Hope (pp. ¢¢ ¢¢ Clark. careful decisions have to be made about how to use it in education. & Chalmers. (2009). For the most part of their education. Analysis . (1994). Press. The 70 . Engelbrecht. & S. P.qualcomm. & Maglio. A. R. E. Retrieved from http://site. 1-23).ac.. 513-549. Cognition Distributed: How Cognitive Technology Extends Our Minds (pp. (2011). Dror. a piece of paper and maybe a dictionary or an outdated (graphi4 de Lange. Het Semantic Web en netwerktechnologische cognitieve uitbreidingen. Smart. Hendler. This critical attitude towards the use of technology is very understandable. Furthermore. D.. (2008). and the future of human intelligence. can offer a valuable perspective in this discussion. (1998). (1999). Minds. Rorty. Retrieved from http://www. Supersizing the Mind. Embodiment. E.com/labs/4d-anatomy ¢¢ The general view on the use of technology in education is quite different from the view expressed in this article though. O. (2012). ¢¢ Clark. Oxford: Oxford Univ.. (2008). P. ¢¢ daqri. there should be an active discussion on the goals of education and what technologies students can use to reach these goals. ¢¢ Rorty. P. 10-23. J. A. Technologies. students still only get to use some basic technologies: a pen. ¢¢ Qualcomm.this task of storing information could be off-loaded to an external source which is constantly available to us at low information access costs. (2013). REFEREnCES ¢¢ Blomberg.. (2003).. Action and Cognitive Extension. Network-Enabled Cognition: The Contribution of Social and Technological Networks to Human Cognition . Braines.Realising the Vision for UK Education. System Upgrade . ¢¢ ¢¢ ¢¢ Dror. (2010 (In Press)). (2012). Oxford: Oxford Univ. New York USA: Basic Books.com/wikisofie ¢¢ Dennett.. In R. Do socio-technical systems cognise? 2nd Symposium on Computing and Philosophy . in which external resources have an active role in the cognitive process. To come to these decisions. 4D ANATOMY. Cognitive science . D. & Harnad. & Shadbolt. Harnad. Retrieved from http://www. a lot of research on the use of technology in the learning process is needed. Digital technology is developing very rapidly. N. Natural-Born Cyborgs. (2008). R. P. An extended view of the mind. Big Birds words. Offloading Cognition onto Cognitive Technology. The Extended Mind. In I. (1996). 114-126).uk. Press.. London: Penguin Books.daqri. D. cal) calculator5. A. robindelange. London: tel. S. D. Cognition and the Web. I. Kinds of Minds: Toward an Understanding of Consciousness. Education as Socialization and as Individualization. Smart.

2. from University of Southampton . one might argue that the example does not really show AR. M. He is a freelance video producer and science communicator. 3. 5. you would probably not understand the meaning directly and need to look up more information. The initial paper ‘The Extended Mind’ was written by Andy Clark and David Chalmers. Besides his studies.com 71 . 2011. If the word would be jargon of a field you are not familiar with for example. Because Clark has written many other papers and books on this subject. (2011. November). This characteristic of AR of overlay ing the physical world with virtual objects is not really present in this scenario. The hardware in these devices stays roughly the same. thereby increasing the costs of information access. soton. Thinking Beyond the Brain: Educating and Building. However. Moore’s law seems to be failing here.robindelange. For this reason. thereby showing educational challenges and possibilities. Computational Culture . 4. EnDnOTES 1. He is now a student at the Media Technology MSc program at Leiden University and is particularly interested in technologically extended cognition. he is looking for funding to do a PhD within his field of interest. The information access costs of looking up a word in the dictionary go through the roof. I will refer to Clark further on. For more information: www. Robin has taken part in several entrepreneurial projects.ac. Retrieved June 24.uk/16649/1/Network-Enabled_Cognitionv17. even remains the same price! Robin de Lange Robin de Lange has a bachelor’s degree in Physics and Philosophy and has followed courses on Artificial Intelligence. Most notably. At the moment. it does use certain AR technologies intensively to provide context-sensitive information to the user who interacts with the world.Extended Mind and Network-Enabled Cognition.School of Electronics and Computer Science: http://eprints.ecs. For his graduation research project he is developing an Augmented Reality application that supports the graphical solving of mathematical equations. from the Standpoint of Extended Cognition.pdf ¢¢ Wheeler. he was the co-owner of a company that specialized in homework guidance and tutoring.

HOW DID WE DO IT: WhICh AUGmEnTED REALITY SOFTWARE DOES ThE AR LAB USE? Wim van Eck 72 .

For this overview we created four categories: ‘easy-to-use for mobile devices’. At the AR Lab we use a variety of augmented reality programs to realize our projects. First of all. ‘easy-to-use for desktop augmented reality’. Easy-to-use for mobile devices Since the AR Lab is based at the Royal Academy of Art The Hague most of our projects are developed together with our students. THE PAINTING 'ISAAC BLESSING JACOB' BY THE DUTCH PAINTER GOVERT FLINCK (16151660) HAS BEEN AUGMENTED TO GIVE INSIGHT INTO THE PAINTERS PRACTICE AND OFFERS SCIENTIFIC DATA IN A PLAYFUL WAY. ‘augmented reality software for interactions’ and ‘head-mounted augmented reality’. NETHERLANDS) ON MAY 17TH. I will give an overview of the software we use and why we use it. IS DEVELOPED BY THE AR LAB USING VUFORIA AND UNITY 3D. there are so many programs available it is difficult to actually have tried them all. or we supervise 73 . This doesn’t mean we claim that this is the best software and that there are no alternatives. and updates and new programs pop up all the time. THE APPLICATION WILL PREMIER AT MUSEUM CATHARIJNECOVENT (UTRECHT. Often we are asked “What is the best augmented reality software?” which is a difficult question to answer. COMMISSIONED BY THE RCE (CULTURAL HERITAGE AGENCY OF THE NETHERLANDS). Secondly.THIS APPLICATION. choosing which software to use really depends on what you want to achieve in the end. we merely aim to give an insight into our daily workflow.

aurasma. and BuildAR (http://www. you only need to register to be able to access it. still prefer to present their creations on Fig. This well documented application makes it easier to precisely position your virtual objects. but there is a ‘demo’ watermark on your video-background and you won’t be able to save your scene (Figure 2). desktop augmented reality software is getting scarcer. but it is definitely good enough for most projects. BuildAR is based on the world-renowned ARToolKit. however. so easy-to-use software is preferable. and also to import images and videos which are not on your mobile device. they often have their own smartphone or tablet and are very keen to develop for it. Aurasma (www. 1 not the best around. and photographs and videos which are stored on the device can be used to augment the chosen scene.com/partners).buildar. Cinema 4d. Many of these students don’t have a technical background. all they need is their mobile device. fast camera movements will result in loss of tracking. 2 a large screen. co.nz) is the software we use most often.aurasma. Furthermore. Markers can be generated on the spot by taking pictures with the device. though the OSX version lacks some options and seems to be less stable.The app provides clear information for every step you have to take. there are quite some versions which can be a bit confusing. When downloading BuildAR. but students who want to develop their own application. it’s more difficult to import 3D models from. adds an easy-to-use graphical interface. You can use this version for free. It runs on both Windows and OSX. The tracking quality of Aurasma is Fig. It is very easy though to use video with transparency. Some of our students. The ‘BuildAR Pro 2 Trial Download’ is the newest version for Windows and features both marker tracking and image based tracking. for example. removing any technical barriers (Figure 1). as well as 3D models. By downloading the free Aurasma app from the iOS App Store or Google Play store students can immediately begin creating their first project. Easy-to-use for desktop augmented reality Nowadays most software companies seem to focus on mobile devices. It is a pity that Aurasma only seems to give good support for 3ds Max and Maya. When more possibilities are needed there is also a free online application (www.com) quickly became our favourite choice for this group of students. ‘BuildAR Pro Mac 74 . a not very common option which can give great results.

there are many tutorials around and the Unity forum is a great source for information. Also. but many of our students don’t mind this logo so much though. the new version can augment your scene with images. you will need to buy the Unity iOS or Android add-on to be able to export to a mobile device. Android (Eclipse) or use it as a plugin for Unity (www. The only limitation is a ‘HITLabNZ’ logo. this issue of AR[t] features more information about this system.com). Unity (Figure 3) is a popular game engine offering excellent visual quality on mobile devices.Trial Download’ is the OSX version and only features marker tracking. there is the ‘BuildAR Free version Download (2008 version)’. BuildAR is easy-to-use. it very easy to import 3d-models including animations from almost all 3d-packages. video. However. In issue 2 of AR[t] we introduced our latest augmented reality headset named Marty and its tracking software. You can natively develop for iOS (Xcode). Semiconductor company Qualcomm offers Vuforia for free and Unity has a free version as well. audio and 3D models. Unity makes Head-mounted augmented reality For head mounted based augmented reality we have solely used software and hardware developed by TU Delft. Besides image markers there are also frame markers available. AR software for interactions A program we often use at the AR Lab ourselves is Vuforia (https://www.vuforia.unity3d. Vuforia’s image based tracking is extremely stable.com) allowing you to easily export to iOS and Android. the tracking quality stays good even if the tracking image is partly occluded or when there is little light available. Fig. Adding interactions through scripting is relatively easy. Lastly. which is only available for Windows and is free of charge. which use a pattern of black and white cubes positioned around the image. the demo-limitations are the same as in the Windows version. 3 75 .

2011 76 .AR[t] PICK ImmATERIALS BY DESIGn STUDIO OnFORmATIVE AnD ChRISTOphER WARnOW.

COm) AnD ThE COmpUTATIOnAL DESIGnER ChRISTOphER WARnOW (ChRISTOphERWARnOW.com/portfolio/?p=244 http://www.OnFORmATIVE.IMMATERiALS IS ThE RESULT OF A COLLABORATIOn BETWEEn ThE DESIGn STUDIO OnFORmATIVE (WWW. ThE RESULTInG FORmS DEpICT pOSSIBLE DATA SETS AnD EXAmInE ThE DESIGn pOSSIBILITIES BETWEEn TEChnOID hOLOGRAmS AnD pERSOnAL nOTES.COm). Further information: ¢¢ ¢¢ http://christopherwarnow. ThE pROJECT EXpLORES hOW InFORmATIOn CAn BE InTEGRATED InTO phYSICAL SpACE.com/work/immaterials/ 77 .onformative. USInG ThE LIGhT pAInTInG TEChnIqUE. DATA IS pLACED In A ROOm.

studying New Media and Digital Culture. His main interests and fields of research are: real-time embedded­ image processing. EDWIn VAn DER HEIDE Leiden University evdheide@liacs. space and interaction.vahrmeijer@kabk. real-time 3d animation. His interests are. Maritime and Materials Engineering (3ME). YOLAnDE KOLSTEE  Royal Academy of Art (KABK) Y. She holds the post of Lector (Dutch for researcher in professional universities) in the ­ field of ‘Innovative Visualisation Techniques in higher Art Education’ for the Royal Academy of Art.Jonker@tudelft. Esmé Vahrmeijer is the graphic designer and webmaster of the AR Lab. and models for perceptualization. parallel image processing architectures. Her interests are in graphic design.nl ESmÉ VAhRmEIJER Royal Academy of Art (KABK) e. Pieter Jonker is Professor at  Delft University of Technology.nl MARIAnA KnIVETOn Royal Academy of Art (KABK) m. 78 . Mariana took up editing duties for this current issue. His main tasks are developing Augmented Reality projects. scientific creativity. game design and creative research.nl HAnnA SChRAFFEnBERGER Leiden University hkschraf@liacs.nl Wim van Eck is the 3D animation specialist of the AR Lab. she is a part time student at the Royal Academy of Art (KABK) and runs her own graphic design studio Ooxo. Faculty Mechanical.ContribUtors  WIm VAn ECK Royal Academy of Art (KABK) w. Besides her work at the AR Lab. PIETER JOnKER Delft University of Technology P.kniveton@kabk. typography. Maarten Lamers is assistant professor at the Leiden Institute of Advanced Computer Science (LIACS) and board member of the Media Technology MSc program. The Hague. Since september 2012 she has worked as an intern at the Research Department IVT and the AR Lab. web design. bio-hybrid computer games.vaneck@kabk.P. Her research interests include interaction in interactive art and (non-visual) Augmented Reality. supporting and supervising students and creating 3d content. After a brief stint as a cover model for AR[t] #2. Specializations include social robotics.nl Yolande Kolstee is head of the AR Lab since 2006. MAARTEn LAmERS Leiden University lamers@liacs. Beside’s running his own studio he’s part-time assistant professor at Leiden University (LIACS / Media Technology MSc programme) and heading the Spatial Interaction Lab at the ArtScience Interfaculty of the Royal Conservatoire and Arts Academy in The Hague. robot learning and Augmented Reality.Kolstee@kabk. photography and education.nl Hanna Schraffenberger works as a researcher and PhD student at the Leiden Institute of Advanced Computer Science (LIACS) and at the AR Lab in The Hague. among others. robot vision.nl Mariana Kniveton is currently a master student at Utrecht Universtity.nl Edwin van der Heide is an artist and researcher in the field of sound.

His research interests are design pedagogy.antalruhl.c. special collections and business.com Jouke Verlinden is assistant professor at the section of computer aided design engineering at the Faculty of Industrial Design Engineering.com OLIVER PERCIVALL Antal Ruhl is a media artist with a background in design. Lotte would again like to work in the domains that combine art and technology. Oliver Percivall works in I. and interaction. Alejandro Veliz Reyes is a teaching assistant and PhD student on digital architectural design at the University of Salford (United Kingdom). These objects vary from kinetic sculptures to interactive installations.T Project Management.com Robin de Lange is a student at the Media Technology MSc program at Leiden University and part-time entrepreneur.wordpress. science and art. ALEJANDRO VELIZ REYES University of Salford ROBIN DE LANGE www.theheavyprojects. NEXT ISSUE The next issue of AR[t] will be out in the fourth quarter of 2013.verlinden@tudelft. augmented reality. he leads the “Augmented Matter in Context” lab that focuses on blend between bits and atoms for design and creativity. With a background in virtual reality and interaction design.lottedereus. He is looking for funding to do a PhD on technologically extended cognition. This interest has resulted in the science fiction novel "The Augmented Star".robindelange.JOUKE VERLINDEN Delft University of Technology j.nl GUEST CONTRIBUTORS BC “HEAVY” BIERMANN The Heavy Projects www. science. IT. He is interested in where technology could take us. Heavy creates innovative interfaces between digital design and physical worlds in ways that that provoke the imagination and challenge existing styles of art.nl MATT RAMIREZ teamscarlet. 79 . design. and collaborative technologies for design ANTAL RUHL www. Matt Ramirez has over 15 years experience in web design and e-learning content development for a variety of subject areas including medicine. He creates objects that let us rethink our environment. BC "Heavy" is founder of The Heavy Projects [and its collaborative spin-off Re+Public].com Lotte de Reus recently received her Master’s degree in Design for Interaction at the Faculty of Industrial Design Engineering at Delft University of Technology. LOTTE DE REUS www. In the future.