03

may 2013

AUGMENTED REALITY, ART AND TECHNOLOGY

Re+Public: Re+Imagining Public Space
BC “Heavy” Biermann

Pre-Digital Augmented Reality
Maarten H. Lamers Vestibular Stimulation

as an Augmented Reality Layer?
Antal Ruhl

AR[t]
Magazine about Augmented Reality, art and technology

MAY 2013
2

HOW & NOSm MURAL AUGmEnT, ThE HEAVY PROJECTS, ARTICLE On pAGE 8

3

Colophon
ISSN
2213-2481

COnTACT
The Augmented Reality Lab (AR Lab) Royal Academy of Art, The Hague (Koninklijke Academie van Beeldende Kunsten) Prinsessegracht 4 2514 AN The Hague The Netherlands +31 (0)70 3154795 www.arlab.nl info@arlab.nl

EDITORIAL TEAm
Hanna Schraffenberger, Mariana Kniveton, Yolande Kolstee, Jouke Verlinden

COnTRIBUTORS
AR LAB & PARTnERS: Wim van Eck, Edwin van der Heide, Pieter Jonker, Maarten Lamers, Maaike Roozenburg GUEST COnTRIBUTORS: Alejandro Veliz Reyes, Antal Ruhl, Lotte de Reus, Matt Ramirez, Oliver Percivall, Robin de Lange, BC “Heavy” Biermann

GRAphIC DESIGn
Esmé Vahrmeijer

pRInTInG
Klomp Reproka, Amersfoort

COVER
Our AR[t]y cover is a work by Royal Academy of Art student Donna van West who participated in the Smart Replica’s project, see: www.donnavanwest.nl

www.arlab.nl

4

Table of contents 30 WELCOmE to AR[t] 50 WhO OWnS ThE SpACE 2 Yolande Kolstee 06 08 16 18 24 26 30 34 40 44 50 56 62 64 72 76 RE+PUBLIC: RE+ImAGInInG PUBLIC SpACE BC “Heavy” Biermann — PhD ThE MISADVEnTURES In AR Oliver Percivall AUGmEnTED pEDAGOGIES Alejandro Veliz Reyes MARTY. ThE nEW AFFORDABLE hEADSET — PART 2 Pieter Jonker and Yolande Kolstee A STUDY In SCARLET Matt Ramirez SUBJECT: InTERVIEW From: Hanna Schraffenberger To: Lev Manovich InTRODUCInG AR TO FIRST YEAR GRAphIC DESIGn STUDEnTS Yolande Kolstee PRE-DIGITAL AR Maarten H. Lamers AUGmEnTED EDUCATIOn Robin de Lange VESTIBULAR STImULATIOn AS An AR LAYER? Antal Ruhl HOW DID WE DO IT Wim van Eck AR[t] PICK: ImmATERIALS Editors' Choice BELIEVABILITY Edwin van der Heide NOnLInEAR STORIES TOLD BY CUpS AnD SAUCERS Lotte de Reus 5 .

FINE ART STUDENTS SHOW THEIR WORK ON THE POP-UP GALLERY TWO GRAPHIC DESIGN STUDENTS (DAVE POPPING AND GABOR KEREKES) MADE A MOIRÉ-PATTERN AS A DECODING KEY 6 .

in the attempt to leverage AR to allow artists to make incursions into public spaces. On this website. and information about our experiments in the cultural domain. on the edge of art and technology. The results are a wide variety of implementations and reflections in different creative contexts. In Re+Public’s article we can read about blurring private properties’ boundaries. Please feel free to contact us to tell us what caught your eye. scientists. we have chosen 'Immaterials' — the result of a collaboration between the onformative design studio and Christopher Warnow. In this issue. the magazine about Augmented Reality. We feature both articles with a philosophical perspective and articles with a more technical point of view. Should you like to contribute to issue #4. special attention is given to education: Experiences of bringing AR into the classroom are presented in separate articles by Alejandro Veliz Reyes. via the same URL www. which might lead to an item there. describing some AR software programmes which are widely available. well-known among many people since the publication of his book ‘The Language of New Media’ in 2001. These drawings & 3D model will be downloadable from our website. In our new section AR[t] Pick. We are confidant you will enjoy this issue. we feature a short science fiction sequel by Oliver Percivall. art and technology! In this issue we present articles by contributors from all over the world who are involved in stretching the borders of augmented reality. and in ways they were previously physically unable to do.. Robin de Lange considers the long-term ramifications of extending both the mind and cognition itself with AR. Wim van Eck continues his AR tutorials in the series ‘How did we do it’. we share artworks that caught our eye. We invite you to visit our new website. to print out at your local 3D print facility! Hanna Schraffenberger sets about interviewing Lev Manovich. Head of AR Lab 7 In the second part of this issue. Maarten Lamers takes us back to pre-digital AR with his story on Pepper’s Ghost and at the other end of the spectrum Antal Ruhl explores the potential of using vestibular stimulation in order to create new AR experiences. A recurring topic is set out by Yolande Kolstee: the legal ramifications of AR initiated in AR[t] #2.. We are very pleased to introduce to you the improved version of Marty: the video see-through AR head-up display. Matt Ramirez and Yolande Kolstee. Crucial to AR experiences is the concept of believability. which will be posted on our website soon! .arlab. which will provide you with all the information on our artistic and technical research — but also that of fellow researchers. Yolande Kolstee.WELCOME. to the third edition of AR[t].nl. look out for our call for contributions. in the part News Picks we post short news items on AR artists. explored by Edwin van der Heide. Furthermore. Lotte de Reus discusses a spatial audio intervention to enhance museum exhibits. events and developments.

OUTDOOR ADVERTISING: AR | AD TAKEOVER (NYC. photo by WILL SHERMAN BC “HEAVY” BIERMANN — PHD RE+PUBLIC: RE+IMAGINING PUBLIC SPACE 8 . RON ENGLISH. 2011).

and buildings. Re+Public has developed an experimental mobile device application that digitally resurfaces three specific areas of public space: outdoor advertising. murals. and augmented reality (AR) in particular. Re+Public is dedicated to using emerging media technology. OUTDOOR ADVERTISING: AR | AD TAKEOVER (NYC. With this goal in mind. to alter current expectations of our public media environment generally dictated by property ownership. Re+Public seeks to leverage AR in an effort to allow artists to make incursions into public spaces in ways they were previously physically unable to do. this article focuses on these domains and demonstrates how Re+Public has used AR to transcend current private property boundaries. Blurring private property boundaries. Seeking to understand a profit-driven governmentality and its affect on 9 . 2011) AR presents unique opportunities to creatively problematize the political and economic systems that shape the awareness of both individuals and the public writ large.RE+PUBLIC A creative collaboration between The Heavy Projects (Los Angeles) and the Public Ad Campaign (New York City). As such. or a willingness to break the law. the ability to pay for its usage. which lies at the heart of our endeavor to re+imagine public space as a more open visual commons.

Specifically. In 2012. To this end. PosterBoy. and OX. artists who have contributed to the Wynwood Walls include Os Gemeos. they might start demanding a better version of public messaging than the billboard default. In New York City. which commercial ads do not necessarily have to dominate. and Swoon among many others. digital world. 10 . Once individuals experience this AR version of reality. Dr. D. Retna. In 2008. we augmented ads in Times Square with artistic content. users are able to see the former murals. In other words. The Bowery Mural Wall is an outdoor mural exhibition space in Manhattan. we used AR to rupture public space with a new kind of artistic interactivity. It is precisely this kind of spatial “aura” that distinguishes AR from other types of emerging media technology. Beginning with the 25th and 26th Street complex of six separate buildings. AR permits users to see these murals as if they were actually back on the wall. in situ. Kenny Scharf. and Shepard Fairey. Goldman endeavored to create a center that developed the area’s pedestrian potential. Invader. the mural series commenced with a recreation of Keith Haring’s famous 1982 mural followed by work by such recognized artists as Os Gemeos. real estate developer and arts supporter Tony Goldman started the Bowery Mural with Jeffery Deitch and Deitch Projects. In addiIn 2009. in the space. Re+Public investigated how civic authorities allow certain private parties to profit while preventing or discouraging other forms of public media production. they move in perspectival relation to the viewer. Our digital infiltration into public space and takeover of commercial ads created a place of dialogic interaction rather than a monologic con- sumptive message. Goldman Properties and Tony Goldman who was looking to transform the industrial warehouse district of Miami also conceived the Wynwood Walls. removes the viewer from the space by placing them squarely in the absent.public space. Saner & Sego. Viewing previous murals online. AR is an incremental step towards showing the public an alternate view of their landscape. John Fekner. We foresee AR mobile device technology as a first step in the transformation of public space into an arena shaped by user created content. Nunca. In other words. Owned by Goldman Properties since 1984. for example. Re+Public used AR to resurrect murals that once existed on the wall. by pointing a mobile device at the present mural. While users could certainly view the previous murals online. Re+Public was commissioned by the Wynwood Walls to create an AR experience. 2012) and Wynwood Walls (Miami. users could trigger web-based information related to the showcased artists whose work has historically addressed commercial advertising in public space such as: Ron English. we envision AR as the first step in the evolution of better tools of expression that can democratize public media production. whereas other technologies tend to disconnect the viewer from their immediate physical surroundings. OUtdoor MUrals: Bowery Wall (NYC. at both the Bowery and Wynwood Walls sites. Faile. AR connects the digital with the physical in an intimately “present” way. The AR | AD Takeover used street level ads and billboards to trigger a curated digital art installation that displayed on mobile devices. 2012) In contradistinction to the use of AR to problematize the consumptive monologue of outdoor advertising. that the current artist has painted over. Aiko. as Drawing from an international pool of talent. Barry McGee. During Art Basel 2012.

Finally.OUTDOOR MURALS: BOWERY WALL (NYC. rather abstract 3D environment and permitted users to both pull apart and reconstruct the mural elements. In addition to an immersive garden with a bridge. 11 . it is arguable that the AR assets represent original works in that they contain a sufficiently new visual expression of ideas. we created a 3D Kabuki theater that allowed users to walk into a digitized version of the AR is an incremental step towards showing the public an alternate view of their landscape. With the How and Nosm. RESURRECTED KEITH HARING tion to resurrecting a Shepard Fairey mural that he recently painted over with a new mural in tribute to the recently deceased Tony Goldman. we worked directly with MOMO and collaborated on an AR version of his indoor mural at the Nicelook Gallery on site. we wrestled with this new type of work and wondered if it constituted a new mode of art. we built the mural shapes in 3D and animated them to extend out of the wall and placed them both on the ground and above the wall. we made it appear as if the paint colors were draining out of the mural. over and above those embodied in the 2D mural. with the McGinness. interactive environments for four other murals by How and Nosm. which commercial ads do not necessarily have to dominate. and flowing waterfall. In considering our deployment at the Wynwood Walls in particular. stream. With the Retna. we created another immersive. Ryan McGinness. Aiko. 2012). the AR overlay used the 2D paintings as feature tracking markers and source material to produce an original expres- sion of creative content. In the case of each traditional mural. Additionally. and Retna. Re+Public created 3D. Without pretending to discern any immediate resolution. mural where all of the elements were separated in Z-space.

Retna MUral AUgment. fUll view Retna MUral AUgment. screenshot 12 .

In Los Angeles. we used AR to provide an artistic rendering of the re+imagined building by projecting it into a Blade Runner style future. murals. WILLIAMSBURG ART & DESIGN BUILDINGS (NYC. instead of placing converted art on the structure. In this way. It is our hope that these early entrants will help create experiences that consider art and design as an important part of the way the public adopts this technology. using AR. made it possible for MOMO to put his art on buildings that he could not have accessed in his traditional 2D format. the digital overlay will become a much more seamless and natural part of our daily existence. we really see the city as a canvas that allows for a multiplicity of voices to enter into our visual landscape. we digitally resurfaced. Converting MOMO’s 2D designs into digital 3D models optimized for mobile. This city vision type of deployment potentially provides more practical architectural and urban planning uses and maintains our notion that the AR experience should be spatially relevant in order to maintain the physi- cal aura that may have drawn the viewer to the building in the first place. we placed his art on both the Pearl Paint and the Williamsburg Art & Historical Center buildings in New York City. we converted the Bradbury Building. With the coming advancements in wearable computing. we chose three buildings that had a particular cultural significance and. Working with muralist MOMO. Both structures have a long “art history” in the city and AR allowed us to blur the lines between private and public space. rather than the current commercial hegemony. and their unique architecture much in the same way. 2012) In addition to using both outdoor ads and murals as the markers that trigger our AR deployments. 2012) BRADBURY BUILDING (LA. Re+Public seeks to continue to deploy AR in an effort to democratize access to our shared visual environment and alter the current expectations of urban media in accomplishing our core mission of re+imagining public space. the AR assets should maintain some logical connection to the building or space upon which we have attached them. in some meaningful way. we created the first example of what we refer to as “city visions. or “skinned” physical buildings in urban centers by overlaying 3D content onto the physical environment. To this end. It is vital to the health of any city that its inhabitants are able to participate. we have also experimented with using buildings. or entire buildings. site of many interior shots in the film Blade Runner (1982) into a futuristic version of itself. in our attempt to use AR to re+imagine public space.AUGMENTED ARCHITECTURE: PEARL PAINT. In other words. Ultimately. FINAL REMARKS Whether it is outdoor ads.” Specifically. in the visual urban messaging systems that surround them. WILLIAMSBURG ART & HISTORICAL CENTER BUILDING AUGMENTED WITH MOMO URBAN ART 13 .

BC “Heavy” Biermann Deriving his pseudonym from his penchant for philosophical discussion. Since 2007. RESURRECTED SHEPARD FAIREY. Heavy creates innovative interfaces between digital design and physical worlds in ways that that provoke the imagination and challenge existing styles of art. After finally giving up his painfully amateur skateboarding career due to a bum right knee… BC plans to use his extra time continuing to examine meaningful ways to fuse tech + creativity. As a kind of synthesis between scholarly inquiry and emerging media. academia. and the arts. Building upon existing technological and theoretical frameworks. 2012). art and semiotics in public space. OUTDOOR MURALS: WYNWOOD WALLS (MIAMI. and Saint Louis. With a PhD in Humanities [Intermedia Analysis] from the Universiteit van Amsterdam. Prague. BC “Heavy” Biermann possesses an interdisciplinary background that comprises technology. design. BC has internationally presented his academic work. which explores augmented reality. BC founded The Heavy Projects [and its collaborative spin-off Re+Public] to investigate how the fusion of creativity and technology can uncover new modes of relaying ideas. photo by JORDAN SEILER 14 . BC has worked as both a university professor and a tech developer in Anaheim. and interaction.

15 .

Chatzimichali. cramped shoulders from long computer days and the sighing. however. the air. Hetty van Emmerik. If they are valuable we will publish about it in our next AR[t] magazine. October 9-12. For more information: info@arlab. “why can laying on bed or in an armchair not the ceiling or the white wall. still many companies do not understand that you need two camera’s to see virtual objects in 3D and two camera’s to track (salient) keypoints in 3D in order to track your head pose while walking around with your headset in any unknown environment. Verlinden In proceedings of the IEEE International Conference on Systems. In the mean time be our guest and contribute to AR to come up with your own solutions. Alaska. with the aim to do research on co-operative AR. no assumptions on scale can be made and two cameras that see that same feature are necessary. Frido E. Smulders. subject to copyright. Due to a lack of manpower and funds this might. Man and Cybernetics. It was designed by Niels Mulder from studio RAAR on assignment of the TU-Delft. Keep an eye on our website as we will also link to the software to do 3D pose tracking based on natural feature tracking. partner of the AR Lab. which has another face mask. Wim Gijselaers. ThE nEW AFFORDABLE hEADSET — PART 2 Pieter Jonker AND Yolande Kolstee In AR[t] 2 we announced the new MARTY videosee-through headset as a design based on the Sony HMZ T1. But also lower backpain. One of the aims of the AR Lab is to stimulate AR in the world by putting our developments in public domain. Segers. or anything at any time be my display?” DISCLAImER The industrial design of Marty is meant to do experiments by researchers. so anyone with access to rapid prototyping facilities can reproduce Marty for his/her own scientific or artistic research. Mien S. It is. Piet Van den Bossche. meaning that no large scale industrial production can be based on this design without the consent of the copyright owner. natural features like dominant corners or lines in the scene are tracked. We are also aware of Google glasses and other equipment and we welcome those developments! However.0 that is based on the HMZ T2. AR-toolkit like markers can do with one camera as the size of the marker is known and REFEREnCES ¢¢ Bridging the multiple reality gap: Application of Augmented Reality in new product development. Jonker. the geek that walked around with an AR display. When the new AR Lab website will be in the air at about the same time as this AR[t] 3 magazine is published. hence there is extra knowledge on scale. 2011 16 . take a while. Jouke C. R. artists and designers in the area of AR and stimulate industry to come up with affordable AR equipment. Anchorage. We are well aware that the Sony HMTZ T2 is on the market. In markerless systems.nl. USA. Anna P. however. We are aiming at adapting Marty to version 2. Pieter P. we will post the design files as well as a photo series how to assemble the Marty. And finally: Yes we were already decades ago inspired by fighter pilots with heads-up display and of course Steve Mann.MARTY.

ARLAB.NL) WE WILL SOON POST A PHOTO SERIES ON HOW TO ASSEMBLE THE MARTY HEADSET. THE NEW AFFORDABLE HEADSET.MARTY. Image by FERENC MOLNAR ON OUR NEW WEBSITE (WWW. 17 .

I’d like to interview you even more. So I’ll challenge myself to challenge you.S. I finally also read The Language of New Media.WORDpRESS. LEV. After my last few interviews. I realized that I would like to interview you about Augmented Reality for the AR[t] magazine.FORWARDED mESSAGE ---------FROm: HAnnA SChRAFFEnBERGER <HAnnA@ARLAB. Maybe we can print my questions in issue 3 and your answers in issue 4? 18 MAnOVICh. ThE LAnGUAGE OF nEW mEDIA. I work at the Augmented Reality Lab in The Hague and I am one of the editors of the AR[t] magazine. ThE MIT pRESS.JEWIShphILOSOphYpLACE.S. A short time ago. P.---------. 2001. my supervisor (Edwin van der Heide) told me that I could/should be more critical towards my interview partners. Hanna P.COm Dear Lev. ImAGE SOURCE: WWW.P. When I read your article The Poetics of Augmented Space. .nL> DATE: 2013/4/3 15:53 SUBJECT: InTERVIEW AR[t] mAGAZInE TO: LEV MAnOVICh Maybe you remember me from Facebook. As a consequence. So I hope you’ll agree to an interview for the magazine? Best regards.

the user is presented with auditory information that relates to the immediate surrounding space. monitoring. Wouldn’t you? Augmented Space What is special about AR compared to other forms of Augmented Space? In your article The Poetics of Augmented Space you discuss the concept of Augmented Space. one gets to listen to a mix of edited sounds that blend in with the sounds of the surroundings. Augmented Space refers to all those physical spaces that are overlaid with dynamic information such as shopping malls and entertainment centers that are filled with electronic screens and all those places where one can access information wirelessly on phones. I would call this Augmented Reality. It would be great if you’d address the topic once more. In The Poetics of Augmented Space you describe AR as ‘the laying of dynamic and context-specific information over the visual field of a user’. In contrast to ‘typical’ visual AR. you mention Janet Cardiff’s audio walks as great examples of laying information over physical space. While navigating the environment. And secondly. you mention several other technological developments in the context of Augmented Space. ubiquitous computing. Firstly. Personally.Augmented Reality What is Augmented Reality? To begin with. for example. Besides AR. because our readers might not have read your article. 19 . These walks are designed for specific walking routes. tablets or laptops. as well as spoken narrative elements and instructions such as where to go and what to look at. In The Poetics of Augmented Space. among which. because I think that this point of view unnecessarily limits AR to the visual sense. I would like to ask you what you consider Augmented Reality (AR) to be.

However. but nevertheless. But besides these two options. In AR. the information and my spatial surroundings aren’t perceived as a single gestalt. More specifically. something virtual augments something real. Do you think that all forms of augmentation bring along an augmentation of space or influence our experience of the immediate surrounding space? What else can be augmented? Something I really like about your article is that you see augmentation as an idea and a practice rather than a collection of technologies. Let’s imagine that my phone registered that I have been sitting still for a long time and reminds me to take a short break to stretch my legs. relates to one individual in the space (me). independent layer of content. I might consider it an augmentation of the activity (not moving. I am doubtful whether our experience of a space is affected by this kind of information. What is augmented depends on what the additional content relates to. In our view. but something else. a song is just a song. we have considered things like augmented objects. and has nothing to do with the surrounding space. that nothing is happening there. This information Information and space — one coherent gestalt? In The Poetics of Augmented Space you raise a question that intrigues me a lot. there are more possibilities. For example. I am curious whether you’d agree. the newsreader might tell me about a traffic jam and thereby inform me about my immediate physical space. pattern of work. So judging from my experience of listening to the radio. I’d even say that often it’s not the space that is augmented. you have only discussed the augmentation of space. augmented content and augmented activities. augmented perception. to the activity the person is performing (sitting still). focus of attention or interests of their user. space is one of the possibilities. Edwin (my supervisor) and I have recently given this topic a lot of thought and we were fascinated by the questions: “What is actually augmented in Augmented Reality? What else can (we imagine to) be augmented?” We came up with the answer. something virtual augments something real. Most of the time. but I don’t consider it an augmentation of space. nor does it relate to my environment. but likewise. just to find. Is AR just one of many related recent phenomena that play a role in overlaying the physical space with information? What’s special about AR compared to other forms of Augmented Space? that in AR. so far. However. For example. you mention software that performs tasks according to the mood. most often. the virtual augments that to which it relates. information and the surrounding space can be perceived as one single mixed thing as well as independently. The voice of the newsreader doesn’t mix with the voice of my colleague. 20 . Do the real space and the dynamically presented information add up to one single coherent phenomenological gestalt or are they processed as separate layers? I am a bit of a sound-person and it has always fascinated me that sometimes the sounds of a radio seem to mix in with environmental sounds. I don’t think augmentation is limited to a space or an environment. sitting still) or an augmentation of the user (me). the ticking of a red streetlight might perfectly mix in with the rhythm of the song that is currently playing. but I don’t think it has anything to do with the surrounding space. augmented humans. there is a relationship between both. the sound of the radio just exists as a separate. an event could sound so real and so nearby. Hence. that I’d turn around.tangible interfaces and smart objects. Listening to a radio play. For example. But of course. I was wondering whether you have considered other manifestations of augmentation as well. Here.

Virtual Reality. current manifestations of AR might still bring something new and special to the table. smell like them. when striving for realism. You note that new technological developments illustrate how unrealistic the previous existing images were. It’s the same physical space. physical environment. So some questions I’d like you to answer with respect to Augmented Space are: When are information and space perceived independently from each other — would you still call these occurrences Augmented Space? When are information and space perceived as separate but related layers? And when and why do they add up to one single gestalt? equally to renaissance paintings and to modern computer displays. websites and CD-ROMs — to cinema. I’d like to ask it again with respect to the history of new media. the virtual leaves the realm of virtual space and enters our real physical environments — with respect to that the images might be experienced as more realistic than ever… Will AR take the quest for realism to a new level? I can imagine. Often. the screen was used to separate two absolutely different spaces. when we consider the concept of AR. there are more and more so-called AR applications. For example. Is this a fundamental change in visual culture? AR & the quest for realism The quest for realism in computer graphics New Media One of the main questions I want to ask you is: What makes Augmented Reality special? I have posed that question with respect to other forms of augmented space.I think the same is true for Augmented Space. we find examples that date back centuries.g. AR technologies and new media works that work with AR. How about the relation between AR and cinema? 21 . Will photorealism be traded in for a form of realism that encompasses all senses? Do you think new media will develop towards a more multimodal form? AR & the second space From The Language of New Media. which contains both virtual and real elements. disappear or morph into each other in an otherwise real. a virtual bird is sitting on a real tree) there is no second space. The 3D models that are usually integrated in real space don't come close to the level of photorealism we know from cinema. what’s special about it and what isn’t? is something that has always bored me. the virtual things that appear to exist in our physical space should not only look like real things — ideally they also feel like them.g. I don’t think of AR as a recent phenomenon. glass and special lighting in order to let objects seem to appear. even when they don’t add up to one phenomenological gestalt. Personally. It uses a second room. you relate different forms of new media — e. An example of ancient AR is the Pepper’s Ghost trick (which is discussed by Maarten Lamers on page 24). taste like them and behave like them. If we look at contemporary AR and compare that with other forms of new media. I understood that throughout media history. visual AR could be considered a huge step back. I was wondering: How does AR fit in the widespread aspiration towards realism? On the one hand. When we imagine a typical AR scenario in which virtual objects are integrated into a real scene (e. information and space might be related. Of course. At the same time they remind us that current images will also be superseded. However. On the other hand. But even if the concept isn’t new. this function of the screen applies AR & cinema In The Language of New Media.

L. (2001). Could we say that when working with AR.I’m certainly not a cinema expert..) Does visual AR use similar concepts as cinema? Does cinema use other techniques to create fictional realities that are not (yet) used in AR? Does AR use techniques that might be adapted by cinema in the future? lationship to something real. Are there certain aspects of Augmented Reality you consider especially interesting for future experiments and explorations? AR & future research References 22 . this can be seen as an analogy to compositing in films: an attempt to blend the virtual and the real into a seamless whole ‘augmented’ reality. The poetics of augmented space. Visual Communication. for storytelling? As you point out. Let’s apply this database theory to a typical AR scenario in which virtual objects (seem to) appear in a real environment. we could say the real world serves as a database index for those virtual elements.” In AR. Do we have montage in AR as well? (You give the example of montage within a shot. The database might hold a virtual chair. (2006). (Technically speaking. all new media works are databases. and emotional dissonance between them. when creating a new media work. The MIT press. you did not only One of the main claims in The Language of New Media is that at their basis. So the only way to create a pure database is to spatialise it. AR research is very concerned with registering virtual objects in real space. virtual elements are distributed in real space. We can see this as a database filled with virtual objects. For example. provide a theory of new media. you also pointed your readers towards aspects of new media that were still relatively unexplored at that time and you suggested directions for practical experimentation. 5(2). The language of new media. The same could easily be done in AR. e.g. 219-240. montage aims to create visual. we cannot assume that elements will form a narrative when they are accessed in an arbitrary order. You argue that what artists or designers do. where an image of a dream appears over a man’s sleeping head.) AR as spatialized databases With The Language of New Media. they will be inevitably ordered.. As far as I understand it. stylistic. L. Manovich. distributing the elements in space. but I guess most of what we see in visual AR has been present in cinema for a long time. In The Language of New Media you write about the elements of a database: “If the elements exist in one dimension (time of a film. Can we understand this as a pure database? What are the consequences of working with spatialized elements? What are the inherent limitations and possibilities when working with this form? (I can imagine it has consequences. a virtual pen and a virtual painting. artists and designers create a database for an existing interface? I have one more question about databases.) What is the interface to access the database? Is it my phone? What does the artist create? I think it is usually the virtual content and its re¢¢ ¢¢ Manovich. Do you agree? You oppose compositing to montage: while compositing aims to blend different elements into a single gestalt. is constructing an interface to such a database. semantic. These virtual objects are displayed as part of a real room when a user views the augmented environment with a smartphone. list on a page).

. distributing the elements in space.“[. The Language of New Media 23 Image COURTESY OF LEV MAnOVICh .” Lev Manovich.] the only way to create a pure database is to spatialise it..

Besides meeting Mickey Mouse.. by 26 years! In 1978 my parents took me and my brother to Disneyland. I was most impressed by the augmented reality of Pepper’s Ghost...Disneyla nd. I mentioned how the “Mozzies” mobile game of 2004 was my earliest encounter with augmented reality. I lied. 1978 n Valentij rten a a M PRE-DIGITAL AUGMENTED REALITY MAARTEN H. LAMERS In the first issue of this magazine. With no computer required. 24 ..

facing each other. render and interact with virtual content. a mechanical funhouse car drove us through Disney’s “Haunted Mansion” ride. most realistically. But good old-fashioned Pepper’s Ghost projection is what augments our reality with that content. In effect. of which I can remember only one thing: the car stopped. He explained something about glass. its use is apparent. send a short description to lamers@liacs.net lboUrke rke. . the ghost wasn’t really between us. Disney augmented the reality that we hold our mirrored image to be: pre-digital augmented reality. rotated 90 degrees to the right. and on the seat between us appeared the scariest moving ghost ever! Instantly our heads turned.Image coU rtesy of D avid http Llewellyn ://ravenmanor . Naturally. The mirror showed us some weird illusion. Thank God. paU PaUl BoU . Who would have guessed that such basic illusionary tricks are crucial to what we now consider cutting-edge technology? In fact. if you know other pre-digital augmented reality techniques.com Sitting beside my brother. he explained what is known as Pepper’s Ghost: a simple but clever technique that creates holographic scenes in 2 or 3 dimensions.nl. 25 Image of coUrtesy . and help me put AR in perspective. in current head-up see-through displays Pepper’s Ghost technology is less apparent but nonetheless used in the same fashion. Combining this with a large mirror. However. In Microsoft Research’s HoloDesk (see research. Scary stuff. Pepper’s Ghost technique was first described in the 16th century and later refined by John Pepper around 1860.com/projects/holodesk/) project. Actually. darkness and reflecting light. but also as part of modern optical see-through AR technology.microsoft. It is still used in amuse- ment parks and museums today. facing a large mirror. To me it is interesting that we still rely on John Pepper’s idea to add digital content to our optical reality. I asked my dad how the illusion worked. digital technology lets us define.

Galvanic Vestibular Stimulation (GVS) stimulates the human vestibular system (balance organ) by means of electric stimulation. but never has it been used as an extra sensory input. usually an anode and a cathode. Since I also wanted to investigate what the effect would be to our balance system. 26 . Two electrodes. I used my body orientation to alter my balance. are placed on the mastoids (the bones behind the ear). When I became interested in galvanic vestibular stimulation I realized that this could become a whole new form of AR. To find out whether GVS can be used as a new form of AR. This technique has been around for a long time and many studies have been done in order to learn more about the human balance system. In order to use GVS as a form of augmented reality. I needed a connection to the physical world. I constructed a device and developed a series of experiments. Applying a specific current to those mastoids will result in an altered balance perception related to both to the intensity and the direction of the current.ImAGE BY KRISTIn SmITh VESTIBULAR STImULATIOn AS An AUGmEnTED REALITY LAYER? Antal RUhl Over the past few years the field of Augmented Reality has started to include non-visual forms of augmentation to their scope.

it would push me back to the center. There were two possible modi for the control unit. while modus B could amplify it. it would push me further to the left. Is this a form of augmented reality or is it an alter- Conditions The first step of my research project was measuring the effect of an altered balance. and to make sure this wouldn’t influence the results. In other words. The headphones are in fact merely rings around the ears. Testing this simple setup immediately revealed some potential AR applications of vestibular stimulation. The data from the accelerometer is sent to the control unit. Without even using external data to augment an extra layer to reality I created an altered world based on data from the physical world. The middle part is left uncovered to retain the full use of users’ auditive orientation. so it took more effort to move around. if I was tilted to the left in modus A. In modus B. I realized that. it felt like my resistance was really low since the device backed up every movement I made. This provided me with the setup as shown in Figure 1. own orientation. it had the opposite effect. The control unit calculates the appropriate intensity and direction of the current in real-time and provides feedback to the electrodes. in modus B. when put in modus A. Modus A could counteract my balance. I mounted an accelerometer on top of my head-mounted GVS device in order to measure its orientation. The GVS device counteracted all my movements. The electrodes are incorporated in a pair of headphones to make sure they are pressed against the mastoids properly. it felt like I was moving through a liquid or a thick syrup-like medium. based on my 27 . While testing.GALVANIC VESTIBULAR STIMULATION CAN BE USED TO ALTER BALANCE PERCEPTION Setup To do so.

we can already create an enriched environment. Can this be used as an interface? I demonstrated a system in which the real world environment. This system could enhance a simulated environment by distorting the balance organ based on the simulators’ virtual orientation. could we use this system to add computer generated data to an extra sensory layer of our environment? The easy answer is: Yes! It’s not hard to imagine that you can enrich game play with wave motions for example. improve my balancing skills etc. Using only these two simple orientationbased modi. walking over a balance beam. which could be used to train divers or astronauts who work in other environments. However. and nausea. and letting them experience their surroundings in a modified fashion. Using electrode gel and covering the electrodes with sponges soaked in a salt-water solution did the trick. I found that in certain cirnate reality? Be that as it may. This might be of use for people with an impaired balance system or in those situations when accurate balance is crucial. before we can use this. I was wondering whether I can diminish or amplify motion sickness. But these issues might all be overcome with the right execution. which is the broader field of manipulating users’ sensory perception through a wearable device. (in this case body orientation. Although we have to be very careful about drawing conclusions from these results (given the self-experimentation and singlesubject constraints). This creates a broad range of possibilities. For example. for now. cumstances my balance actually improved while using this device.) is measured. I have used this device for over two weeks to see if there was any progress. Or simulate the G-forces in a racing game’s sharp corners. perhaps more Figure 1: Galvanic VestibUlar StimUlation device troublesome. walking to targets on the street. there are some practical considerations to keep in mind. both are not very userfriendly. But you can also think about fighter pilots who fly in simulators that don’t alter pilots’ balance when flying upside down.The tests that I have done focused on enhanced performances in everyday life situations. experience I had was the adapta- 28 . Tests involved bus rides. During the testing phase. Using my self-constructed device. The skin’s resistance is an important factor in getting the electric pulses down to the desired area. modified and used as an input to alter the actuators (in this case the electrodes which in turn alter my own balance). Another. But a more practical solution must be found. But can we also use it in a more traditional AR environment? In other words. playing Wii Balance Board games and many other examples. but for what use? The examples point towards simulations of the physical world. I experienced some burn marks of the skin around the mastoids. it does fall within the definition of mediated reality. Other issues were the bright light flashes I experienced while testing.

can be found here: www.com/media_tech/paper. but they always share the goal to intrigue people. and removed it after that time. But when I used an alternating pulse. giving off the impression of moving through a low friction environment. Los Angeles. science and art. You might remember that study from the 1960’s where a test subject was wearing reversal goggles. thus. Creative in Motion is a creative brand activation studio: www. published a scientific paper. Media Technology at Leiden University and Design and Media Arts at UCLA. Experiments with Galvanic Vestibular Stimulation in Daily Activities: An Informal Look into its Suitability for HCI. Antal studied Industrial Design at the The Hague University of Applied Sciences. my body had to adapt to this ‘new’ non-stimulated situation.pdf Antal Ruhl Antal Ruhl is a media artist with a background in design. And it did so in two ways. but at a much quicker pace. Arnhem. the subject saw the world upside down because the body had adapted to this new situation. it gave me the experience as if I was moving through a liquid. festivals and event. When I used a constant pulse on the electrodes (a constant current in the same direction) the effect diminished quite soon. Apparently I was much more sensitive to a change in stimulation than to the actual stimulation itself. Your body is extremely good at adapting to new situations and environments. The Netherlands.creativeinmotion. This article is based on research presented at Chi Sparks 2011 (Chi Nederland 15th conference). While using GVS for augmented reality purposes is still in a research stage. travelled the world and currently works a freelance artist/ designer. the effect was constant. After removing the goggles.antalruhl. Antal has also started a company to develop interactive installation for brands. when I was wearing the device while it was set in modus A. Antal worked at distinguished design companies in Amsterdam and Barcelona. The same effect occurred with my balance organ. Sometimes they are purely conceptual or formal and sometimes they serve a more commercial purpose. Secondly. and only changed the pulse-width to affect my balance. June 23. it did so quite convincingly. After a few days the brain adapted to the newly displayed environment and reversed the image back to normal. it felt like I had switched it to modus B.com 29 .nl For an overview of his work or to contact Antal please visit: www. While my body needed some time to get used to my ‘new and improved’ balance. I saw that. has had several (national and international) performances and exhibition. The presented paper. 2011.antalruhl.tion effect. which are very important design-considerations if we want to use a GVS device as an interface. So when I turned it off. After removing the GVS device I experienced the exact opposite. His work can be described as visually and technically attractive with a focus on natural and physical phenomena. when I was wearing the device for about fifteen minutes. the possibilities are endless if you use your imagination. Interactive and playful objects are much better in holding someone´s attention while they convey their message”. These objects vary from kinetic sculptures to interactive installations. “Using the technological possibilities at hand we can enrich our work and create an engaging object. Antal creates objects that let us rethink our environment.

while not real. The story is believable. The e-mail is believable. the story can be entirely fictional and it nevertheless draws us in without questioning its realness or truthfulness. The examples from the bank’s e-mail as well as the novel show us that believability is actually independent from something being real or fake. and we enjoy it. or in other words. but is in fact fake. An important factor for a story to be believable is that we can relate ourselves to it (or the story relates to us). We’ve learned to get suspicious and need to verify the credibility of the story.BELIEVABILITY Edwin van der Heide Believability is something we deal with on a continuous basis. independent from something being real or ‘virtual’. When we read a novel. appears to be authentic. it pretends to be real. we enter a situation in which the content appears to be believable. When we receive an e-mail that claims to be from our bank with the request to enter our account information on ‘their’ website in order to upgrade its security. What we also learn from 30 .

more specifically. but it also triggered me to think about the believability of an artwork and. or never go to bed. PittsbUrgh. I realized that the believability (of the behavior) of an artwork was not (yet) seen as a fundamental topic and might deserve its own study and experimentation. time-travel. 2009 . We can ‘simply’ imagine these things and believe them. the believability of the behavior of an interactive artwork (as opposed to the believability of a robot’s behavior).Spatial SoUnds (100dB at 100km/h) at Wood Street Galeries. The reason that it fascinated me is that it made me not only think about believability in the context of robots. When I was writing my paper about the interactive art installation Spatial Sounds (100dB at 100km/h) for the Third International conference on HumanRobot Personal Relationships (HRPR) I was introduced to a topic addressed by Kerstin Dautenhahn (2007) that fascinated me: The believability of a Robot. In a novel we can meet creatures from Mars. 31 . USA.Image coUrtesy StUdio Edwin van der Heide the example of the novel is that things that happen in the story don’t have to be possible in reality.

Soon after I had that idea I had to think about a full automatic coffee machine that grinds the beans and steams fresh milk for each individual cup of coffee that it makes. for example. welcome and/ or thank the user for using it. We imagine that the robot has a certain amount of intelligence and. imagine that the coffee machine made that coffee especially for us things start to change. Nevertheless. however. it has to be useful and perform tasks around the house. It is interesting to question whether these machines indeed make us believe they show affection.e. How do the two differ from each other? With the robot we can imagine that he read our mind and therefore made a coffee for us. we don’t imagine any intelligence and we simply think of it as a machine without affection for us. even ‘feels’ affection for us. Robots are often made to look and behave like humans (the humanoid) and the coffee machine isn’t. If we. perhaps. i. a robot (or interactive installation) doesn’t have to represent something (else). It can be an abstract work that is believable on its own. it allows us to somehow identify ourselves with it. There is another possible difference between the coffee machine and the robot making coffee. In the HRPR paper about Spatial Sounds (100dB at 100km/h) I put it like this: “The installation can be seen as a non-verbal abstract robot and does not imitate an animal or human-like look or behavior.e. i.” I tried to imagine an example and came up with the idea of a robot that makes and brings you a cup of coffee. This might be the reason that in Japan a lot of machines talk or include animations to. if it lasts or wears off. and if so. Spatial Sounds (100dB at 100km/h) is an example of a believable robot in the sense that the visitors believe they 32 . in a manner that is believable and acceptable to humans.Kerstin Dautenhahn has an interest in socially intelligent robots: “A robot companion in a home environment needs to ‘do the right things’. It is a machine-like object but does not resemble existing machines. but it also has to ‘do the things right’. In the case of the coffee machine. It’s by imagining that we can change what we believe and thereby turn a machine into a believable affective robot. Does this then mean that the representation is a requirement for believability? No.

Socially intelligent robots: dimensions of human–robot interaction. Philosophical Transactions of the Royal Society B: Biological Sciences.understand the behavior of the installation and find it worthwhile to interact with. we’ve learned that something doesn’t have to represent something. 679-704. it can be abstract and nevertheless be believable. Abstract Affective Robotics. K. (2011)..leiden. A. 362(1480). 2011). 27-33. ¢¢ de Rooij. while we keep on believing other things that we know are fake. We have also learned that something doesn't have to be possible in order to be believable. ¢¢ van der Heide. completely lose their believability. http://mediatechnology. Spatial Sounds (100dB at 100km/h) in the Context of Human Robot Personal Relationships. What aspects make combinations of.. H. 59. The aspect of believability is so strong that people accept the installation as a real being and want to interact with it over and over” (van der Heide. References ¢¢ Dautenhahn. Human-Robot Personal Relationships (HRPR 2010). and interactions between. (2007). E. the real and the virtual in augmented reality believable? We have learned that something doesn’t have to be true to be believable. LNICST Vol. (2010). J. 2006 — Image coUrtesy StUdio Edwin van der Heide 33 . The former makes me believe that believability forms an interesting perspective to think about what we can imagine in augmented realities! There are certain things we believe in that. Interesting to read in this context is Alwin de Rooij’s (2010) graduation research project for the Media Technology Master program on ‘Abstract Affective Robotics’. Japan. I’m curious how the abovementioned thoughts apply to augmented realities. Verbeek F.edu/ research/theses/abstract-affective-robotics Spatial SoUnds (100dB at 100km/h) at DAF Tokyo. In: Lamers M. once we discover they are fake. Besides that.

In other words. Depicted in figure 1. let alone use them. in a showcase with extra textual information added to it. Museum visitors can never touch the objects.NOnLInEAR STORIES TOLD BY CUpS AnD SAUCERS Smart Replic as with responsive 3D aUdio Lotte de ReUs. and ending with the small scale production of porcelain in the Netherlands. The objects that are central in this project are seven teacups and saucers that are part of the collection of Museum Boijmans Van Beuningen. but to analyse. New methods of access and new digitization strategies based on the study of the relationship between the ‘bits’ and ‘atoms’ are being developed. something that needs reconsideration in our present time where the ‘experience’ is essential. the goal is not to make the most realistic copy of the original. each of these teacup and saucer sets represents a milestone in the Dutch history of porcelain. in the shape of a 3D print. 34 . 2012): a new kind of reproduction. now. Completed as a graduation project. As a result. Here we present the design project of Lotte de Reus in connection with this paradigm shift. it presents an auditory environment to augment the artefact in an unobtrusive and non-linear way. These are currently on display in a new exhibition on design and pre-industrial design. visitors ‘scan’ the displayed objects from a distance. Starting with the first import of porcelain from China in the seventeenth century by the Dutch East Indies Company. Maaike RoozenbUrg Introduction In museum exhibitions historical objects are usually shown by visual display. JoUke Verlinden. To provide a way around this situation. the so-called ‘smart replica’ was proposed in the previous issue of Ar[t] (Roozenburg. followed by the invention of Delft’s blue as an attempt to copy Chinese porcelain. We hypothesize that by using these 3D imaging techniques the value of our cultural heritage can be increased. communicate and enhance those qualities of the historical artefact that are the most meaningful to us. that stretches the boundaries of the replica’s concept as an autonomous object based on a historical artefact.

The engagement of visitors is limited as they are not experts on the particular subject of the exhibition. and integrity. In the vision of the curator. including museum shops. They draw audiences in through exhibitions and related facilities. Concept of tangible interaction with responsive audio For the seven teacups and saucers that have been chosen to represent the history of the Dutch quest for porcelain. satisfaction. while the information passively waits for the visitor act upon it. rituals. the collection’s past and the like. The artefacts shown and experienced in an exhibition construct a powerful medium which instils visitors with thoughts they perhaps have not yet fostered. This means that a museum requires the visitor to reach out for information. For this design project our aim was to convey the following qualities of interaction: intrigue. 35 . These types of encounters could correspond with respectively a museum or a theme park. paying a visit to an exhibition enables the visitor to have an intimate encounter with the artefacts on display (van Dongen. thereby increasing the opportunity to reflect and learn even after the visit. the exhibitions of Boijmans Van Beuningen can be characterised as enriching experiences: an encounter in which the visitor is conscious of the artefact and the (hi)story it represents. as shown in Table 1. This storytelling model categorises the information in three layers: engagement. 2012). The participants walked into a room. the ECR-model is used to classify the valuable information (Wong. locations.Active enrichment but not entertainment? Museums and their collections house a specific group of objects with their corresponding historical data. Augmented matter — the mixture between bits and atoms — allows novel interaction techniques to embody these networks of information. a focus group that visited the museum exactly proved this problem case in hand. Because of the passive character of the objects and their corresponding information. In the project discussed here. consisting of stories. networks of information associated with the object. Knowledge and culture become more accessible because of the actual presence of the objects. –Visitors’ overall interaction with Boijmans’ exhibitions can mainly be described as scanning. context and reference. went on to the next. full of untold stories. These qualities feed the resulting research questions: How can museums anticipate and facili- Figure 1. THE SEVEN CUPS AND SAUCERS picture by HUGO BES In the spectrum of recreational activities. Our hypothesis is that more active means will lead to a more comprehensive museum experience. materials. leaving the exhibition behind. 2012). An important part of the collection are the nonlinear narratives. restaurants and so forth. understanding. started at a random showcase. and tate the active assemblage of old and new stories – and how do these stories refer back to the replica’s original? How can digital databases be employed in linking smart replicas to their collections? On a philosophical level: does the original still attract interest? In the case of pre-industrial utensils such as the teacups and saucers this question is very relevant. looked at an object briefly. it takes effort on the part of the visitors to maintain concentration.

Information a. These audio clips give the impression that they are experiencing more: it is as if their auditory attention has doubled their visual attention (Erens. When the visitor picks up a replica. Triggered by proximity to object. 2012). Proposed dialogUe between visitor and exhibition. the spatial audio of a narrator starts. Table 1. it attracts the visitor to go to other objects accompanied by a spatial soundscape. When the visitor returns home. every teacup and saucer holds a series of three audio clips. e. The visitor can put down an object and pick up another object. It provides more detailed and interpretive information about the work. c. the moment when a visitor has some immediate sensory. Together with an app that is made available. The narrator tells about the role the specific objects (“audio story”). b. depending on user location / viewing angle. In the first phase. Close proximity to object. Background information (text and images). Context: Draw the visitors’ attention. The app contains more background information on porcelain and the objects. while retaining the soundscape. visitors are attracted to participate in the experience by other visitors who are listening to the 3D audio clips. available after the exhibition. Interaction characteristics Spatialised audio. Triggered by handling an object. Enabling spatial audio As discussed in Table 1. connecting the narrative with what is visualised about the object. Identification data (text. Reference: Gives the visitor the opportunity to draw conclusions and connect to related resources. Information plaque. Exhibition app. she can consult the app at her convenience. the moment when a visitor has some immediate sensory. Ambient soundscape (audio). the narra- tor focuses on the features of the teacup or saucer (“audio specifics”). The audio clips are played when the visitor shows interest through their actions. emotional or intellectual response to the artefact. d. to browse the additional information for example. Arriving in close proximity of the teacup and saucer. visual). emotional or intellectual response to the artefact. information on the role (audio). proximity to an object and the han- 36 .ECR_Phase Engagement: Draw the visitors’ attention. In this stage the visitor is encouraged to turn and explore the object. Narration 1. the visitor is intrigued and triggered to participate in the exhibition. When the visitor is in the proximity of an object.  Narration 2 — information on specifics that can directly be related to the object (audio). a 3D ambient soundscape that fits in with the history of the specific teacup and saucer will appear — once the first soundscape has been heard.

and speaks directly to the visitor (Erens. The essential observations include: Participants seem to enjoy the ambient soundscape: it triggers the imagination and the recognition that the objects used to be utensils and not art object as they are now. Ambient soundscapes were on or off. SPATIAL AUDIO INTERFACE OF AUDIOSTAGE. With the use of tangible replicas and 3D audio clips. it works more associatively. Figure 2. Information about the teacups and saucers was presented via text or audio clips. The soundscape is the auditory equivalent of an ambient image. Furthermore. scanning behaviour can be transformed into an immersive encounter. typically with so-called earcons which represent specific objects or events. without visual clutter and conserves traditional values on how art should be experienced. non-visual immersive content. even those visitors who preferred text to audio clips. The character of 3D audio relates to the idea that the sounds seem to come from sound sources placed anywhere in a space (a surround sound effect). The technology to spatialise audio was developed two decades ago. yielding four permutations. the brain places the sources of the audio clip in your head. To connect this aspect to the exhibition visitor. An example of a high quality 3D audio clip is “In your head” by Big Orange. We used AudioStage to produce our audio clips with a visual interface. whilst the visitor is holding the object. Here optical or magnetic tracking principles also make sense. Presenting the information via audio. 7-10 February 2013).dling of an object. 140 people interacted with a selection of the configurations presented in pairs. Presenting the information via responsive audio facilitated the visitors to consciously turn the object to find the image that the narrator referred to. When listening through headphones. see Burgess and Verlinden (1993) for example. Auditory feedback enriches the visitors’ experience actively. does not clutter the visual sense. cf. they were asked to choose between the two displays and support their reasoning. position and orientation tracking of the human head as well as the objects at hand is required. 37 . 2012). audio clips allow temporal cues. Figure 2. Preliminary evaluation The core of the concept was tested during the Object design fair (Rotterdam. as they are fit for indoor use. Participants appeared to be pleased with the responsive auditory system.

2. weight. and can be connected to various forms of social networking websites and location-based services. this 1. triggers an immersive encounter in which this sensibility is restored: the passive. Thirdly. it rather extends the existing strengths with emerging technologies such as indoor tracking and spatialised soundscapes. decoration make sense primarily in the context of their functions and relations to other objects. the experience is saved. by the layered structure of the narratives. a satisfactorily quality is propelled. substance. Future work includes experimenting and researching the effect of the design in the environment of a museum: Figure 3.  Implementation of responsive audio tours in the current exhibition context. as well as the people who used them. moulded physical replicas afford what Dutch historian Johan Huizinga has called a “historical sensation”. will lead to a more meaningful experience. The proposed system is by no means the first auditory guide for exhibits. Lastly. 38 . Because the visitor has a direct intellectual sensory dialogue with the object. 2005). In essence this proposal presents a new type of relationship between visitor and object that has interaction qualities equal to a human conversation. it is more or less compatible with existing gear already employed by many museums. tangible interaction with physical replicas and the connection between information and corresponding visuals. Impression of the preliminary evalUation. which can be browsed in a non-linear mode by the visitor. Firstly. one-sided encounter with the objects now becomes an active two-sided encounter.Conclusion Once the teacups and saucers were objects of dailylife and their form. the experience fits in with the integrity values of the museum Boijmans Van Beuningen – the 3D scanned and printed. creating curiosity and making visitors want to engage with the artefacts. Secondly. understanding is nurtured because the encounter is intuitive and the information presented by the artefact responds to the visitor’s body language. this poses an intriguing quality that pulls the visitor in by using 3D audio clips and ambient soundscapes. colour. texture.  Experimentation with augmenting untouchable artwork with ambient soundscapes. Furthermore. the feeling as though you are somehow “in touch” with the past (Ankersmit. Through the app. Combining ambient soundscapes.

39 . Lotte would again like to work in the domains that combine art and technology. ¢¢ Erens. Conversation with Curtis Wong.blogspot. we would like to acknowledge the valuable advice and support by Cilia Erens. Themaparken.php ¢¢ Van Dongen.nl. with special attention to smartphone infrastructure. (2005). ¢¢ Ministerie van Onderwijs.com/ watch?v=enR1Ggbuf_8 ¢¢ Software to render binaural output by visually placing audioclips in 3D: http://www. A. ¢¢ The Futures Channel.longcat.  Creating guidelines on how the concept could be implemented to suit different kinds of objects in the museum (or maybe even utensils in our everyday context). A.en Cultuurwetenschappen. (1993). Cultuur en Wetenschap (2005).nl Acknowledgements We would like to thank Alexandra van Dongen. During the project she was pleasantly surprised by the world of audio. M. D.. Bewaren om teweeg te brengen. a high quality 3D audio example: http://www. Burgess.com/conversations_ archive/wong_conversation. [online] Available at: http:// www.thefutureschannel. 4. (1995). immersive soundscapes and 3D audio now hold a new. J. In July 2012 she started her graduation project for the Smart Replicas project. Interviewed by Lotte de Reus. dierentuinen en musea. Her portfolio can be found at www. Amsterdam. Rotterdam. Dr. Interview on the museum context. An architecture for spatial audio servers. Sublime historical experience. Museale strategie.R. Professor Joris Dik. ¢¢ Halbertsma. curator at the Museum Boijmans Van Beuningen for her constructive collaboration.  Prototyping indoor tracking and interaction sensing possibilities. special interest for her. com/2008/05/new-work-detroit-institute-of-1 ¢¢ ¢¢ In your head.3. Proceedings of Virtual Reality Systems Conference (Fall ‘93). Woerden: Drukkerij Zuidam.lottedereus. (2012). F. C.big-orange. Furthermore.nl Official Smart Replica blog: http://smartreplicas. (2012).youtube. fr/web/en/prods/audiostage ¢¢ Interactive installations regarding preindustrial utensils: http://new. she was driven by a fascination for porcelain and the wish to create effective storytelling experiences. References ¢¢ ¢¢ Ankersmit.pentagram. New technologies such as augmented audio. Lotte de Reus Lotte de Reus recently received her Master’s degree in Design for Interaction at the Faculty of Industrial Design Engineering at Delft University of Technology. 14 December. Rotterdam: Erasmus Centrum voor Kunst. Stanford University Press.C.. (1999). something she had not yet encountered in her studies. Interviewed by Lotte de Reus. Interview on soundscapes. 21 November 2012. Many thanks to DOEN foundation for funding the Smart Replicas project (including the graduation project of Lotte de Reus) and many thanks to Mareco prototyping for their contribution in the form of 3D micro prints. Verlinden. More information ¢¢ Video presentation of this project (headphones required): http://www. Wim Verwaal. In the future. Wolf Song and Ir.

GUIDO LAGERWEIJ 40 40 .Image: REVEALInG IDEnTITY.

everybody has the same access to virtual content. Dutch AR artist. The only thing you need to do 41 . made a lot of people aware that AR could be everywhere. we keep a close look on the developments concerning the invasion of our physical space with virtual information.WhO OWnS ThE SpACE 2 discUssing legal issUes of AR Yolande Kolstee WHAT IS PRESENT IN AR SPACE? As mentioned in ‘Who owns the space’ in AR[t] 2. Although. using information from.185. we don’t have the means with which to recognize the virtual content floating in the air around us. unsure whether or not they are they are talking about you… Sander Veenhof. de facto. For many people. in theory. and discuss some legal aspects thereof. the blog Wassom. I contended that we as passers-by might not be aware of any virtual content at all and. For some this might be a bit frustrating. helps them to realize that this AR art piece encapsulates the entire earth with its 7.678 virtual cubes whose colours can be changed by simply tapping on a mobile phone. seeing his art work Biggar and playing with it for the first time. this is not the case. amongst other sources. In AR[t] 2. when we don’t have the right app (software) and/or mobile (hardware). giving off the impression as if people are gossiping about them in a different language or at a very low tone.463.com.and artistic purposes. and actually is everywhere and that this is a special characteristic of AR for as a new medium for artists.

“Stop the Cyborgs!” Mr Wassom recognized he was not right in all his predictions. above warzones and above peaceful places… Brian Wassom. Since most of the public are not familiar with AR marketing activities like this. again. it will have a significant informative impact. the highly anticipated augmented reality device set to be released later this year. 42 42 PREVALEnCE OF AR John Moe (the host of Marketplace Tech Report and handles web content for the program). some predictions were made and trends were spotted. On March 11. Real immersive AR depends on knowing exactly what our eyes are looking at. ranging from public resistance to ‘adult’ AR (AR porn that is) via licensing. When downloaded. and. The “No Glass” logo that the café published on its website http://the5pointcafe. entering information about the physical world (‘burglary information’ as we called this in the last issue of AR[t]) into the virtual world. on the other hand. This virtual sculpture has true omnipresence.5 out of 5. without any prior screening or approval from an examination committee. we see a concern regarding the possibility to record without the public knowing this.2013. On the Stop the Cyborgs site. as well as above industrial places. What is at stake is. this information is used to position the virtual content there where our eyes are looking at. inserting a lot of information from the physical world. but for him it was more a matter of time than a matter of substantial incorrect prediction. blogger and specialist in AR Law. EYE-TRACKING Another type of warning from Mr Wassom is related to the information derived from tracking our eye-movements to detect were we are looking at. and the girls will come to life allowing you to watch them in a video right there on the street. to AR patents. is to download this app via Layar. Seattle’s 5 Point Cafe became the  first known establishment to publicly ban Google Glass. that there’s no obvious way to know when the device is on or what it’s actually doing (recording or not). Concerns raised range from on the one hand. maybe voluntarily via players of AR games. and negligence. . analysed and even sold to marketing companies.com was developed and released (under a Creative Commons license) by a new London-based group called “Stop the Cyborgs.000 billboards with Augmented Reality markers throughout many cities in the Netherlands. However. Using the eye-movements of the person wearing AR glasses gives extra accuracy. WhAT IS RECORDED ? With the announced introduction of Google glass. the app distributed via Layar works by pointing your smartphone at one of these posters. the group raises a significant concern: namely. Natural feature tracking systems are in rapid development. we can enter a room. wrote in 2011 that “Augmented Reality has been the Next Big Thing for a while now. scoring 1.” The group is composed of three young Londoners who decided to make a public case against Google Glass and other similar devices. in February 2013 an adult magazine has distributed around 1. it floats above holy places in Rome or in the Himalayas. This adult content will certainly appeal to a lot of people and this will definitely result in another large group of people coming into contact with AR. let our camera survey the space. detect the salient points. what happens when this information about our eye-movements is stored in databases. into the virtual world. although it never manages to become the Actual Current Big Thing”. TWO WAY TRAFFIC For 2013.to actually see Biggar. predicted five issues related to AR Law for 2012. and make a virtual grid on which we position our virtual objects or scenes.

It’s not about the tech. We are getting used to it at a nice pace. and will be considered as such.com This quote gives us a direction for discussing the abovementioned AR related legal issues. and still for some. see www. For the time being. we have privacy laws to handle this.and if those laws are not adequate enough. We might come to the preliminary insight that as for now. is possible with our smartphones and is widely spread among youtube videos.Images coUrtesy of GoldRUn. our current laws might be sufficient. via an AR app. I can take our AR Lab.snapsapp. is as simple as assigning correct or incorrect information to any subject. object or space in our physical world.org FALSE STAYS FALSE Recording without the recorded person knowing this. FURThER READInG  Critical blog on law and social media: http://wassom. To add correct or incorrect virtual information to a virtual object or a space. As an example. the concept of AR is completely new. “But if the Google glass will become prevalent” as one the people from ‘Stop the Cyborg’ argues in a conversation with a journalist of arstechnica. An incorrect accusation or incorrect information remains incorrect.com and www. Facebook or in a space around us. we have to change those laws. regardless whether it is posted on Twitter. com “and suddenly everyone is wearing it and this becomes as prevalent as smartphones—you can see it becomes very intrusive very quickly. even closely related people within our Academy. However. we don’t need special AR criminal legislation.goldrUnner2013. The intertwinement of AR in our day-to-day lives is actually quite slow — especially compared to the revolutionary predictions.com  Critical site on wearable technology: http://stopthecyborgs. our actual privacy laws are sufficient enough to deal with privacy concerns in augmented reality. 43 43 . which has been working in the augmented reality field since 2006. it’s about the social culture around it”.

ILLUSTRATOR: PAUL HADCOCK 44 .OLIVER PERCIVALL. ThE AUGmEnTED STAR. BOOK COVER.

The Misadventures in Augmented Reality OLIVER PERCIVALL I FIRST LEARNED ABOUT AUGMENTED REALITY AROUND TWO YEARS AGO. WHAT IF THERE WAS SOME AUGMENTED REALITY TEXT MOST OF US WERE NOT SUPPOSED TO SEE AND YOU HAD A DEVICE THAT WAS ABLE TO READ IT… WOULD YOU FOLLOW ITS INSTRUCTIONS? THIS IDEA FASCINATED ME INTO WRITING A SERIES OF STORIES TITLED THE MISADVENTURES IN AUGMENTED REALITY. I DISCOVERED ITS MAIN USES WERE AS A NAVIGATION TOOL USED IN CITY CENTERS. POINTING PEOPLE FOR EXAMPLE TO PLACES OF INTEREST OR NEAREST TUBE STATIONS. PART ONE FOLLOWS OUR ACCIDENTAL EXPLORER LYNDEN AS HE POSTS HIS ADVENTURES TO AN ONLINE FORUM. THE THOUGHT OF CONCEALED TEXTUAL MARKERS STRUCK A STORY IDEA IN MY MIND. 45 .

the shy inventor type.Part One My uncle was one clever soul. Now there is probably nothing I don’t know about the integrated software algorithms used in Augmented Reality.com and www. the noise levels. A few years ago I wasn’t exactly a technology fanatic although admittedly I had more than a passing interest in new gadgets. There’s no one thing that makes it uniquely different. So my blog entries below are concise. He could probably be best described as a recluse. And because of this I have a story to tell. My uncle made it possible to post directly online from this device. see www. not wanting to be found is my guess and I think I know why.goldrUnner2013. It’s everything. But be warned: Augmented Reality text as we know it was not developed within The Augmented Star so it makes it pretty hard to find your way out. The smell.com Life in this newly discovered domain is a far cry from home. He created two actually. the temperature. Using his ingenious engineering skills he somehow created a VAARR (Very Advanced Augmented Reality Reader) It works just the same as other regular Augmented Reality apps I am led to believe but with one unique difference… It has a setting that once activated. One was entrusted to me. It’s not easy to write long paragraphs on this VAARR. 46 . Images coUrtesy of GoldRUn. But nearly nine years ago now he disappeared from his hometown and has never been seen since. the beings. everything is entirely unfamiliar when you first arrive. You’re immediately plunged into its rich diverse landscapes and to find your way around or locate anything at all you must be in possession of the special device — The VAARR or Very Advanced Augmented Reality Reader. shows unseen Augmented Reality text none of which is detected by any other device. But once you start exploring all the hidden signs and directions it’s impossible to stop. I don’t think he’s lost.snapsapp. But that was nearly three years ago now before I set foot within the Augmented Star.

Blog Entry #3 Lynden Oct 12 2011 Just noticed a glitch with this online forum that I would like to point out to the moderators.’ That’s the first message you see when you arrive in The Augmented Star. From learning a new food recipe to breathtaking advertising campaigns. To find my uncle. It puts things in perspective when vehicles move from 0 to 600 MPH in less time than it takes to chalk a pool cue. those that do not wish to be found or find themselves beyond the confines of earthly boundaries. On the downside crossing the street can be something of a perilous activity. Blog Entry #1 Lynden Apr 12 2011 ‘It’s not about gazing into the future. Every time.The Ultimate Frequency Forum This forum was created for anyone to post to that feels lost. Blog Entry #2 Lynden Jul 12 2011 OK it took me a couple of hours to get acquainted here this time. I’ve come here with a strict agenda. There’s just so much to get distracted by. it’s about seeing the present through a unique lens. Three months later? I think not! 47 . the Augmented Reality here really is something else. And you never arrive at the same place twice. The first post I made earlier was date stamped Apr 12 2011 but the post I made just a couple of hours after that is stamped July 12 2011.

The mysterious side of technology especially intrigued me including the possibilities of where it could take us in ‘What If’ scenarios.cc/augmented-star Amazon Kindle Store. I just learned that a thimbleful of a neutron star would weigh over 100 million tons.T support roles leading up to Project Management. After leaving school I took a basic Computer studies diploma. The Augmented Star is now available on The http://tiny. I feel I’m ready for that science degree now! O. Must be something in the air here as my clothes are becoming dirty and ragged really quickly. L. Although I feel like I have been.Blog Entry #4 Lynden Jan 12 2012 It’s funny because since I’ve been wandering the streets here the last few hours I’ve regretted never going to Cambridge University. The whole place is an encyclopedia of knowledge. Knowledge is fluidly brought to astonishing life everywhere by Augmented Reality. They promote learning a Science degree in Artificial Intelligence can be completed in three months here. jsut follow the QR code on the left! 48 . I set myself the complex task of plotting my science fiction novel and created an alternative fictional world that became The Augmented Star. which eventually led to a career in various I. It appears there is nothing you can’t learn here by waving the VAARR around. Percivall I have always had an interest in technology and gadgets. So equipped with a reasonable understanding of Augmented Reality and enjoying a challenge.

It feels like I’m on the run myself.GOLDRUNNER2013. I have learned some creatures here would even kill to own one. Using my VAARR triggered a full Augmented Reality breakdown of its features from performance figures to finance options and then it invited you to take a virtual test drive in a car simulation game. Wow that’s a curious fact. I’ve met a lot of people here within The Augmented Star and most of them know my uncle but apparently he’s on the run from some bad people. I’ve gotta move quickly. SEE WWW. I saw an advert on a billboard for an automobile just now. End of part one Images COURTESY OF GOLDRUN. The AR campaigns here really resonate with customers in a way that most other ad platforms fall miserably short.Blog Entry #5 Lynden Apr 12 2012 The risk of being struck by a falling meteorite for a human is one occurrence every 9300 years. I’ve come to realize now that I may never be able to leave the Augmented Star. Blog Entry #6 Lynden Jul 12 2012 Blog Entry #7 Lynden Oct 12 2012 Just received some news that was pretty hard to comprehend.SNAPSAPP. Someone is coming. I wonder what the odds are for walking into an Augmented Universe like this one. 49 .COM AND WWW. After an exhausting and bloody battle earlier today. It can read over 150 types of Augmented Reality text and other types too that alternative devices can’t.COM Turns out my VAARR is a pretty valuable commodity here.

Likewise. modifiability. Even if the semantics of augmentation suggests an ‘increment on size or amount’.. This recent conference had. . This design-technology synchronic co-evolution has been historically aligned with the zeitgeist of the architectural practice.AUGmEnTED mODELLInG In ThE DESIGn STUDIO: AUGmEnTED pEDAGOGIES Alejandro Veliz Reyes University of Salford Digital physicality / Physical digitality From very early stages of the design process (sketches and diagrams) up to detailing and construction. the biggest amount of published articles in the history of the eCAADe Association Conference. Also. augmented models. the concept of augmentation has a major role in this work. scanning and reverse engineering methods allow a fluent interaction between models which embeds information and knowledge as the design process proceeds. since the distinction between both cannot be fully depicted in a world where digital information is continuously embedded into ‘real world’ situations. a digital counterpart..augmented reality now gives us the chance to build hybrid. As a result. this section is titled with the topic of the latest conference of the Association of Education and Research on Computer Aided Architectural Design in Europe (eCAADe) held at the Czech Technical University in Prague (September 2012): 50 . tools and techniques merge into tailored modelling flow works according to each project’s requirements. Physical digitality. hence taking advantage of the benefits of physical modelling methods (dexterity skills. The construction of a physical model by computercontrolled machinery requires. augmented reality now gives us the chance to build hybrid. models are built by using several different — yet interrelated techniques. in which new technological developments shape the way we express ourselves. unavoidably. Indeed. manipulability) as well as digital (accuracy. a complex orchestration of users. augmented models. This work claims that beyond physical or digital modelling techniques. such as ubiquitous computing systems or environmentally responsive technologies. in addition to the technical definition of augmented reality. SIGGRAPH or the ISMAR community. Augmented models are those which blend a physical and a digital counterpart in a resulting synchronic manner. and define new design forms and organizational/social schemes. among others). One of the main grips in which this topic is grounded is that the growing academic concern on the dialogs between physical and digital realms in architecture is becoming blurry. so far. developers. it also Digital physicality. and further evidence of this trend is the growing participation of architects and designers in professional events such as Ars Electronica.

synthesis) are mainly developed in architectural education programs around the world. planning. 51 . after a centennial adaptation and evolution the studio teaching scheme and “project-based education around architecture employing the studio system is the most advanced method of teaching complex problem solving that exists”. the architecture studio and its interactions as a subject matter is a quite complex challenge. analysis. physical mock-ups. students. As a result. suggests some enhancement. UNIVERSITY OF SALFORD. says an architectural design instructor while chopping a piece of cardboard off a building model. Actually.TRAINING SESSION ON AUGMENTED REALITY AUTHORING AND AUGMENTED MODELLING. thus naming this work as ‘Augmented Pedagogies’. instructors and observers engage in a dialog based on the use of plans and sections. This scenario might not be unknown for any architecture student. this approach to the augmentation of reality fits with the major aim of educational research which is to enhance and improve educational processes and methods. MODELS IN THE DESIGN STUDIO “It looks better like this”. rendered views of a project. since the studio teaching scheme has been largely acknowledged as the core practice-based module in which both design (composition. In the same design crit session. representing) and high-order cognitive skills (critical thinking. SCHOOL OF THE BUILT ENVIRONMENT. diagrams and sketches. As stated by Allen Cunnigham in 2005.

The usefulness of models within the design studio is clear. Beyond the fact that its construction itself entails the development of technical skills, models also embed design information and knowledge, affecting organizational dynamics (the design critique or peer to peer collaborative work), the creation of students’ ‘toolkits’ or the final presentation of the design solutions, among other benefits. During this research being conducted at the University of Salford (UK), the extent of the impact of augmented models in this complex studio-system is yet to be depicted. Augmented models will be used, therefore, as a way to understand how new technologies impact design education and how can we describe that impact

from a scientific research perspective, that is following the guiding principles of generalizability, communicability and transferability of that resulting knowledge. The deep impact of new digital tools in design pedagogy has been explored recently by design theorists, such as Dr Rivka Oxman. The particularity of the design studio as a research setting is spiced by theoretical underpins that can potentially lead the path to depict this impact. For example, it has been stated that the studio teaching is usually an unstructured process, in which perceptions and interpretations of information and models play a major role in the students’

DRAFT VERSION OF AN AUGMENTED MODEL FOR INTERIOR DESIGN. FURNITURE CAN BE RE-ARRANGED IN A PHYSICAL MODEL OF A HOUSE ROOM (STUDENTS’ WORK).

52

progression in the courses, mostly based on ‘design dialogs’ between students, and students and instructors. Also, digital tools have the potential to not only re-shape the toolkits being used for design, but also mediate in the way design methods are structured, offers new ‘digital materials’ to work with or changes the very nature of the design problems to be faced in different courses. It is not clear, however, how this impacts occurs. The interactions within the studio that make use of representations and models to design are well established ‘rituals’ such as peer-to-peer collaborative activities or the design critique, but the nature of each studio differs from each other. Variables such as the experience of the instructors, the background of the students, the nature of the design problems to be faced or the institutional standpoint turn the studio into highly context-dependant modules. As a result those variables are usually highly controlled and the study of the impact of different technologies is commonly constrained to the description of technical challenges to be solved, the development of new systems/software or metrics of student satisfaction, rather than on the provision of a theoretical account of their impact into this complex teaching/learning process. The lack of a theory that describes how technology re-shapes the studio results in very limited knowledge re-usability and in turn, into very caged and localized pedagogical frameworks that do not allow cross-institutional or cross-disciplinary collaboration, to evaluate the constant infusion and evolution of new digital tools for educational purposes or to re-use a pedagogical approach and its associated knowledge.
IMAGES OF THE TRAINING SESSIONS ON AUGMENTED REALITY AUTHORING AND AUGMENTED MODELLING. STUDENTS OF THE PROGRAMS MSC DIGITAL ARCHITECTURAL DESIGN AND MSC BUILDING INFORMATION MODELLING AND INTEGRATED DESIGN, SCHOOL OF THE BUILT ENVIRONMENT, UNIVERSITY OF SALFORD.

A THEORETICAL APPROACH TOWARDS AUGMENTED PEDAGOGIES
Indeed, there is not fixed methodology to study the impact of digital technologies in design education. Rather, each study requires an own

53

standpoint in terms of validity and fitness to the research problem and the subject matter. As Wanda Orlikowski and Suzanne Iacono (2001) state on their work on information systems theory research, this corresponds to the fact that the use of technologies depend on the context and hence, “there is no single, one-size-fits-all conceptualization of technology that will work for all studies. As a result, IS researchers need to develop the theoretical apparatus that is appropriate for their particular types of investigations, given their questions, focus, methodology, and units of analysis.” In order to overcome this challenge, this ongoing research proposes a theoretical approach to depict the impact of augmented models in design education. By following a grounded theory methodology, observations and recordings are being collected in diverse settings on an attempt to de-

scribe the resulting studio dynamics by using augmented models. Several trainings on augmented reality and augmented modelling have been made at the University of Salford (MSc Digital Architectural Design, MSc in Building Information Modelling and Integrated Design), and two more experimental settings are now being arranged in different European countries. These multiple settings are not only intended to provide a wide view of the subject being studied, but also fits with the current recommendations for theory construction methodologies, since the manipulation and observation of data in many divergent ways and the juxtaposition of different conflicting realities and sources counteracts the tendency of reaching false or incomplete results, or informationprocessing biases of the investigator. This work is expected to be finished by end-2014.

VIEW OF THE STUDENTS’ TOOLKIT DURING THE FIRST AR WORKSHOP AT THE UNIVERSITY OF SALFORD.

54

Related literatUre and soUrces
¢¢ Achten, H., Pavlicek, J., Hulin, J., Matejdan, D. (Eds.). (2012). Digital Physicality / Physical Digitality - Proceedings of the 30th eCAADe Conference - Volume 1 / 2. Czech Technical University in Prague, Faculty of Architecture (Czech Republic) 12-14 September 2012. ¢¢ Buechner, J. (2011). Fictional Entities and Augmented Reality: A Metaphysical Impossibility Result. Journal of Evolution & Technology, 22(Minds and Machines Special Issue), 53-72. ¢¢ Cunningham, A. (2005). Notes on education and research around architecture. The Journal of Architecture, 10(4), 415-441. ¢¢ Eisenhardt, K. M. (1989). Building theories from case study research. Academy of management review, 14(4), 532-550. ¢¢ Orlikowski, W. J., & Iacono, C. S. (2001). Research commentary: Desperately seeking the "IT" in IT research—a call to theorizing the it artifact. Information systems research, 12(2), 121-134. ¢¢ Oxman, R. (2008). Digital architecture as a challenge for design pedagogy: theory, knowledge, models and medium. Design Studies, 29(2), 99-120. I finished my Architecture degree in 2007, in Chile. Today I am a teaching assistant and PhD student on digital architectural design at the University of Salford (United Kingdom) and my current research interests are design pedagogy, augmented reality, and collaborative technologies for design. I have been involved in research activities since 2007, but my background is quite mixed and has involved the domains of sustainable design, buildings in extreme/cold environments, mass customization and digital design. My published work has led me to present on conferences and invited lectures in Chile, United Kingdom, Italy, Belgium, Slovenia, Czech Republic, Canada, among others.

Alejandro Veliz Reyes

55

56 SCARLET PROJECT © UnIVERSITY OF MAnChESTER .

Crafts Study Centre/ Research Associate. Whenever the words “technical innovation” is spoken in education circles. The University of Manchester Rose Lock Special Collections Supervisor University of Sussex MIMAS . If the focus for the student is the technology. on deep rooted pedagogical benefits rather than short-term gimmickry. in education technology should be transparent and not an overpowering driver. particularly where the emphasis is on the teaching material. After all. Many observers have already buried AR as a fleeting fad in education due to its lack of use cases and documented impact studies. perhaps rightly. users do not want to spend time adapting to a new way of learning. the learning experience can be diluted. new technology should integrate seamlessly into Students can view and touch real manuscripts/ editions in conjunction with guided support from trusted sources.A STUDY IN SCARLET MATT RAMIREZ Laura Skilton Learning and Teaching Co-ordinator Mimas. The benefits of using Augmented Reality in Education 57 .UNIVERSITY OF MANCHESTER Jean Vacher Curator. Visual Arts Data Service University for the Creative Arts Introduction Augmented Reality (AR) was identified in the 2011 Horizon Report1 as a key technology trend with potential impact on education. this can inevitably lead to dissatisfaction and resistance. It is always useful when dealing with new methods of delivery. Some of the most persuasive arguments are described below: established learning methods and styles. supporting independent learning. educators are understandably cautious electing to concentrate. In addition. The benefits to student learning should always be central to the introduction of any new technology and AR is no different. as a way of leveraging innovative technology with pedagogical processes. The report provided the catalyst for the SCARLET (Special Collections using Augmented Reality to Enhance Learning and Teaching) project. to be armed with a long list of tangible gains for adopting the technology. Crafts Study Centre University for the Creative Arts Marie-Therese Gramstadt Educational Technologist.

Sources for primary content were ten key editions of ‘The Divine Comedy’ by Dante (between 1472 and 1555) particularly important in terms of publishing and intellectual history and the world-renowned oldest fragment of the Gospel of John. students could simultaneously experience SCARLET (June 2011. whilst enhancing the learning experience by surrounding the book with digitised content. Users retain a very small amount of the information that is delivered. At the start of the project in 2011. and information on related objects. 58 . videos and information to images in the form of “GLUE” based channels. so this experience can be foreign and a barrier to their use of special collections.g. linking 3D models. translations. iOS and Nokia devices would prove decisive in the decision to use Junaio. Junaio was the only AR browser to harness optical tracking functionality. not the technology. while embracing the potential of AR. This is in opposition to traditional didactic methods that are predominantly teacher led.  The learning curve for new users engaging  with mobile AR through browsers is relatively shallow enabling the learning/pedagogy to be the driver. learning technologists and students. such as an early printed book in the library. a national centre of expertise at The University of Manchester. Layering AR on texts/images can encourage  interaction (e. 3D models. maximizing  the opportunity for interaction.  Abstract concepts or ideas that might otherwise be difficult for students to comprehend can be presented through an enhanced learning environment offering access to source historical artefacts and online research in situ. By implementing an object based AR experience. rare books within the controlled conditions of reading rooms. augmented 3D models that overlay the physical image and require user touch gestures to proceed) and spark enthusiasm. manuscripts and the magic of original primary materials. A mixed team was pulled together dedicated to enhancing the student experience through the application of technology including librarians.  AR can harness both asynchronous (emailing tutor questions) and synchronous (discussion with peers) e-learning methods. The SCARLET project. isolated from much of the secondary supporting materials and a growing mass of related digital assets.May 2012) The main challenge with the use of special collections in learning and teaching is that students are required to consult archives. and a slightly larger percentage of what is shown to them. This coupled with an open API and compatibility on Android. academics. but when we become actively involved in an experience. Students are used to having access to electronic information on demand. learners will remember and retain the majority of the information presented to them. SCARLET was led by the Learning and Teaching team at Mimas2. concentrated on delivering the benefits to student learning without being a flag bearer for the technology. preparing them for solo research. encouraging critical response and the adoption of new perspectives and positions.  AR promotes ‘active’ teaching. images.

In addition. planning for essay or presentation). UNIVERSITY OF MANCHESTER. providing a foundation for further investigation and research. UNIVERSITY OF MANCHESTER. SCARLET+ (June 2012 – April 2013) Following the success of the SCARLET project. Simply adding existing web assets to an object is insufficient. directly feeding into the application using an iterative development approach. 59 . further funding was made available to work with two other UK institutions to facilitate and guide them in the development of AR content. student feedback noted that AR experiences should be best used as part of a learning activity (either independent or group based) acting as enabler to achieving a key course objective (e. making them unique and packaged in digestible chunks produces more positive feedback and value. student evaluation was critical. DANTE AR CONTENT © MIMAS. The learning experience was most enhanced by AR when information delivered was FRAGMENT OF THE GOSPEL OF JOHN AR CONTENT © MIMAS. Throughout the project lifespan. Students who had little prior subject knowledge found the app most useful.g. AR was most successful layered over the printed marker instead of signposting to other web based resources already accessible using traditional teaching scaffolds (e. Further funding was made available through the SCARLET+ project whose primary focus was to apply the process and framework to other institutions special collections embedding the methodology using a toolkit5 (University of Sussex and the Craft Study Centre at the University for the Creative Arts). contextual and less generic.g. Emphasising the need to align technology to teaching and learning objectives was paramount from the outset to maximise student benefit and impact. CMS).Evaluation A dominant theme that became evident in the evaluation was that the two academics found differing responses dependent on student user groups. It would also liaise with staff at these universities to transfer the skills learnt and hone the toolkit that was used as a high level framework for replicating similar AR practices in education. The team went on to win the 2nd prize for the ALT learning technologist of the year team award3 and won the Innovation in HE award at Education Innovation 20134.

Mimas have worked with the Manchester Medical School8 helping students to understand pattern recognition. It demonstrates a visual representation of raw materials that are often extremely large in size and difficult to comprehend. building heights and ground usage to name a few. focusing on content from 1980s’ mass observation and 20th century crafts. a multi-disciplinary team approach was adopted. delivering a suite of rich materials especially given the small amount of funding that was available.g. There is vast potential in this field enabling students to examine x-rays/biopsies. bringing their static objects to life. Mimas has the Crafts Study Centre at the University for the Creative Arts7. multi-layered information accurately locating building types. The partner universities involved were the Special Collections at the University of Sussex and 6 which will allow students to access surrounding resources. cannula application). This ensured that the content developed made an impact on learning and teaching as well as enabling AR skills to spread across the institutions. collaborated with colleagues from the Landmap9 service to create an AR experience around the UKMap dataset. Sharing was a key element at the heart 60 . the two projects have presented important findings in the impact of AR in education. This provides a wealth of rich. Working with other subject areas In parallel with SCARLET and SCARLET+. as with SCARLET. UNIVERSITY OF MANCHESTER. In the area of geo-spatial mapping. Both succeeded in providing a showcase for the Special Collections held at their respective institutions using AR. The challenge was to incorporate this tabular data in a visual 3D model that a handheld device could render efficiently. Conclusion To conclude.AR VISUALISATION OF UKMAP DATA © MIMAS. In conjunction with this. layering anatomical information and reinforcing learning with instructional demonstrations (e. a crucial part in decision making processes. early development has started using 3D object recognition in the clinical skills laboratories. This project has both developed Mimas’ understanding of implementing AR in education and embedding best practice and methodology to other institutions. using AR to identify abnormalities. Crucial to the success was ensuring that.

uk/mediawiki/index. special collections and business.uk 61 . science. 6.uk/cms SCARLET Toolkit: http://scarlet. Blender.uk Manchester Medical School: http://www. 13th September 2012: http://teamscarlet. Read more about Matt’s work at http://teamscarlet.com Matt Ramirez Matt Ramirez is currently working on the technical development and support of the JISC funded SCARLET+ Augmented Reality project. this follows on from his involvement in the award winning SCARLET project.mimas. It is particularly effective in explaining abstract concepts visually. Mimas: http://mimas. allowing active based learners to better absorb the transfer of knowledge.g.ac. 9.ac. 3.org/ horizon2011/sections/augmentedreality 2. He has over 15 years experience in web design and e-learning content development for a variety of subject areas including medicine.uk SCARLET team are joint second in learning and teaching awards. This aims to improve the student experience by embracing innovative learning methods and providing cutting edge support materials. 8.uk/library/ specialcollections 7.manchester. Unity to name a few. 5. Flash. It is hoped that the legacy from these small projects will be to inspire others to undertake similar work and display the student led benefits AR can offer.ac. Education Innovation 2013: http://educationinnovation. References 1. IT. iBook/mobile/multimedia development projects and haptics) with the Manchester Medical and Dental Schools.uk Landmap: http://landmap. Matt's role is also concerned with the research and development of new technologies (e.wordpress. Further information: http://teamscarlet.co. when employed well it can capture the attention like few other technological mediums.php/Main_Page Special Collections at the University of Sussex: http://www.sussex.com/2012/09/13/scarletteam-are-joint-runner-up-in-learning-technology-awards 4.of SCARLET and SCARLET+. wordpress. These have used multimedia content authoring tools such as Flex. wordpress.csc. ac.ucreative.ac. AR opens up huge possibilities for creating immersive learning activities. Crafts Study Centre at the University for the Creative Arts: http://www.mms.ac. While it may not be suitable for all students and situations. this coupled with a strong team ethic enabling stakeholders to buy into the long-term vision.nmc. 2011 HORIZON report. section on augmented reality: http://wp.com and @team_scarlet.

in cooperation with the Studium Generale programme. When working with Augmented Reality — a very suitable medium for graphic designers. At various other art academies we see similar projects. Most important. However. both young and digitally savvy. by the way — new elements should be considered in the concept and design phase. one of which is the Scarlet (Special Collections using Augmented Reality to Enhance Learning and Teaching) project. because it will change aspects of their discipline as well as their ‘metier’. and the department involved. aspects are added to the world of graphics regarding the Internet and social media. At the beginning of the Royal Academy of Arts’ current study year. as AR Lab are thoroughly convinced that it is important to let first-year students. Whether a concept is devised with colour pencils. experience augmented reality. in effect. in the UK. we made a head start with the introduction of AR via projects differing in length and with different student groups. is that they want the students to become acquainted with creative or artistic research. freshmen and -women. meaning a short. What is important for the Academy. started to work with their classes of 27 students each (!) together with some third year students as well. the majority of mainstream teachers place analogue and digital techniques at the same level when developing concepts. a programme covering lectures throughout the year. very new.INTRODUCING AR TO FIRST YEAR GRAPHIC DESIGN STUDENTS YOLANDE KOLSTEE Graphic Design students learn how to work with the latest Adobe suit. while. one-day exhibition of the results. at the University for the Creative Arts. The search to get to know what AR is and how it could be a 62 . felt pens or a collage of other material is not the most important issue at that moment. are the courses in which concept development forms the main part. while also taking co-authorship and performance into account. they often have a (underlying) preference for pen and ink based handwork. Photoshop and website development tools. Two Graphic Design teachers. The idea was to work towards a special PopUp Gallery. however. We. With Augmented Reality new.

but two winners were awarded: Jente Hoogeveen en Jorick de Quaatsteniet. We showed them our AR projects and introduced them to e. They were part of the jury that awarded the two ‘best’ projects at the end of the first Pop-Up Gallery. These types of projects show the broad range of ways in which an extra layer on top of the physical world poses questions about what we see. Aurasma — a simple and elegant tool for beginners in the AR app world. We were lucky to have two lecturers who gave their talk on AR as part of the Studium Generale programme preceding the Pop-Up Gallery: AR artist Sander Veenhof and data-visualisation expert Niels Schrader.meaningful technique for graphic designers began comprised two steps. without the aid digital techniques and devices? Following that. It is exactly this assumption that we want to investigate. turned out to be very exciting. The end-results would be displayed during our first Pop-Up Gallery on AR. The variety of final projects was staggering and impressive. What type of problem would they want to solve using AR? With what classification should it work – analogue and/or digital? With this exercise we hoped to widen the concept. and the second step was to learn to play with the idea of adding extra information — both analogue and/or digital — a concept development trajectory of sorts. Finally. while also seeking to come up with new applications and reasons for the use of Augmented Reality. is that they appear to invite spectators to look at the world from another perspective. at first. a publication of all the projects was made with a spread for each student in which the description and intention of their project were incorporated. Or it would be just reality. Not one. Jente won with her project 'Had je dat gedicht?' (roughly translated as 'How does that rhyme with you?'). We provided technical help when students were really eager to get their ideas to work as a fully functioning app. Ultimately. Without digital technology.” In completing the first step of the two step project. A special recommendation was given for the dyslexia project of Janne van Hooff who visualised how people suffering from dyslexia see letters and numbers. This is obvious because the assumption in AR is that reality must be supplemented with digital information. The concept developing part. The same goes for those students who. were not completely convinced about using AR and how they could incorporate this technique in their projects. so to say. 63 . no addition to reality and certainly no interaction would be possible. We asked them to think about the kind of information they wanted to disclose in specific situations. The rationale underlying the research project therefore lies especially in the use of digital techniques to make this extra layer onto reality visible. the idea was to provide an enriching experience for the students and to give them the tools with which they could incorporate AR in their work. Jorick won with his project 'Get your shit together' where animated QR-codes formed text. regardless whether they are analogue or digital. Words emerged by moving strips of black paper over the seemingly incoherent rows of letters. Is Augmented Reality only achievable by the use of digital means? Can hidden information be augmented. the students gave us a huge input of examples they encountered: they found all sorts of material via social media or as search results that they consequently discussed. phones and tablets.g. headsets. The first was a very active YouTube-Google search. These are some tools one associates with Augmented Reality (AR). we invited freshmen and -women from the Graphic Design department to get started with Augmented Reality. step two. What all the students’ projects have in common. In the words of the students’ teachers : “Spectacles. Error messages we are confronted with on a daily basis are transformed into poems in order to take some of the zing out of stressful situations. for the students themselves as well as for us.

64 .

interesting. which uses the latest text recognition algorithms. This way the user is asked to involve their environment in the process of learning words. Another interesting project in development is the Sesame Street app Big Bird’s Words (Qualcomm. 2013). WWW. On the screen you see a 3D model of the human body. or switch on the layer showing the nervous system for example.CREATIVASTOCK. These examples show some of the new forms of interaction and presenting information. With sliders and buttons you can set the transparency of the skin.COM ample is the 4D Anatomy project by daqri (2012). An interesting exImage by MARCELO GERPE. Many of these experiments draw on of the possibilities to explore virtual information spatially. with which developers are trying to create new. where you can explore the physiology of a human being by moving the display device along a piece of paper with markers. just as in many other fields.AUGMENTED EDUCATION HOW AR TECHNOLOGIES EXTEND OUR MINDS ROBIN DE LANGE In the field of education. it gives points to the user. The (young) users of this upcoming app are asked to look for certain words in their home and aim their device at it. and memorable learning experiences. When the device recognizes the word. researchers and developers are experimenting with potential applications of Augmented Reality technologies. 65 .

cal rotation in (b) is actually much faster than the mental rotation. Clark and Chalmers argue. To make this claim plausible. while Inga’s biological memory is functioning properly. In the same manner.” (Clark and Chalmers. thereby simplifying the task. who are remembering how to get to the museum. where the person can choose between using his neural implant that does the rotation operation as fast as the computer does in (b). Clark and Chalmers show that true mental events can extend in the environment as well. In this experiment the reader is asked to consider three cases and to think about how much cognition is present in each case: (a) A person sits in front of a computer screen which displays geometrical shapes and is asked to fit the shapes in the displayed ‘sockets’. In their paper about cognitive performances while playing Tetris. this information may even be seen as part of our cognitive process. by rotating the shapes in his mind. these kinds of cases are actually not as abstract as they might seem: they describe a very well-known real-life situation. Inga is thought to have a belief about the location of the museum. AR technologies could not only lead to new learning methods. under the right circumstances. players were not only physically rotating the shapes to fit the slot. 1994) It is this example of the human capacity to manipulate the environment to solve problems. Furthermore. we would have no hesitation in recognizing as part of the cognitive process. Because of this change of perspective regarding external information sources. By showing how beliefs are not bound by the borders of the body. the latter having some speed advantage. To backup this claim. but could. before she recalls this from her internal memory. which Clark and Chalmers employ to introduce the Parity Principle: “if a part of the world functions as a process EXTEnDED MInD AnD ThE PARITY PRInCIpLE Andy Clark and David Chalmers start their renowned paper ‘The Extended Mind’ with a thought experiment. In certain cases. Kirsh and Maglio found that the physi- which. and in my opinion should. then that part of the world is (so we claim) part of the cognitive process. Otto has Alzheimer’s disease and uses a notebook to serve the function of his memory. Now. namely playing the video game Tetris. were it done in the head. (b) The same situation as in (a). Thereby. also trigger debates about the very goals of education itself. All the examples of cognitive extension that Clark1 gives in his books and papers are not the typical futuristic technologies that come to mind when thinking about humans merging with tech- 66 . Otto can be said to have a belief about the location of the museum before he actually consults his notebook. but now the person can choose either to mentally rotate the shape or physically rotate it by pressing a button. I will first introduce the concept of the Extended Mind. or using ‘old-fashioned’ mental rotation. (Kirsh and Maglio. but were also trying to determine whether the shape fits in the slot. Clark and Chalmers present a thought experiment involving the fictional characters Otto and Inga. 1998) According to the Parity Principle.In this article I will argue that the developments in AR technologies will make digital information sources much more transparently available to us. the notebook can be seen as an extension of Otto’s memory. (c) The same situation in a possible future. the human mind is not bound by the borders of skin and skull.

according to the Parity Principle. ‘you’ is only “the hybrid biotechnological system that now includes the wristwatch as a proper part” (Clark. (This proven transparency of the wrist watch is what makes the development of smart watches interesting. 2003).) Our brains incorporate the world and some of the technologies therein in their cognitive processes in such an intimate way.nology. The technological aid should always be available when needed. we use the pen to manipulate the external memory source. One example he mentions is the use of pen and paper when doing long multiplications. The user has to put physical and cognitive effort into navigating the browser to the right page. During this time these technologies have become ubiquitous. In fact. we often answer “yes”. for example parts of the Web. Furthermore. actually part of this process. according to Clark this is not simply loosely formulated informal language. From this. An interesting question is whether more modern external information sources could obtain the same status as these age-old technologies and play a similar. the paper. The information retrieved from the external source should be directly endorsed and trusted. 3. To calculate the product of two numbers. Subsequently we raise our arm. due to the open character of the Web. it is quite clear that the Web with which we interact on a daily basis cannot be seen as part of our cognition.) Now. The writing utensils play a crucial role in this cognitive process and are therefore. 2. a measure of the combination of time and physical and cognitive effort. we can refer to the three criteria suggested by Clark: 1. look at our watch and see what time it is. From our critical stance we do not immediately believe most information we encounter on the Web something which is. that Clark considers us to be “natural born cyborgs” (Clark. this direct interaction with brains is by no means necessary to become part of the cognitive process (nor are they sufficient for cognitive extension: communicating with technology through a Brain Machine Interface usually still takes too much cognitive effort. active role in cognitive processes. and we are wearing our watch. Now. Although the possibilities of Brain-Machine Interfaces and neural implants such as in case (c) offer very exciting new ways of communicating with technology. Another example shows that it has become common to talk about the information that is in some of our technologies as if part of our own knowledge. then wait for the downloading of the page 67 . 2003). we use an algorithm that divides the process of multiplying arbitrary large numbers into very simple steps. 2008). the information access costs when retrieving information from the Web is way too high. the technologies which Clark considers as cognitive extensions of our cyborg minds are hardly identified as technology anymore. probably a wise attitude. More precisely: the information access costs. should be as low as with an equivalent function of the brain (Smart et al. actually become parts of our minds? Could digital information sources actually become parts of our minds? To begin to answer this question and decide whether a part of the world should (temporarily) be seen as part of the cognitive process. By writing down figures in certain locations. You actually do know what time it is. Could digital information sources. They have become socially accepted and actually shaped culture itself. wrist watches have been around for many decades. writing utensils even for centuries. The external resource should be directly available without difficulty. When somebody asks us on the street whether we know what time it is.

the information access costs of retrieving information from the Web are way too high to be considered as part of the cognitive process. tion knows when you’re busy driving for example and doesn’t bother you then. telecommunication service providers keep improving their networks to decrease download times.2 More so. technology is ever-evolving and many of the present advancements will make information available to us in more reliable. Now. An application running in the background does speech and text recognition and keeps track of words you (the biological ‘you’) don’t know the meaning of. Of course. clear description of the word pops up immediately in the corner of your field of view. that they could lower information access cost so significantly? Of course Head-Up Displays (HUDs) play a great part in this. information can be presented to the user at all LOWERInG InFORmATIOn ACCESS COSTS However. user interfaces are becoming more intuitive. Despite of what many critics expected. In the widespread current way of interacting. But what is it with AR technologies. The technological developments that are associated with Augmented Reality especially have the potential to lower the information access costs for digital sources considerably. and a short. quicker and more intuitive ways. know the meaning of a certain word that is not in your biological memory. Processors in smartphones are becoming faster. in a very real sense. You’ve been using this application for a few months now and it’s starting to have quite a good sense of when to present you with the meaning of a word you encounter. Let us consider the following scenario to get a better idea: For a few years now you have your own AR glasses. would you say that you know the meaning of this word? I can imagine that you — after you get more and more used to the device and have experienced this situation a few times already — might say ‘yes’. batteries are getting better. it has become socially quite acceptable to be wearing a Head-Up Display at all times. When it does indeed become ordinary to wear HUDs. by eliminating the physical effort of getting your smartphone from your pocket and having to hold it in within your view. similar to the situation with the wrist watch.and scan through the text to find the information he needs. I think you might start to feel like you really do know it. when a friend (who is not really into new technology and rather asks a friend to help him) asks you whether you 68 . etc. the applica- The main goal of education should be to train the technologically extended cognitive system.

I suggest that under certain conditions. it could be decided to store certain information that we want to remember in an easily accessible. these digital information sources could then be seen as proper parts of our hybrid minds. EXTEnDED COGnITIOn AnD EDUCATIOn In the previous I have introduced the concept of extended cognition and explored the possibilities of digital technology for cognitive extension. creating a very natural. SEE hTTp://DAqRI. digital information can be placed over the world. personalized information sources are able to compete with mental resources in terms of costs of information access. at the exact moment when it is needed. personalized cloud of knowledge. By doing so. intuitive interface. Another important aspect of AR is the use of information from different sensors and smart algorithms doing image and speech recognition. By combining these. Furthermore. According to the Parity Principle. which is of course the main idea of AR3. possibilities are created to present information in context-sensitive ways responding both to the environment and the user. One could argue from this perspective that the main goal of education should be to train the technologically extended cognitive system.COm 69 . you can interact with digital information in similar ways to how you interact with the physical world. Adopting this philosophical perspective can have huge consequences for a field like education. Instead of trying to store all information in biological memory by endless repetition. Image COURTESY OF GAIA DEmpSEY AnD DAqRI.times. well designed. By accepting digital external information sources as a proper part of memory. These are the characteristics of AR that create the potential of making digital information much more transparently available to us.

Minds. Retrieved from http://www. & Harnad. (2003). 10-23. Press. For the most part of their education. In I.daqri. A. An extended view of the mind. (2012). On distinguishing epistemic from pragmatic action. ¢¢ Rorty. D.. (2008). Het Semantic Web en netwerktechnologische cognitieve uitbreidingen. Natural-Born Cyborgs.com/labs/4d-anatomy ¢¢ The general view on the use of technology in education is quite different from the view expressed in this article though. Retrieved from http://www. & Shadbolt. cal) calculator5. a lot of research on the use of technology in the learning process is needed. Furthermore. 1-23). S.com/ media/videos/big-birds-words-demo-reelvideo ¢¢ ¢¢ Clark. System Upgrade . P. Digital technology is developing very rapidly.qualcomm. (1994). & Maglio. To come to these decisions. Embodiment.this task of storing information could be off-loaded to an external source which is constantly available to us at low information access costs. there should be an active discussion on the goals of education and what technologies students can use to reach these goals. Cognition Distributed: How Cognitive Technology Extends Our Minds (pp. R. D. ¢¢ ¢¢ Clark. Kinds of Minds: Toward an Understanding of Consciousness. (1999). Technologies. in which external resources have an active role in the cognitive process. Philosophy and Social Hope (pp. 513-549. Harnad. N. Braines. P. & Chalmers. (2011). E. A. can offer a valuable perspective in this discussion. Network-Enabled Cognition: The Contribution of Social and Technological Networks to Human Cognition . O. ESRC/EPSRC Technology Enhanced Learning Programme. ¢¢ ¢¢ ¢¢ Dror. D. E. students still only get to use some basic technologies: a pen. Smart. D. Oxford: Oxford Univ.. Kirsh. 4D ANATOMY.. (1998). (2013). A. 114-126). Smart. (2008). J. Cognition and the Web. (2008). (2010 (In Press)). ¢¢ daqri. Analysis . The Extended Mind.com/wikisofie ¢¢ Dennett. I. The 70 . Engelbrecht. Press. P. London: tel. ¢¢ Clark. Action and Cognitive Extension. Supersizing the Mind. REFEREnCES ¢¢ Blomberg. Oxford: Oxford Univ. New York USA: Basic Books. P.. Education as Socialization and as Individualization.ac. Do socio-technical systems cognise? 2nd Symposium on Computing and Philosophy . This critical attitude towards the use of technology is very understandable. Cognitive science ..Realising the Vision for UK Education. (2012). robindelange. & S.. (2009). London: Penguin Books.uk. ¢¢ Qualcomm. a piece of paper and maybe a dictionary or an outdated (graphi4 de Lange. Dror.. careful decisions have to be made about how to use it in education. R. and the future of human intelligence. (1996). Hendler. Retrieved from http://site. Rorty. Big Birds words. In R. Offloading Cognition onto Cognitive Technology.

pdf ¢¢ Wheeler. (2011. If the word would be jargon of a field you are not familiar with for example. Moore’s law seems to be failing here. I will refer to Clark further on. Retrieved June 24. he is looking for funding to do a PhD within his field of interest. 2. For his graduation research project he is developing an Augmented Reality application that supports the graphical solving of mathematical equations. 5.com 71 .Extended Mind and Network-Enabled Cognition.uk/16649/1/Network-Enabled_Cognitionv17. Robin has taken part in several entrepreneurial projects. M. The initial paper ‘The Extended Mind’ was written by Andy Clark and David Chalmers. from University of Southampton . thereby increasing the costs of information access. he was the co-owner of a company that specialized in homework guidance and tutoring. At the moment. The hardware in these devices stays roughly the same. Computational Culture . Most notably. 4. Besides his studies. 3. 2011. Thinking Beyond the Brain: Educating and Building.School of Electronics and Computer Science: http://eprints. For this reason.ac. you would probably not understand the meaning directly and need to look up more information. from the Standpoint of Extended Cognition. For more information: www. thereby showing educational challenges and possibilities. He is now a student at the Media Technology MSc program at Leiden University and is particularly interested in technologically extended cognition. He is a freelance video producer and science communicator.ecs. Because Clark has written many other papers and books on this subject. This characteristic of AR of overlay ing the physical world with virtual objects is not really present in this scenario. it does use certain AR technologies intensively to provide context-sensitive information to the user who interacts with the world. EnDnOTES 1. The information access costs of looking up a word in the dictionary go through the roof. one might argue that the example does not really show AR. November). soton. However. even remains the same price! Robin de Lange Robin de Lange has a bachelor’s degree in Physics and Philosophy and has followed courses on Artificial Intelligence.robindelange.

HOW DID WE DO IT: WhICh AUGmEnTED REALITY SOFTWARE DOES ThE AR LAB USE? Wim van Eck 72 .

or we supervise 73 . and updates and new programs pop up all the time. ‘easy-to-use for desktop augmented reality’. Secondly. THE APPLICATION WILL PREMIER AT MUSEUM CATHARIJNECOVENT (UTRECHT. there are so many programs available it is difficult to actually have tried them all. Often we are asked “What is the best augmented reality software?” which is a difficult question to answer. ‘augmented reality software for interactions’ and ‘head-mounted augmented reality’. we merely aim to give an insight into our daily workflow. Easy-to-use for mobile devices Since the AR Lab is based at the Royal Academy of Art The Hague most of our projects are developed together with our students. At the AR Lab we use a variety of augmented reality programs to realize our projects. First of all. I will give an overview of the software we use and why we use it. IS DEVELOPED BY THE AR LAB USING VUFORIA AND UNITY 3D. This doesn’t mean we claim that this is the best software and that there are no alternatives. choosing which software to use really depends on what you want to achieve in the end.THIS APPLICATION. COMMISSIONED BY THE RCE (CULTURAL HERITAGE AGENCY OF THE NETHERLANDS). For this overview we created four categories: ‘easy-to-use for mobile devices’. NETHERLANDS) ON MAY 17TH. THE PAINTING 'ISAAC BLESSING JACOB' BY THE DUTCH PAINTER GOVERT FLINCK (16151660) HAS BEEN AUGMENTED TO GIVE INSIGHT INTO THE PAINTERS PRACTICE AND OFFERS SCIENTIFIC DATA IN A PLAYFUL WAY.

you only need to register to be able to access it. 2 a large screen. it’s more difficult to import 3D models from. It is a pity that Aurasma only seems to give good support for 3ds Max and Maya. though the OSX version lacks some options and seems to be less stable. When more possibilities are needed there is also a free online application (www.buildar. The tracking quality of Aurasma is Fig. BuildAR is based on the world-renowned ARToolKit. Markers can be generated on the spot by taking pictures with the device. Aurasma (www.com/partners). they often have their own smartphone or tablet and are very keen to develop for it. By downloading the free Aurasma app from the iOS App Store or Google Play store students can immediately begin creating their first project. adds an easy-to-use graphical interface. and also to import images and videos which are not on your mobile device. co. all they need is their mobile device. Cinema 4d. for example.com) quickly became our favourite choice for this group of students. Furthermore. but it is definitely good enough for most projects. however. ‘BuildAR Pro Mac 74 . 1 not the best around. It is very easy though to use video with transparency. Some of our students. This well documented application makes it easier to precisely position your virtual objects. and BuildAR (http://www. there are quite some versions which can be a bit confusing. still prefer to present their creations on Fig. but students who want to develop their own application. a not very common option which can give great results.aurasma. When downloading BuildAR. The ‘BuildAR Pro 2 Trial Download’ is the newest version for Windows and features both marker tracking and image based tracking.The app provides clear information for every step you have to take. Many of these students don’t have a technical background. but there is a ‘demo’ watermark on your video-background and you won’t be able to save your scene (Figure 2). so easy-to-use software is preferable. You can use this version for free. as well as 3D models. removing any technical barriers (Figure 1).aurasma. and photographs and videos which are stored on the device can be used to augment the chosen scene. desktop augmented reality software is getting scarcer. Easy-to-use for desktop augmented reality Nowadays most software companies seem to focus on mobile devices. It runs on both Windows and OSX.nz) is the software we use most often. fast camera movements will result in loss of tracking.

You can natively develop for iOS (Xcode). you will need to buy the Unity iOS or Android add-on to be able to export to a mobile device. 3 75 . In issue 2 of AR[t] we introduced our latest augmented reality headset named Marty and its tracking software. Lastly. this issue of AR[t] features more information about this system. the new version can augment your scene with images. BuildAR is easy-to-use.Trial Download’ is the OSX version and only features marker tracking. there is the ‘BuildAR Free version Download (2008 version)’. Unity makes Head-mounted augmented reality For head mounted based augmented reality we have solely used software and hardware developed by TU Delft. Unity (Figure 3) is a popular game engine offering excellent visual quality on mobile devices. Adding interactions through scripting is relatively easy. Android (Eclipse) or use it as a plugin for Unity (www. Fig. which is only available for Windows and is free of charge. it very easy to import 3d-models including animations from almost all 3d-packages. the demo-limitations are the same as in the Windows version. but many of our students don’t mind this logo so much though. audio and 3D models.unity3d. Semiconductor company Qualcomm offers Vuforia for free and Unity has a free version as well. However.vuforia. video. there are many tutorials around and the Unity forum is a great source for information. Besides image markers there are also frame markers available. The only limitation is a ‘HITLabNZ’ logo.com). Also. Vuforia’s image based tracking is extremely stable. AR software for interactions A program we often use at the AR Lab ourselves is Vuforia (https://www. which use a pattern of black and white cubes positioned around the image. the tracking quality stays good even if the tracking image is partly occluded or when there is little light available.com) allowing you to easily export to iOS and Android.

AR[t] PICK ImmATERIALS BY DESIGn STUDIO OnFORmATIVE AnD ChRISTOphER WARnOW. 2011 76 .

COm).onformative. DATA IS pLACED In A ROOm.OnFORmATIVE.COm) AnD ThE COmpUTATIOnAL DESIGnER ChRISTOphER WARnOW (ChRISTOphERWARnOW. ThE RESULTInG FORmS DEpICT pOSSIBLE DATA SETS AnD EXAmInE ThE DESIGn pOSSIBILITIES BETWEEn TEChnOID hOLOGRAmS AnD pERSOnAL nOTES. ThE pROJECT EXpLORES hOW InFORmATIOn CAn BE InTEGRATED InTO phYSICAL SpACE. Further information: ¢¢ ¢¢ http://christopherwarnow. USInG ThE LIGhT pAInTInG TEChnIqUE.IMMATERiALS IS ThE RESULT OF A COLLABORATIOn BETWEEn ThE DESIGn STUDIO OnFORmATIVE (WWW.com/portfolio/?p=244 http://www.com/work/immaterials/ 77 .

ContribUtors  WIm VAn ECK Royal Academy of Art (KABK) w. Faculty Mechanical. Maritime and Materials Engineering (3ME). bio-hybrid computer games. supporting and supervising students and creating 3d content. Esmé Vahrmeijer is the graphic designer and webmaster of the AR Lab. scientific creativity. Specializations include social robotics. robot learning and Augmented Reality. Her interests are in graphic design. EDWIn VAn DER HEIDE Leiden University evdheide@liacs. YOLAnDE KOLSTEE  Royal Academy of Art (KABK) Y. web design. Pieter Jonker is Professor at  Delft University of Technology. His main interests and fields of research are: real-time embedded­ image processing.nl Mariana Kniveton is currently a master student at Utrecht Universtity. photography and education.nl ESmÉ VAhRmEIJER Royal Academy of Art (KABK) e. real-time 3d animation. Beside’s running his own studio he’s part-time assistant professor at Leiden University (LIACS / Media Technology MSc programme) and heading the Spatial Interaction Lab at the ArtScience Interfaculty of the Royal Conservatoire and Arts Academy in The Hague. robot vision. She holds the post of Lector (Dutch for researcher in professional universities) in the ­ field of ‘Innovative Visualisation Techniques in higher Art Education’ for the Royal Academy of Art. among others.nl Wim van Eck is the 3D animation specialist of the AR Lab.vahrmeijer@kabk. His main tasks are developing Augmented Reality projects. space and interaction.P. The Hague. Besides her work at the AR Lab. PIETER JOnKER Delft University of Technology P. 78 .nl Hanna Schraffenberger works as a researcher and PhD student at the Leiden Institute of Advanced Computer Science (LIACS) and at the AR Lab in The Hague.nl HAnnA SChRAFFEnBERGER Leiden University hkschraf@liacs. MAARTEn LAmERS Leiden University lamers@liacs.kniveton@kabk.nl MARIAnA KnIVETOn Royal Academy of Art (KABK) m. Her research interests include interaction in interactive art and (non-visual) Augmented Reality.vaneck@kabk. Mariana took up editing duties for this current issue. His interests are.Kolstee@kabk. game design and creative research. and models for perceptualization.nl Edwin van der Heide is an artist and researcher in the field of sound.nl Yolande Kolstee is head of the AR Lab since 2006. studying New Media and Digital Culture. she is a part time student at the Royal Academy of Art (KABK) and runs her own graphic design studio Ooxo. typography. Since september 2012 she has worked as an intern at the Research Department IVT and the AR Lab. Maarten Lamers is assistant professor at the Leiden Institute of Advanced Computer Science (LIACS) and board member of the Media Technology MSc program. After a brief stint as a cover model for AR[t] #2. parallel image processing architectures.Jonker@tudelft.

com OLIVER PERCIVALL Antal Ruhl is a media artist with a background in design. Matt Ramirez has over 15 years experience in web design and e-learning content development for a variety of subject areas including medicine. science. NEXT ISSUE The next issue of AR[t] will be out in the fourth quarter of 2013.wordpress.com Jouke Verlinden is assistant professor at the section of computer aided design engineering at the Faculty of Industrial Design Engineering.nl MATT RAMIREZ teamscarlet.JOUKE VERLINDEN Delft University of Technology j. LOTTE DE REUS www.antalruhl. He creates objects that let us rethink our environment. he leads the “Augmented Matter in Context” lab that focuses on blend between bits and atoms for design and creativity. science and art.T Project Management. These objects vary from kinetic sculptures to interactive installations.com Robin de Lange is a student at the Media Technology MSc program at Leiden University and part-time entrepreneur. Alejandro Veliz Reyes is a teaching assistant and PhD student on digital architectural design at the University of Salford (United Kingdom). augmented reality.c. design.theheavyprojects.nl GUEST CONTRIBUTORS BC “HEAVY” BIERMANN The Heavy Projects www. IT.verlinden@tudelft. He is interested in where technology could take us. Oliver Percivall works in I. Heavy creates innovative interfaces between digital design and physical worlds in ways that that provoke the imagination and challenge existing styles of art. special collections and business.robindelange.lottedereus. Lotte would again like to work in the domains that combine art and technology. and interaction. He is looking for funding to do a PhD on technologically extended cognition. This interest has resulted in the science fiction novel "The Augmented Star".com Lotte de Reus recently received her Master’s degree in Design for Interaction at the Faculty of Industrial Design Engineering at Delft University of Technology. His research interests are design pedagogy. 79 . ALEJANDRO VELIZ REYES University of Salford ROBIN DE LANGE www. BC "Heavy" is founder of The Heavy Projects [and its collaborative spin-off Re+Public]. In the future. With a background in virtual reality and interaction design. and collaborative technologies for design ANTAL RUHL www.

Sign up to vote on this title
UsefulNot useful