Haskell and Yesod

Chapters

Basics Introduction Haskell Basics Shakespearean Templates Widgets Yesod Typeclass Routing and Handlers Forms Sessions Persistent Deploying your Webapp d!anced o R"ST#ul $ontent o Yesod%s &onads o uthentication and uthori'ation o Sca##olding and the Site Template o Internationali'ation o $reating a Subsite "(amples o Blog) i*+n, authentication, authori'ation, and database o Wiki) markdo-n, chat subsite, e!ent source o .S/0 Web Ser!ice o $ase Study) Sphin(1based Search ppendices o monad1control o $onduit o Web pplication Inter#ace o Settings Types o http1conduit o (ml1conduit o "numerator Package
o o o o o o o o o o o

Introduction
Since -eb programming began, people ha!e been trying to make the de!elopment process a more pleasant one2 s a community, -e ha!e continually pushed ne- techni3ues to try and sol!e some o# the lingering di##iculties o# security threats, the stateless nature o# HTTP, the multiple languages 4HT&5, $SS, .a!ascript6 necessary to create a po-er#ul -eb application, and more2 Yesod attempts to ease the -eb de!elopment process by playing to the strengths o# the Haskell programming language2 Haskell%s strong compile1time guarantees o# correctness not only encompass types7 re#erential transparency ensures that -e don%t ha!e any unintended side e##ects2 Pattern matching on algebraic data types can help guarantee -e%!e accounted #or e!ery possible case2 By building upon Haskell, entire classes o# bugs disappear2 8n#ortunately, using Haskell isn%t enough2 The -eb, by its !ery nature, is not type sa#e2 "!en the simplest case o# distinguishing bet-een an integer and string is impossible) all data on the -eb is trans#erred as ra- bytes, e!ading our best e##orts at type sa#ety2 "!ery app -riter is le#t -ith the task o# !alidating all input2 I call this problem the boundary issue) as much as your application is type sa#e on the inside, e!ery boundary -ith the outside -orld still needs to be saniti'ed2

Type Safety
This is -here Yesod comes in2 By using high1le!el declarati!e techni3ues, you can speci#y the e(act input types you are e(pecting2 nd the process -orks the other -ay as -ell) using a process o# type1sa#e 8R5s, you can make sure that the data you send out is also guaranteed to be -ell #ormed2 The boundary issue is not 9ust a problem -hen dealing -ith the client) the same problem e(ists -hen persisting and loading data2 /nce again, Yesod sa!es you on the boundary by per#orming the marshaling o# data #or you2 You can speci#y your entities in a high1le!el de#inition and remain bliss#ully ignorant o# the details2

Concise
We all kno- that there is a lot o# boilerplate coding in!ol!ed in -eb applications2 Where!er possible, Yesod tries to use Haskell%s #eatures to sa!e your #ingers the -ork)

The #orms library reduces the amount o# code used #or common cases by le!eraging the pplicati!e type class2 Routes are declared in a !ery terse #ormat, -ithout sacri#icing type sa#ety2 Seriali'ing your data to and #rom a database is handled automatically !ia code generation2

In Yesod, -e ha!e t-o kinds o# code generation2 To get your pro9ect started, -e pro!ide a sca##olding tool to set up your #ile and #older structure2 Ho-e!er, most code generation is done at compile time !ia meta programming2 This means your generated code -ill ne!er get stale, as a simple library upgrade -ill bring all your generated code up1to1date2 But #or those -ho like to stay in control, and kno- e(actly -hat their code is doing, you can al-ays run closer to the compiler and -rite all your code yoursel#2

Performance
Haskell%s main compiler, the ;H$, has ama'ing per#ormance characteristics, and is impro!ing all the time2 This choice o# language by itsel# gi!es Yesod a large per#ormance ad!antage o!er other o##erings2 But that%s not enough) -e need an architecture designed #or per#ormance2 /ur approach to templates is one e(ample) by allo-ing HT&5, $SS and .a!aScript to be analy'ed at compile time, Yesod both a!oids costly disk I</ at runtime and can optimi'e the rendering o# this code2 But the architectural decisions go deeper) -e use ad!anced techni3ues such as conduits and builders in the underlying libraries to make sure our code runs in constant memory, -ithout e(hausting precious #ile handles and other resources2 By o##ering high1le!el abstractions, you can get highly compressed and properly cached $SS and .a!aScript2 Yesod%s #lagship -eb ser!er, Warp, is the #astest Haskell -eb ser!er around2 When these t-o pieces o# technology are combined, it produces one o# the #astest -eb application deployment solutions a!ailable2

Modular
Yesod has spa-ned the creation o# do'ens o# packages, most o# -hich are usable in a conte(t outside o# Yesod itsel#2 /ne o# the goals o# the pro9ect is to contribute back to the community as much as possible7 as such, e!en i# you are not planning on using Yesod in your ne(t pro9ect, a large portion o# this book may still be rele!ant #or your needs2 /# course, these libraries ha!e all been designed to integrate -ell together2 8sing the Yesod Frame-ork should gi!e you a strong #eeling o# consistency throughout the !arious PIs2

A solid foundation
I remember once seeing a PHP #rame-ork ad!ertising support #or 8TF1+2 This struck me as surprising) you mean ha!ing 8TF1+ support isn%t automatic= In the Haskell -orld, issues like character encoding are already -ell addressed and #ully supported2 In #act, -e usually ha!e the opposite problem) there are a number o# packages pro!iding po-er#ul and -ell1designed support #or the problem2 The Haskell community is constantly pushing the boundaries #inding the cleanest, most e##icient solutions #or each challenge2

>

The do-nside o# such a po-er#ul ecosystem is the comple(ity o# choice2 By using Yesod, you -ill already ha!e most o# the tools chosen #or you, and you can be guaranteed they -ork together2 /# course, you al-ays ha!e the option o# pulling in your o-n solution2 s a real1li#e e(ample, Yesod and Hamlet 4the de#ault templating language6 use bla'e1builder #or te(tual content generation2 This choice -as made because bla'e pro!ides the #astest inter#ace #or generating 8TF1+ data2 nyone -ho -ants to use one o# the other great libraries out there, such as te(t, should ha!e no problem dropping it in2

Introduction to Haskell
Haskell is a po-er#ul, #ast, type1sa#e, #unctional programming language2 This book takes as an assumption that you are already #amiliar -ith most o# the basics o# Haskell2 There are t-o -onder#ul books #or learning Haskell, both o# -hich are a!ailable #or reading online)

5earn You a Haskell #or ;reat ;ood? Real World Haskell

Yesod relies on a #e- #eatures in Haskell that most introductory tutorials do not co!er2 Though you -ill rarely need to understand ho- these -ork, it%s al-ays best to start o## -ith a good appreciation #or -hat your tools are doing2 These are co!ered in the ne(t chapter2

Haskell
In order to use Yesod, you%re going to ha!e to kno- at least the basics o# Haskell2 dditionally, Yesod uses some #eatures o# Haskell that aren%t co!ered in most introductory te(ts2 While this book assumes the reader has a basic #amiliarity -ith Haskell, this chapter is intended to #ill in the gaps2 I# you are already #luent in Haskell, #eel #ree to completely skip this chapter2 lso, i# you -ould pre#er to start o## by getting your #eet -et -ith Yesod, you can al-ays come back to this chapter later as a re#erence2 I# you are looking #or a more thorough introduction to Haskell, I -ould recommend either Real World Haskell or 5earn You a Haskell2

Terminology
"!en #or those #amiliar -ith Haskell as a language, there can sometimes be some con#usion about terminology2 5et%s establish some base terms that -e can use throughout this book2 Data type This is one o# the core building blocks #or a strongly typed language like Haskell2 Some data types, like Int, can be treated as primiti!e !alues, -hile other data types -ill build on top o# these to create more complicated !alues2 For e(ample, you might represent a person -ith)
data Person = Person Text Int

Here, the Text -ould gi!e the person%s name, and the Int -ould gi!e the person%s age2 Due to its simplicity, this speci#ic e(ample type -ill recur throughout the book2 There are essentially three -ays you can create a ne- data type) type declaration such as type GearCount = Int merely creates a synonym #or an e(isting type2 The type system -ill do nothing to pre!ent you #rom using an Int -here you asked #or a GearCount2 8sing this can make your code more sel#1documenting2 newtype declaration such as newtype Make = Make Text2 In this case, you cannot accidently use a Text in place o# a Make7 the compiler -ill stop you2 The ne-type -rapper al-ays disappears during compilation, and -ill introduce no o!erhead2 data declaration, such as Person abo!e2 You can also create lgebraic Data Types 4 DTs6, such as data Vehicle = Bicycle GearCount | Car Make Model2 Data constructor In our e(amples abo!e, Person, Make, Bicycle, and Car are all data constructors2 Type constructor In our e(amples abo!e, Person, Make, and Vehicle are all type constructors2 Type !ariables $onsider the data type data Maybe a = Just a | othin!2 In this case, a is a type !ariable2

Tools
There are t-o main tools you%ll need to Haskell de!elopment2 The ;lasgo- Haskell $ompiler 4;H$6 is the standard Haskell compiler, and the only one o##icially supported by Yesod2 You%ll also need $abal, -hich is the standard Haskell build tool2 0ot only do -e use $abal #or building our local code, but it can automatically do-nload and install dependencies #rom Hackage, the Haskell package repository2 I# you%re on Windo-s or &ac, it is strongly recommended to do-nload the Haskell Plat#orm2 /n 5inu(, many distributions include the Haskell Plat#orm in their repositories2 /n Debian1 based systems, #or e(ample, you can get started by running sudo apt"!et install haskell"plat#or$2 I# your distribution does not include the Haskell Plat#orm, you can install it manually by #ollo-ing the instructions on the Haskell Plat#orm%s page2 /ne important tool you%ll need to update is ale(2 The Haskell Plat#orm includes !ersion :, -hile the .a!ascript mini#ier Yesod uses, h9smin, re3uires !ersion three2 Be sure to cabal install ale( a#ter getting set up -ith the Haskell Plat#orm, or you%ll run into error messages about the language19a!ascript package2 Some people like to li!e in the bleeding edge and install the latest !ersion o# ;H$ be#ore it is a!ailable in the Haskell Plat#orm2 We try to keep Yesod up1to1date -ith all current !ersions o# ;H$, but -e only o##icially support the Haskell Plat#orm2 I# you do go the route o# manually install ;H$, here are a #e- notes) You%ll need to install some additional build tools, ale( and happy in particular2 &ake sure to install all o# the re3uired $ libraries2 /n Debian1based systems, you -ould need to run) A

sudo apt"!et install libedit"de% libbsd"de% lib!$p&"de% 'lib(!"de% #ree!lut&"de%

Regardless o# ho- you%!e installed your tools, you should sure to put cabal%s bin #older in your P)T* !ariable2 /n &ac and 5inu(, this -ill be +*,M-./cabal.bin and on Windo-s it -ill be 0)PP1)T)02cabal2bin2
cabal

has lots o# di##erent options a!ailable, but #or no-, 9ust try out t-o commands) cabal update -ill do-nload the most recent list o# packages #rom Hackage2 cabal install yesod1plat#orm -ill install Yesod and all its dependencies2

&any people in the community pre#er to per#orm sandbo(ed builds o# their Haskell packages, -hich pre!ents your install Yesod #rom breaking e(isting packages, or packages you install in the #uture #rom breaking your Yesod install2 I -on%t go into details on ho- to use these in the book, but the t-o most commonly used tools are cabal1de! and !irthualen!2

Language Pragmas
;H$ -ill run by de#ault in something !ery close to HaskellB+ mode2 It also ships -ith a large number o# language e(tensions, allo-ing more po-er#ul type classes, synta( changes, and more2 There are multiple -ays to tell ;H$ to turn on these e(tensions2 For most o# the code snippets in this book, you%ll see language pragmas, -hich look like this)
3"4 5) G6)G- My5an!ua!e-xtension 4"7

These should al-ays appear at the top o# your source #ile2 dditionally, there are t-o other common approaches)

/n the ;H$ command line, pass an e(tra argument 1C&y5anguage"(tension2 In your cabal #ile, add an extensions block2

I personally ne!er use the ;H$ command line argument approach2 It%s a personal pre#erence, but I like to ha!e my settings clearly stated in a #ile2 In general it%s recommended to a!oid putting e(tensions in your cabal #ile7 ho-e!er, in the Yesod sca##olded site -e speci#ically use this approach to a!oid the boilerplate o# speci#ying the same language pragmas in e!ery source #ile2 We%ll end up using 3uite a #e- language e(tensions in this book 4the sca##olding uses **62 We -ill not co!er the meaning o# all o# them2 Instead, please see the ;H$ documentation2

O erloaded Strings
What%s the type o# 8hello8= Traditionally, it%s 9trin!, -hich is de#ined as type 9trin! = :Char;2 8n#ortunately, there are a number o# limitations -ith this)

It%s a !ery ine##icient implementation o# te(tual data2 We need to allocate e(tra memory #or each cons cell, plus the characters themsel!es each take up a #ull machine -ord2 Sometimes -e ha!e string1like data that%s not actually te(t, such as Byte9trin!s and HT&52

To -ork around these limitations, ;H$ has a language e(tension called ,%erloaded9trin!s2 When enabled, literal strings no longer ha!e the monomorphic type 9trin!7 instead, they ha!e the type Is9trin! a =< a, -here Is9trin! is de#ined as)
class Is9trin! a where #ro$9trin! == 9trin! "< a

There are Is9trin! instances a!ailable #or a number o# types in Haskell, such as Text 4a much more e##icient packed 9trin! type6, Byte9trin!, and *t$l2 Eirtually e!ery e(ample in this book -ill assume that this language e(tension is turned on2 8n#ortunately, there is one dra-back to this e(tension) it can sometimes con#use ;H$%s type checker2 Imagine -e ha!e)
3"4 5) G6)G- ,%erloaded9trin!s> Type9ynony$Instances> ?lexibleInstances 4"7 i$port 1ata/Text @TextA class 1o9o$ethin! a where so$ethin! == a "< I, @A instance 1o9o$ethin! 9trin! where so$ethin! B = put9tr5n 89trin!8 instance 1o9o$ethin! Text where so$ethin! B = put9tr5n 8Text8 $y?unc == I, @A $y?unc = so$ethin! 8hello8

Will the program print out 9trin! or Text= It%s not clear2 So instead, you%ll need to gi!e an e(plicit type annotation to speci#y -hether 8hello8 should be treated as a 9trin! or Text2

Type !amilies
The basic idea o# a type #amily is to state some association bet-een t-o di##erent types2 Suppose -e -ant to -rite a #unction that -ill sa#ely take the #irst element o# a list2 But -e don%t -ant it to -ork 9ust on lists7 -e%d like it to treat a Byte9trin! like a list o# CordDs2 To do so, -e need to introduce some associated type to speci#y -hat the contents o# a certain type are2
3"4 5) i$port i$port i$port G6)G- Type?a$ilies> ,%erloaded9trin!s 4"7 1ata/Cord @CordDA Euali#ied 1ata/Byte9trin! as 9 1ata/Byte9trin!/CharD @A "" !et an orphan Is9trin! instance

class 9a#e*ead a where type Content a

sa#e*ead == a "< Maybe @Content aA instance 9a#e*ead :a; where type Content :a; = a sa#e*ead :; = othin! sa#e*ead @x=BA = Just x instance 9a#e*ead 9/Byte9trin! where type Content 9/Byte9trin! = CordD sa#e*ead bs | 9/null bs = othin! | otherwise = Just + 9/head bs $ain == I, @A $ain = do print + sa#e*ead @88 == 9trin!A print + sa#e*ead @8hello8 == 9trin!A print + sa#e*ead @88 == 9/Byte9trin!A print + sa#e*ead @8hello8 == 9/Byte9trin!A

The ne- synta( is the ability to place a type inside o# a class and instance2 We can also use data instead, -hich -ill create a ne- datatype instead o# re#erence an e(isting one2 There are other -ays to use associated types outside the conte(t o# a typeclass2 Ho-e!er, in Yesod, all o# our associated types are in #act part o# a type class2 For more in#ormation on type #amilies, see the Haskell -iki page2

Template Haskell
Template Haskell 4TH6 is an approach to code generation2 We use it in Yesod in a number o# places to reduce boilerplate, and to ensure that the generated code is correct2 Template Haskell is essentially Haskell -hich generates a Haskell bstract Synta( Tree 4 ST62 There%s actually more po-er in TH than that, as it can actually introspect code2 We don%t use these #acilities in Yesod, ho-e!er2 Writing TH code can be tricky, and un#ortunately there isn%t !ery much type sa#ety in!ol!ed2 You can easily -rite TH that -ill generate code that -on%t compile2 This is only an issue #or the de!elopers o# Yesod, not #or its users2 During de!elopment, -e use a large collection o# unit tests to ensure that the generated code is correct2 s a user, all you need to do is call these already e(isting #unctions2 For e(ample, to include an e(ternally de#ined Hamlet template, you can -rite)
+@ha$let?ile 8$y#ile/ha$let8A

4Hamlet is discussed in the Shakespeare chapter26 The dollar sign immediately #ollo-ed by parantheses tell ;H$ that -hat #ollo-s is a Template Haskell #unction2 The code inside is then run by the compiler and generates a Haskell ST, -hich is then compiled2 nd yes, it%s e!en possible to go meta -ith this2

nice trick is that TH code is allo-ed to per#orm arbitrary I, actions, and there#ore -e can place some input in e(ternal #iles and ha!e it parsed at compile time2 /ne e(ample usage is to ha!e compile1time checked HT&5, $SS, and .a!ascript templates2 I# your Template Haskell code is being used to generate declarations, and is being placed at the top le!el o# our #ile, -e can lea!e o## the dollar sign and parentheses2 In other -ords)
3"4 5) G6)G- Te$plate*askell 4"7 "" or$al #unction declaration> nothin! special $y?unction = /// "" Include so$e T* code +@$yThCodeA "" ,r eEui%alently $yThCode

It can be use#ul to see -hat code is being generated by Template Haskell #or you2 To do so, you should use the "ddu$p"splices ;H$ option2 There are many other #eatures o# Template Haskell not co!ered here2 For more in#ormation, see the Haskell -iki page2

"uasi"uotes
GuasiGuotes 4GG6 are a minor e(tension o# Template Haskell that let us embed arbitrary content -ithin our Haskell source #iles2 For e(ample, -e mentioned pre!iously the ha$let?ile TH #unction, -hich reads the template contents #rom an e(ternal #ile2 We also ha!e a 3uasi13uoter named ha$let that takes the content inline)
3"4 5) G6)G- FuasiFuotes 4"7 :ha$let|Gp<This is Euasi"Euoted *a$let/|;

The synta( is set o## using s3uare brackets and pipes2 The name o# the 3uasi13uoter is gi!en bet-een the opening bracket and the #irst pipe, and the content is gi!en bet-een the pipes2 Throughout the book, -e -ill o#ten times use the GG1approach o!er a TH1po-ered e(ternal #ile since the #ormer is simpler to copy1and1paste2 Ho-e!er, in production, e(ternal #iles are recommended #or all but the shortest o# inputs as it gi!es a nice separation o# the non1Haskell synta( #rom your Haskell code2

Summary
You don%t need to be an e(pert in Haskell to use Yesod, a basic #amiliarity -ill su##ice2 This chapter hope#ully ga!e you 9ust enough e(tra in#ormation to #eel more com#ortable #ollo-ing the rest o# the book2

#asics
The #irst step -ith any ne- technology is getting it running2 The goal o# this chapter is to get you started -ith a simple Yesod application, and co!er some o# the basic concepts and terminology2

Hello $orld
5et%s get this book started properly) a simple -eb page that says Hello World)
3"4 5) G6)G- Type?a$ilies> FuasiFuotes> MultiPara$TypeClasses> Te$plate*askell> ,%erloaded9trin!s 4"7 i$port Hesod data *elloCorld = *elloCorld $kHesod 8*elloCorld8 :parseIoutes| . *o$eI G-T |; instance Hesod *elloCorld !et*o$eI == *andler Iep*t$l !et*o$eI = de#ault5ayout :wha$let|*ello CorldJ|; $ain == I, @A $ain = warp1ebu! &KKK *elloCorld

I# you sa!e that code in helloworld/hs and run it -ith runhaskell helloworld/hs, you%ll get a -eb ser!er running on port >HHH2 I# you point your bro-ser to http)<<localhost)>HHH, you%ll get the #ollo-ing HT&5)
GJ1,CTHP- ht$l< Ght$l<Ghead<Gtitle<G.title<G.head<Gbody<*ello CorldJG.body<G.ht$l<

We%ll re#er back to this e(ample through the rest o# the chapter2

%outing
5ike most modern -eb #rame-orks, Yesod #ollo-s a #ront controller pattern2 This means that e!ery re3uest to a Yesod application enters at the same point and is routed #rom there2 s a contrast, in systems like PHP and SP you usually create a number o# di##erent #iles, and the -eb ser!er automatically directs re3uests to the rele!ant #ile2 In addition, Yesod uses a declarati!e style #or speci#ying routes2 In our e(ample abo!e, this looked like)
$kHesod 8*elloCorld8 :parseIoutes| . *o$eI G-T |; $kHesod is a Template Haskell #unction, and parseIoutes

is a GuasiGuoter2

*H

In "nglish, all this means is) In the HelloWorld application, create one route2 I%d like to call it *o$eI, it should listen #or re3uests to . 4the root o# the application6, and should ans-er G-T re3uests2 We call *o$eI a resource, -hich is -here the IRI su##i( comes #rom2 The R su##i( on resource names is simply con!ention, but it%s a #airly uni!ersally #ollo-ed con!ention2 It makes it 9ust a bit easier to read and understand code2 The $kHesod TH #unction generates 3uite a bit o# code here) a route data type, a dispatch #unction, and a render #unction2 We%ll look at this in more detail in the routing chapter2 But by using the "ddu$p"splices ;H$ option, -e can get an immediate look at the generated code2 much cleaned up !ersion o# it is)
instance IenderIoute *elloCorld where data Ioute *elloCorld = *o$eI deri%in! @9how> -E> IeadA renderIoute *o$eI = @:;> :;A instance Hesod1ispatch *elloCorld *elloCorld where yesod1ispatch $aster sub toMaster appLKL appLKM $ethod pieces = case dispatch pieces o# Just # "< # $aster sub toMaster appLKL appLKM $ethod othin! "< appLKL where dispatch = Hesod/Ioutes/1ispatch/to1ispatch : Hesod/Ioutes/1ispatch/Ioute :; ?alse on*o$e ; on*o$e :; = Just + 2$aster sub toMaster BappLKL appLKM $ethod "< case $ethod o# 8G-T8 "< yesodIunner @#$ap chooseIep !et*o$eIA $aster sub @Just *o$eIA toMaster B "< appLKM *o$eI !et*o$eI = return @A $ain == I, @A $ain = return @A

Some o# that -ill likely not make sense yet2 In particular, the implementation o# yesod1ispatch is a bit hairy to accomodate di##erent dispatch approaches and #it the model necessary #or our high1per#ormance routing structures2 Ho-e!er, the IenderIoute implementation -ith its associated data type should already gi!e you a good #eel #or -hat%s going on under the sur#ace2

**

Handler function
So -e ha!e a route named *o$eI, and it responds to G-T re3uests2 Ho- do you de#ine your response= You -rite a handler #unction2 Yesod #ollo-s a standard naming scheme #or these #unctions) it%s the lo-er case method name 4e2g2, G-T becomes !et6 #ollo-ed by the route name2 In this case, the #unction name -ould be !et*o$eI2 &ost o# the code you -rite in Yesod li!es in handler #unctions2 This is -here you process user input, per#orm database 3ueries and create responses2 In our simple e(ample, -e create a response using the de#ault5ayout #unction2 This #unction -raps up the content it%s gi!en in your site%s template2 By de#ault, it produces an HT&5 #ile -ith a doctype and ht$l, head and body tags2 s -e%ll see in the Yesod typeclass chapter, this #unction can be o!erridden to do much more2 In our e(ample, -e pass :wha$let|*ello CorldJ|; to de#ault5ayout2 wha$let is another 3uasi13uoter2 In this case, it con!erts Hamlet synta( into a Widget2 Hamlet is the de#ault HT&5 templating engine in Yesod2 Together -ith its siblings $assius, 5ucius and .ulius, you can create HT&5, $SS and .a!ascript in a #ully type1sa#e and compile1time1checked manner2 We%ll see much more about this in the Shakespeare chapter2 Widgets are another cornerstone o# Yesod2 They allo- you to create modular components o# a site consisting o# HT&5, $SS and .a!ascript and reuse them throughout your site2 We%ll get into more detail on them in the -idgets chapter2

The !oundation
The -ord IHelloWorldI sho-s up a number o# times in our e(ample2 "!ery Yesod application has a #oundation datatype2 This datatype must be an instance o# the Yesod typeclass, -hich pro!ides a central place #or declaring a number o# di##erent settings controlling the e(ecution o# our application2 In our case, this datatype is pretty boring) it doesn%t contain any in#ormation2 0onetheless, the #oundation is central to ho- our e(ample runs) it ties together the routes -ith the instance declaration and lets it all be run2 We%ll see throughout this book that the #oundation pops up in a -hole bunch o# places2 But #oundations don%t ha!e to be boring) they can be used to store lots o# use#ul in#ormation, usually stu## that needs to be initiali'ed at program launch and used throughout2 Some !ery common e(amples are)

database connection pool2 Settings loaded #rom a con#ig #ile2 n HTTP connection manager2

By the -ay, the -ord Yesod 4JKLM6 means foundation in Hebre-2

*:

%unning
/nce again -e mention *elloCorld in our main #unction2 /ur #oundation contains all the in#ormation -e need to route and respond to re3uests in our application7 no- -e 9ust need to con!ert it into something that can run2 use#ul #unction #or this in Yesod is warp1ebu!, -hich runs the Warp -ebser!er -ith debug output enabled on the speci#ied port 4here, it%s >HHH62 /ne o# the #eatures o# Yesod is that you aren%t tied do-n to a single deployment strategy2 Yesod is built on top o# the Web pplication Inter#ace 4W I6, allo-ing it to run on Fast$;I, S$;I, Warp, or e!en as a desktop application using the Webkit library2 We%ll discuss some o# these options in the deployment chapter2 nd at the end o# this chapter, -e -ill e(plain the de!elopment ser!er2 Warp is the premiere deployment option #or Yesod2 It is a light-eight, highly e##icient -eb ser!er de!eloped speci#ically #or hosting Yesod2 It is also used outside o# Yesod #or other Haskell de!elopment 4both #rame-ork and non1#rame-ork applications6, as -ell as a standard #ile ser!er in a number o# production en!ironments2

%esources and type&safe '%Ls

In our hello -orld, -e de#ined 9ust a single resource 4*o$eI62 -eb application is usually much more e(citing -ith more than one page on it2 5et%s take a look)
3"4 5) G6)G- Type?a$ilies> FuasiFuotes> MultiPara$TypeClasses> Te$plate*askell> ,%erloaded9trin!s 4"7 i$port Hesod data 5inks = 5inks $kHesod 85inks8 :parseIoutes| . *o$eI G-T .pa!e( Pa!e(I G-T .pa!eN Pa!eNI G-T |; instance Hesod 5inks !et*o$eI = de#ault5ayout :wha$let|Ga hre#=O3Pa!e(I7<Go to pa!e (J|; !etPa!e(I = de#ault5ayout :wha$let|Ga hre#=O3Pa!eNI7<Go to pa!e NJ|; !etPa!eNI = de#ault5ayout :wha$let|Ga hre#=O3*o$eI7<Go ho$eJ|; $ain = warp1ebu! &KKK 5inks

/!erall, this is !ery similar to Hello World2 /ur #oundation is no- 5inks instead o# *elloCorld, and in addition to the *o$eI resource, -e%!e added Pa!e(I and Pa!eNI2 s such, -e%!e also added t-o more handler #unctions) !etPa!e(I and !etPa!eNI2 The only truly ne- #eature is inside the wha$let 3uasi13uotation2 We%ll del!e into synta( in the Shakespeare chapter, but -e can see that)

*>

Ga hre#=O3Pa!e(I7<Go to pa!e (J

creates a link to the Pa!e(I resource2 The important thing to note here is that Pa!e(I is a data constructor2 By making each resource a data constructor, -e ha!e a #eature called type1sa#e 8R5s2 Instead o# splicing together strings to create 8R5s, -e simply create a plain old Haskell !alue2 By using at1sign interpolation 4O3///76, Yesod automatically renders those !alues to te(tual 8R5s be#ore sending things o## to the user2 We can see ho- this is implemented by looking again at the 1ddump1splices output)
instance IenderIoute 5inks where data Ioute 5inks = *o$eI | Pa!e(I | Pa!eNI deri%in! @9how> -E> IeadA renderIoute *o$eI = @:;> :;A renderIoute Pa!e(I = @:8pa!e(8;> :;A renderIoute Pa!eNI = @:8pa!eN8;> :;A $ain == I, @A $ain = return @A

In the Ioute associated type #or 5inks, -e ha!e additional constructors #or Pa!e(I and Pa!eNI2 We also no- ha!e a better glimpse o# the return !alues #or returnIoute2 The #irst part o# the tuple gi!es the path pieces #or the gi!en route2 The second part gi!es the 3uery string parameters7 #or almost all use cases, this -ill be an empty list2 It%s hard to o!er1estimate the !alue o# type1sa#e 8R5s2 They gi!e you a huge amount o# #le(ibility and robustness -hen de!eloping your application2 You can mo!e 8R5s around at -ill -ithout e!er breaking links2 In the routing chapter, -e%ll see that routes can take parameters, such as a blog entry 8R5 taking the blog post ID2 5et%s say you -ant to s-itch #rom routing on the numerical post ID to a year<month<slug setup2 In a traditional -eb #rame-ork, you -ould need to go through e!ery single re#erence to your blog post route and update appropriately2 I# you miss one, you%ll ha!e @H@s at runtime2 In Yesod, all you do is update your route and compile) ;H$ -ill pinpoint e!ery single line o# code that needs to be corrected2

The scaffolded site

Installing Yesod -ill gi!e you both the Yesod library, as -ell as a yesod e(ecutable2 This e(ecutable accepts a #e- commands, but the #irst one you%ll -ant to be ac3uainted -ith is yesod init2 It -ill ask you some 3uestions, and then generate a #older containing the de#ault sca##olded site2 Inside that #older, you can run cabal install ""only"dependencies to build any e(tra dependencies 4such as your database backends6, and then yesod de%el to run your site2 The sca##olded site gi!es you a lot o# best practices out o# the bo(, setting up #iles and dependencies in a time1tested approach used by most production Yesod sites2 Ho-e!er, all this con!enience can get in the -ay o# actually learning Yesod2 There#ore, most o# this book -ill a!oid the sca##olding tool, and instead deal directly -ith Yesod as a library2 We -ill co!er the structure o# the sca##olded site in more detail later2 *@

(e elopment ser er
/ne o# the ad!antages interpreted languages ha!e o!er compiled languages is #ast prototyping) you sa!e changes to a #ile and hit re#resh2 I# -e -ant to make any changes to our Yesod apps abo!e, -e%ll need to call runhaskell #rom scratch, -hich can be a bit tedious2 Fortunately, there%s a solution to this) yesod de%el automatically rebuilds and reloads your code #or you2 This can be a great -ay to de!elop your Yesod pro9ects, and -hen you%re ready to mo!e to production, you still get to compile do-n to incredibly e##icient code2 The Yesod sca##olding automatically sets things up #or you2 This gi!es you the best o# both -orlds) rapid prototyping and #ast production code2 It%s a little bit more in!ol!ed to set up your code to be used by yesod de!el, so our e(amples -ill 9ust use warp1ebu!2 But -hen you%re ready to make your real1-orld applications, yesod de!el -ill be -aiting #or you2

Summary
"!ery Yesod application is built around a #oundation datatype2 We associate some resources -ith that datatype and de#ine some handler #unctions, and Yesod handles all o# the routing2 These resources are also data constructors, -hich lets us ha!e type1sa#e 8R5s2 By being built on top o# W I, Yesod applications can run -ith a number o# di##erent backends2 warp1ebu! is an easy -ay to get started, as it%s included -ith Yesod2 For rapid de!elopment, yesod de%el is a good choice2 nd -hen you%re ready to mo!e to production, you ha!e Warp as a high1per#ormance option2 When de!eloping in Yesod, -e get a number o# choices #or coding style) 3uasi13uotation or e(ternal #iles, warp1ebu! or yesod de%el, and so on2 The e(amples in this book -ill tend to-ards using the choices that are easiest to copy1and1paste, but the more po-er#ul options -ill be a!ailable -hen you start building real Yesod applications2

#asics
The #irst step -ith any ne- technology is getting it running2 The goal o# this chapter is to get you started -ith a simple Yesod application, and co!er some o# the basic concepts and terminology2

Hello $orld
5et%s get this book started properly) a simple -eb page that says Hello World)
3"4 5) G6)G- Type?a$ilies> FuasiFuotes> MultiPara$TypeClasses> Te$plate*askell> ,%erloaded9trin!s 4"7 i$port Hesod data *elloCorld = *elloCorld

*A

$kHesod 8*elloCorld8 :parseIoutes| . *o$eI G-T |; instance Hesod *elloCorld !et*o$eI == *andler Iep*t$l !et*o$eI = de#ault5ayout :wha$let|*ello CorldJ|; $ain == I, @A $ain = warp1ebu! &KKK *elloCorld

I# you sa!e that code in helloworld/hs and run it -ith runhaskell helloworld/hs, you%ll get a -eb ser!er running on port >HHH2 I# you point your bro-ser to http)<<localhost)>HHH, you%ll get the #ollo-ing HT&5)
GJ1,CTHP- ht$l< Ght$l<Ghead<Gtitle<G.title<G.head<Gbody<*ello CorldJG.body<G.ht$l<

We%ll re#er back to this e(ample through the rest o# the chapter2

%outing
5ike most modern -eb #rame-orks, Yesod #ollo-s a #ront controller pattern2 This means that e!ery re3uest to a Yesod application enters at the same point and is routed #rom there2 s a contrast, in systems like PHP and SP you usually create a number o# di##erent #iles, and the -eb ser!er automatically directs re3uests to the rele!ant #ile2 In addition, Yesod uses a declarati!e style #or speci#ying routes2 In our e(ample abo!e, this looked like)
$kHesod 8*elloCorld8 :parseIoutes| . *o$eI G-T |; $kHesod is a Template Haskell #unction, and parseIoutes

is a GuasiGuoter2

In "nglish, all this means is) In the HelloWorld application, create one route2 I%d like to call it *o$eI, it should listen #or re3uests to . 4the root o# the application6, and should ans-er G-T re3uests2 We call *o$eI a resource, -hich is -here the IRI su##i( comes #rom2 The R su##i( on resource names is simply con!ention, but it%s a #airly uni!ersally #ollo-ed con!ention2 It makes it 9ust a bit easier to read and understand code2 The $kHesod TH #unction generates 3uite a bit o# code here) a route data type, a dispatch #unction, and a render #unction2 We%ll look at this in more detail in the routing chapter2 But by using the "ddu$p"splices ;H$ option, -e can get an immediate look at the generated code2 much cleaned up !ersion o# it is)
instance IenderIoute *elloCorld where data Ioute *elloCorld = *o$eI deri%in! @9how> -E> IeadA

*D

renderIoute *o$eI = @:;> :;A instance Hesod1ispatch *elloCorld *elloCorld where yesod1ispatch $aster sub toMaster appLKL appLKM $ethod pieces = case dispatch pieces o# Just # "< # $aster sub toMaster appLKL appLKM $ethod othin! "< appLKL where dispatch = Hesod/Ioutes/1ispatch/to1ispatch : Hesod/Ioutes/1ispatch/Ioute :; ?alse on*o$e ; on*o$e :; = Just + 2$aster sub toMaster BappLKL appLKM $ethod "< case $ethod o# 8G-T8 "< yesodIunner @#$ap chooseIep !et*o$eIA $aster sub @Just *o$eIA toMaster B "< appLKM *o$eI !et*o$eI = return @A $ain == I, @A $ain = return @A

Some o# that -ill likely not make sense yet2 In particular, the implementation o# yesod1ispatch is a bit hairy to accomodate di##erent dispatch approaches and #it the model necessary #or our high1per#ormance routing structures2 Ho-e!er, the IenderIoute implementation -ith its associated data type should already gi!e you a good #eel #or -hat%s going on under the sur#ace2

Handler function
So -e ha!e a route named *o$eI, and it responds to G-T re3uests2 Ho- do you de#ine your response= You -rite a handler #unction2 Yesod #ollo-s a standard naming scheme #or these #unctions) it%s the lo-er case method name 4e2g2, G-T becomes !et6 #ollo-ed by the route name2 In this case, the #unction name -ould be !et*o$eI2 &ost o# the code you -rite in Yesod li!es in handler #unctions2 This is -here you process user input, per#orm database 3ueries and create responses2 In our simple e(ample, -e create a response using the de#ault5ayout #unction2 This #unction -raps up the content it%s gi!en in your site%s template2 By de#ault, it produces an HT&5 #ile -ith a doctype and ht$l, head and body tags2 s -e%ll see in the Yesod typeclass chapter, this #unction can be o!erridden to do much more2 In our e(ample, -e pass :wha$let|*ello CorldJ|; to de#ault5ayout2 wha$let is another 3uasi13uoter2 In this case, it con!erts Hamlet synta( into a Widget2 Hamlet is the de#ault

*F

HT&5 templating engine in Yesod2 Together -ith its siblings $assius, 5ucius and .ulius, you can create HT&5, $SS and .a!ascript in a #ully type1sa#e and compile1time1checked manner2 We%ll see much more about this in the Shakespeare chapter2 Widgets are another cornerstone o# Yesod2 They allo- you to create modular components o# a site consisting o# HT&5, $SS and .a!ascript and reuse them throughout your site2 We%ll get into more detail on them in the -idgets chapter2

The !oundation
The -ord IHelloWorldI sho-s up a number o# times in our e(ample2 "!ery Yesod application has a #oundation datatype2 This datatype must be an instance o# the Yesod typeclass, -hich pro!ides a central place #or declaring a number o# di##erent settings controlling the e(ecution o# our application2 In our case, this datatype is pretty boring) it doesn%t contain any in#ormation2 0onetheless, the #oundation is central to ho- our e(ample runs) it ties together the routes -ith the instance declaration and lets it all be run2 We%ll see throughout this book that the #oundation pops up in a -hole bunch o# places2 But #oundations don%t ha!e to be boring) they can be used to store lots o# use#ul in#ormation, usually stu## that needs to be initiali'ed at program launch and used throughout2 Some !ery common e(amples are)

database connection pool2 Settings loaded #rom a con#ig #ile2 n HTTP connection manager2

By the -ay, the -ord Yesod 4JKLM6 means foundation in Hebre-2

%unning
/nce again -e mention *elloCorld in our main #unction2 /ur #oundation contains all the in#ormation -e need to route and respond to re3uests in our application7 no- -e 9ust need to con!ert it into something that can run2 use#ul #unction #or this in Yesod is warp1ebu!, -hich runs the Warp -ebser!er -ith debug output enabled on the speci#ied port 4here, it%s >HHH62 /ne o# the #eatures o# Yesod is that you aren%t tied do-n to a single deployment strategy2 Yesod is built on top o# the Web pplication Inter#ace 4W I6, allo-ing it to run on Fast$;I, S$;I, Warp, or e!en as a desktop application using the Webkit library2 We%ll discuss some o# these options in the deployment chapter2 nd at the end o# this chapter, -e -ill e(plain the de!elopment ser!er2 Warp is the premiere deployment option #or Yesod2 It is a light-eight, highly e##icient -eb ser!er de!eloped speci#ically #or hosting Yesod2 It is also used outside o# Yesod #or other Haskell de!elopment 4both #rame-ork and non1#rame-ork applications6, as -ell as a standard #ile ser!er in a number o# production en!ironments2 *+

%esources and type&safe '%Ls

In our hello -orld, -e de#ined 9ust a single resource 4*o$eI62 -eb application is usually much more e(citing -ith more than one page on it2 5et%s take a look)
3"4 5) G6)G- Type?a$ilies> FuasiFuotes> MultiPara$TypeClasses> Te$plate*askell> ,%erloaded9trin!s 4"7 i$port Hesod data 5inks = 5inks $kHesod 85inks8 :parseIoutes| . *o$eI G-T .pa!e( Pa!e(I G-T .pa!eN Pa!eNI G-T |; instance Hesod 5inks !et*o$eI = de#ault5ayout :wha$let|Ga hre#=O3Pa!e(I7<Go to pa!e (J|; !etPa!e(I = de#ault5ayout :wha$let|Ga hre#=O3Pa!eNI7<Go to pa!e NJ|; !etPa!eNI = de#ault5ayout :wha$let|Ga hre#=O3*o$eI7<Go ho$eJ|; $ain = warp1ebu! &KKK 5inks

/!erall, this is !ery similar to Hello World2 /ur #oundation is no- 5inks instead o# *elloCorld, and in addition to the *o$eI resource, -e%!e added Pa!e(I and Pa!eNI2 s such, -e%!e also added t-o more handler #unctions) !etPa!e(I and !etPa!eNI2 The only truly ne- #eature is inside the wha$let 3uasi13uotation2 We%ll del!e into synta( in the Shakespeare chapter, but -e can see that)
Ga hre#=O3Pa!e(I7<Go to pa!e (J

creates a link to the Pa!e(I resource2 The important thing to note here is that Pa!e(I is a data constructor2 By making each resource a data constructor, -e ha!e a #eature called type1sa#e 8R5s2 Instead o# splicing together strings to create 8R5s, -e simply create a plain old Haskell !alue2 By using at1sign interpolation 4O3///76, Yesod automatically renders those !alues to te(tual 8R5s be#ore sending things o## to the user2 We can see ho- this is implemented by looking again at the 1ddump1splices output)
instance IenderIoute 5inks where data Ioute 5inks = *o$eI | Pa!e(I | Pa!eNI deri%in! @9how> -E> IeadA renderIoute *o$eI = @:;> :;A renderIoute Pa!e(I = @:8pa!e(8;> :;A renderIoute Pa!eNI = @:8pa!eN8;> :;A $ain == I, @A $ain = return @A

In the Ioute associated type #or 5inks, -e ha!e additional constructors #or Pa!e(I and Pa!eNI2 We also no- ha!e a better glimpse o# the return !alues #or returnIoute2 The #irst

*B

part o# the tuple gi!es the path pieces #or the gi!en route2 The second part gi!es the 3uery string parameters7 #or almost all use cases, this -ill be an empty list2 It%s hard to o!er1estimate the !alue o# type1sa#e 8R5s2 They gi!e you a huge amount o# #le(ibility and robustness -hen de!eloping your application2 You can mo!e 8R5s around at -ill -ithout e!er breaking links2 In the routing chapter, -e%ll see that routes can take parameters, such as a blog entry 8R5 taking the blog post ID2 5et%s say you -ant to s-itch #rom routing on the numerical post ID to a year<month<slug setup2 In a traditional -eb #rame-ork, you -ould need to go through e!ery single re#erence to your blog post route and update appropriately2 I# you miss one, you%ll ha!e @H@s at runtime2 In Yesod, all you do is update your route and compile) ;H$ -ill pinpoint e!ery single line o# code that needs to be corrected2

The scaffolded site

Installing Yesod -ill gi!e you both the Yesod library, as -ell as a yesod e(ecutable2 This e(ecutable accepts a #e- commands, but the #irst one you%ll -ant to be ac3uainted -ith is yesod init2 It -ill ask you some 3uestions, and then generate a #older containing the de#ault sca##olded site2 Inside that #older, you can run cabal install ""only"dependencies to build any e(tra dependencies 4such as your database backends6, and then yesod de%el to run your site2 The sca##olded site gi!es you a lot o# best practices out o# the bo(, setting up #iles and dependencies in a time1tested approach used by most production Yesod sites2 Ho-e!er, all this con!enience can get in the -ay o# actually learning Yesod2 There#ore, most o# this book -ill a!oid the sca##olding tool, and instead deal directly -ith Yesod as a library2 We -ill co!er the structure o# the sca##olded site in more detail later2

(e elopment ser er
/ne o# the ad!antages interpreted languages ha!e o!er compiled languages is #ast prototyping) you sa!e changes to a #ile and hit re#resh2 I# -e -ant to make any changes to our Yesod apps abo!e, -e%ll need to call runhaskell #rom scratch, -hich can be a bit tedious2 Fortunately, there%s a solution to this) yesod de%el automatically rebuilds and reloads your code #or you2 This can be a great -ay to de!elop your Yesod pro9ects, and -hen you%re ready to mo!e to production, you still get to compile do-n to incredibly e##icient code2 The Yesod sca##olding automatically sets things up #or you2 This gi!es you the best o# both -orlds) rapid prototyping and #ast production code2 It%s a little bit more in!ol!ed to set up your code to be used by yesod de!el, so our e(amples -ill 9ust use warp1ebu!2 But -hen you%re ready to make your real1-orld applications, yesod de!el -ill be -aiting #or you2

:H

Summary
"!ery Yesod application is built around a #oundation datatype2 We associate some resources -ith that datatype and de#ine some handler #unctions, and Yesod handles all o# the routing2 These resources are also data constructors, -hich lets us ha!e type1sa#e 8R5s2 By being built on top o# W I, Yesod applications can run -ith a number o# di##erent backends2 warp1ebu! is an easy -ay to get started, as it%s included -ith Yesod2 For rapid de!elopment, yesod de%el is a good choice2 nd -hen you%re ready to mo!e to production, you ha!e Warp as a high1per#ormance option2 When de!eloping in Yesod, -e get a number o# choices #or coding style) 3uasi13uotation or e(ternal #iles, warp1ebu! or yesod de%el, and so on2 The e(amples in this book -ill tend to-ards using the choices that are easiest to copy1and1paste, but the more po-er#ul options -ill be a!ailable -hen you start building real Yesod applications2

Shakespearean Templates
Yesod uses the Shakespearean #amily o# template languages as its standard approach to HT&5, $SS and .a!ascript creation2 This language #amily shares some common synta(, as -ell as o!erarching principles)

s little inter#erence to the underlying language as possible, -hile pro!iding con!eniences -here unobtrusi!e2 $ompile1time guarantees on -ell1#ormed content2 Static type sa#ety, greatly helping the pre!ention o# CSS 4cross1site scripting6 attacks2 utomated checking o# !alid 8R5s, -hene!er possible, through type1sa#e 8R5s2

There is nothing inherently tying Yesod to these languages, or the other -ay around) each can be used independently o# the other2 This chapter -ill address these template languages on their o-n, -hile the remainder o# the book -ill use them to enhance Yesod application de!elopment2

Synopsis
There are #our main languages at play) Hamlet is an HT&5 templating language, .ulius is #or .a!ascript, and $assius and 5ucius are both #or $SS2 Hamlet and $assius are both -hitespace1sensiti!e #ormats, using indentation to denote nesting2 By contrast, 5ucius is a superset o# $SS, keeping $SS%s braces #or denoting nesting2 .ulius is a simple passthrough language #or producing .a!ascript7 the only added #eature is !ariable interpolation2

Hamlet )HTML*
+doctype M Ght$l< Ghead<

:*

Gtitle<43pa!eTitle7 " My 9ite Glink rel=stylesheet hre#=O39tylesheet7< Gbody< Gh( /pa!e"title<43pa!eTitle7 Gp<*ere is a list o# your #riends= +i# null #riends Gp<9orry> I lied> you donPt ha%e any #riends/ +else Gul< +#orall ?riend na$e a!e G" #riends Gli<43na$e7 @43a!e7 years oldA G#ooter<Q3copyri!ht7

Cassius )CSS*
4$yid color= 43red7 #ont"si'e= 43body?ont9i'e7 #oo bar ba' back!round"i$a!e= url@O3MyBack!roundI7A

Lucius )CSS*
section/blo! 3 paddin!= (e$R border= (px solid 4KKKR h( 3 color= 43headin!Color7R 7 7

+ulius )+a ascript*

+@#unction@A3 +@8section/43sectionClass78A/hide@AR +@84$ybutton8A/click@#unction@A3docu$ent/location = 8O39o$eIouteI78R7AR Q3addBlin!7 7AR

Types
Be#ore -e 9ump into synta(, let%s take a look at the !arious types in!ol!ed2 We mentioned in the introduction that types help protect us #rom CSS attacks2 For e(ample, let%s say that -e ha!e an HT&5 template that should display someone%s name7 it might look like this)
Gp<*ello> $y na$e is 43na$e7 43///7 is ho- -e do !ariable interpolation

in Shakespeare2

What should happen to name, and -hat should its datatype be= nai!e approach -ould be to use a Text !alue, and insert it !erbatim2 But that -ould gi!e us 3uite a problem -hen

::

na$e=8Gscript src=Phttp=..ne#arious/co$.e%il/SsP<G.script<82 to be able to entity1encode the name, so that G becomes TltR2

What -e -ant is

n e3ually nai!e approach is to simply entity1encode e ery piece o# te(t that gets embedded2 What happens -hen you ha!e some pree(isting HT&5 generated #rom another process= For e(ample, on the Yesod -ebsite, all Haskell code snippets are run through a colori'ing #unction that -raps up -ords in appropriate span tags2 I# -e entity escaped e!erything, code snippets -ould be completely unreadable? Instead, -e ha!e an *t$l datatype2 In order to generate an *t$l !alue, -e ha!e t-o options #or PIs) the To*t$l typeclass pro!ides a -ay to con!ert 9trin! and Text !alues into *t$l, !ia its to*t$l #unction, automatically escaping entities along the -ay2 This -ould be the approach -e%d -ant #or the name abo!e2 For the code snippet e(ample, -e -ould use the pre"scaped #amily o# #unctions2 When you use !ariable interpolation in Hamlet 4the HT&5 Shakespeare language6, it automatically applies a to*t$l call to the !alue inside2 So i# you interpolate a 9trin!, it -ill be entity1escaped2 But i# you pro!ide an *t$l !alue, it -ill appear unmodi#ied2 In the code snippet e(ample, -e might interpolate -ith something like 43pre-scapedText $y*askell*t$l72 The *t$l datatype, as -ell as the #unctions mentioned, are all pro!ided by the bla'e1html package2 This allo-s Hamlet to interact -ith all other bla'e1html packages, and lets Hamlet pro!ide a general solution #or producing bla'e1html !alues2 lso, -e get to take ad!antage o# bla'e1html%s ama'ing per#ormance2 Similarly, -e ha!e Css<ToCss, as -ell as Ja%ascript<ToJa%ascript2 These pro!ide some compile1time sanity checks that -e ha!en%t accidently stuck some HT&5 in our $SS2 /ne other ad!antage on the $SS side is some helper datatypes #or colors and units2 For e(ample)
/red 3 color= 43colorIed7 7

Please see the Haddock documentation #or more details2

Type&safe '%Ls
Possibly the most uni3ue #eature in Yesod is type1sa#e 8R5s, and the ability to use them con!eniently is pro!ided directly by Shakespeare2 8sage is nearly identical to !ariable interpolation, -e 9ust use the at1sign 4N6 instead o# the hash 4O62 We%ll co!er the synta( later7 #irst, let%s clari#y the intuition2 Suppose -e ha!e an application -ith t-o routes) http)<<e(ample2com<pro#ile<home is the homepage, and http)<<e(ample2com<display<time displays the current time2 nd let%s say -e -ant to link #rom the homepage to the time2 I can think o# three di##erent -ays o# constructing the 8R5) *2 :2 >2 s a relati!e link) 22<display<time s an absolute link, -ithout a domain) <display<time s an absolute link, -ith a domain) http)<<e(ample2com<display<time :>

There are problems -ith each approach) the #irst -ill break i# either 8R5 changes2 lso, it%s not suitable #or all use cases7 RSS and tom #eeds, #or instance, re3uire absolute 8R5s2 The second is more resilient to change than the #irst, but still -on%t be acceptable #or RSS and tom2 nd -hile the third -orks #ine #or all use cases, you%ll need to update e!ery single 8R5 in your application -hene!er your domain name changes2 You think that doesn%t happen o#ten= .ust -ait till you mo!e #rom your de!elopment to staging and #inally production ser!er2 But more importantly, there is one huge problem -ith all approaches) i# you change your routes at all, the compiler -on%t -arn you about the broken links2 0ot to mention that typos can -reak ha!oc as -ell2 The goal o# type1sa#e 8R5s is to let the compiler check things #or us as much as possible2 In order to #acilitate this, our #irst step must be to mo!e a-ay #rom plain old te(t, -hich the compiler doesn%t understand, to some -ell de#ined datatypes2 For our simple application, let%s model our routes -ith a sum type)
data MyIoute = *o$e | Ti$e

Instead o# placing a link like <display<time in our template, -e can use the Ti$e constructor2 But at the end o# the day, HT&5 is made up o# te(t, not data types, so -e need some -ay to con!ert these !alues to te(t2 We call this a 8R5 rendering #unction, and a simple one is)
renderMyIoute == MyIoute "< Text renderMyIoute *o$e = 8http=..exa$ple/co$.pro#ile.ho$e8 renderMyIoute Ti$e = 8http=..exa$ple/co$.display.ti$e8

8R5 rendering #unctions are actually a bit more complicated than this2 They need to address 3uery string parameters, handle records -ithin the constructor, and more intelligently handle the domain name2 But in practice, you don%t need to -orry about this, since Yesod -ill automatically create your render #unctions2 The one thing to point out is that the type signature is actually a little more complicated to handle 3uery strings)
type Fuery = :@Text> TextA; type Iender url = url "< Fuery "< Text renderMyIoute == Iender MyIoute renderMyIoute *o$e B = /// renderMyIoute Ti$e B = ///

/P, -e ha!e our render #unction, and -e ha!e type1sa#e 8R5s embedded in the templates2 Ho- does this #it together e(actly= Instead o# generating an *t$l 4or Css or Ja%ascript6 !alue directly, Shakespearean templates actually produce a #unction, -hich takes this render #unction and produces HT&52 To see this better, let%s ha!e a 3uick 4#ake6 peek at hoHamlet -ould -ork under the sur#ace2 Supposing -e had a template)
Ga hre#=O3Ti$e7<The ti$e

this -ould translate roughly into the Haskell code)

2render "< $concat :8Ga hre#=P8> render Ti$e> 8P<The ti$eG.a<8;

Synta,

:@

ll Shakespearean languages share the same interpolation synta(, and are able to utili'e type1 sa#e 8R5s2 They di##er in the synta( speci#ic #or their target language 4HT&5, $SS, or .a!ascript62

Hamlet Synta,
Hamlet is the most sophisticated o# the languages2 0ot only does it pro!ide synta( #or generating HT&5, it also allo-s #or basic control structures) conditionals, looping, and maybes2

Tags
/b!iously tags -ill play an important part o# any HT&5 template language2 In Hamlet, -e try to stick !ery close to e(isting HT&5 synta( to make the language more com#ortable2 Ho-e!er, instead o# using closing tags to denote nesting, -e use indentation2 So something like this in HT&5)
Gbody< Gp<9o$e para!raph/G.p< Gul< Gli<Ite$ (G.li< Gli<Ite$ NG.li< G.ul< G.body<

-ould be

Gbody< Gp<9o$e para!raph/ Gul< Gli<Ite$ ( Gli<Ite$ N

In general, -e #ind this to be easier to #ollo- than HT&5 once you get accustomed to it2 The only tricky part comes -ith dealing -ith -hitespace be#ore and a#ter tags2 For e(ample, let%s say you -ant to create the HT&5
Gp<Para!raph Gi<italicG.i< end/G.p<

We -ant to make sure that there is a -hitespace preser!ed a#ter the -ord IParagraphI and be#ore the -ord IendI2 To do so, -e use t-o simple escape characters)
Gp< Para!raph 4 Gi<italic 2 end/

The -hitespace escape rules are actually !ery simple) *2 I# the #irst non1space character in a line is a backslash, the backslash is ignored2 :2 I# the last character in a line is a hash, it is ignored2 /ne other thing2 Hamlet does not escape entities -ithin its content2 This is done on purpose to allo- e(isting HT&5 to be more easily copied in2 So the e(ample abo!e could also be -ritten as)
Gp<Para!raph Gi<italicG.i< end/

:A

0otice that the #irst tag -ill be automatically closed by Hamlet, -hile the inner IiI tag -ill not2 You are #ree to use -hiche!er approach you -ant, there is no penalty #or either choice2 Be a-are, ho-e!er, that the only time you use closing tags in Hamlet is #or such inline tags7 normal tags are not closed2

Interpolation
What -e ha!e so #ar is a nice, simpli#ied HT&5, but it doesn%t let us interact -ith our Haskell code at all2 Ho- do -e pass in !ariables= Simple) -ith interpolation)
Ghead< Gtitle<43title7

The hash #ollo-ed by a pair o# braces denotes aria-le interpolation2 In the case abo!e, the title !ariable #rom the scope in -hich the template -as called -ill be used2 5et me state that again) Hamlet automatically has access to the !ariables in scope -hen it%s called2 There is no need to speci#ically pass !ariables in2 You can apply #unctions -ithin an interpolation2 You can use string and numeric literals in an interpolation2 You can use 3uali#ied modules2 Both parentheses and the dollar sign can be used to group statements together2 nd at the end, the to*t$l #unction is applied to the result, meaning any instance o# To*t$l can be interpolated2 Take, #or instance, the #ollo-ing code2
"" Just i!nore the EuasiEuote stu## #or now> and that sha$let thin!/ "" It will be explained later/ 3"4 5) G6)G- FuasiFuotes 4"7 i$port Text/*a$let @sha$letA i$port Text/Bla'e/Ienderer/9trin! @render*t$lA i$port 1ata/Char @to5owerA i$port 1ata/5ist @sortA data Person = Person 3 na$e == 9trin! > a!e == Int 7 $ain == I, @A $ain = put9tr5n + render*t$l :sha$let| Gp<*ello> $y na$e is 43na$e person7 and I a$ 43show + a!e person7/ Gp< 5etPs do so$e #unny stu## with $y na$e= 4 Gb<43sort + $ap to5ower @na$e personA7 Gp<,h> and in M years IPll be 43show @@UA M @a!e personAA7 years old/ |; where person = Person 8Michael8 NV

What about our much1touted type1sa#e 8R5s= They are almost identical to !ariable interpolation in e!ery -ay, e(cept they start -ith an at1sign 4N6 instead2 In addition, there is embedding !ia a caret 4Q6 -hich allo-s you to embed another template o# the same type2 The ne(t code sample demonstrates both o# these2
3"4 5) G6)G- FuasiFuotes 4"7 3"4 5) G6)G- ,%erloaded9trin!s 4"7

:D

i$port Text/*a$let @*t$l6rl> ha$letA i$port Text/Bla'e/Ienderer/9trin! @render*t$lA i$port 1ata/Text @TextA data MyIoute = *o$e render == MyIoute "< :@Text> TextA; "< Text render *o$e B = 8.ho$e8 #ooter == *t$l6rl MyIoute #ooter = :ha$let| G#ooter< Ieturn to 4 Ga hre#=O3*o$e7<*o$epa!e / |; $ain == I, @A $ain = put9tr5n + render*t$l + :ha$let| Gbody< Gp<This is $y pa!e/ Q3#ooter7 |; render

Attri-utes
In that last e(ample, -e put an hre# attribute on the IaI tag2 5et%s elaborate on the synta()

You can ha!e interpolations -ithin the attribute !alue2 The e3uals sign and !alue #or an attribute are optional, 9ust like in HT&52 So Ginput type=checkbox checked< is per#ectly !alid2 There are t-o con!enience attributes) #or id, you can use the hash, and #or classes, the period2 In other -ords, Gp 4para!raphid /class( /classN<2 While 3uotes around the attribute !alue are optional, they are re3uired i# you -ant to embed spaces2 You can add an attribute optionally by using colons2 To make a checkbo( only checked i# the !ariable is$hecked is True, you -ould -rite Ginput type=checkbox =isChecked=checked<2 To ha!e a paragraph be optionally red, you could use Gp =isIed=style=8color=red8<2

Conditionals
"!entually, you%ll -ant to put in some logic in your page2 The goal o# Hamlet is to make the logic as minimalistic as possible, pushing the hea!y li#ting into Haskell2 s such, our logical statements are !ery basic222 so basic, that it%s i#, elsei#, and else2
+i# is)d$in Gp<Celco$e to the ad$in section/ +elsei# is5o!!edIn Gp<Hou are not the ad$inistrator/ +else Gp<I donPt know who you are/ Please lo! in so I can decide i# you !et access/

:F

ll the same rules o# normal interpolation apply to the content o# the conditionals2

May-e
Similarly, -e ha!e a special construct #or dealing -ith &aybe !alues2 This could technically be dealt -ith using i#, isJust and #ro$Just, but this is more con!enient and a!oids partial #unctions2
+$aybe na$e G" $aybe a$e Gp<Hour na$e is 43na$e7 +nothin! Gp<I donPt know your na$e/

In addition to simple identi#iers, you can use a #e- other, more complicated !alues on the le#t hand side, such as constructors and tuples2
+$aybe Person #irst a$e last a$e G" $aybePerson Gp<Hour na$e is 43#irst a$e7 43last a$e7

The right1hand1side #ollo-s the same rules as interpolations, allo- !ariables, #unction application, and so on2

!orall
nd -hat about looping o!er lists= We ha!e you co!ered there too)
+i# null people Gp< o people/ +else Gul< +#orall person G" people Gli<43person7

Case
Pattern matching is one o# the great strengths o# Haskell2 Sum types let you cleanly model many real1-orld types, and case statements let you sa#ely match, letting the compiler -arn you i# you missed a case2 Hamlet gi!es you the same po-er2
+case #oo +o# 5e#t bar Gp<It was le#t= 43bar7 +o# Ii!ht ba' Gp<It was ri!ht= 43ba'7

$ith
Rounding out our statements, -e ha!e with2 It%s basically 9ust a con!enience #or declaring a synonym #or a long e(pression2

:+

+with #oo G" so$e %ery @lon! u!lyA expression that + should only + happen once Gp<But IP$ !oin! to use 43#oo7 $ultiple ti$es/ 43#oo7

(octype
5ast bit o# syntactic sugar) the doctype statement2 We ha!e support #or a number o# di##erent !ersions o# a doctype, though -e recommend +doctype M #or modern -eb applications, -hich generates GJ1,CTHP- ht$l<2
+doctype M Ght$l< Ghead< Gtitle<*a$let is )weso$e Gbody< Gp<)ll done/

There is an older and still supported synta() three e(clamation points 4JJJ62 You may still see this in code out there2 We ha!e no plans to remo!e support #or this, but in general #ind the +doctype approach easier to read2

Cassius Synta,
$assius is the original $SS template language2 It uses simple -hitespace rules to delimit blocks, making braces and semicolons unnecessary2 It supports both !ariable and 8R5 interpolation, but not embedding2 The synta( is !ery straight1#or-ard)
4banner border= (px solid 43bannerColor7 back!round"i$a!e= url@O3BannerI$a!eI7A

Lucius Synta,
While $assius uses a modi#ied, -hitespace1sensiti!e synta( #or $SS, 5ucius is true to the original2 You can take any $SS #ile out there and it -ill be a !alid 5ucius #ile2 There are, ho-e!er, a #e- additions to 5ucius)

5ike $assius, -e allo- both !ariable and 8R5 interpolation2 $SS blocks are allo-ed to nest2 You can declare !ariables in your templates2

Starting the -ith second point) let%s say you -ant to ha!e some special styling #or some tags -ithin your article2 In plain ol% $SS, you%d ha!e to -rite)
article code 3 back!round"color= !reyR 7 article p 3 text"indent= Ne$R 7 article a 3 text"decoration= noneR 7

In this case, there aren%t that many clauses, but ha!ing to type out article each time is still a bit o# a nuisance2 Imagine i# you had a do'en or so o# these2 0ot the -orst thing in the -orld, but a bit o# an annoyance2 5ucius helps you out here)
article 3

:B

code 3 back!round"color= !reyR 7 p 3 text"indent= Ne$R 7 a 3 text"decoration= noneR 7

Ha!ing 5ucius !ariables allo-s you to a!oid repeating yoursel#2 to de#ine a commonly used color)

simple e(ample -ould be

Otextcolor= 4cccR .W Sust because we hate our users W. body 3 color= 43textcolor7 7 a=link> a=%isited 3 color= 43textcolor7 7

/ther than that, 5ucius is identical to $SS2

+ulius Synta,
.ulius is the simplest o# the languages discussed here2 In #act, some might e!en say it%s really 9ust .a!ascript2 .ulius allo-s the three #orms o# interpolation -e%!e mentioned so #ar, and other-ise applies no trans#ormations to your content2 I# you use .ulius -ith the sca##olded Yesod site, you may notice that your .a!ascript is automatically mini#ied2 This is not a #eature o# .ulius7 instead, Yesod uses the h9smin package to mini#y .ulius output2

Calling Shakespeare
The 3uestion o# course arises at some point) ho- do I actually use this stu##= There are three di##erent -ays to call out to Shakespeare #rom your Haskell code) Guasi3uotes Guasi3uotes allo- you to embed arbitrary content -ithin your Haskell, and #or it to be con!erted into Haskell code at compile time2 "(ternal #ile In this case, the template code is in a separate #ile -hich is re#erenced !ia Template Haskell2 Reload mode Both o# the abo!e modes re3uire a #ull recompile to see any changes2 In reload mode, your template is kept in a separate #ile and re#erenced !ia Template Haskell2 But at runtime, the e(ternal #ile is reparsed #rom scratch each time2Reload mode is not a!ailable #or Hamlet, only #or $assius, 5ucius and .ulius2 There are too many sophisticated #eatures in Hamlet that rely directly on the Haskell compiler and could not #easible be reimplemented at runtime2 /ne o# the #irst t-o approaches should be used in production2 They both embed the entirety o# the template in the #inal e(ecutable, simpli#ying deployment and increasing per#ormance2 The ad!antage o# the 3uasi3uoter is the simplicity) e!erything stays in a single #ile2 For short templates, this can be a !ery good #it2 Ho-e!er, in general, the e(ternal #ile approach is recommended because)

>H

It #ollo-s nicely in the tradition o# separate logic #rom presentation2 You can easily s-itch bet-een e(ternal #ile and debug mode -ith some simple $PP macros, meaning you can keep rapid de!elopment and still achie!e high per#ormance in production2

Since these are special GuasiGuoters and Template Haskell #unctions, you need to be sure to enable the appropriate language e(tensions and use correct synta(2 You can see a simple e(ample o# each in the #igures2

"uasi.uoter
3"4 5) 3"4 5) i$port i$port i$port G6)G- ,%erloaded9trin!s 4"7 "" wePre usin! Text below G6)G- FuasiFuotes 4"7 Text/*a$let @*t$l6rl> ha$letA 1ata/Text @TextA Text/Bla'e/Ienderer/9trin! @render*t$lA

data MyIoute = *o$e | Ti$e | 9tylesheet render render render render == MyIoute "< :@Text> TextA; "< Text *o$e B = 8.ho$e8 Ti$e B = 8.ti$e8 9tylesheet B = 8.style/css8

te$plate == Text "< *t$l6rl MyIoute te$plate title = :ha$let| +doctype M Ght$l< Ghead< Gtitle<43title7 Glink rel=stylesheet hre#=O39tylesheet7< Gbody< Gh(<43title7 |; $ain == I, @A $ain = put9tr5n + render*t$l + te$plate 8My Title8 render

/,ternal file
3"4 5) 3"4 5) 3"4 5) i$port i$port i$port G6)G- ,%erloaded9trin!s 4"7 "" wePre usin! Text below G6)G- Te$plate*askell 4"7 G6)G- CPP 4"7 "" to control production %ersus debu! Text/5ucius @Css6rl> lucius?ile> lucius?ile1ebu!> renderCssA 1ata/Text @TextA Euali#ied 1ata/Text/5a'y/I, as T5I,

data MyIoute = *o$e | Ti$e | 9tylesheet render render render render == MyIoute "< :@Text> TextA; "< Text *o$e B = 8.ho$e8 Ti$e B = 8.ti$e8 9tylesheet B = 8.style/css8

te$plate == Css6rl MyIoute

>*

4i# PI,16CTI, te$plate = +@lucius?ile 8te$plate/lucius8A 4else te$plate = +@lucius?ile1ebu! 8te$plate/lucius8A 4endi# $ain == I, @A $ain = T5I,/put9tr5n + renderCss + te$plate render "" Ote$plate/lucius #oo 3 bar= ba' 7

The naming scheme #or the #unctions is !ery consistent2 5anguage Hamlet $assius 5ucius .ulius Guasi3uoter "(ternal #ile Reload ha$let?ile N/A hamlet
cassius lucius Sulius cassius?ile cassius?ileIeload lucius?ile lucius?ileIeload Sulius?ile Sulius?ileIeload

Alternate Hamlet Types

So #ar, -e%!e seen ho- to generate an *t$l6rl !alue #rom Hamlet, -hich is a piece o# HT&5 -ith embedded type1sa#e 8R5s2 There are currently three other !alues -e can generate using Hamlet) plain HT&5, HT&5 -ith 8R5s and internationali'ed messages, and -idgets2 That last one -ill be co!ered in the -idgets chapter2 To generate plain HT&5 -ithout any embedded 8R5s, -e use Isimpli#ied HamletI2 There are a #e- changes)

We use a di##erent set o# #unctions, pre#i(ed -ith an IsI2 So the 3uasi3uoter is sha$let and the e(ternal #ile #unction is sha$let?ile2 Ho- -e pronounce those is still up #or debate2 0o 8R5 interpolation is allo-ed2 Doing so -ill result in a compile1time error2 "mbedding 4the caret1interpolator6 no longer allo-s arbitrary *t$l6rl !alues2 The rule is that the embedded !alue must ha!e the same type as the template itsel#, so in this case it must be *t$l2 That means that #or sha$let, embedding can be completely replaced -ith normal !ariable interpolation 4-ith a hash62

Dealing -ith internationali'ation 4i*+n6 in Hamlet is a bit complicated2 Hamlet supports i*+n !ia a message datatype, !ery similar in concept and implementation to a type1sa#e 8R52 s a moti!ating e(ample, let%s say -e -ant to ha!e an application that tells you hello and homany apples you ha!e eaten2 We could represent those messages -ith a datatype2
data Ms! = *ello | )pples Int

0e(t, -e -ould -ant to be able to con!ert that into something human1readable, so -e de#ine some render #unctions)
render-n!lish render-n!lish render-n!lish render-n!lish == Ms! "< Text *ello = 8*ello8 @)pples KA = 8Hou did not buy any apples/8 @)pples (A = 8Hou bou!ht ( apple/8

>:

render-n!lish @)pples iA = T/concat :8Hou bou!ht 8> T/pack + show i> 8 apples/8;

0o- -e -ant to interpolate those &sg !alues directly in the template2 For that, -e use underscore interpolation2
+doctype M Ght$l< Ghead< Gtitle<i(Dn Gbody< Gh(<B3*ello7 Gp<B3)pples count7

This kind o# a template no- needs some -ay to turn those !alues into HT&52 So 9ust like type1sa#e 8R5s, -e pass in a render #unction2 To represent this, -e de#ine a ne- type synonym)
type Iender url = url "< :@Text> TextA; "< Text type Translate $s! = $s! "< *t$l type *t$l6rlI(Dn $s! url = Translate $s! "< Iender url "< *t$l t this point, you can pass render-n!lish, render9panish, or renderXlin!on

to this template, and it -ill generate nicely translated output 4depending, o# course, on the 3uality o# your translators62 The complete program is)
3"4 5) 3"4 5) i$port i$port i$port i$port i$port G6)G- FuasiFuotes 4"7 G6)G- ,%erloaded9trin!s 4"7 1ata/Text @TextA Euali#ied 1ata/Text as T Text/*a$let @*t$l6rlI(Dn> iha$letA Text/Bla'e @to*t$lA Text/Bla'e/Ienderer/9trin! @render*t$lA

data MyIoute = *o$e | Ti$e | 9tylesheet render6rl render6rl render6rl render6rl == MyIoute "< :@Text> TextA; "< Text *o$e B = 8.ho$e8 Ti$e B = 8.ti$e8 9tylesheet B = 8.style/css8

data Ms! = *ello | )pples Int render-n!lish render-n!lish render-n!lish render-n!lish render-n!lish apples/8; == Ms! "< Text *ello = 8*ello8 @)pples KA = 8Hou did not buy any apples/8 @)pples (A = 8Hou bou!ht ( apple/8 @)pples iA = T/concat :8Hou bou!ht 8> T/pack + show i> 8

te$plate == Int "< *t$l6rlI(Dn Ms! MyIoute te$plate count = :iha$let| +doctype M Ght$l< Ghead< Gtitle<i(Dn Gbody< Gh(<B3*ello7 Gp<B3)pples count7 |; $ain == I, @A $ain = put9tr5n + render*t$l

>>

+ @te$plate MA @to*t$l / render-n!lishA render6rl

Other Shakespeare
In addition to HT&5, $SS and .a!ascript helpers, there is also some more general1purpose Shakespeare a!ailable2 shakespeare1te(t pro!ides a simple -ay to create interpolated strings, much like people are accustomed to in scripting languages like Ruby and Python2 This package%s utility is de#initely not limited to Yesod2
3"4 5) i$port i$port i$port i$port G6)G- FuasiFuotes> ,%erloaded9trin!s 4"7 Text/9hakespeare/Text Euali#ied 1ata/Text/5a'y/I, as T5I, 1ata/Text @TextA Control/Monad @#orMBA

data Ite$ = Ite$ 3 ite$ a$e == Text > ite$Fty == Int 7 ite$s ite$s : > ; == :Ite$; = Ite$ 8apples8 M Ite$ 8bananas8 (K

$ain == I, @A $ain = #orMB ite$s + 2ite$ "< T5I,/put9tr5n :lt|Hou ha%e 43show + ite$Fty ite$7 43ite$ a$e ite$7/|;

Some 3uick points about this simple e(ample)

0otice that -e ha!e three di##erent te(tual datatypes in!ol!ed 49trin!, strict Text and la'y Text62 They all play together -ell2 We use a 3uasi3uoter named lt, -hich generates la'y te(t2 There is also st2 lso, there are longer names #or these 3uasi3uoters 4ltext and stext62

0eneral %ecommendations
Here are some general hints #rom the Yesod community on ho- to get the most out o# Shakespeare2

For actual sites, use e(ternal #iles2 For libraries, it%s /P to use 3uasi3uoters, assuming they aren%t too long2 Patrick Brisbin has put together a Eim code highlighter that can help out immensely2 You should almost al-ays start Hamlet tags on their o-n line instead o# embedding start<end tags a#ter an e(isting tag2 The only e(ception to this is the occasional Gi< or Gb< tag inside a large block o# te(t2

>@

$idgets
/ne o# the challenges in -eb de!elopment is that -e ha!e to coordinate three di##erent client1 side technologies) HT&5, $SS and .a!ascript2 Worse still, -e ha!e to place these components in di##erent locations on the page) $SS in a style tag in the head, .a!ascript in a script tag in the head, and HT&5 in the body2 nd ne!er mind i# you -ant to put your $SS and .a!ascript in separate #iles? In practice, this -orks out #airly nicely -hen building a single page, because -e can separate our structure 4HT&56, style 4$SS6 and logic 4.a!ascript62 But -hen -e -ant to build modular pieces o# code that can be easily composed, it can be a headache to coordinate all three pieces separately2 Widgets are Yesod%s solution to the problem2 They also help -ith the issue o# including libraries, such as 9Guery, one time only2 /ur #our template languages1 Hamlet, $assius, 5ucius and .ulius1 pro!ide the ra- tools #or constructing your output2 Widgets pro!ide the glue that allo-s them to -ork together seamlessly2

Synopsis
!etIootI = de#ault5ayout + do setTitle 8My Pa!e Title8 toCid!et :lucius| h( 3 color= !reenR 7 |; add9criptIe$ote 8https=..aSax/!oo!leapis/co$.aSax.libs.SEuery.(/V/N.SEuery/$in/Ss8 toCid!et :Sulius| +@#unction@A 3 +@8h(8A/click@#unction@A3 alert@8Hou clicked on the headin!J8AR 7AR 7AR |; toCid!et*ead :ha$let| G$eta na$e=keywords content=8so$e sa$ple keywords8<|; toCid!et :ha$let| Gh(<*erePs one way o# includin! content |; :wha$let| GhN<*erePs another |; toCid!etBody :Sulius| alert@8This is included in the body itsel#8AR |; $ain = warp1ebu! &KKK C

This produces the #ollo-ing HT&5 4indentation added6)

GJ1,CTHP- ht$l< Ght$l< Ghead< Gtitle<My Pa!e TitleG.title< Gstyle<h( 3 color = !reen 7G.style< Gscript src=8https=..aSax/!oo!leapis/co$.aSax.libs.SEuery.(/V/N.SEuery/$in/Ss8<G.sc ript< Gscript< +@#unction@A 3

>A

+@8h(8A/click@#unction@A3 alert@8Hou clicked on the headin!J8AR 7AR 7AR G.script< G$eta na$e=8keywords8 content=8so$e sa$ple keywords8< G.head< Gbody< Gh(<*erePs one way o# includin! content G.h(< GhN<*erePs another G.hN< Gscript< alert@8This is included in the body itsel#8AR G.script< G.body< G.ht$l<

$hat1s in a $idget2
t a !ery super#icial le!el, an HT&5 document is 9ust a bunch o# nested tags2 This is the approach most HT&5 generation tools take) you simply de#ine hierarchies o# tags and are done -ith it2 But let%s imagine that I -ant to -rite a component o# a page #or displaying the na!bar2 I -ant this to be Iplug and playI) I simply call the #unction at the right time, and the na!bar is inserted at the correct point in the hierarchy2 This is -here our super#icial HT&5 generation breaks do-n2 /ur na!bar likely consists o# some $SS and .a!aScript in addition to HT&52 By the time -e call the na!bar #unction, -e ha!e already rendered the Ghead< tag, so it is too late to add a ne- Gstyle< tag #or our $SS declarations2 8nder normal strategies, -e -ould need to break up our na!bar #unction into three parts) HT&5, $SS and .a!aScript, and make sure that -e al-ays call all three pieces2 Widgets take a di##erent approach2 Instead o# !ie-ing an HT&5 document as a monolithic tree o# tags, -idgets see a number o# distinct components in the page2 In particular)

The title "(ternal stylesheets "(ternal .a!ascript $SS declarations .a!ascript code rbitrary Ghead< content rbitrary Gbody< content

Di##erent components ha!e di##erent semantics2 For e(ample, there can only be one title, but there can be multiple e(ternal scripts and stylesheets2 Ho-e!er, those e(ternal scripts and stylesheets should only be included once2 rbitrary head and body content, on the other hand, has no limitation 4someone may -ant to ha!e #i!e lorem ipsum blocks a#ter all62 The 9ob o# a -idget is to hold onto these disparate components and apply proper logic #or combining di##erent -idgets together2 This consists o# things like taking the #irst title set and ignoring others, #iltering duplicates #rom the list o# e(ternal scripts and stylesheets, and concatenating head and body content2

Constructing $idgets
>D

In order to use -idgets, you%ll ob!iously need to be able to get your hands on them2 The most common -ay -ill be !ia the ToCid!et typeclass, and its toCid!et method2 This allo-s you to con!ert your Shakespearean templates directly to a Cid!et) Hamlet code -ill appear in the body, .ulius scripts inside a Gscript< tag in the head, and $assius and 5ucius in a Gstyle< tag2 You can actually o!erride the de#ault beha!ior and ha!e the script and style code appear in a separate #ile2 The sca##olded site pro!ides this #or you automatically2 dditionally, -e%ll see in the Yesod typeclass chapter ho- to turn on asynchronous script loading, -hich -ill place your script content at the end o# the body2 But -hat i# you -ant to add some G$eta< tags, -hich need to appear in the head= /r i# you -ant some .a!ascript to appear in the body instead o# the head= For these purposes, Yesod pro!ides t-o additional type classes) ToCid!et*ead and ToCid!etBody2 These -ork e(actly as they seem they should2 In addition, there are a number o# other #unctions #or creating speci#ic kinds o# Widgets) setTitle Turns some HT&5 into the page title2 add$assius&edia, add5ucius&edia Works the same as toWidget, but takes an additional parameter to indicate -hat kind o# media this applies to2 8se#ul #or creating print stylesheets, #or instance2 addStylesheet dds a re#erence, !ia a Glink< tag, to an e(ternal stylesheet2 Takes a type1sa#e 8R52 addStylesheetRemote Same as add9tylesheet, but takes a normal 8R52 8se#ul #or re#erring to #iles hosted on a $D0, like ;oogle%s 9Guery 8I $SS #iles2 addScript dds a re#erence, !ia a Gscript< tag, to an e(ternal script2 Takes a type1sa#e 8R52 addScriptRemote Same as add9cript, but takes a normal 8R52 8se#ul #or re#erring to #iles hosted on a $D0, like ;oogle%s 9Guery2

Com-ining $idgets
The -hole idea o# -idgets is to increase composability2 You can take these indi!idual pieces o# HT&5, $SS and .a!ascript, combine them together into something more complicated, and then combine these larger entities into complete pages2 This all -orks naturally through the Monad instance o# Cid!et, meaning you can use do1notation to compose pieces together2

Com-ining $idgets
$yCid!et( = do toCid!et :ha$let|Gh(<My Title|; toCid!et :lucius|h( 3 color= !reen 7 |; $yCid!etN = do setTitle 8My Pa!e Title8

>F

add9criptIe$ote 8http=..www/exa$ple/co$.script/Ss8 $yCid!et = do $yCid!et( $yCid!etN "" or> i# you want $yCid!etP = $yCid!et( << $yCid!etN I# you%re so inclined, there%s also a Monoid instance o# Cid!et, meaning you can use $concat or a Criter monad to build things up2 In my e(perience, it%s easiest and most natural to 9ust

use do1notation2

0enerate I(s
I# -e%re really going #or true code reuse here, -e%re e!entually going to run into name con#licts2 5et%s say that there are t-o helper libraries that both use the class name I#ooI to a##ect styling2 We -ant to a!oid such a possibility2 There#ore, -e ha!e the newIdent #unction2 This #unction automatically generates a -ord that is uni3ue #or this handler2

'sing ne3Ident
!etIootI = de#ault5ayout + do headerClass G" li#t newIdent toCid!et :ha$let|Gh( /43headerClass7<My *eader|; toCid!et :lucius| /43headerClass7 3 color= !reenR 7 |; $ain = warp1ebu! &KKK C

You might be -ondering) -hat does li#t mean= Cid!et is a monad trans#ormer, sitting on top o# a *andler2 newIdent is a #unction o# a *andler, so -e need to Ili#tI the #unction #rom the *andler layer to the Cid!et layer to use it2 We can actually use this same approach to per#orm comple( actions, like database 3ueries, #rom -ithin a -idget2 We%ll co!er that -hen -e discuss Yesod%s monads2

3hamlet
5et%s say you%!e got a #airly standard Hamlet template, that embeds another Hamlet template to represent the #ooter)
pa!e = :ha$let| Gp<This is $y pa!e/ I hope you enSoyed it/ Q3#ooter7 |; #ooter = :ha$let| G#ooter< Gp<ThatPs all #olksJ |;

That -orks #ine i# the #ooter is plain old HT&5, but -hat i# -e -ant to add some style= Well, -e can easily spice up the #ooter by turning it into a Widget)

>+

#ooter = do toCid!et :lucius| #ooter 3 #ont"wei!ht= boldR text"ali!n= center 7 |; toCid!et :ha$let| G#ooter< Gp<ThatPs all #olksJ |;

But no- -e%!e got a problem) a Hamlet template can only embed another Hamlet template7 it kno-s nothing about a Widget2 This is -here wha$let comes in2 It takes e(actly the same synta( as normal Hamlet, and !ariable 4OR222S6 and 8R5 4NR222S6 interpolation are unchanged2 But embedding 4QR222S6 takes a Cid!et, and the #inal result is a Cid!et2 To use it, -e can 9ust do)
pa!e = :wha$let| Gp<This is $y pa!e/ I hope you enSoyed it/ Q3#ooter7 |;

There is also wha$let?ile, i# you -ould pre#er to keep your template in a separate #ile2 The sca##olded site has an e!en more con!enient #unction, wid!et?ile, -hich -ill also include your 5ucius, $assius, and .ulius #iles automatically2 We%ll co!er that in the sca##olding chapter2

Types
You may ha!e noticed that I%!e been a!oiding type signatures so #ar2 That%s because there%s a little bit o# a complication in!ol!ed here2 t the most basic le!el, all you need to kno- is that there%s a type synonym called Cid!et -hich you -ill almost al-ays use2 The technical details #ollo-, but don%t -orry i# it%s a little ha'y2 There isn%t actually a Cid!et type de#ined in the Yesod libraries, since the e(act meaning o# it changes bet-een sites2 Instead, -e ha!e a more general type GCid!et sub $aster a2 The #irst t-o parameters gi!e the sub and master #oundation types, respecti!ely2 The #inal parameter is the contained !alue, 9ust like any Monad has2 So -hat%s the deal -ith that sub<master stu##= Well, -hen you%re -riting some reusable code, such as a $R8D application, you can -rite it as a subsite that can be embedded -ithin any other Yesod application2 In such a case, -e need to keep track o# in#ormation #or both the sub and master sites2 The simplest e(ample is #or the type1sa#e 8R5s) Yesod needs to kno- hoto take a route #or your $R8D subsite and turn it into a route #or the master site so that it can be properly rendered2 Ho-e!er, that sub<master distinction only e!er matters -hen you%re interacting -ith subsites2 When you%re -riting your standard response code, you%re dealing -ith 9ust your application, and so the sub and master sites -ill be the same2 Since this is the most common case, the sca##olded site declares a type synonym to help you out2 5et%s say your #oundation type is &y$ool pp, it -ill de#ine type Cid!et = GCid!et MyCool)pp MyCool)pp @A2 There#ore, -e can get some !ery user1#riendly type signatures on our -idgets)
#ooter == Cid!et

>B

#ooter = do toCid!et :lucius| #ooter 3 #ont"wei!ht= boldR text"ali!n= center 7 |; toCid!et :ha$let| G#ooter< Gp<ThatPs all #olksJ |; pa!e == Cid!et pa!e = :wha$let| Gp<This is $y pa!e/ I hope you enSoyed it/ Q3#ooter7 |;

I# you%!e been paying close attention, you might be con#used2 We used li#t on Cid!et in the ID generation e(ample abo!e, but GCid!et isn%t actually a monad trans#ormer2 What%s going on here= Well, in older !ersions o# Yesod, it was a trans#ormer around the *andler type2 8n#ortunately, this led to di##icult1to1parse error messages2 s a result, GCid!et is no- a newtype -rapper that hides a-ay its monad1trans#ormer essence2 But -e still -ant to be able to li#t #unctions #rom the inner *andler monad2 To sol!e this, Yesod pro!ides an alternate, more general li#t #unction that -orks #or both standard MonadTrans instances, and special newtype -rappers like GCid!et2 s a result, you can pretend like GCid!et is a standard trans#ormer, -hile still getting to keep your nice error message2 /ne last point) 9ust like -e ha!e the breakdo-n bet-een Cid!et and GCid!et, -e ha!e a similar breakdo-n bet-een *andler and G*andler2

'sing $idgets
It%s all -ell and good that -e ha!e these beauti#ul Widget datatypes, but ho- e(actly do -e turn them into something the user can interact -ith= The most commonly used #unction is de#ault5ayout, -hich essentially has the type signature Cid!et "< *andler Iep*t$l2 4I say IessentiallyI because o# the -hole G*andler issue26 Iep*t$l is a datatype containing some ra- HT&5 output ready to be sent o!er the -ire2 is actually a typeclass method, -hich can be o!erridden #or each application2 This is ho- Yesod apps are themed2 So -e%re still le#t -ith the 3uestion) -hen -e%re inside de#ault5ayout, ho- do -e un-rap a Cid!et= The ans-er is wid!etToPa!eContent2 5et%s look at some 4simpli#ied6 types)
de#ault5ayout wid!etToPa!eContent == Cid!et "< *andler @Pa!eContent urlA data Pa!eContent url = Pa!eContent 3 pa!eTitle == *t$l > pa!e*ead == *t$l6rl url > pa!eBody == *t$l6rl url 7

This is getting closer to -hat -e need2 We no- ha!e direct access to the HT&5 making up the head and body, as -ell as the title2 t this point, -e can use Hamlet to combine them all together into a single document, along -ith our site layout, and -e use ha$letToIep*t$l to

@H

render that Hamlet result into actual HT&5 that%s ready to be sho-n to the user2 The ne(t #igure demonstrates this process2

'sing 3idgetToPageContent
$y5ayout == GCid!et s My)pp @A "< G*andler s My)pp Iep*t$l $y5ayout wid!et = do pc G" wid!etToPa!eContent wid!et ha$letToIep*t$l :ha$let| +doctype M Ght$l< Ghead< Gtitle<43pa!eTitle pc7 G$eta charset=ut#"D< Gstyle<body 3 #ont"#a$ily= %erdana 7 Q3pa!e*ead pc7 Gbody< Garticle< Q3pa!eBody pc7 |; instance Hesod My)pp where de#ault5ayout = $y5ayout $ain = warp1ebu! &KKK My)pp

You may ha!e noticed that -e used GCid!et and G*andler instead o# Cid!et and *andler2 That%s because de#ault5ayout is a method that can be called by subsites to ensure that they get the same styling as the master site2 There#ore, -e need to keep our types #le(ible here2 This is all -ell and good, but there%s one thing that bothers me) that style tag2 There are a #e- problems -ith it)

8nlike 5ucius or $assius, it doesn%t get compile1time checked #or correctness2 ;ranted that the current e(ample is !ery simple, but in something more complicated -e could get into character escaping issues2 We%ll no- ha!e t-o style tags instead o# one) the one produced by $y5ayout, and the one generated in the pa!e*ead based on the styles set in the -idget2

We ha!e one more trick in our bag to address this) -e apply some last1minute ad9ustments to the -idget itsel# be#ore calling wid!etToPa!eContent2 It%s actually !ery easy to do) -e 9ust use do1notation again, as in 5ast1&inute Widget d9ustment2

Last&Minute $idget Ad4ustment

$y5ayout == GCid!et s My)pp @A "< G*andler s My)pp Iep*t$l $y5ayout wid!et = do pc G" wid!etToPa!eContent + do wid!et toCid!et :lucius| body 3 #ont"#a$ily= %erdana 7 |; ha$letToIep*t$l :ha$let| +doctype M Ght$l< Ghead<

@*

Gtitle<43pa!eTitle pc7 G$eta charset=ut#"D< Q3pa!e*ead pc7 Gbody< Garticle< Q3pa!eBody pc7 |; instance Hesod My)pp where de#ault5ayout = $y5ayout $ain = warp1ebu! &KKK My)pp

Summary
The basic building block o# each page is a -idget2 Indi!idual snippets o# HT&5, $SS, and .a!ascript can be turned into -idgets !ia the polymorphic toCid!et #unction2 8sing do1 notation, you can combine these indi!idual -idgets into larger -idgets, e!entually containing all the content o# your page2 8n-rapping these -idgets is usually per#ormed -ithin the de#ault5ayout #unction, -hich can be used to apply a uni#ied look1and1#eel to all your pages2

Yesod Typeclass
"!ery one o# our Yesod applications re3uires an instance o# the Hesod typeclass2 So #ar, -e%!e only seen de#ault5ayout2 In this chapter, -e%ll e(plore the meaning o# many o# the methods o# the Hesod typeclass2 The Hesod typeclass gi!es us a central place #or de#ining settings #or our application2 "e!erything else has a de#ault de#inition -hich is usually the right thing2 But in order to build a po-er#ul, customi'ed application, you%ll usually end up -anting to o!erride at least a #e- o# these methods2

%endering and Parsing '%Ls

We%!e already mentioned ho- Yesod is able to automatically render type1sa#e 8R5s into a te(tual 8R5 that can be inserted into an HT&5 page2 5et%s say -e ha!e a route de#inition that looks like)
$kHesod 8My)pp8 :parseIoutes| .so$e.path 9o$ePathI G-T ;

I# -e place 9o$ePathI into a hamlet template, ho- does Yesod render it= Yesod al-ays tries to construct absolute 8R5s2 This is especially use#ul once -e start creating C&5 sitemaps and tom #eeds, or sending emails2 But in order to construct an absolute 8R5, -e need to kno- the domain name o# the application2

@:

You might think -e could get that in#ormation #rom the user%s re3uest, but -e still need to deal -ith ports2 nd e!en i# -e get the port number #rom the re3uest, are -e using HTTP or HTTPS= nd e!en i# you kno- that, such an approach -ould mean that, depending on hothe user submitted a re3uest -ould generate di##erent 8R5s2 For e(ample, -e -ould generate di##erent 8R5s depending i# the user connected to Ie(ample2comI or I---2e(ample2comI2 For Search "ngine /ptimi'ation, -e -ant to be able to consolidate on a single canonical 8R52 nd #inally, Yesod doesn%t make any assumption about where you host your application2 For e(ample, I may ha!e a mostly static site 4http)<<static2e(ample2com<6, but I%d like to stick a Yesod1po-ered Wiki at <-iki<2 There is no reliable -ay #or an application to determine -hat subpath it is being hosted #rom2 So instead o# doing all o# this guess-ork, Yesod needs you to tell it the application root2 8sing the -iki e(ample, you -ould -rite your Hesod instance as)
instance Hesod MyCiki where approot B = 8http=..static/exa$ple/co$.wiki8 "" ?IYM- this is out"o#" date

0otice that there is no trailing slash there2 0e(t, -hen Yesod -ants to construct a 8R5 #or 9o$ePathI, it determines that the relati!e path #or 9o$ePathI is .so$e.path, appends that to your approot and creates http=..static/exa$ple/co$.wiki.so$e.path2 This also e(plains our cryptic approot B = 88 FIC&") #or our e(amples in the book, -e%re al-ays ser!ing #rom the root o# the domain 4in our case, localhost62 By using an empty string, 9o$ePathI renders to .so$e.path, -hich -orks 9ust #ine2 In real li#e applications, ho-e!er, you should use a real application root2 The #irst argument to approot FIC&" is the site #oundation2 This means that you could load your approot at program initiali'ation, store it in the #oundation, and then retrie!e it2 This is in #act -hat the sca##olded site does, using a Y &5 con#ig #ile2 nd by the -ay, the sca##olded site can load di##erent settings #or de!eloping, testing, staging, and production builds, so you can easily test on one domain1 like localhost1 and ser!e #rom a di##erent domain2 To reiterate) e!en though #or the simple cases in this book, the #irst argument to approot is usually ignored, in real li#e code it usually isn%t2 We also need to keep that argument so that Haskell%s type system can determine -hich instance o# Hesod to use in grabbing the approot2

4oinPath
In order to con!ert a type1sa#e 8R5 into a te(t !alue, Yesod uses t-o helper #unctions2 The #irst is the renderIoute method o# the IenderIoute typeclass2 "!ery type1sa#e 8R5 is an instance o# this typeclass2 renderIoute con!erts a !alue into a list o# path pieces2 For e(ample, our 9o$ePathI #rom abo!e -ould be con!erted into :8so$e8> 8path8;2

@>

ctually, renderIoute produces both the path pieces and a list o# 3uery1string parameters2 The de#ault instances o# renderIoute al-ays pro!ide an empty list o# 3uery string parameters2 Ho-e!er, it is possible to o!erride this2 /ne notable case is the static subsite, -hich puts a hash o# the #ile contents in the 3uery string #or caching purposes2 The other #unction is the SoinPath method o# the Yesod typeclass2 This #unction takes #our arguments) the #oundation !alue, the application root, a list o# path segments and a list o# 3uery string parameters, and returns a te(tual 8R52 The de#ault implementation does the Iright thingI) it separates the path pieces by #or-ard slashes, prepends the application root and appends the 3uery string2 I# you are happy -ith de#ault 8R5 rendering, you should not need to modi#y it2 Ho-e!er, i# you -ant to modi#y 8R5 rendering to do things like append a trailing slash, this -ould be the place to do it2

cleanPath
The #lip side to SoinPath is cleanPath2 5et%s look at ho- it gets used in the dispatch process) *2 The path in#o re3uested by the user is split into a series o# path pieces2 :2 We pass the path pieces to the cleanPath #unction2 >2 I# cleanPath indicates a redirect 4a 5e#t response6, then a >H* response is sent to the client2 This is used to #orce canonical 8R5s 4eg, remo!e e(tra slashes62 @2 /ther-ise, -e try to dispatch using the response #rom cleanPath 4a Ii!ht62 I# this -orks, -e return a response2 /ther-ise, -e return a @H@2 This combination allo-s subsites to retain #ull control o# ho- their 8R5s appear, yet allo-s master sites to ha!e modi#ied 8R5s2 s a simple e(ample, let%s see ho- -e could modi#y Yesod to al-ays produce trailing slashes on 8R5s)
3"4 5) G6)G- Type?a$ilies> FuasiFuotes> MultiPara$TypeClasses> Te$plate*askell> ,%erloaded9trin!s 4"7 i$port Hesod i$port etwork/*TTP/Types @encodePathA i$port Bla'e/Byte9trin!/Builder/Char/6t#D @#ro$TextA i$port Euali#ied 1ata/Text as T i$port Euali#ied 1ata/Text/-ncodin! as Ti$port Control/)rrow @@WWWAA i$port 1ata/Monoid @$appendA data 9lash = 9lash $kHesod 89lash8 :parseIoutes| . IootI G-T .#oo ?ooI G-T |; instance Hesod 9lash where SoinPath B ar piecesP EsP = #ro$Text ar Z$appendZ encodePath pieces Es where Es = $ap @T-/encode6t#D WWW !oA EsP

@@

!o 88 = othin! !o x = Just + T-/encode6t#D x pieces = piecesP UU :88; "" Ce want to keep canonical 6I5s/ There#ore> i# the 6I5 is $issin! a "" trailin! slash> redirect/ But the e$pty set o# pieces always stays the "" sa$e/ cleanPath B :; = Ii!ht :; cleanPath B s | dropChile @not / T/nullA s == :88; = "" the only e$pty strin! is the last one Ii!ht + init s "" 9ince SoinPath will append the $issin! trailin! slash> we si$ply "" re$o%e e$pty pieces/ | otherwise = 5e#t + #ilter @not / T/nullA s !etIootI = de#ault5ayout :wha$let| Gp< Ga hre#=O3IootI7<IootI Gp< Ga hre#=O3?ooI7<?ooI |; !et?ooI = !etIootI $ain = warp1ebu! &KKK 9lash

First, let%s look at our SoinPath implementation2 This is copied almost !erbatim #rom the de#ault Yesod implementation, -ith one di##erence) -e append an e(tra empty string to the end2 When dealing -ith path pieces, an empty string -ill append another slash2 So adding an e(tra empty string -ill #orce a trailing slash2 is a little bit trickier2 First, -e check #or the empty path like be#ore, and i# so pass it through as1is2 We use Right to indicate that a redirect is not necessary2 The ne(t clause is actually checking #or t-o di##erent possible 8R5 issues)
cleanPath

There is a double slash, -hich -ould sho- up as an empty string in the middle o# our paths2 There is a missing trailing slash, -hich -ould sho- up as the last piece not being an empty string2

ssuming neither o# those conditions hold, then only the last piece is empty, and -e should dispatch based on all but the last piece2 Ho-e!er, i# this is not the case, -e -ant to redirect to a canonical 8R52 In this case, -e strip out all empty pieces and do not bother appending a trailing slash, since SoinPath -ill do that #or us2

defaultLayout
&ost -ebsites like to apply some general template to all o# their pages2 de#ault5ayout is the recommended approach #or this2 While you could 9ust as easily de#ine your o-n #unction and call that instead, -hen you o!erride de#ault5ayout all o# the Yesod1generated pages 4error pages, authentication pages6 automatically get this style2

@A

/!erriding is !ery straight1#or-ard) -e use wid!etToPa!eContent to con!ert a Cid!et to a title, head tags and body tags, and then use ha$letToIep*t$l to con!ert a Hamlet template into a Iep*t$l2 We can e!en add e(tra -idget components, like a 5ucius template2 #rom -ithin de#ault5ayout2 n e(ample should make this all clear)
de#ault5ayout contents = do Pa!eContent title headTa!s bodyTa!s G" wid!etToPa!eContent + do toCid!et :cassius| 4body #ont"#a$ily= sans"seri# 4wrapper width= [VKpx $ar!in= K auto |; addCid!et contents ha$letToIep*t$l :ha$let| +doctype M Ght$l< Ghead< Gtitle<43title7 Q3headTa!s7 Gbody< Gdi% id=8wrapper8< Q3bodyTa!s7 |; !etIootI = de#ault5ayout + do setTitle 8Ioot test8 toCid!et :cassius|body color= red |; toCid!et :ha$let|Gh(<*ello|; $ain = warp1ebu! LKKK 5ayout

getMessage
"!en though -e ha!en%t co!ered sessions yet, I%d like to mention !etMessa!e here2 common pattern in -eb de!elopment is setting a message in one handler and displaying it in another2 For e(ample, i# a user P,9Ts a #orm, you may -ant to redirect him<her to another page along -ith a IForm submission completeI message2 This is commonly kno-n as Post<Redirect<;et2 To #acilitate this, Yesod comes built in -ith a pair o# #unctions) setMessa!e sets a message in the user session, and !etMessa!e retrie!es the message 4and clears it, so it doesn%t appear a second time62 It%s recommended that you put the result o# !etMessa!e into your de#ault5ayout2 For e(ample)
de#ault5ayout contents = do Pa!eContent title headTa!s bodyTa!s G" wid!etToPa!eContent contents $$s! G" !etMessa!e ha$letToIep*t$l :ha$let| +doctype M Ght$l<

@D

|; !etIootI = de#ault5ayout :wha$let|Ga hre#=8O3Ms!I78<$essa!e|; !etMs!I = setMessa!e 8#oo8 << redirect IootI << return @A $ain = warp1ebu! LKKK 5ayout

Ghead< Gtitle<43title7 Q3headTa!s7 Gbody< +$aybe $s! G" $$s! Gdi% 4$essa!e<43$s!7 Q3bodyTa!s7

We%ll co!er !etMessa!e<setMessa!e in more detail -hen -e discuss sessions2

Custom error pages

/ne o# the marks o# a pro#essional -eb site is a properly designed error page2 Yesod gets you a long -ay there by automatically using your de#ault5ayout #or displaying error pages2 But sometimes, you%ll -ant to go e!en #urther2 For this, you%ll -ant to o!erride the error*andler method)
error*andler ot?ound = #$ap chooseIep + de#ault5ayout + do setTitle 8IeEuest pa!e not located8 toCid!et :ha$let| Gh(< ot ?ound Gp<Ce apolo!i'e #or the incon%enience> but the reEuested pa!e could not be located/ |; error*andler other = de#ault-rror*andler other !etIootI = de#ault5ayout :wha$let|*ello Corld|; $ain = warp1ebu! LKKK 5ayout

Here -e speci#y a custom @H@ error page2 We can also use the de#ault-rror*andler -hen -e don%t -ant to -rite a custom handler #or each error type2 Due to type constraints, -e need to start o## our methods -ith #$ap chooseIep, but other-ise you can -rite a typical handler #unction2 In #act, you could e!en use special responses like redirects)
error*andler ot?ound = redirect IootI error*andler other = de#ault-rror*andler other !etIootI = de#ault5ayout :wha$let|*ello Corld|; $ain = warp1ebu! LKKK 5ayout

"!en though you can do this, I don%t actually recommend such practices2 @H@2

@H@ should be a

/,ternal CSS and +a ascript

The #unctionality described here is automatically included in the sca##olded site, so you don%t need to -orry about implementing this yoursel#2

@F

/ne o# the most po-er#ul, and most intimidating, methods in the Yesod typeclass is add9taticContent2 Remember that a Widget consists o# multiple components, including $SS and .a!ascript2 Ho- e(actly does that $SS<.S arri!e in the user%s bro-ser= By de#ault, they are ser!ed in the Ghead< o# the page, inside Gstyle< and Gscript< tags, respecti!ely2 That might be simple, but it%s #ar #rom e##icient2 "!ery page load -ill no- re3uire loading up the $SS<.S #rom scratch, e!en i# nothing changed? What -e really -ant is to store this content in an e(ternal #ile and then re#er to it #rom the HT&52 This is -here add9taticContent comes in2 It takes three arguments) the #ilename e(tension o# the content 4css or Ss6, the mime1type o# the content 4text.css or text.Sa%ascript6 and the content itsel#2 It -ill then return one o# three possible results) 0othing 0o static #ile sa!ing occurred7 embed this content directly in the HT&52 This is the de#ault beha!ior2 .ust 45e#t Te(t6 This content -as sa!ed in an e(ternal #ile, and use the gi!en te(tual link to re#er to it2 .ust 4Right 4Route a, Guery66 Same, but no- use a type1sa#e 8R5 along -ith some 3uery string parameters2 The 5e#t result is use#ul i# you -ant to store your static #iles on an e(ternal ser!er, such as a $D0 or memory1backed ser!er2 The Ii!ht result is more commonly used, and ties in !ery -ell -ith the static subsite2 This is the recommended approach #or most applications, and is pro!ided by the sca##olded site by de#ault2 You might be -ondering) i# this is the recommended approach, -hy isn%t it the de#ault= The problem is that it makes a number o# assumptions that don%t uni!ersally hold) your application has a static subsite, and the location o# your static #iles2 The sca##olded add9taticContent does a number o# intelligent things to help you out)

It automatically mini#ies your .a!ascript using the h9smin package2 It names the output #iles based on a hash o# the #ile contents2 This means you can set your cache headers to #ar in the #uture -ithout #ears o# stale content2 lso, since #ilenames are based on hashes, you can be guaranteed that a #ile doesn%t need to be -ritten i# a #ile -ith the same name already e(ists2 The sca##old code automatically checks #or the e(istence o# that #ile, and a!oids the costly disk I</ o# a -rite i# it%s not necessary2

Smarter Static !iles

;oogle recommends an important optimi'ation) ser!e static #iles #rom a separate domain2 The ad!antage to this approach is that cookies set on your main domain are not sent -hen retrie!ing static #iles, thus sa!ing on a bit o# band-idth2 To #acilitate this, -e ha!e the urlIender,%erride method2 This method intercepts the normal 8R5 rendering and sets a special !alue #or some routes2 For e(ample, the sca##olding de#ines this method as) @+

urlIender,%erride y @9taticI sA = Just + uncurry @SoinPath y @9ettin!s/staticIoot + settin!s yAA + renderIoute s urlIender,%erride B B = othin!

This means that static routes are ser!ed #rom a special static root, -hich you can con#igure to be a di##erent domain2 This is a great e(ample o# the po-er and #le(ibility o# type1sa#e 8R5s) -ith a single line o# code you%re able to change the rendering o# static routes throughout all o# your handlers2

Authentication5Authori6ation
For simple applications, checking permissions inside each handler #unction can be a simple, con!enient approach2 Ho-e!er, it doesn%t scale -ell2 "!entually, you%re going to -ant to ha!e a more declarati!e approach2 &any systems out there de#ine $5s, special con#ig #iles, and a lot o# other hocus1pocus2 In Yesod, it%s 9ust plain old Haskell2 There are three methods in!ol!ed) isWriteRe3uest Determine i# the current re3uest is a IreadI or I-riteI operations2 By de#ault, Yesod #ollo-s R"ST#ul principles, and assumes G-T, *-)1, ,PTI, 9, and TI)C- re3uests are read1only, -hile all others are can -rite2 is uthori'ed Takes a route 4i2e2, type1sa#e 8R56 and a boolean indicating -hether or not the re3uest is a -rite re3uest2 It returns an )uthIesult, -hich can ha!e one o# three !alues)
)uthori'ed )uthenticationIeEuired 6nauthori'ed

By de#ault, it returns )uthori'ed #or all re3uests2 authRoute I# is)uthori'ed returns )uthenticationIeEuired, then redirect to the gi!en route2 I# no route is pro!ided 4the de#ault6, return a @H> IPermission DeniedI message2 These methods tie in nicely -ith the yesod1auth package, -hich is used by the sca##olded site to pro!ide a number o# authentication options, such as /penID, Bro-serID, email, username and T-itter2 We%ll co!er more concrete e(amples in the auth chapter2

Some Simple Settings

0ot e!erything in the Yesod typeclass is complicated2 Some methods are simple #unctions2 5et%s 9ust go through the list) encryptPey Yesod uses client1side sessions, -hich are stored in encrypted, cryptographically1 hashed cookies2 Well, as long as you pro!ide an encryption key2 I# this #unction returns 0othing, then sessions are disabled2 This can be a use#ul optimi'ation on sites that don%t need session #acilities, as it a!oids an encrypt<decrypt pair on each @B

re3uest2The combination o# encryption and hashing guarantees t-o properties) the session payload is tamper1proo#, and is opa3ue2 "ncryption -ithout hashing -ould allo- a user to randomly change the cookie data and still ha!e it accepted by the ser!er, -hile hashing -ithout encryption -ould allo- inspection o# the data2 clientSessionDuration Ho- long a session should last #or2 By de#ault, this is t-o hours2 sessionIp ddress By de#ault, sessions are tied to an indi!idual IP address2 I# your users are sitting behind a pro(y ser!er, this can cause trouble -hen their IP suddenly changes2 This setting lets you disable this security #eature2 cookiePath What paths -ithin your current domain to set cookies #or2 The de#ault is I<I, and -ill almost al-ays be correct2 /ne e(ception might be -hen you%re ser!ing #rom a subpath -ithin a domain 4like our -iki e(ample abo!e62 ma(imum$ontent5ength To pre!ent Denial o# Ser!er 4DoS6 attacks, Yesod -ill limit the si'e o# re3uest bodies2 Some o# the time, you%ll -ant to bump that limit #or some routes 4e2g2, a #ile upload page62 This is -here you%d do that2 yepnope.s You can speci#y the location o# the yepnope .a!ascript library2 I# this is gi!en, then yepnope -ill be used to asynchronously load all o# the .a!ascript on your page2

Summary
The Yesod typeclass has a number o# o!errideable methods that allo- you to con#igure your application2 They are all optional, and pro!ide sensible de#aults2 By using built1in Yesod constructs like de#ault5ayout and !etMessa!e, you%ll get a consistent look1and1#eel throughout your site, including pages automatically generated by Yesod such as error pages and authentication2 We ha!en%t co!ered all the methods in the Yesod typeclass in this chapter2 For a #ull listing o# methods a!ailable, you should consult the Haddock documentation2

%outing and Handlers

I# -e look at Yesod as a &odel1Eie-1$ontroller #rame-ork, routing and handlers make up the controller2 For contrast, let%s describe t-o other routing approaches used in other -eb de!elopment en!ironments)

Dispatch based on #ile name2 This is ho- PHP and SP -ork, #or e(ample2 Ha!e a centrali'ed routing #unction that parses routes based on regular e(pressions2 D9ango and Rails #ollo- this approach2

Yesod is closer in principle to the latter techni3ue2 "!en so, there are signi#icant di##erences2 Instead o# using regular e(pressions, Yesod matches on pieces o# a route2 Instead o# ha!ing a one1-ay route1to1handler mapping, Yesod has an intermediate data type 4called the route datatype, or a type1sa#e 8R56 and creates t-o1-ay con!ersion #unctions2

AH

$oding this more ad!anced system manually is tedious and error prone2 There#ore, Yesod de#ines a Domain Speci#ic 5anguage 4DS56 #or speci#ying routes, and pro!ides Template Haskell #unctions to con!ert this DS5 to Haskell code2 This chapter -ill e(plain the synta( o# the routing declarations, gi!e you a glimpse o# -hat code is generated #or you, and e(plain the interaction bet-een routing and handler #unctions2

%oute Synta,
Instead o# trying to shoe1horn route declarations into an e(isting synta(, Yesod%s approach is to use a simpli#ied synta( designed 9ust #or routes2 This has the ad!antage o# making the code not only easy to -rite, but simple enough #or someone -ith no Yesod e(perience to read and understand the sitemap o# your application2 basic e(ample o# this synta( is)
. IootI G-T .blo! Blo!I G-T P,9T .blo!.4Blo!Id Blo!PostI G-T P,9T .static 9taticI 9tatic !et9tatic

The ne(t #e- sections -ill e(plain the #ull details o# -hat goes on in the route declaration2

Pieces
/ne o# the #irst thing Yesod does -hen it gets a re3uest is split up the re3uested path into pieces2 The pieces are tokeni'ed at all #or-ard slashes2 For e(ample)
toPieces 8.8 = :; toPieces 8.#oo.bar.ba'.8 = :8#oo8> 8bar8> 8ba'8> 88;

You may notice that there are some #unny things going on -ith trailing slashes, or double slashes 4I<#oo<<bar<<I6, or a #e- other things2 Yesod belie!es in ha!ing canonical 8R5s7 i# someone re3uests a 8R5 -ith a trailing slash, or -ith a double slash, they automatically get a redirect to the canonical !ersion2 This ensures you ha!e one 8R5 #or one resource, and can help -ith your search rankings2 What this means #or you is that you needn%t concern yoursel# -ith the e(act structure o# your 8R5s) you can sa#ely think about pieces o# a path, and Yesod automatically handles intercalating the slashes and escaping problematic characters2 I#, by the -ay, you -ant more #ine1tuned control o# ho- paths are split into pieces and 9oined together again, you%ll -ant to look at the cleanPath and SoinPath methods in the Yesod typeclass chapter2

Types of Pieces
When you are declaring your routes, you ha!e three types o# pieces at your disposal) A*

Static This is a plain string that must be matched against precisely in the 8R52 Dynamic single This is a single piece 4ie, bet-een t-o #or-ard slashes6, but can be a user1submitted !alue2 This is the primary method o# recei!ing e(tra user input on a page re3uest2 These pieces begin -ith a hash 4O6 and are #ollo-ed by a data type2 The datatype must be an instance o# PathPiece2 Dynamic multi The same as be#ore, but can recei!e multiple pieces o# the 8R52 This must al-ays be the last piece in a resource pattern2 It is speci#ied by an asterisk 4T6 #ollo-ed by a datatype, -hich must be an instance o# PathMultiPiece2 &ulti pieces are not as common as the other t-o, though they are !ery important #or implementing #eatures like static trees representing #ile structure or -ikis -ith arbitrary hierarchies2 5et us take a look at some standard kinds o# resource patterns you may -ant to -rite2 Starting simply, the root o# an application -ill 9ust be .2 Similarly, you may -ant to place your F G at .pa!e.#aE2 0o- let%s say you are going to -rite a Fibonacci -ebsite2 You may construct your 8R5s like .#ib.4Int2 But there%s a slight problem -ith this) -e do not -ant to allo- negati!e numbers or 'ero to be passed into our application2 Fortunately, the type system can protect us)
newtype atural = atural Int deri%in! @9how> Iead> -E> u$> ,rdA instance PathPiece atural where toPathPiece @ atural iA = T/pack + show i #ro$PathPiece s = case reads + T/unpack s o# @i> 88A=B | i G ( "< othin! | otherwise "< Just + atural i :; "< othin! $kHesod 8?ibs8 :parseIoutes| .#ibs.4 atural ?ibsI G-T |; instance Hesod ?ibs #ibs = ( = ( = 'ipCith @UA #ibs @tail #ibsA !et?ibsI == atural "< G*andler ?ibs ?ibs IepPlain !et?ibsI @ atural iA = return + IepPlain + toContent + show + #ibs JJ @i " (A $ain = warp1ebu! &KKK ?ibs

/n line * -e de#ine a simple ne-type -rapper around Int to protect oursel!es #rom in!alid input2 We can see that PathPiece is a typeclass -ith t-o methods2 toPathPiece does nothing more than con!ert to a Text2 #ro$PathPiece attempts to con!ert a Text to our datatype, returning othin! -hen this con!ersion is impossible2 By using this datatype, -e can ensure that our handler #unction is only e!er gi!en natural numbers, allo-ing us to once again use the type system to battle the boundary issue2 In a real li#e application, -e -ould also -ant to ensure -e ne!er accidently constructed an in!alid atural !alue internally to our app2 To do so, -e could use an approach like smart constructors2 For the purposes o# this e(ample, -e%!e kept the code simple2

A:

De#ining a PathMultiPiece is 9ust as simple2 5et%s say -e -ant to ha!e a Wiki -ith at least t-o le!els o# hierarchy7 -e might de#ine a datatype such as)
data Pa!e = Pa!e Text Text :Text; "" N or $ore instance PathMultiPiece Pa!e where toPathMultiPiece @Pa!e x y 'A = x = y = ' #ro$PathMultiPiece @x=y='A = Just + Pa!e x y ' #ro$PathMultiPiece B = othin! $ain = return @A

%esource name
"ach resource pattern also has a name associated -ith it2 That name -ill become the constructor #or the type sa#e 8R5 datatype associated -ith your application2 There#ore, it has to start -ith a capital letter2 By con!ention, these resource names all end -ith a capital R2 There is nothing #orcing you to do this, it is 9ust common practice2 The e(act de#inition o# our constructor depends upon the resource pattern it is attached to2 Whate!er datatypes are included in single and multi pieces o# the pattern become arguments to the datatype2 This gi!es us a *1to1* correspondence bet-een our type sa#e 8R5 !alues and !alid 8R5s in our application2 This doesn%t necessarily mean that every !alue is a -orking page, 9ust that it is is a potentially !alid 8R52 s an e(ample, that !alue PersonI 8Michael8 may not resol!e to a !alid page i# there is no &ichael in the database2 5et%s get some real e(amples going here2 I# you had the resource patterns .person.4Text named PersonI, .year.4Int named HearI and .pa!e.#aE named ?aEI, you -ould end up -ith a route data type roughly looking like)
data MyIoute = PersonI Text | HearI Int | ?aEI

I# a user re3uests the relati!e 8R5 o# .year.NKK\, Yesod -ill con!ert it into the !alue HearI NKK\2 .person.Michael becomes PersonI 8Michael8 and .pa!e.#aE becomes ?aEI2 /n the other hand, .year.two"thousand"nine, .person.$ichael.snoy$an and .pa!e.?)F -ould all result in @H@ errors -ithout e!er seeing your code2

Handler specification
The last piece o# the pu''le -hen declaring your resources is ho- they -ill be handled2 There are three options in Yesod)

single handler #unction #or all re3uest methods on a gi!en route2 separate handler #unction #or each re3uest method on a gi!en route2 ny other re3uest method -ill generate a @HA Bad &ethod response2 You -ant to pass o## to a subsite2

A>

The #irst t-o can be easily speci#ied2 single handler #unction -ill be a line -ith 9ust a resource pattern and the resource name, such as .pa!e.#aE ?aEI2 In this case, the handler #unction must be named handle?aEI2 separate handler #or each re3uest method -ill be the same, plus a list o# re3uest methods2 The re3uest methods must be all capital letters2 For e(ample, .person.49trin! PersonI G-T P,9T 1-5-T-2 In this case, you -ould need to de#ine three handler #unctions) !etPersonI, postPersonI and deletePersonI2 Subsites are a !ery use#ulU but complicatedU topic in Yesod2 We -ill co!er -riting subsites later, but using them is not too di##icult2 The most commonly used subsite is the static subsite, -hich ser!es static #iles #or your application2 In order to ser!e static #iles #rom .static, you -ould need a resource line like)
.static 9taticI 9tatic !et9tatic

In this line, .static 9ust says -here in your 8R5 structure to ser!e the static #iles #rom2 There is nothing magical about the -ord static, you could easily replace it -ith .$y.non" dyna$ic.#iles2 The ne(t -ord, 9taticI, gi!es the resource name2 The ne(t t-o -ords are -hat speci#y that -e are using a subsite2 9tatic is the name o# the subsite #oundation datatype, and !et9tatic is a #unction that gets a 9tatic !alue #rom a !alue o# your master #oundation datatype2 5et%s not get too caught up in the details o# subsites no-2 We -ill look more closely at the static subsite in the sca##olded site chapter2

(ispatch
/nce you ha!e speci#ied your routes, Yesod -ill take care o# all the pesky details o# 8R5 dispatch #or you2 You 9ust need to make sure to pro!ide the appropriate handler #unctions2 For subsite routes, you do not need to -rite any handler #unctions, but you do #or the other t-o2 We mentioned the naming rules abo!e 4My*andlerI G-T becomes !etMy*andlerI, My,ther*andlerI becomes handleMy,ther*andlerI62 0o- -e need the type signature2

%eturn Type
5et%s look at a simple handler #unction)
$kHesod 89i$ple8 :parseIoutes| . *o$eI G-T |; !et*o$eI == *andler Iep*t$l !et*o$eI = de#ault5ayout :wha$let|Gh(<This is si$ple |; instance Hesod 9i$ple $ain = warp1ebu! &KKK 9i$ple

A@

5ook at the type signature o# !et*o$eI2 The #irst component is *andler2 *andler is a special monad that all handler #unctions li!e in2 It pro!ides access to re3uest in#ormation, let%s you send redirects, and lots o# other stu## -e%ll get to soon2 0e(t -e ha!e Iep*t$l2 When -e discuss representations -e -ill e(plore the why o# things more7 #or no-, -e are 9ust interested in the how2 s you might guess, Iep*t$l is a datatype #or HT&5 responses2 nd as you also may guess, -eb sites need to return responses besides HT&52 $SS, .a!ascript, images, C&5 are all necessities o# a -ebsite2 There#ore, the return !alue o# a handler #unction can be any instance o# *asIeps2 is a po-er#ul concept that allo-s Yesod to automatically choose the correct representation o# your data based on the client re3uest2 For no-, -e -ill #ocus 9ust on simple instances such as Iep*t$l, -hich only pro!ide one representation2
*asIeps

Arguments
0ot e!ery route is as simple as the *o$eI -e 9ust de#ined2 Take #or instance our PersonI route #rom earlier2 The name o# the person needs to be passed to the handler #unction2 This translation is !ery straight1#or-ard, and hope#ully intuiti!e2 For e(ample)
$kHesod 8)r!s8 :parseIoutes| .person.4Text PersonI G-T .year.4Inte!er.$onth.4Text.day.4Int 1ateI .wiki.WTexts CikiI G-T |; !etPersonI == Text "< *andler Iep*t$l !etPersonI na$e = de#ault5ayout :wha$let|Gh(<*ello 43na$e7J|; handle1ateI == Inte!er "< Text "< Int "< *andler IepPlain "" text.plain handle1ateI year $onth day = return + IepPlain + toContent + T/concat :$onth> 8 8> T/pack + show day> 8> 8> T/pack + show year; !etCikiI == :Text; "< *andler IepPlain !etCikiI = return / IepPlain / toContent / T/unwords instance Hesod )r!s $ain = warp1ebu! &KKK )r!s

The arguments ha!e the types o# the dynamic pieces #or each route, in the order speci#ied2 lso, notice ho- -e are able to use both Iep*t$l and IepPlain2

The Handler Monad

The !ast ma9ority o# code you -rite in Yesod sits in the *andler monad2 I# you are approaching this #rom an &E$ 4&odel1Eie-1$ontroller6 background, your *andler code is the $ontroller2 Some important points to kno- about *andler)

AA

It is an instance o# MonadI,, so you can run any I/ action in your handlers -ith li#tI,2 By the -ay, li#tI, is e(ported by the Hesod module #or your con!enience2 5ike Cid!et, *andler is a #ake1monad1trans#ormer2 It -raps around a IesourceT I, monad2 We discuss this type at length in the conduits appendi(, but #or no-, -e%ll 9ust say it let%s you sa#ely allocate resources2 By I#akeI, I mean you can%t use the standard li#t #unction pro!ided by the trans#ormers package, you must use the Yesod1supplied one 49ust like -ith -idgets62 *andler is 9ust a type synonym around G*andler2 G*andler let%s you speci#y e(actly -hich subsite and master site you%re using2 The *andler synonym says that the sub and master sites are your application%s type2 *andler pro!ides a lot o# di##erent #unctionality, such as) o Pro!iding re3uest in#ormation2 o Peeping a list o# the e(tra response headers you%!e added2 o llo-ing you to modi#y the user%s session2 o Short1circuiting responses, #or redirecting, sending static #iles, or reporting errors2

The remainder o# this chapter -ill gi!e a brie# introduction to some o# the most common #unctions li!ing in the *andler monad2 I am speci#ically not co!ering any o# the session #unctions7 that -ill be addressed in the sessions chapter2

Application Information
There are a number o# #unctions that return in#ormation about your application as a -hole, and gi!e no in#ormation about indi!idual re3uests2 Some o# these are) getYesod Returns your applicaton #oundation !alue2 I# you store con#iguration !alues in your #oundation, you -ill probably end up using this #unction a lot2 getYesodSub ;et the subsite #oundation !alue2 8nless you are -orking in a subsite, this -ill return the same !alue as !etHesod2 get8rlRender Returns the 8R5 rendering #unction, -hich con!erts a type1sa#e 8R5 into a Text2 &ost o# the time1 like -ith Hamlet1 Yesod calls this #unction #or you, but you may occassionally need to call it directly2 get8rlRenderParams !ariant o# !et6rlIender that con!erts both a type1sa#e 8R5 and a list o# 3uery1 string parameters2 This #unction handles all percent1encoding necessary2

%e.uest Information
The most common in#ormation you -ill -ant to get about the current re3uest is the re3uested path, the 3uery string parameters and P/STed #orm data2 The #irst o# those is dealt -ith in the routing, as described abo!e2 The other t-o are best dealt -ith using the #orms module2 That said, you -ill sometimes need to get the data in a more ra- #ormat2 For this purpose, Yesod e(poses the IeEuest datatype along -ith the !etIeEuest #unction to retrie!e it2 This AD

gi!es you access to the #ull list o# ;"T parameters, cookies, and pre#erred languages2 There are some con!enient #unctions to make these lookups easier, such as lookupGetPara$, lookupCookie and lan!ua!es2 For ra- access to the P/ST parameters, you should use runIeEuest2 I# you need e!en more ra- data, like re3uest headers, you can use waiIeEuest to access the Web pplication Inter#ace 4W I6 re3uest !alue2 See the W I appendi( #or more details2

Short Circuiting
The #ollo-ing #unctions immediately end e(ecution o# a handler #unction and return a result to the user2 redirect Sends a redirect response to the user 4a >H> response62 I# you -ant to use a di##erent response code 4e2g2, a permanent >H* redirect6, you can use redirectCith2Yesod uses a >H> response #or HTTP<*2* clients, and a >H: response #or HTTP<*2H clients2 You can read up on this sordid saga in the HTTP spec2 notFound Return a @H@ response2 This can be use#ul i# a user re3uests a database !alue that doesn%t e(ist2 permissionDenied Return a @H> response -ith a speci#ic error message2 in!alid rgs @HH response -ith a list o# in!alid arguments2 sendFile Sends a #ile #rom the #ilesystem -ith a speci#ied content type2 This is the pre#erred -ay to send static #iles, since the underlying W I handler may be able to optimi'e this to a send#ile system call2 8sing read?ile #or sending static #iles should not be necessary2 sendResponse Send a normal *asIeps response -ith a :HH status code2 This is really 9ust a con!enience #or -hen you need to break out o# some deeply nested code -ith an immediate response2 sendWaiResponse When you need to get lo-1le!el and send out a ra- W I response2 This can be especially use#ul #or creating streaming responses or a techni3ue like ser!er1sent e!ents2

%esponse Headers
set$ookie Set a cookie on the client2 Instead o# taking an e(piration date, this #unction takes a cookie duration in minutes2 Remember, you -on%t see this cookie using lookupCookie until the following re3uest2 delete$ookie Tells the client to remo!e a cookie2 /nce again, lookupCookie -ill not re#lect this change until the ne(t re3uest2 AF

setHeader Set an arbitrary response header2 set5anguage Set the pre#erred user language, -hich -ill sho- up in the result o# the lan!ua!es #unction2 cacheSeconds Set a $ache1$ontrol header to indicate ho- many seconds this response can be cached2 This can be particularly use#ul i# you are using !arnish on your ser!er2 ne!er"(pires Set the "(pires header to the year :H>F2 You can use this -ith content -hich should ne!er e(pire, such as -hen the re3uest path has a hash !alue associated -ith it2 already"(pired Sets the "(pires header to the past2 e(pires t Sets the "(pires header to the speci#ied date<time2

Summary
Routing and dispatch is arguably the core o# Yesod) it is #rom here that our type1sa#e 8R5s are de#ined, and the ma9ority o# our code is -ritten -ithin the *andler monad2 This chapter co!ered some o# the most important and central concepts o# Yesod, so it is important that you properly digest it2 This chapter also hinted at a number o# more comple( Yesod topics that -e -ill be co!ering later2 But you should be able to -rite some !ery sophisticated -eb applications -ith 9ust the kno-ledge you ha!e learned up until here2

!orms
I%!e mentioned the boundary issue already) -hene!er data enters or lea!es an application, -e need to !alidate it2 Probably the most di##icult place this occurs is #orms2 $oding #orms is comple(7 in an ideal -orld, -e%d like a solution that addresses the #ollo-ing problems)

"nsure data is !alid2 &arshal string data in the #orm submission to Haskell datatypes2 ;enerate HT&5 code #or displaying the #orm2 ;enerate .a!ascript to do clientside !alidation and pro!ide more user1#riendly -idgets, such as date pickers2 Build up more comple( #orms by combining together simpler #orms2 utomatically assign names to our #ields that are guaranteed to be uni3ue2

The yesod1#orm package pro!ides all these #eatures in a simple, declarati!e PI2 It builds on top o# Yesod%s -idgets to simpli#y styling o# #orms and applying .a!ascript appropriately2 nd like the rest o# Yesod, it uses Haskell%s type system to make sure e!erything is -orking correctly2

A+

Synopsis
3"4 5) G6)G- FuasiFuotes> Te$plate*askell> MultiPara$TypeClasses> ,%erloaded9trin!s> Type?a$ilies 4"7 i$port Hesod i$port Hesod/?or$/JEuery i$port 1ata/Ti$e @1ayA i$port 1ata/Text @TextA i$port Control/)pplicati%e @@G+<A> @GW<AA data 9ynopsis = 9ynopsis $kHesod 89ynopsis8 :parseIoutes| . IootI G-T .person PersonI P,9T |; instance Hesod 9ynopsis "" Tells our application to use the standard -n!lish $essa!es/ "" I# you want i(Dn> then you can supply a translatin! #unction instead/ instance IenderMessa!e 9ynopsis ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e "" )nd tell us where to #ind the SFuery libraries/ CePll Sust use the de#aults> "" which point to the Goo!le C1 / instance HesodJEuery 9ynopsis "" The datatype we wish to recei%e #ro$ the #or$ data Person = Person 3 person a$e == Text > personBirthday == 1ay > person?a%oriteColor == Maybe Text > person-$ail == Text > personCebsite == Maybe Text 7 deri%in! 9how "" 1eclare the #or$/ The type si!nature is a bit inti$idatin!> but herePs the "" o%er%iew= "" "" W The *t$l para$eter is used #or encodin! so$e extra in#or$ation/ 9ee the "" discussion re!ardin! run?or$Get and run?or$Post below #or #urther "" explanation/ "" "" W Ce ha%e the sub and $aster site types> as usual/ "" "" W ?or$Iesult can be in three states= ?or$Missin! @no data a%ailableA> "" ?or$?ailure @in%alid dataA and ?or$9uccess "" "" W The Cid!et is the %iewable #or$ to place into the web pa!e/ "" "" ote that the sca##olded site pro%ides a con%enient ?or$ type synony$> "" so that our si!nature could be written as= "" "" < person?or$ == ?or$ Person

AB

"" "" ?or our purposes> itPs !ood to see the lon! %ersion/ person?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Person> Cid!etA person?or$ = render1i%s + Person G+< areE text?ield 8 a$e8 othin! GW< areE @SEuery1ay?ield de# 3 SdsChan!eHear = True "" !i%e a year dropdown > SdsHearIan!e = 8(\KK="M8 "" (\KK till #i%e years a!o 7A 8Birthday8 othin! GW< aopt text?ield 8?a%orite color8 othin! GW< areE e$ail?ield 8-$ail address8 othin! GW< aopt url?ield 8Cebsite8 othin! "" The G-T handler displays the #or$ !etIootI == *andler Iep*t$l !etIootI = do "" Generate the #or$ to be displayed @wid!et> enctypeA G" !enerate?or$Post person?or$ de#ault5ayout :wha$let| Gp<The wid!et !enerated contains only the contents o# the #or$> not the #or$ ta! itsel#/ 9o/// G#or$ $ethod=post action=O3PersonI7 enctype=43enctype7< Q3wid!et7 Gp<It also doesnPt include the sub$it button/ Ginput type=sub$it< |; "" The P,9T handler processes the #or$/ I# it is success#ul> it displays the "" parsed person/ ,therwise> it displays the #or$ a!ain with error $essa!es/ postPersonI == *andler Iep*t$l postPersonI = do @@result> wid!etA> enctypeA G" run?or$Post person?or$ case result o# ?or$9uccess person "< de#ault5ayout :wha$let|Gp<43show person7|; B "< de#ault5ayout :wha$let| Gp<In%alid input> letPs try a!ain/ G#or$ $ethod=post action=O3PersonI7 enctype=43enctype7< Q3wid!et7 Ginput type=sub$it< |; $ain == I, @A $ain = warp1ebu! &KKK 9ynopsis

7inds of !orms
Be#ore 9umping into the types themsel!es, -e should begin -ith an o!er!ie- o# the di##erent kinds o# #orms2 There are three categories) pplicati!e These are the most commonly used 4it%s -hat appeared in the synopsis62 pplicati!e gi!es us some nice properties o# letting error messages coallesce together and keep a !ery high1le!el, declarati!e approach2 4For more in#ormation on applicati!e code, see the Haskell -iki26 &onadic DH

more po-er#ul alternati!e to applicati!e2 While this allo-s you more #le(ibility, it does so at the cost o# being more !erbose2 8se#ul i# you -ant to create #orms that don%t #it into the standard t-o1column look2 Input 8sed only #or recei!ing input2 Does not generate any HT&5 #or recei!ing the user input2 8se#ul #or interacting -ith e(isting #orms2 In addition, there are a number o# di##erent !ariables that come into play #or each #orm and #ield you -ill -ant to set up)

Is the #ield re3uired or optional= Should it be submitted -ith ;"T or P/ST= Does it ha!e a de#ault !alue, or not=

n o!erriding goal is to minimi'e the number o# #ield de#initions and let them -ork in as many conte(ts as possible2 /ne result o# this is that -e end up -ith a #e- e(tra -ords #or each #ield2 In the synopsis, you may ha!e noticed things like areE and that e(tra othin! parameter2 We%ll co!er -hy all o# those e(ist in the course o# this chapter, but #or no- reali'e that by making these parameters e(plicit, -e are able to reuse the indi!iduals #ields 4like intField6 in many di##erent -ays2 3uick note on naming con!entions2 "ach #orm type has a one1letter pre#i( 4 , & and I6 -hich is used in a #e- places, such as saying &Form2 We also use re3 and opt to mean re3uired and optional2 $ombining these, -e create a re3uired applicati!e #ield -ith areE, or an optional input #ield -ith iopt2

Types
The Yesod2Form2Types module declares a #e- types2 5et%s start o## -ith some simple helpers) "nctype The encoding type, either 6rl-ncoded or Multipart2 This datatype declares an instance o# To*t$l, so you can use the enctype directly in Hamlet2 "n! &aps a parameter name to a list o# !alues2 File"n! &aps a parameter name to the associated uploaded #ile2 Ints s mentioned in the introduction, yesod"#or$ automatically assigns a uni3ue name to each #ield2 Ints is used to keep track o# the ne(t number to assign2 FormResult Has one o# three possible states) ?or$Missin! i# no data -as submitted, ?or$?ailure i# there -as an error parsing the #orm 4e2g2, missing a re3uired #ield, in!alid content6, or ?or$9uccess i# e!erything -ent smoothly2 0e(t -e ha!e three datatypes used #or de#ining indi!idual #ields2 #ield is a single piece o# in#ormation, such as a number, a string or an email address2 Fields are combined together to build #orms2 D*

Field De#ines t-o pieces o# #unctionality) ho- to parse the te(t input #rom a user into a Haskell !alue, and ho- to create the -idget to be displayed to the user2 yesod"#or$ de#ines a number o# indi!idual Fields in Yesod2Form2Fields2 FieldSettings Basic in#ormation on ho- a #ield should be displayed, such as the display name, an optional tooltip, and possibly hardcoded id and na$e attributes2 4I# none are pro!ided, they are automatically generated26?ield9ettin!s pro!ides an Is9trin! instance, so -hen you need to pro!ide a ?ield9ettin!s !alue, you can actually type in a literal string2 That%s ho- -e interacted -ith it in the synopsis2 FieldEien intermediate #ormat containing a bunch o# !ie- in#ormation on a #ield2 This is hardly e!er used directly by the user, -e%ll see more details later2 nd #inally, -e get to the important stu##) the #orms themsel!es2 There are three types #or this) M?or$ is #or monadic #orms, )?or$ #or applicati!e and I?or$ 4declared in IForm6 #or input2 M?or$ is actually a type synonym #or a monad stack that pro!ides the #ollo-ing #eatures)
Ieader monad gi!ing us the parameters 4-n% and ?ile-n%6, the master site argument and the list o# languages the user supports2 The last t-o are used #or i*+n 4more on this later62 Criter monad keeping track o# the -nctype2 #orm -ill al-ays be 6rl-ncoded, unless there is a #ile input #ield, -hich -ill #orce us to use multipart instead2 9tate monad holding an Ints to keep track o# the ne(t uni3ue name to produce2

n )?or$ is pretty similar2 Ho-e!er, there are a #e- ma9or di##erences)

It produces a list o# ?ieldViews2 This allo-s us to keep an abstract idea o# the #orm display, and then at the end o# the day choose an appropriate #unction #or laying it out on the page2 In the synopsis, -e used render1i%s, -hich creates a bunch o# di! tags2 nother option -ould be renderTable2 It does not pro!ide a Monad instance2 The goal o# )pplicati%e is to allo- the entire #orm to run, grab as much in#ormation on each #ield as possible, and then create the #inal result2 This cannot -ork in the conte(t o# Monad2

n I?or$ is e!en simpler) it returns either a list o# error messages or a result2

Con erting
IBut -ait a minute,I you say2 IYou said the synopsis uses applicati!e #orms, but I%m sure the type signature said M?or$2 Shouldn%t it be &onadic=I That%s true, the #inal #orm -e produced -as monadic2 But -hat really happened is that -e con!erted an applicati!e #orm to a monadic one2 gain, our goal is to reuse code as much as possible, and minimi'e the number o# #unctions in the PI2 nd &onadic #orms are more po-er#ul than pplicati!e, i# more clumsy, so anything that can be e(pressed in an pplicati!e #orm could also be e(pressed in a &onadic

D:

#orm2 There are t-o core #unctions that help out -ith this) a#or$To?or$ con!erts any applicati!e #orm to a monadic one, and #or$To)?or$ con!erts certain kinds o# monadic #orms to applicati!e #orms2 IBut -ait another minute,I you insist2 II didn%t see any a#or$To?or$?I lso true2 The render1i%s #unction takes care o# that #or us2

Create A!orms
0o- that I%!e 4hope#ully6 con!inced you that in our synopsis -e -ere really dealing -ith applicati!e #orms, let%s ha!e a look and try to understand ho- these things get created2 5et%s take a simple e(ample)
data Car = Car 3 carModel == Text > carHear == Int 7 deri%in! 9how car)?or$ == )?or$ 9ynopsis 9ynopsis Car car)?or$ = Car G+< areE text?ield 8Model8 othin! GW< areE int?ield 8Hear8 othin! car?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Car> Cid!etA car?or$ = renderTable car)?or$ !etCarI == *andler Iep*t$l !etCarI = do @@result> wid!etA> enctypeA G" run?or$Get car?or$ case result o# ?or$9uccess car "< de#ault5ayout :wha$let|Gp<43show car7|; B "< de#ault5ayout :wha$let| G#or$ $ethod=!et action=O3CarI7 enctype=43enctype7< Gtable< Q3wid!et7 Ginput type=sub$it< |;

Here, -e%!e e(plicitly split up applicati!e and monadic #orms2 In car)?or$, -e use the G+< and GW< operators2 This should not be surprising7 these are almost al-ays used in applicati!e1 style code2 nd -e ha!e one line #or each record in our Car datatype2 Perhaps unsurprisingly, -e ha!e a text?ield #or the Text record, and an int?ield #or the Int record2 5et%s look a bit more closely at the areE #unction2 Its 4simpli#ied6 type signature is ?ield a "< ?ield9ettin!s "< Maybe a "< )?or$ a2 So that #irst argument is going to determine the datatype o# this #ield, ho- to parse it, and ho- to render it2 The ne(t argument, ?ield9ettin!s, tells us the label, tooltip, name and ID o# the #ield2 In this case, -e%re using the pre!iously1mentioned Is9trin! instance o# ?ield9ettin!s2 nd -hat%s up -ith that Maybe a= It pro!ides the optional de#ault !alue2 For e(ample, i# -e -ant our #orm to #ill in I:HHFI as the de#ault car year, -e -ould use areE int?ield 8Hear8

D>

@Just NKK[A2

We can e!en take this to the ne(t le!el, and ha!e a #orm that takes an optional parameter gi!ing the de#ault !alues2

!orm 3ith default alues

car)?or$ == Maybe Car "< )?or$ 9ynopsis 9ynopsis Car car)?or$ $car = Car G+< areE text?ield 8Model8 @carModel G+< $carA GW< areE int?ield 8Hear8 @carHear G+< $carA car?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Car> Cid!etA car?or$ = renderTable @car)?or$ + Just + Car 8?orte8 NK(KA !etCarI == *andler Iep*t$l !etCarI = do @@result> wid!etA> enctypeA G" run?or$Get car?or$ case result o# ?or$9uccess car "< de#ault5ayout :wha$let|Gp<43show car7|; B "< de#ault5ayout :wha$let| G#or$ $ethod=!et action=O3CarI7 enctype=43enctype7< Gtable< Q3wid!et7 Ginput type=sub$it< |;

Optional fields
Suppose -e -anted to ha!e an optional #ield 4like the car color62 ll -e do instead is use the aopt #unction2

Optional fields
data Car = Car 3 carModel == Text > carHear == Int > carColor == Maybe Text 7 deri%in! 9how car)?or$ == )?or$ 9ynopsis 9ynopsis Car car)?or$ = Car G+< areE text?ield 8Model8 othin! GW< areE int?ield 8Hear8 othin! GW< aopt text?ield 8Color8 othin! car?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Car> Cid!etA car?or$ = renderTable car)?or$ !etCarI == *andler Iep*t$l !etCarI = do @@result> wid!etA> enctypeA G" run?or$Get car?or$ case result o# ?or$9uccess car "< de#ault5ayout :wha$let|Gp<43show car7|; B "< de#ault5ayout :wha$let|

D@

G#or$ $ethod=!et action=O3CarI7 enctype=43enctype7< Gtable< Q3wid!et7 Ginput type=sub$it< |;

nd like re3uired #ields, the last argument is the optional de#ault !alue2 Ho-e!er, this has t-o layers o# &aybe -rapping2 This may seem redundant 4and it is6, but it makes it much easier to -rite code that takes an optional de#ault #orm parameter, such as in the ne(t e(ample2

(efault optional fields

data Car = Car 3 carModel == Text > carHear == Int > carColor == Maybe Text 7 deri%in! 9how car)?or$ == Maybe Car "< )?or$ car)?or$ $car = Car G+< areE text?ield 8Model8 GW< areE int?ield 8Hear8 GW< aopt text?ield 8Color8 9ynopsis 9ynopsis Car @carModel G+< $carA @carHear G+< $carA @carColor G+< $carA

car?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Car> Cid!etA car?or$ = renderTable + car)?or$ + Just + Car 8?orte8 NK(K + Just 8!ray8 !etCarI == *andler Iep*t$l !etCarI = do @@result> wid!etA> enctypeA G" run?or$Get car?or$ case result o# ?or$9uccess car "< de#ault5ayout :wha$let|Gp<43show car7|; B "< de#ault5ayout :wha$let| G#or$ $ethod=!et action=O3CarI7 enctype=43enctype7< Gtable< Q3wid!et7 Ginput type=sub$it< |;

8alidation
Ho- -ould -e make our #orm only accept cars created a#ter *BBH= I# you remember, -e said abo!e that the ?ield itsel# contained the in#ormation on -hat is a !alid entry2 So all -e need to do is -rite a ne- ?ield, right= Well, that -ould be a bit tedious2 Instead, let%s 9ust modi#y an e(isting one)
car)?or$ == Maybe Car "< )?or$ 9ynopsis 9ynopsis Car car)?or$ $car = Car G+< areE text?ield 8Model8 @carModel G+< $carA GW< areE carHear?ield 8Hear8 @carHear G+< $carA GW< aopt text?ield 8Color8 @carColor G+< $carA where errorMessa!e == Text

DA

errorMessa!e = 8Hour car is too old> !et a new oneJ8 carHear?ield = check %alidateHear int?ield %alidateHear y | y G (\\K = 5e#t errorMessa!e | otherwise = Ii!ht y car?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Car> Cid!etA car?or$ = renderTable + car)?or$ + Just + Car 8?orte8 NK(K + Just 8!ray8 !etCarI == *andler Iep*t$l !etCarI = do @@result> wid!etA> enctypeA G" run?or$Get car?or$ case result o# ?or$9uccess car "< de#ault5ayout :wha$let|Gp<43show car7|; B "< de#ault5ayout :wha$let| G#or$ $ethod=!et action=O3CarI7 enctype=43enctype7< Gtable< Q3wid!et7 Ginput type=sub$it< |;

The trick here is the check #unction2 It takes a #unction 4%alidateHear6 that returns either an error message or a modi#ied #ield !alue2 In this e(ample, -e ha!en%t modi#ied the !alue at all2 That is usually going to be the case2 This kind o# checking is !ery common, so -e ha!e a shortcut)
carHear?ield = checkBool @<= (\\KA errorMessa!e int?ield car?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Car> Cid!etA car?or$ = renderTable + car)?or$ + Just + Car 8?orte8 NK(K + Just 8!ray8 !etCarI == *andler Iep*t$l !etCarI = do @@result> wid!etA> enctypeA G" run?or$Get car?or$ case result o# ?or$9uccess car "< de#ault5ayout :wha$let|Gp<43show car7|; B "< de#ault5ayout :wha$let| G#or$ $ethod=!et action=O3CarI7 enctype=43enctype7< Gtable< Q3wid!et7 Ginput type=sub$it< |;

takes t-o parameters) a condition that must be #ul#illed, and an error message to be displayed i# it -as not2
checkBool

You may ha!e noticed the e(plicit Text type signature on errorMessa!e2 In the presence o# ,%erloaded9trin!s, this is necessary2 In order to support i*+n, messages can ha!e many di##erent datatypes, and ;H$ has no -ay o# determining -hich instance o# Is9trin! you intended to use2 It%s great to make sure the car isn%t too old2 But -hat i# -e -ant to make sure that the year speci#ied is not #rom the #uture= In order to look up the current year, -e%ll need to run some I,2 For such circumstances, -e%ll need checkM)

DD

carHear?ield = checkM inPast + checkBool @<= (\\KA errorMessa!e int?ield inPast y = do thisHear G" li#tI, !etCurrentHear return + i# y G= thisHear then Ii!ht y else 5e#t @8Hou ha%e a ti$e $achineJ8 == TextA !etCurrentHear == I, Int !etCurrentHear = do now G" !etCurrentTi$e let today = utct1ay now let @year> B> BA = toGre!orian today return + #ro$Inte!er year car?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Car> Cid!etA car?or$ = renderTable + car)?or$ + Just + Car 8?orte8 NK(K + Just 8!ray8 !etCarI == *andler Iep*t$l !etCarI = do @@result> wid!etA> enctypeA G" run?or$Get car?or$ case result o# ?or$9uccess car "< de#ault5ayout :wha$let|Gp<43show car7|; B "< de#ault5ayout :wha$let| G#or$ $ethod=!et action=O3CarI7 enctype=43enctype7< Gtable< Q3wid!et7 Ginput type=sub$it< |;

is a #unction that -ill return an -ither result2 Ho-e!er, it uses a *andler monad2 We use li#tI, !etCurrentHear to get the current year and then compare it against the user1supplied year2 lso, notice ho- -e can chain together multiple !alidators2
inPast

Since the checkM !alidator runs in the *andler monad, it has access to a lot o# the stu## you can normally do in Yesod2 This is especially use#ul #or running database actions, -hich -e%ll co!er in the Persistent chapter2

More sophisticated fields

/ur color entry #ield is nice, but it%s not e(actly user1#riendly2 What -e really -ant is a drop1 do-n list2

(rop&do3n lists
data Car = Car 3 carModel == Text > carHear == Int > carColor == Maybe Color 7 deri%in! 9how data Color = Ied | Blue | Gray | Black deri%in! @9how> -E> -nu$> BoundedA

DF

car)?or$ == Maybe Car "< )?or$ 9ynopsis 9ynopsis Car car)?or$ $car = Car G+< areE text?ield 8Model8 @carModel G+< $carA GW< areE carHear?ield 8Hear8 @carHear G+< $carA GW< aopt @select?ield5ist colorsA 8Color8 @carColor G+< $carA where colors == :@Text> ColorA; colors = :@8Ied8> IedA> @8Blue8> BlueA> @8Gray8> GrayA> @8Black8> BlackA; errorMessa!e == Text errorMessa!e = 8Hour car is too old> !et a new oneJ8 carHear?ield = checkM inPast + checkBool @<= (\\KA errorMessa!e int?ield inPast y = do thisHear G" li#tI, !etCurrentHear return + i# y G= thisHear then Ii!ht y else 5e#t @8Hou ha%e a ti$e $achineJ8 == TextA !etCurrentHear == I, Int !etCurrentHear = do now G" !etCurrentTi$e let today = utct1ay now let @year> B> BA = toGre!orian today return + #ro$Inte!er year car?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Car> Cid!etA car?or$ = renderTable + car)?or$ + Just + Car 8?orte8 NK(K + Just Black !etCarI == *andler Iep*t$l !etCarI = do @@result> wid!etA> enctypeA G" run?or$Get car?or$ case result o# ?or$9uccess car "< de#ault5ayout :wha$let|Gp<43show car7|; B "< de#ault5ayout :wha$let| G#or$ $ethod=!et action=O3CarI7 enctype=43enctype7< Gtable< Q3wid!et7 Ginput type=sub$it< |;

takes a list o# pairs2 The #irst item in the pair is the te(t displayed to the user in the drop1do-n list, and the second item is the actual Haskell !alue2 /# course, the code abo!e looks really repetiti!e7 -e can get the same result using the "num and Bounded instance ;H$ automatically deri!es #or us2
select?ield5ist

'ses /num and #ounded

data Car = Car 3 carModel == Text > carHear == Int > carColor == Maybe Color 7 deri%in! 9how

D+

data Color = Ied | Blue | Gray | Black deri%in! @9how> -E> -nu$> BoundedA car)?or$ == Maybe Car "< )?or$ 9ynopsis 9ynopsis Car car)?or$ $car = Car G+< areE text?ield 8Model8 @carModel G+< $carA GW< areE carHear?ield 8Hear8 @carHear G+< $carA GW< aopt @select?ield5ist colorsA 8Color8 @carColor G+< $carA where colors = $ap @pack / show TTT idA + :$inBound//$axBound; errorMessa!e == Text errorMessa!e = 8Hour car is too old> !et a new oneJ8 carHear?ield = checkM inPast + checkBool @<= (\\KA errorMessa!e int?ield inPast y = do thisHear G" li#tI, !etCurrentHear return + i# y G= thisHear then Ii!ht y else 5e#t @8Hou ha%e a ti$e $achineJ8 == TextA !etCurrentHear == I, Int !etCurrentHear = do now G" !etCurrentTi$e let today = utct1ay now let @year> B> BA = toGre!orian today return + #ro$Inte!er year car?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Car> Cid!etA car?or$ = renderTable + car)?or$ + Just + Car 8?orte8 NK(K + Just Black !etCarI == *andler Iep*t$l !etCarI = do @@result> wid!etA> enctypeA G" run?or$Get car?or$ case result o# ?or$9uccess car "< de#ault5ayout :wha$let|Gp<43show car7|; B "< de#ault5ayout :wha$let| G#or$ $ethod=!et action=O3CarI7 enctype=43enctype7< Gtable< Q3wid!et7 Ginput type=sub$it< |; :$inBound//$axBound; gi!es us a list o# all the di##erent Color !alues2 and TTT 4a2k2a, the #an1out operator6 to turn that into a list o# pairs2

We then apply a $ap

Some people pre#er radio buttons to drop1do-n lists2 Fortunately, this is 9ust a one1-ord change2 For e(ample, see Radio buttons

%adio -uttons
data Car = Car 3 carModel == Text > carHear == Int > carColor == Maybe Color 7 deri%in! 9how

DB

data Color = Ied | Blue | Gray | Black deri%in! @9how> -E> -nu$> BoundedA car)?or$ == Maybe Car "< )?or$ 9ynopsis 9ynopsis Car car)?or$ $car = Car G+< areE text?ield 8Model8 @carModel G+< $carA GW< areE carHear?ield 8Hear8 @carHear G+< $carA GW< aopt @radio?ield5ist colorsA 8Color8 @carColor G+< $carA where colors = $ap @pack / show TTT idA + :$inBound//$axBound; errorMessa!e == Text errorMessa!e = 8Hour car is too old> !et a new oneJ8 carHear?ield = checkM inPast + checkBool @<= (\\KA errorMessa!e int?ield inPast y = do thisHear G" li#tI, !etCurrentHear return + i# y G= thisHear then Ii!ht y else 5e#t @8Hou ha%e a ti$e $achineJ8 == TextA !etCurrentHear == I, Int !etCurrentHear = do now G" !etCurrentTi$e let today = utct1ay now let @year> B> BA = toGre!orian today return + #ro$Inte!er year car?or$ == *t$l "< M?or$ 9ynopsis 9ynopsis @?or$Iesult Car> Cid!etA car?or$ = renderTable + car)?or$ + Just + Car 8?orte8 NK(K + Just Black !etCarI == *andler Iep*t$l !etCarI = do @@result> wid!etA> enctypeA G" run?or$Get car?or$ case result o# ?or$9uccess car "< de#ault5ayout :wha$let|Gp<43show car7|; B "< de#ault5ayout :wha$let| G#or$ $ethod=!et action=O3CarI7 enctype=43enctype7< Gtable< Q3wid!et7 Ginput type=sub$it< |;

%unning forms
t some point, -e%re going to need to take our beauti#ul #orms and produce some results2 There are a number o# di##erent #unctions a!ailable #or this, each -ith its o-n purpose2 I%ll go through them, starting -ith the most common2 runFormPost This -ill run your #orm against any submitted P,9T parameters2 I# this is not a P,9T submission, it -ill return a ?or$Missin!2 This automatically inserts a security token as a hidden #orm #ield to a!oid $SRF attacks2 runForm;et

FH

Same as run?or$Post, #or ;"T parameters2 In order to distinguish a normal G-T page load #rom a G-T submission, it includes an e(tra Bhasdata hidden #ield in the #orm2 runFormPost0o0once Same as run?or$Post, but does not include 4or re3uire6 the $SRF security token2 generateFormPost Instead o# binding to e(isting P,9T parameters, acts as i# there are none2 This can be use#ul -hen you -ant to generate a ne- #orm a#ter a pre!ious #orm -as submitted, such as in a -i'ard2 generateForm;et Same as !enerate?or$Post, but #or G-T2 The return type #rom the #irst three is @@?or$Iesult a> Cid!etA> -nctypeA2 The Cid!et -ill already ha!e any !alidation errors and pre!iously submitted !alues2

i9:n
There ha!e been a #e- re#erences to i*+n in this chapter2 The topic -ill get more thorough co!erage in its o-n chapter, but since it has such a pro#ound e##ect on yesod"#or$, I -anted to gi!e a brie# o!er!ie-2 The idea behind i*+n in Yesod is to ha!e data types represent messages2 "ach site can ha!e an instance o# IenderMessa!e #or a gi!en datatype -hich -ill translate that message based on a list o# languages the user accepts2 s a result o# all this, there are a #e- things you should be a-are o#)

There is an automatic instance o# IenderMessa!e #or Text in e!ery site, so you can 9ust use plain strings i# you don%t care about i*+n support2 Ho-e!er, you may need to use e(plicit type signatures occassionally2 yesod"#or$ e(presses all o# its messages in terms o# the ?or$Messa!e datatype2 There#ore, to use yesod"#or$, you%ll need to ha!e an appropriate IenderMessa!e instance2 simple one that uses the de#ault "nglish translations -ould be)
instance IenderMessa!e My)pp ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e

This is pro!ided automatically by the sca##olded site2

Monadic !orms
/#ten times, a simple #orm layout is ade3uate, and applicati!e #orms e(cel at this approach2 Sometimes, ho-e!er, you%ll -ant to ha!e a more customi'ed look to your #orm2

A non&standard form layout

For these use cases, monadic #orms #it the bill2 They are a bit more !erbose than their applicati!e cousins, but this !erbosity allo-s you to ha!e complete control o!er -hat the #orm -ill look like2 In order to generate the #orm abo!e, -e could code something like this2 F*

3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> FuasiFuotes> Te$plate*askell> MultiPara$TypeClasses 4"7 i$port Hesod i$port Control/)pplicati%e i$port 1ata/Text @TextA data M?or$-xa$ple = M?or$-xa$ple $kHesod 8M?or$-xa$ple8 :parseIoutes| . IootI G-T |; instance Hesod M?or$-xa$ple instance IenderMessa!e M?or$-xa$ple ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e data Person = Person 3 person a$e == Text> person)!e == Int 7 deri%in! 9how person?or$ == *t$l "< M?or$ M?or$-xa$ple M?or$-xa$ple @?or$Iesult Person> Cid!etA person?or$ extra = do @na$eIes> na$eViewA G" $reE text?ield 8this is not used8 othin! @a!eIes> a!eViewA G" $reE int?ield 8neither is this8 othin! let personIes = Person G+< na$eIes GW< a!eIes let wid!et = do toCid!et :lucius| 443#%Id a!eView7 3 width= &e$R 7 |; :wha$let| 43extra7 Gp< *ello> $y na$e is 4 Q3#%Input na$eView7 2 and I a$ 4 Q3#%Input a!eView7 2 years old/ 4 Ginput type=sub$it %alue=8Introduce $ysel#8< |; return @personIes> wid!etA !etIootI == *andler Iep*t$l !etIootI = do @@res> wid!etA> enctypeA G" run?or$Get person?or$ de#ault5ayout :wha$let| Gp<Iesult= 43show res7 G#or$ enctype=43enctype7< Q3wid!et7 |; $ain == I, @A $ain = warp1ebu! &KKK M?or$-xa$ple

Similar to the applicati!e areE, -e use $reE #or monadic #orms2 4 nd yes, there%s also $opt #or optional #ields26 But there%s a big di##erence) $reE gi!es us back a pair o# !alues2 Instead o# hiding a-ay the FieldEie- !alue and automatically inserting it into a -idget, -e get the control to insert it as -e see #it2 F:

has a number o# pieces o# in#ormation2 The most important is #%Input, -hich is the actual #orm #ield2 In this e(ample, -e also use #%Id, -hich gi!es us back the HT&5 id attribute o# the input tag2 In our e(ample, -e use that to speci#y the -idth o# the #ield2
?ieldView

You might be -ondering -hat the story is -ith the Ithis is not usedI and Ineither is thisI !alues2 $reE takes a ?ield9ettin!s as its second argument2 Since ?ield9ettin!s pro!ides an Is9trin! instance, the strings are essentially e(panded by the compiler to)
#ro$9trin! 8this is not used8 == ?ield9ettin!s 3 #s5abel = 8this is not used8 > #sTooltip = othin! > #sId = othin! > #s a$e = othin! > #sClass = :; 7 In the case o# applicati!e #orms, the #s5abel and #sTooltip

!alues are used -hen constructing your HT&52 In the case o# monadic #orms, Yesod does not generate any o# the I-rapperI HT&5 #or you, and there#ore these !alues are ignored2 Ho-e!er, -e still keep the ?ield9ettin!s parameter to allo- you to o!erride the id and na$e attributes o# your #ields i# desired2 The other interesting bit is the extra !alue2 G-T #orms include an e(tra #ield to indicate that they ha!e been submitted, and P,9T #orms include a security tokens to pre!ent $SRF attacks2 I# you don%t include this e(tra hidden #ield in your #orm, Yesod -ill not accept it2 /ther than that, things are pretty straight1#or-ard2 We create our personIes !alue by combining together the na$eIes and a!eIes !alues, and then return a tuple o# the person and the -idget2 nd in the !etIootI #unction, e!erything looks 9ust like an applicati!e #orm2 In #act, you could s-ap out our monadic #orm -ith an applicati!e one and the code -ould still -ork2

Input forms
pplicati!e and monadic #orms handle both the generation o# your HT&5 code and the parsing o# user input2 Sometimes, you only -ant to do the latter, such as -hen there%s an already1e(isting #orm in HT&5 some-here, or i# you -ant to generate a #orm dynamically using .a!ascript2 In such a case, you%ll -ant input #orms2 These -ork mostly the same as applicati!e and monadic #orms, -ith some di##erences)

You use runInputPost and runInputGet2 You use ireE and iopt2 These #unctions no- only take t-o arguments) the #ield type and the name 4i2e2, HT&5 na$e attribute6 o# the #ield in 3uestion2 #ter running a #orm, it returns the !alue2 It doesn%t return a -idget or an encoding type2 I# there are any !alidation errors, the page returns an Iin!alid argumentsI error page2

You can use input #orms to recreate the pre!ious e(ample2 0ote, ho-e!er, that the input !ersion is less user #riendly2 I# you make a mistake in an applicati!e or monadic #orm, you -ill be brought back to the same page, -ith your pre!iously entered !alues in the #orm, and F>

an error message e(planing -hat you need to correct2 With input #orms, the user simply gets an error message2
3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> FuasiFuotes> Te$plate*askell> MultiPara$TypeClasses 4"7 i$port Hesod i$port Control/)pplicati%e i$port 1ata/Text @TextA data Input = Input $kHesod 8Input8 :parseIoutes| . IootI G-T .input InputI G-T |; instance Hesod Input instance IenderMessa!e Input ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e data Person = Person 3 person a$e == Text> person)!e == Int 7 deri%in! 9how !etIootI == *andler Iep*t$l !etIootI = de#ault5ayout :wha$let| G#or$ action=O3InputI7< Gp< My na$e is 4 Ginput type=text na$e=na$e< 2 and I a$ 4 Ginput type=text na$e=a!e< 2 years old/ 4 Ginput type=sub$it %alue=8Introduce $ysel#8< |; !etInputI == *andler Iep*t$l !etInputI = do person G" runInputGet + Person G+< ireE text?ield 8na$e8 GW< ireE int?ield 8a!e8 de#ault5ayout :wha$let|Gp<43show person7|; $ain == I, @A $ain = warp1ebu! &KKK Input

Custom fields
The #ields that come built1in -ith Yesod -ill likely co!er the !ast ma9ority o# your #orm needs2 But occassionally, you%ll need something more speciali'ed2 Fortunately, you can create ne- #orms in Yesod yoursel#2 The ?ield datatype has t-o records) #ieldParse takes a list o# !alues submitted by the user and returns one o# three results)

n error message saying !alidation #ailed2 The parsed !alue2 0othing, indicating that no data -as supplied2

F@

That last case might sound surprising) shouldn%t Yesod automatically kno- that no in#ormation is supplied -hen the input list is empty= Well, no actually2 $heckbo(es, #or instance, indicate an unchecked state by sending in an empty list2 lso, -hat%s up -ith the list= Shouldn%t it be a Maybe= Well, that%s also not the case2 With grouped checkbo(es and multi1select lists, you%ll ha!e multiple -idgets -ith the same name2 We also use this trick in our e(ample belo-2 The second record is #ieldView, and it renders a -idget to display to the user2 This #unction has #our arguments) the id attribute, the na$e attribute, the result and a Bool indicating i# the #ield is re3uired2 What did I mean by result= It%s actually an -ither, gi!ing either the unparsed input 4-hen parsing #ailed6 or the success#ully parsed !alue2 int?ield is a great e(ample o# ho- this -orks2 I# you type in @:, the !alue o# result -ill be Ii!ht LN2 But i# you type in turtle, the result -ill be 5e#t 8turtle82 This lets you put in a !alue attribute on your input tag that -ill gi!e the user a consistent e(perience2 s a small e(ample, -e%ll create a ne- #ield type that is a pass-ord con#irm #ield2 This #ield has t-o te(t inputs1 both -ith the same name attribute1 and returns an error message i# the !alues don%t match2 0ote that, unlike most #ields, it does not pro!ide a !alue attribute on the input tags, as you don%t -ant to send back a user1entered pass-ord in your HT&5 e er2
passwordCon#ir$?ield == ?ield sub $aster Text passwordCon#ir$?ield = ?ield 3 #ieldParse = 2rawVals "< case rawVals o# :a> b; | a == b "< return + Ii!ht + Just a | otherwise "< return + 5e#t 8Passwords donPt $atch8 :; "< return + Ii!ht othin! B "< return + 5e#t 8Hou $ust enter two %alues8 > #ieldView = 2id)ttr na$e)ttr B eIesult isIeE "< :wha$let| Ginput id=43id)ttr7 na$e=43na$e)ttr7 type=password< Gdi%<Con#ir$= Ginput id=43id)ttr7"con#ir$ na$e=43na$e)ttr7 type=password< |; 7 !etIootI == *andler Iep*t$l !etIootI = do @@res> wid!etA> enctypeA G" run?or$Get + render1i%s + areE passwordCon#ir$?ield 8Password8 othin! de#ault5ayout :wha$let| Gp<Iesult= 43show res7 G#or$ enctype=43enctype7< Q3wid!et7 Ginput type=sub$it %alue=8Chan!e password8< |; $ain == I, @A $ain = warp1ebu! &KKK Password

FA

Summary
Forms in Yesod are broken up into three groups2 pplicati!e is the most common, as it pro!ides a nice user inter#ace -ith an easy1to1use PI2 &onadic #orms gi!e you more po-er, but are harder to use2 Input #orms are intended -hen you 9ust -ant to read data #rom the user, not generate the input -idgets2 There are a number o# di##erent ?ields pro!ided by Yesod out1o#1the1bo(2 In order to use these in your #orms, you need to indicate the kind o# #orm and -hether the #ield is re3uired or optional2 The result is si( helper #unctions) areE, aopt, $reE, $opt, ireE, and iopt2 Forms ha!e signi#icant po-er a!ailable2 They can automatically insert .a!ascript to help you le!erage nicer 8I controls, such as a 9Guery 8I date picker2 Forms are also #ully i*+n1ready, so you can support a global community o# users2 nd -hen you ha!e more speci#ic needs, you can slap on some !alidation #unctions to an e(isting #ield, or -rite a ne- one #rom scratch2

Sessions
HTTP is a stateless protocol2 While some !ie- this as a disad!antage, ad!ocates o# R"ST#ul -eb de!elopment laud this as an plus2 When state is remo!ed #rom the picture, it is easier to scale applications, caching can happen automatically, and many other nice side e##ects occur2 You can dra- many parallels -ith the non1mutable nature o# Haskell in general2 s much as possible, R"ST#ul applications should a!oid storing state about an interaction -ith a client2 Ho-e!er, it is sometimes una!oidable2 Features like shopping carts are the classic e(ample, but other more mundane interactions like proper login handling can be greatly enhanced by proper usage o# sessions2 This chapter -ill describe ho- Yesod stores session data, ho- you can access this data, and some special #unctions to help you make the most o# sessions2

Clientsession
/ne o# the earliest packages spun o## #rom Yesod -as clientsession2 This package uses encryption and signatures to store data in a client1side cookie2 The encryption pre!ents the user #rom inspecting the data, and the signature ensures that the session can be neither hi9acked nor tampered -ith2 It might sound like a bad idea #rom an e##iciency standpoint to store data in a cookie) a#ter all, this means that the data must be sent on e!ery re3uest2 Ho-e!er, in practice, clientsession can be a great boon #or per#ormance2

0o ser!er side database lookup is re3uired to ser!ice a re3uest2 We can easily scale hori'ontally) each re3uest contains all the in#ormation -e need to send a response2

FD

To a!oid undue band-idth o!erhead, production sites can ser!e their static content #rom a separate domain name to a!oid the o!erhead o# transmitting the session cookie #or each re3uest2

Storing megabytes o# in#ormation in the session -ill be a bad idea2 But #or that matter, most session implementations recommend against such practices2 I# you really need massi!e storage #or a user, it is best to store a lookup key in the session, and put the actual data in a database2 ll o# the interaction -ith clientsession is handled by Yesod internally, but there are a #espots -here you can t-eak the beha!ior 9ust a bit2

Controlling sessions
There are three #unctions in the Yesod typeclass that control ho- sessions -ork2 encryptPey returns the encryption key used2 By de#ault, it -ill take this #rom a local #ile, so that sessions can persist bet-een database shutdo-ns2 This #ile -ill be automatically created and #illed -ith random data i# it does not e(ist2 nd i# you o!erride this #unction to return othin!, sessions -ill be disabled2 Why disable sessions= They do introduce a per#ormance o!erhead2 8nder normal circumstances, this o!erhead is minimal, especially compared to database access2 Ho-e!er, -hen dealing -ith !ery basic tasks, the o!erhead can become noticeable2 But be care#ul about disabling sessions) this -ill also disable such #eatures as $SRF 4$ross1Site Re3uest Forgery6 protection2 The ne(t #unction is clientSessionDuration2 This #unction gi!es the number o# minutes that a session should be acti!e2 The de#ault is *:H 4: hours62 This !alue ends up a##ecting the session cookie in t-o -ays) #irstly, it determines the e(piration date #or the cookie itsel#2 &ore importantly, ho-e!er, the session e(piration timestamp is encoded inside the session signature2 When Yesod decodes the signature, it checks i# the date is in the past7 i# so, it ignores the session !alues2 "!ery time Yesod sends a response to the client, it sends an updated session cookie -ith a ne- e(pire date2 This -ay, e!en i# you do not change the session !alues themsel!es, a session -ill not time out i# the user continues to bro-se your site2 nd this leads !ery nicely to the last #unction) sessionIp ddress2 By de#ault, Yesod also encodes the client%s IP address inside the cookie to pre!ent session hi9acking2 In general, this is a good thing2 Ho-e!er, some ISPs are kno-n #or putting their users behind pro(ies that re-rite their IP addresses, sometimes changing the source IP in the middle o# the session2 I# this happens, and you ha!e sessionIp)ddress enabled, the user%s session -ill be reset2 Turning this setting to ?alse -ill allo- a session to continue under such circumstances, at the cost o# e(posing a user to session hi9acking2

FF

Session Operations
5ike most #rame-orks, a session in Yesod is a key1!alue store2 The base session PI boils do-n to #our #unctions) lookup9ession gets a !alue #or a key 4i# a!ailable6, !et9ession returns all o# the key<!alue pairs, set9ession sets a !alue #or a key, and delete9ession clears a !alue #or a key2
3"4 5) G6)G- Type?a$ilies> FuasiFuotes> Te$plate*askell> MultiPara$TypeClasses> ,%erloaded9trin!s 4"7 i$port Hesod i$port Control/)pplicati%e @@G+<A> @GW<AA i$port Euali#ied Ceb/Client9ession as C9 data 9ession-xa$ple = 9ession-xa$ple $kHesod 89ession-xa$ple8 :parseIoutes| . Ioot G-T P,9T |; !etIoot == *andler Iep*t$l !etIoot = do sess G" !et9ession ha$letToIep*t$l :ha$let| G#or$ $ethod=post< Ginput type=text na$e=key< Ginput type=text na$e=%al< Ginput type=sub$it< Gh(<43show sess7 |; postIoot == *andler @A postIoot = do @key> $%alA G" runInputPost + @>A G+< ireE text?ield 8key8 GW< iopt text?ield 8%al8 case $%al o# othin! "< delete9ession key Just %al "< set9ession key %al li#tI, + print @key> $%alA redirect Ioot instance Hesod 9ession-xa$ple where "" Make the session ti$eout ( $inute so that itPs easier to play with $ake9essionBackend B = do key G" C9/!etXey C9/de#aultXey?ile return + Just + client9essionBackend key ( instance IenderMessa!e 9ession-xa$ple ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e $ain == I, @A $ain = warp1ebu! &KKK 9ession-xa$ple

Messages
/ne usage o# sessions pre!iously alluded to is messages2 They come to sol!e a common problem in -eb de!elopment) the user per#orms a P,9T re3uest, the -eb app makes a change, F+

and then the -eb app -ants to simultaneously redirect the user to a ne- page and send the user a success message2 4This is kno-n as Post<Redirect<;et26 Yesod pro!ides a pair o# #unctions to make this !ery easy) setMessa!e stores a !alue in the session, and !etMessa!e both reads the !alue most recently put into the session, and clears the old !alue so it does not accidently get displayed t-ice2 It is recommended to ha!e a call to !etMessa!e in de#ault5ayout so that any a!ailable message is sho-n to a user immediately, -ithout ha!ing to remember to add !etMessa!e calls to e!ery handler2
3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> Te$plate*askell> FuasiFuotes> MultiPara$TypeClasses 4"7 i$port Hesod data Messa!es = Messa!es $kHesod 8Messa!es8 :parseIoutes| . IootI G-T .set"$essa!e 9etMessa!eI P,9T |; instance Hesod Messa!es where de#ault5ayout wid!et = do pc G" wid!etToPa!eContent wid!et $$s! G" !etMessa!e ha$letToIep*t$l :ha$let| +doctype M Ght$l< Ghead< Gtitle<43pa!eTitle pc7 Q3pa!e*ead pc7 Gbody< +$aybe $s! G" $$s! Gp<Hour $essa!e was= 43$s!7 Q3pa!eBody pc7 |; instance IenderMessa!e Messa!es ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e !etIootI == *andler Iep*t$l !etIootI = de#ault5ayout :wha$let| G#or$ $ethod=post action=O39etMessa!eI7< My $essa!e is= 4 Ginput type=text na$e=$essa!e< Ginput type=sub$it< |; post9etMessa!eI == *andler @A post9etMessa!eI = do $s! G" runInputPost + ireE text?ield 8$essa!e8 setMessa!e + to*t$l $s! redirect IootI $ain == I, @A $ain = warp1ebu! &KKK Messa!es

FB

Initial page load; no message

<e3 message entered in te,t -o,

After form su-mit; message appears at top of page

After refresh; the message is cleared

'ltimate (estination
0ot to be con#used -ith a horror #ilm, this concept is used internally in yesod1auth2 Suppose a user re3uests a page that re3uires authentication2 I# the user is not yet logged in, you need to send him<her to the login page2 -ell1designed -eb app -ill then send them back to the first page they requested2 That%s -hat -e call the ultimate destination2 sends the user to the ultimate destination set in his<her session, clearing that !alue #rom the session2 It takes a de#ault destination as -ell, in case there is no destination set2 For setting the session, there are three options)
redirect6lt1est set6lt1est sets the destination to the gi!en 8R5 set6lt1estCurrent sets the destination to the currently re3uested 8R52 set6lt1estIe#erer sets the destination based on the Ie#erer header 4the

page that

led the user to the current page62

+H

5et%s look at a small sample app2 It -ill allo- the user to set his<her name in the session, and then tell the user his<her name #rom another route2 I# the name hasn%t been set yet, the user -ill be redirected to the set name page, -ith an ultimate destination set to come back to the current page2
3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> Te$plate*askell> FuasiFuotes> MultiPara$TypeClasses 4"7 i$port Hesod data 6lt1est = 6lt1est $kHesod 86lt1est8 :parseIoutes| . IootI G-T .setna$e 9et a$eI G-T P,9T .sayhello 9ay*elloI G-T |; instance Hesod 6lt1est instance IenderMessa!e 6lt1est ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e !etIootI = de#ault5ayout :wha$let| Gp< Ga hre#=O39et a$eI7<9et your na$e Gp< Ga hre#=O39ay*elloI7<9ay hello |; "" 1isplay the set na$e #or$ !et9et a$eI = de#ault5ayout :wha$let| G#or$ $ethod=post< My na$e is 4 Ginput type=text na$e=na$e< / 4 Ginput type=sub$it %alue=89et na$e8< |; "" Ietrei%e the sub$itted na$e #ro$ the user post9et a$eI == *andler @A post9et a$eI = do "" Get the sub$itted na$e and set it in the session na$e G" runInputPost + ireE text?ield 8na$e8 set9ession 8na$e8 na$e "" )#ter we !et a na$e> redirect to the ulti$ate destination/ "" I# no destination is set> de#ault to the ho$epa!e redirect6lt1est IootI !et9ay*elloI = do "" 5ookup the na$e %alue set in the session $na$e G" lookup9ession 8na$e8 case $na$e o# othin! "< do "" o na$e in the session> set the current pa!e as "" the ulti$ate destination and redirect to the "" 9et a$e pa!e set6lt1estCurrent setMessa!e 8Please tell $e your na$e8 redirect 9et a$eI

+*

Just na$e "< de#ault5ayout :wha$let| Gp<Celco$e 43na$e7 |; $ain == I, @A $ain = warp1ebu! &KKK 6lt1est

Summary
Sessions are the number one -ay -e bypass the statelessness imposed by HTTP2 We shouldn%t consider this an escape hatch to per#orm -hate!er actions -e -ant) statelessness in -eb applications is a !irtue, and -e should respect it -hene!er possible2 Ho-e!er, there are speci#ic cases -here it is !ital to retain some state2 The session PI in Yesod is !ery simple2 It pro!ides a key1!alue store, and a #econ!enience #unctions built on top #or common use cases2 I# used properly, -ith small payloads, sessions should be an unobtrusi!e part o# your -eb de!elopment2

Persistent
Forms deal -ith the boundary bet-een the user and the application2 nother boundary -e need to deal -ith is bet-een the application and the storage layer2 Whether it be a SG5 database, a Y &5 #ile, or a binary blob, odds are you ha!e to -ork to get your storage layer to accept your application datatypes2 Persistent is Yesod%s ans-er to data storage1 a type1sa#e, uni!ersal data store inter#ace #or Haskell2 Haskell has many di##erent database bindings a!ailable2 Ho-e!er, most o# these ha!e little kno-ledge o# a schema and there#ore do not pro!ide use#ul static guarantees2 They also #orce database1dependent PIs and data types on the programmer2 Haskellers ha!e attempted a more re!olutionary route o# creating Haskell speci#ic data stores to get around these #la-s that allo- one to easily store any Haskell type2 These options are great #or certain use cases, but they constrain one to the storage techni3ues pro!ided by the library, do not inter#ace -ell -ith other languages, and the #le(ibility can also mean one must -rite reams o# code #or 3uerying data2 In contrast, Persistent allo-s us to choose among e(isting databases that are highly tuned #or di##erent data storage use cases, interoperate -ith other programming languages, and to use a sa#e and producti!e 3uery inter#ace2 Persistent #ollo-s the guiding principles o# type sa#ety and concise, declarati!e synta(2 Some other nice #eatures are)

Database1agnostic2 There is #irst class support #or PostgreSG5, SG5ite and &ongoDB, -ith e(perimental $ouchDB and &ySG5 support in the -orks2 By being non1relational in nature, -e simultaneously are able to support a -ider number o# storage layers and are not constrained by some o# the per#ormance bottlenecks incurred through 9oins2 ma9or source o# #rustration in dealing -ith SG5 databases is changes to the schema2 Persistent can automatically per#orm database migrations2

+:

Synopsis
3"4 5) 4"7 3"4 5) i$port i$port i$port i$port G6)G- FuasiFuotes> Te$plate*askell> Type?a$ilies> ,%erloaded9trin!s G6)G- G)1Ts> ?lexibleContexts 4"7 1atabase/Persist 1atabase/Persist/9Elite 1atabase/Persist/T* Control/Monad/I,/Class @li#tI,A

share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person na$e 9trin! a!e Int Maybe deri%in! 9how Blo!Post title 9trin! authorId PersonId deri%in! 9how |; $ain == I, @A $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration $i!rate)ll SohnId G" insert + Person 8John 1oe8 + Just &M SaneId G" insert + Person 8Jane 1oe8 othin! insert + Blo!Post 8My #r(st pKst8 SohnId insert + Blo!Post 8,ne $ore #or !ood $easure8 SohnId oneJohnPost G" select5ist :Blo!Post)uthorId ==/ SohnId; :5i$itTo (; li#tI, + print @oneJohnPost == :-ntity Blo!Post;A Sohn G" !et SohnId li#tI, + print @Sohn == Maybe PersonA delete SaneId deleteChere :Blo!Post)uthorId ==/ SohnId;

Sol ing the -oundary issue

Suppose you are storing in#ormation on people in a SG5 database2 Your table might look something like)
CI-)T- T)B5- Person@id 9-II)5 PIIM)IH X-H> na$e V)IC*)I I T-G-IA ,T 655> a!e

nd i# you are using a database like PostgreSG5, you can be guaranteed that the database -ill ne!er store some arbitrary te(t in your age #ield2 4The same cannot be said o# SG5ite, but let%s #orget about that #or no-26 To mirror this database table, you -ould likely create a Haskell datatype that looks something like)
data Person = Person 3 person a$e == Text

+>

> person)!e == Int 7

It looks like e!erything is type sa#e) the database schema matches our Haskell datatypes, the database ensures that in!alid data can ne!er make it into our data store, and e!erything is generally a-esome2 Well, until)

You -ant to pull data #rom the database, and the database layer gi!es you the data in an untyped #ormat2 You -ant to #ind e!eryone older than >:, and you accidently -rite Ithirtyt-oI in your SG5 statement2 ;uess -hat) that -ill compile 9ust #ine, and you -on%t #ind out you ha!e a problem until runtime2 You decide you -ant to #ind the #irst *H people alphabetically2 0o problem222 until you make a typo in your SG52 /nce again, you don%t #ind out until runtime2

In dynamic languages, the ans-ers to these issues is unit testing2 For e!erything that can go -rong, make sure you -rite a test case2 But as I am sure you are a-are by no-, that doesn%t 9i!e -ell -ith the Yesod approach to things2 We like to take ad!antage o# Haskell%s strong typing to sa!e us -here!er possible, and data storage is no e(ception2 So the 3uestion remains) ho- can -e use Haskell%s type system to sa!e the day=

Types
5ike routing, there is nothing intrinsically di##icult about type1sa#e data access2 It 9ust re3uires a lot o# monotonous, error prone, boiler plate code2 s usual, this means -e can use the type system to keep us honest2 nd to a!oid some o# the drudgery, -e%ll use a sprinkling o# Template Haskell2 "arlier !ersions o# Persistent made much hea!ier usage o# Template Haskell2 Starting -ith H2D, there is a ne- architecture inspired by the groundhog package2 This approach uses phantom types to carry a lot o# the burden2 is the basic building block o# Persistent2 It is a sum type that can represent data that gets sent to and #rom a database2 Its de#inition is)
PersistValue data PersistValue = | | | | | | | | | | | #or Mon!o1B backend PersistText Text PersistByte9trin! Byte9trin! PersistIntVL IntVL Persist1ouble 1ouble PersistBool Bool Persist1ay 1ay PersistTi$e,#1ay Ti$e,#1ay Persist6TCTi$e 6TCTi$e Persist ull Persist5ist :PersistValue; PersistMap :@T/Text> PersistValueA; Persist?orei!nXey Byte9trin! "" Q intended especially

+@

"ach Persistent backend needs to kno- ho- to translate the rele!ant !alues into something the database can understand2 Ho-e!er, it -ould be a-k-ard do ha!e to e(press all o# our data simply in terms o# these basic types2 The ne(t layer is the Persist?ield typeclass, -hich de#ines ho- an arbitrary Haskell datatype can be marshaled to and #rom a PersistValue2 Persist?ield correlates to a column in a SG5 database2 In our person e(ample abo!e, name and age -ould be our Persist?ields2 To tie up the user side o# the code, our last typeclass is Persist-ntity2 n instance o# Persist"ntity correlates -ith a table in a SG5 database2 This typeclass de#ines a number o# #unctions and some associated types2 To re!ie-, -e ha!e the #ollo-ing correspondence bet-een Persistent and SG5) SG5 Datatypes 4E R$H R, I0T";"R, etc6 $olumn Table Persistent PersistEalue PersistField Persist"ntity

Code 0eneration
In order to ensure that the Persist"ntity instances match up properly -ith your Haskell datatypes, Persistent takes responsibility #or both2 This is also good #rom a DRY 4Don%t Repeat Yoursle#6 perspecti!e) you only need to de#ine your entities once2 5et%s see a 3uick e(ample)
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> Te$plate*askell> ,%erloaded9trin!s> G)1Ts 4"7 i$port 1atabase/Persist i$port 1atabase/Persist/T* i$port 1atabase/Persist/9Elite i$port Control/Monad/I,/Class @li#tI,A $kPersist sEl9ettin!s :persist| Person na$e 9trin! a!e Int deri%in! 9how |; $ain = return @A

We use a combination o# Template Haskell and Guasi1Guotation 4like -hen de#ining routes6) persist is a 3uasi13uoter -hich con!erts a -hitespace1sensiti!e synta( into a list o# entity de#initions2 4You can also declare your entities in a separate #ile using persistFile26 mkPersist takes that list o# entities and declares)

/ne Haskell datatype #or each entity2 Persist-ntity instance #or each datatype de#ined2

The e(ample abo!e generates code that looks like the #ollo-ing)
3"4 5) G6)G- Type?a$ilies> Generali'ed ewtype1eri%in!> ,%erloaded9trin!s> G)1Ts 4"7

+A

i$port i$port i$port i$port i$port i$port

1atabase/Persist 1atabase/Persist/9tore 1atabase/Persist/9Elite 1atabase/Persist/-ntity1e# Control/Monad/I,/Class @li#tI,A Control/)pplicati%e

data Person = Person 3 person a$e == 9trin! > person)!e == Int 7 deri%in! @9how> Iead> -EA type PersonId = Xey 9ElPersist Person instance Persist-ntity Person where "" ) Generali'ed )l!ebraic 1atatype @G)1TA/ "" This !i%es us a type"sa#e approach to $atchin! #ields with "" their datatypes/ data -ntity?ield Person typ where PersonId == -ntity?ield Person PersonId Person a$e == -ntity?ield Person 9trin! Person)!e == -ntity?ield Person Int type Persist-ntityBackend Person = 9ElPersist toPersist?ields @Person na$e a!eA = : 9o$ePersist?ield na$e > 9o$ePersist?ield a!e ; #ro$PersistValues :na$eValue> a!eValue; = Person G+< #ro$PersistValue na$eValue GW< #ro$PersistValue a!eValue #ro$PersistValues B = 5e#t 8In%alid #ro$PersistValues input8 "" In#or$ation on each #ield> used internally to !enerate 9F5 state$ents persist?ield1e# PersonId = ?ield1e# @*askell a$e 8Id8A @1B a$e 8id8A @?TTypeCon othin! 8PersonId8A :; persist?ield1e# Person a$e = ?ield1e# @*askell a$e 8na$e8A @1B a$e 8na$e8A @?TTypeCon othin! 89trin!8A :; persist?ield1e# Person)!e = ?ield1e# @*askell a$e 8a!e8A @1B a$e 8a!e8A @?TTypeCon othin! 8Int8A :; data 6niEue Person typ = I!noreThis entity1e# = unde#ined hal#1e#ined = unde#ined persist6niEueTo?ield a$es = unde#ined persist6niEueToValues = unde#ined persist6niEueXeys = unde#ined persistId?ield = unde#ined $ain == I, @A

+D

$ain = return @A

s you might e(pect, our Person datatype closely matches the de#inition -e ga!e in the original Template Haskell !ersion2 We also ha!e a ;enerali'ed lgebraic Datatype 4; DT6 -hich gi!es a separate constructor #or each #ield2 This ; DT encodes both the type o# the entity and the type o# the #ield2 We use its constructors throughout Persistent, such as to ensure that -hen -e apply a #ilter, the types o# the #iltering !alue match the #ield2 We can use the generated Person type like any other Haskell type, and then pass it o## to other Persistent #unctions2
$ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do $ichaelId G" insert + Person 8Michael8 NV $ichael G" !et $ichaelId li#tI, + print $ichael

We start o## -ith some standard database connection code2 In this case, -e used the single1 connection #unctions2 Persistent also comes built in -ith connection pool #unctions, -hich -e -ill generally -ant to use in production2 In this e(ample, -e ha!e seen t-o #unctions) insert creates a ne- record in the database and returns its ID2 5ike e!erything else in Persistent, IDs are type sa#e2 We%ll get into more details o# ho- these IDs -ork later2 So -hen you call insert + Person 8Michael8 NM, it gi!es you a !alue back o# type PersonId2 The ne(t #unction -e see is !et, -hich attempts to load a !alue #rom the database using an Id2 In Persistent, you ne!er need to -orry that you are using the key #rom the -rong table) trying to load up a di##erent entity 4like *ouse6 using a PersonId -ill ne!er compile2

PersistStore
/ne last detail is le#t une(plained #rom the pre!ious e(ample) -hat are those with9EliteConn and run9ElConn #unctions doing, and -hat is that monad that our database actions are running in= ll database actions need to occur -ithin an instance o# Persist9tore2 s its name implies, e!ery data store 4PostgreSG5, SG5ite, &ongoDB6 has an instance o# Persist9tore2 This is -here all the translations #rom PersistValue to database1speci#ic !alues occur, -here SG5 3uery generation happens, and so on2 s you can imagine, e!en though Persist9tore pro!ides a sa#e, -ell1typed inter#ace to the outside -orld, there are a lot o# database interactions that could go -rong2 Ho-e!er, by testing this code automatically and thoroughly in a single location, -e can centrali'e our error1prone code and make sure it is as bug1#ree as possible2 creates a single connection to a database using its supplied connection string2 For our test cases, -e -ill use =$e$ory=, -hich uses an in1memory database2 run9ElConn uses that connection to run the inner action2 Both SG5ite and PostgreSG5 share the same instance o# Persist9tore) 9ElPersist2
with9EliteConn

+F

There are actually a #e- other typeclasses) Persist6pdate and PersistFuery2 Di##erent typeclasses pro!ide di##erent #unctionality, -hich allo-s us to -rite backends that use simpler data stores 4e2g2, Redis6 e!en though they can%t pro!ide us all the high1le!el #unctionality a!ailable in Persistent2 /ne important thing to note is that e!erything -hich occurs inside a single call to run9ElConn runs in a single transaction2 This has t-o important implications)

For many databases, committing a transaction can be a costly acti!ity2 By putting multiple steps into a single transaction, you can speed up code dramatically2 I# an e(ception is thro-n any-here inside a single call to run9ElConn, all actions -ill be rolled back 4assuming your backend has rollback support62

Migrations
I%m sorry to tell you, but so #ar I ha!e lied to you a bit) the e(ample #rom the pre!ious section does not actually -ork2 I# you try to run it, you -ill get an error message about a missing table2 For SG5 databases, one o# the ma9or pains can be managing schema changes2 Instead o# lea!ing this to the user, Persistent steps in to help, but you ha!e to ask it to help2 5et%s see -hat this looks like)
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> Te$plate*askell> ,%erloaded9trin!s> G)1Ts> ?lexibleContexts 4"7 i$port 1atabase/Persist i$port 1atabase/Persist/T* i$port 1atabase/Persist/9Elite i$port Control/Monad/I,/Class @li#tI,A share :$kPersist sEl9ettin!s> $k9a%e 8entity1e#s8; :persist| Person na$e 9trin! a!e Int deri%in! 9how |; $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration + $i!rate entity1e#s @unde#ined == PersonA "" this line added= thatPs itJ $ichaelId G" insert + Person 8Michael8 NV $ichael G" !et $ichaelId li#tI, + print $ichael

With this one little code change, Persistent -ill automatically create your Person table #or you2 This split bet-een runMi!ration and $i!rate allo-s you to migrate multiple tables simultaneously2 This -orks -hen dealing -ith 9ust a #e- entities, but can 3uickly get tiresome once -e are dealing -ith a do'en entities2 Instead o# repeating yoursel#, Persistent pro!ides a helper #unction, $kMi!rate)

++

3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> Te$plate*askell> ,%erloaded9trin!s> G)1Ts> ?lexibleContexts 4"7 i$port 1atabase/Persist i$port 1atabase/Persist/9Elite i$port 1atabase/Persist/T* share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person na$e 9trin! a!e Int deri%in! 9how Car color 9trin! $ake 9trin! $odel 9trin! deri%in! 9how |; $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration $i!rate)ll

is a Template Haskell #unction -hich creates a ne- #unction that -ill automatically call $i!rate on all entities de#ined in the persist block2 The share #unction is 9ust a little helper that passes the in#ormation #rom the persist block to each Template Haskell #unction and concatenates the results2
$kMi!rate

Persistent has !ery conser!ati!e rules about -hat it -ill do during a migration2 It starts by loading up table in#ormation #rom the database, complete -ith all de#ined SG5 datatypes2 It then compares that against the entity de#inition gi!en in the code2 For the #ollo-ing cases, it -ill automatically alter the schema)

The datatype o# a #ield changed2 Ho-e!er, the database may ob9ect to this modi#ication i# the data cannot be translated2 #ield -as added2 Ho-e!er, i# the #ield is not null, no de#ault !alue is supplied 4-e%ll discuss de#aults later6 and there is already data in the database, the database -ill not allo- this to happen2 #ield is con!erted #rom not null to null2 In the opposite case, Persistent -ill attempt the con!ersion, contingent upon the database%s appro!al2 brand ne- entity is added2

Ho-e!er, there are some cases that Persistent -ill not handle)

Field or entity renames) Persistent has no -ay o# kno-ing that InameI has no- been renamed to I#ull0ameI) all it sees is an old #ield called name and a ne- #ield called #ull0ame2 Field remo!als) since this can result in data loss, Persistent by de#ault -ill re#use to per#orm the action 4you can #orce the issue by using runMi!ration6nsa#e instead o# runMi!ration, though it is not recommended62

runMi!ration -ill print out the migrations it is running on stderr 4you can bypass this by using runMi!ration9ilent62 Whene!er possible, it uses )5T-I T)B5- calls2 Ho-e!er, in

+B

SG5ite, )5T-I T)B5- has !ery limited abilities, and there#ore Persistent must resort to copying the data #rom one table to another2 Finally, i# instead o# performing a migration, you -ant Persistent to gi!e you hints about -hat migrations are necessary, use the printMi!ration #unction2 This #unction -ill print out the migrations -hich runMi!ration -ould per#orm #or you2 This may be use#ul #or per#orming migrations that Persistent is not capable o#, #or adding arbitrary SG5 to a migration, or 9ust to log -hat migrations occurred2

'ni.ueness
In addition to declaring #ields -ithin an entity, you can also declare uni3ueness constraints2 typical e(ample -ould be re3uiring that a username be uni3ue2

'ni.ue 'sername
6ser userna$e Text 6niEue6serna$e userna$e

While each #ield name must begin -ith a lo-ercase letter, the uni3ueness constraints must begin -ith an uppercase letter2
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> Te$plate*askell> ,%erloaded9trin!s> G)1Ts> ?lexibleContexts 4"7 i$port 1atabase/Persist i$port 1atabase/Persist/9Elite i$port 1atabase/Persist/T* i$port 1ata/Ti$e i$port Control/Monad/I,/Class @li#tI,A share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person #irst a$e 9trin! last a$e 9trin! a!e Int Person a$e #irst a$e last a$e deri%in! 9how |; $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration $i!rate)ll insert + Person 8Michael8 89noy$an8 NV $ichael G" !etBy + Person a$e 8Michael8 89noy$an8 li#tI, + print $ichael

To declare a uni3ue combination o# #ields, -e add an e(tra line to our declaration2 Persistent kno-s that it is de#ining a uni3ue constructor, since the line begins -ith a capital letter2 "ach #ollo-ing -ord must be a #ield in this entity2

BH

The main restriction on uni3ueness is that it can only be applied non1null #ields2 The reason #or this is that the SG5 standard is ambiguous on ho- uni3ueness should be applied to 655 4e2g2, is 655= 655 true or #alse=62 Besides that ambiguity, most SG5 engines in #act implement rules -hich -ould be contrary to -hat the Haskell datatypes anticipate 4e2g2, PostgreSG5 says that 655= 655 is #alse, -hereas Haskell says othin! == othin! is True62 In addition to pro!iding nice guarantees at the database le!el about consistency o# your data, uni3ueness constraints can also be used to per#orm some speci#ic 3ueries -ithin your Haskell code, like the !etBy demonstrated abo!e2 This happens !ia the 6niEue associated type2 In the e(ample abo!e, -e end up -ith a ne- constructor)
Person a$e == 9trin! "< 9trin! "< 6niEue Person

"ueries
Depending on -hat your goal is, there are di##erent approaches to 3uerying the database2 Some commands 3uery based on a numeric ID, -hile others -ill #ilter2 Gueries also di##er in the number o# results they return) some lookups should return no more than one result 4i# the lookup key is uni3ue6 -hile others can return many results2 Persistent there#ore pro!ides a #e- di##erent 3uery #unctions2 s usual, -e try to encode as many in!ariants in the types as possible2 For e(ample, a 3uery that can return only H or * results -ill use a Maybe -rapper, -hereas a 3uery returning many results -ill return a list2

!etching -y I(
The simplest 3uery you can per#orm in Persistent is getting based on an ID2 Since this !alue may or may not e(ist, its return type is -rapped in a Maybe2

'sing get
personId G" insert + Person 8Michael8 89noy$an8 NV $aybePerson G" !et personId case $aybePerson o# othin! "< li#tI, + put9tr5n 8Just kiddin!> not really there8 Just person "< li#tI, + print person

This can be !ery use#ul #or sites that pro!ide 8R5s like <person<A2 Ho-e!er, in such a case, -e don%t usually care about the Maybe -rapper, and 9ust -ant the !alue, returning a @H@ message i# it is not #ound2 Fortunately, the get@H@ #unction helps us out here2 We%ll go into more details -hen -e see integration -ith Yesod2

!etching -y uni.ue constraint

is almost identical to !et, e(cept it takes a uni3ueness constraint instead o# an ID it takes a 8ni3ue !alue2
!etBy

B*

'sing get#y
personId G" insert + Person 8Michael8 89noy$an8 NV $aybePerson G" !etBy + 6niEue a$e 8Michael8 89noy$an8 case $aybePerson o# othin! "< li#tI, + put9tr5n 8Just kiddin!> not really there8 Just person "< li#tI, + print person

5ike !etLKL, there is also a !etByLKL #unction2

Select functions
&ost likely, you%re going to -ant more po-er#ul 3ueries2 You%ll -ant to #ind e!eryone o!er a certain age7 all cars a!ailable in blue7 all users -ithout a registered email address2 For this, you need one o# the select #unctions2 ll the select #unctions use a similar inter#ace, -ith slightly di##erent outputs) Function Returns

9ource containing all the IDs and !alues #rom the database2 This allo-s you to -rite streaming code2We co!er 9ources in detail in the conduits appendi(2 selectSource dditionally, there%s another #unction called select9ourceConn -hich allo-s you more control o# connection allocation2 We use this in the Sphin( case study2 list containing all the IDs and !alues #rom the database2 ll records -ill be select5ist loaded into memory2 selectFirst Takes 9ust the #irst ID and !alue #rom the database, i# a!ailable selectPeys Returns only the keys, -ithout the !alues, as a 9ource2

is the most commonly used, so -e -ill co!er it speci#ically2 8nderstanding the others should be tri!ial a#ter that2
select5ist

takes t-o arguments) a list o# ?ilters, and a list o# 9elect,pts2 The #ormer is -hat limits your results based on characteristics7 it allo-s #or e3uals, less than, is member o#, and such2 9elect,pts pro!ides #or three di##erent #eatures) sorting, limiting output to a certain number o# ro-s, and o##setting results by a certain number o# ro-s2
select5ist

The combination o# limits and o##sets is !ery important7 it allo-s #or e##icient pagination in your -ebapps2 5et%s 9ump straight into an e(ample o# #iltering, and then analy'e it2
people G" select5ist :Person)!e </ NM> Person)!e G=/ &K; :; li#tI, + print people

s simple as that e(ample is, -e really need to co!er three points)

B:

*2 Person)!e is a constructor #or an associated phantom type2 That might sound scary, but -hat%s important is that it uni3uely identi#ies the IageI column o# the IpersonI table, and that it kno-s that the age #ield is an Int2 4That%s the phantom part26 :2 We ha!e a bunch o# Persistent #iltering operators2 They%re all pretty straight1#or-ard) 9ust tack a period to the end o# -hat you%d e(pect2 There are three gotchas here, I%ll e(plain belo-2 >2 The list o# #ilters is 0Ded together, so that our constraint means Iage is greater than :A 0D age is less than or e3ual to >HI2 We%ll describe /Ring later2 The one operator that%s surprisingly named is Inot e3uals2I We use J=/, since .=/ is used #or updates 4#or Idi!ide1and1setI, described later62 Don%t -orry) i# you use the -rong one, the compiler -ill catch you2 The other t-o surprising operators are the Iis memberI and Iis not memberI2 They are, respecti!ely, G"/ and .G"/ 4both end -ith a period62 nd regarding /Rs, -e use the ||/ operator2 For e(ample)
people G" select5ist @ :Person)!e </ NM> Person)!e G=/ &K; ||/ :Person?irst a$e .G"/ :8)da$8> 8Bonny8;; ||/ @:Person)!e ==/ MK; ||/ :Person)!e ==/ VK;A A :; li#tI, + print people

This 4completely nonsensical6 e(ample means) #ind people -ho are :D1>H, inclusi!e, /R -hose names are neither dam or Bonny, /R -hose age is either AH or DH2

SelectOpt
ll o# our select5ist calls ha!e included an empty list as the second parameter2 That speci#ies no options, meaning) sort ho-e!er the database -ants, return all results, and don%t skip any results2 9elect,pt has #our constructors that can be used to change all that2 sc Sort by the gi!en column in ascending order2 This uses the same phantom type as #iltering, such as Person)!e2 Desc Same as )sc, in descending order2 5imitTo Takes an Int argument2 /nly return up to the speci#ied number o# results2 /##setBy Takes an Int argument2 Skip the speci#ied number o# results2 The #ollo-ing code de#ines a #unction that -ill break do-n results into pages2 It returns all people aged *+ and o!er, and then sorts them by age 4oldest person #irst62 For people -ith the same age, they are sorted alphabetically by last name, then #irst name2
results?orPa!e pa!e u$ber = do let resultsPerPa!e = (K select5ist : Person)!e <=/ (D

B>

; : > > > > ;

1esc Person)!e )sc Person5ast a$e )sc Person?irst a$e 5i$itTo resultsPerPa!e ,##setBy + @pa!e u$ber " (A W resultsPerPa!e

$ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration9ilent $i!rate)ll personId G" insert + Person 8Michael8 89noy$an8 NV results?orPa!e ( <<= li#tI, / print

Manipulation
Guerying is only hal# the battle2 We also need to be able to add data to and modi#y e(isting data in the database2

Insert
It%s all -ell and good to be able to play -ith data in the database, but ho- does it get there in the #irst place= The ans-er is the insert #unction2 You 9ust gi!e it a !alue, and it gi!es back an ID2 t this point, it makes sense to e(plain a bit o# the philosophy behind Persistent2 In many other /R& solutions, the datatypes used to hold data are opa3ue) you need to go through their de#ined inter#aces to get at and modi#y the data2 That%s not the case -ith Persistent) -e%re using plain old lgebraic Data Types #or the -hole thing2 This means you still get all the great bene#its o# pattern matching, currying and e!erything else you%re used to2 Ho-e!er, there are a #e- things -e can't do2 For one, there%s no -ay to automatically update !alues in the database e!ery time the record is updated in Haskell2 /# course, -ith Haskell%s normal stance o# purity and immutability, this -ouldn%t make much sense any-ay, so I don%t shed any tears o!er it2 Ho-e!er, there is one issue that ne-comers are o#ten bothered by) -hy are IDs and !alues completely separate= It seems like it -ould be !ery logical to embed the ID inside the !alue2 In other -ords, instead o# ha!ing)
data Person = Person 3 na$e == 9trin! 7

ha!e

data Person = Person 3 personId == PersonId> na$e == 9trin! 7

Well, there%s one problem -ith this right o## the bat) ho- do -e do an insert= I# a Person needs to ha!e an ID, and -e get the ID by inserting, and an insert needs a Person, -e ha!e an impossible loop2 We could sol!e this -ith unde#ined, but that%s 9ust asking #or trouble2 /P, you say, let%s try something a bit sa#er)
data Person = Person 3 personId == Maybe PersonId> na$e == 9trin! 7

B@

I de#initely pre#er insert + Person othin! 8Michael8 to insert + Person unde#ined 8Michael82 nd no- our types -ill be much simpler, right= For e(ample, select5ist could return a simple :Person; instead o# that ugly :-ntity 9ElPersist Person;2 -ntity is a datatype that ties together both the ID and !alue o# an entity2 Since IDs can be di##erent based on backend, -e also need to pro!ide the Persistent backend -e%re using2 The datatype -ntity 9ElPersist Person can be read as Ithe ID and !alue o# a person stored in a SG5 database2I The problem is that the IuglinessI is incredibly use#ul2 Ha!ing -ntity 9ElPersist Person makes it ob!ious, at the type le!el, that -e%re dealing -ith a !alue that e(ists in the database2 5et%s say -e -ant to create a link to another page that re3uires the PersonId 4not an uncommon occurrence as -e%ll discuss later62 The -ntity 9ElPersist Person #orm gi!es us unambiguous access to that in#ormation7 embedding PersonId -ithin Person -ith a Maybe -rapper means an e(tra runtime check #or Just, instead o# a more error1proo# compile time check2 Finally, there%s a semantic mismatch -ith embedding the ID -ithin the !alue2 The Person is the !alue2 T-o people are identical 4in the conte(t o# a database6 i# all their #ields are the same2 By embedding the ID in the !alue, -e%re no longer talking about a person, but about a ro- in the database2 "3uality is no longer really e3uality, it%s identity) is this the same person, as opposed to an e3ui!alent person2 In other -ords, there are some annoyances -ith ha!ing the ID separated out, but o!erall, it%s the right approach, -hich in the grand scheme o# things leads to better, less buggy code2

'pdate
0o-, in the conte(t o# that discussion, let%s think about updating2 The simplest -ay to update is)
let $ichael = Person 8Michael8 NV $ichael)#terBirthday = $ichael 3 person)!e = N[ 7 But that%s not actually updating anything, it%s 9ust creating a ne- Person

!alue based on the old one2 When -e say update, -e%re not talking about modi#ications to the !alues in Haskell2 4We better not be o# course, since Haskell data types are immutable26 Instead, -e%re looking at -ays o# modi#ying ro-s in a table2 nd the simplest -ay to do that is -ith the update #unction2
personId G" insert + Person 8Michael8 89noy$an8 NV update personId :Person)!e =/ N[; results?orPa!e ( <<= li#tI, / print

takes t-o arguments) an ID, and a list o# 6pdates2 The simplest update is assignment, but it%s not al-ays the best2 What i# you -ant to increase someone%s age by *, but you don%t ha!e their current age= Persistent has you co!ered)
update ha%eBirthday personId = update personId :Person)!e U=/ (; $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do

BA

runMi!ration9ilent $i!rate)ll personId G" insert + Person 8Michael8 89noy$an8 NV update personId :Person)!e =/ N[; ha%eBirthday personId results?orPa!e ( <<= li#tI, / print

nd as you might e(pect, -e ha!e all the basic mathematical operators) U=/, "=/, W=/, and .=/ 4#ull stop62 These can be con!enient #or updating a single record, but they are also essential #or proper $ID guarantees2 Imagine the alternati!e) pull out a Person, increment the age, and update the ne- !alue2 I# you ha!e t-o threads<processes -orking on this database at the same time, you%re in #or a -orld o# hurt 4hint) race conditions62 Sometimes you%ll -ant to update many #ields at once 4gi!e all your employees a AV pay increase, #or e(ample62 updateChere takes t-o parameters) a list o# #ilters, and a list o# updates to apply2
updateChere :Person?irst a$e ==/ 8Michael8; :Person)!e W=/ N; "" itPs been a lon! day

/ccassionally, you%ll 9ust -ant to completely replace the !alue in a database -ith a di##erent !alue2 For that, you use 4surprise6 the replace #unction2
personId G" insert + Person 8Michael8 89noy$an8 NV replace personId + Person 8John8 81oe8 NK update personId :Person)!e =/ N[; ha%eBirthday personId updateChere :Person?irst a$e ==/ 8Michael8; :Person)!e W=/ N; "" itPs been a lon! day results?orPa!e ( <<= li#tI, / print

(elete
s much as it pains us, sometimes -e must part -ith our data2 To do so, -e ha!e three #unctions) delete Delete based on an ID deleteBy Delete based on a uni3ue constraint deleteWhere Delete based on a set o# #ilters
personId G" insert + Person 8Michael8 89noy$an8 NV delete personId deleteBy + 6niEue a$e 8Michael8 89noy$an8 deleteChere :Person?irst a$e ==/ 8Michael8;

We can e!en use deleteWhere to -ipe out all the records in a table, -e 9ust need to gi!e some hints to ;H$ as to -hat table -e%re interested in)
deleteChere @:; == :?ilter Person;A

BD

Attri-utes
So #ar, -e ha!e seen a basic synta( #or our persist blocks) a line #or the name o# our entities, and then an indented line #or each #ield -ith t-o -ords) the name o# the #ield and the datatype o# the #ield2 Persistent handles more than this) you can assign an arbitrary list o# attributes a#ter the #irst t-o -ords on a line2 Suppose -e -ant to ha!e a Person entity -ith an 4optional6 age and the timestamp o# -hen he<she -as added to the system2 For entities already in the database, -e -ant to 9ust use the current date1time #or that timestamp2
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> Te$plate*askell> ,%erloaded9trin!s> G)1Ts> ?lexibleContexts 4"7 i$port 1atabase/Persist i$port 1atabase/Persist/9Elite i$port 1atabase/Persist/T* i$port 1ata/Ti$e i$port Control/Monad/I,/Class share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person na$e 9trin! a!e Int Maybe created 6TCTi$e de#ault=now@A deri%in! 9how |; $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do ti$e G" li#tI, !etCurrentTi$e runMi!ration $i!rate)ll insert + Person 8Michael8 @Just NVA ti$e insert + Person 8Gre!8 othin! ti$e

is a built in, single -ord attribute2 It makes the #ield optional2 In Haskell, this means it is -rapped in a Maybe2 In SG5, it makes the column nullable2
Maybe

The de#ault attribute is backend speci#ic, and uses -hate!er synta( is understood by the database2 In this case, it uses the database%s built1in now@A #unction2 Suppose that -e no-ant to add a #ield #or a person%s #a!orite programming language)
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> Te$plate*askell> ,%erloaded9trin!s> G)1Ts> ?lexibleContexts 4"7 i$port 1atabase/Persist i$port 1atabase/Persist/9Elite i$port 1atabase/Persist/T* i$port 1ata/Ti$e share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person na$e 9trin!

BF

a!e Int Maybe created 6TCTi$e de#ault=now@A lan!ua!e 9trin! de#ault=P*askellP deri%in! 9how |; $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration $i!rate)ll The de#ault attribute has absolutely no impact on the Haskell

code itsel#7 you still need to #ill in all !alues2 This -ill only a##ect the database schema and automatic migrations2 We need to surround the string -ith single 3uotes so that the database can properly interpret it2 Finally, Persistent can use double 3uotes #or containing -hite space, so i# -e -ant to set someone%s de#ault home country to be "l Sal!ador)
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> Te$plate*askell> ,%erloaded9trin!s> G)1Ts> ?lexibleContexts 4"7 i$port 1atabase/Persist i$port 1atabase/Persist/9Elite i$port 1atabase/Persist/T* i$port 1ata/Ti$e share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person na$e 9trin! a!e Int Maybe created 6TCTi$e de#ault=now@A lan!ua!e 9trin! de#ault=P*askellP country 9trin! 8de#ault=P-l 9al%adorP8 deri%in! 9how |; $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration $i!rate)ll

/ne last trick you can do -ith attributes is to speci#y the names to be used #or the SG5 tables and columns2 This can be con!enient -hen interacting -ith e(isting databases2
share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person sEl=the"person"table #irst a$e 9trin! sEl=#irstBna$e last a$e 9trin! sEl=#ld5ast a$e a!e Int Gt 1esc 8sEl=The )!e o# the Person8 6niEue a$e #irst a$e last a$e deri%in! 9how |; results?orPa!e pa!e u$ber = do let resultsPerPa!e = (K select5ist : Person)!e <=/ (D ; : 1esc Person)!e > )sc Person5ast a$e > )sc Person?irst a$e > 5i$itTo resultsPerPa!e > ,##setBy + @pa!e u$ber " (A W resultsPerPa!e ;

B+

$ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration $i!rate)ll personId G" insert + Person 8Michael8 89noy$an8 NV results?orPa!e ( <<= li#tI, / print

%elations
Persistent allo-s re#erences bet-een your data types in a manner that is consistent -ith supporting non1SG5 databases2 We do this by embedding an ID in the related entity2 So i# a person has many cars)
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> Te$plate*askell> ,%erloaded9trin!s> G)1Ts> ?lexibleContexts 4"7 i$port 1atabase/Persist i$port 1atabase/Persist/9Elite i$port 1atabase/Persist/T* i$port Control/Monad/I,/Class @li#tI,A i$port 1ata/Ti$e share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person na$e 9trin! deri%in! 9how Car ownerId PersonId -E na$e 9trin! deri%in! 9how |; $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration $i!rate)ll bruce G" insert + Person 8Bruce Cayne8 insert + Car bruce 8Bat Mobile8 insert + Car bruce 8Porsche8 "" this could !o on a while cars G" select5ist :Car,wnerId ==/ bruce; :; li#tI, + print cars

8sing this techni3ue, you can de#ine one1to1many relationships2 To de#ine many1to1many relationships, -e need a 9oin entity, -hich has a one1to1many relationship -ith each o# the original tables2 It is also a good idea to use uni3ueness constraints on these2 For e(ample, to model a situation -here -e -ant to track -hich people ha!e shopped in -hich stores)
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> Te$plate*askell> ,%erloaded9trin!s> G)1Ts> ?lexibleContexts 4"7 i$port 1atabase/Persist i$port 1atabase/Persist/9Elite i$port 1atabase/Persist/T* i$port 1ata/Ti$e share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person na$e 9trin! 9tore

BB

na$e 9trin! Person9tore personId PersonId storeId 9toreId 6niEuePerson9tore personId storeId |; $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration $i!rate)ll bruce G" insert + Person 8Bruce Cayne8 $ichael G" insert + Person 8Michael8 tar!et G" insert + 9tore 8Tar!et8 !ucci G" insert + 9tore 8Gucci8 se%en-le%en G" insert + 9tore 8["((8 insert + Person9tore bruce !ucci insert + Person9tore bruce se%en-le%en insert + Person9tore $ichael tar!et insert + Person9tore $ichael se%en-le%en

Closer look at types

So #ar, -e%!e spoken about Person and PersonId -ithout really e(plaining -hat they are2 In the simplest sense, #or a SG51only system, the PersonId could 9ust be type PersonId = IntVL2 Ho-e!er, that means there is nothing binding a PersonId at the type le!el to the Person entity2 s a result, you could accidently use a PersonId and get a Car2 In order to model this relationship, -e use phantom types2 So, our ne(t nai!e step -ould be)
newtype Xey entity = Xey IntVL type PersonId = Xey Person

Prior to Persistent H2D, -e used associated types instead o# phantom types2 You could sol!e the problem in that direction as -ell, but phantoms -orked out better2 nd that -orks out really -ell, until you get to a backend that doesn%t use IntD@ #or its IDs2 nd that%s not 9ust a theoretical 3uestion7 &ongoDB uses Byte9trin!s instead2 So -hat -e need is a key !alue that can contain an Int and a Byte9trin!2 Seems like a great time #or a sum type)
data Xey entity = XeyInt IntVL | XeyByte9trin! Byte9trin!

But that%s 9ust asking #or trouble2 0e(t -e%ll ha!e a backend that uses timestamps, so -e%ll need to add another constructor to Xey2 This could go on #or a -hile2 Fortunately, -e already ha!e a sum type intended #or representing arbitrary data) PersistValue)
newtype Xey entity = Xey PersistValue

But this has another problem2 5et%s say -e ha!e a -eb application that takes an ID as a parameter #rom the user2 It -ill need to recei!e that parameter as Text and then try to con!ert it to a Xey2 Well, that%s simple) -rite a #unction to con!ert a Text to a PersistValue, and then -rap the result in the Xey constructor, right=

*HH

Wrong2 We tried this, and there%s a big problem2 We end up getting Xeys that could ne!er be2 For e(ample, i# -e%re dealing -ith SG5, a key must be an integer2 But the approach described abo!e -ould allo- arbitrary te(tual data in2 The result -as a bunch o# AHH ser!er errors as the database choked on comparing an integer column to te(t2 So -hat -e need is a -ay to con!ert te(t to a Xey, but ha!e it dependent on the rules o# the backend in 3uestion2 nd once phrased that -ay, the ans-er is simple) 9ust add another phantom2 The real, actual de#inition o# Xey in Persistent is)
newtype Xey backend entity = Xey 3 unXey == PersistValue 7

This -orks great) -e can ha!e a Text "< Xey Mon!o1B entity #unction and a Text "< Xey 9ElPersist entity #unction, and e!erything runs smoothly2 But no- -e ha!e a neproblem) relations2 5et%s say -e -ant to represent blogs and blog posts2 We -ould use the entity de#inition)
Blo! Post title Text title Text blo!Id Blo!Id

But -hat -ould that look like in terms o# our Xey datatype=
data Blo! = Blo! 3 blo!Title == Text 7 data Post = Post 3 postTitle == Text> postBlo!Id == Xey Gwhat !oes here]< Blo! 7

We need something to #ill in as the backend2 In theory, -e could hardcode this to 9ElPersist, or Mon!o, but then our datatypes -ill only -ork #or a single backend2 For an indi!idual application, that might be acceptable, but -hat about libraries de#ining datatypes to be used by multiple applications, using multiple backends= So things got a little more complicated2 /ur types are actually)
data Blo!Generic backend = Blo! 3 blo!Title == Text 7 data PostGeneric backend = Post 3 postTitle == Text> postBlo!Id == Xey backend @Blo!Generic backendA 7

0otice that -e still keep the short names #or the constructors and the records2 Finally, to gi!e a simple inter#ace #or normal code, -e de#ine some type synonyms)
type type type type Blo! = Blo!Id Post = PostId Blo!Generic 9ElPersist = Xey 9ElPersist Blo! PostGeneric 9ElPersist = Xey 9ElPersist Post

nd no, 9ElPersist isn%t hard1coded into Persistent any-here2 That sEl9ettin!s parameter you%!e been passing to $kPersist is -hat tells us to use 9ElPersist2 &ongo code -ill use $on!o9ettin!s instead2

*H*

This might be 3uite complicated under the sur#ace, but user code hardly e!er touches this2 5ook back through this -hole chapter) not once did -e need to deal -ith the Xey or Generic stu## directly2 The most common place #or it to pop up is in compiler error messages2 So it%s important to be a-are that this e(ists, but it shouldn%t a##ect you on a day1to1day basis2

Custom !ields
/ccassionally, you -ill -ant to de#ine a custom #ield to be used in your datastore2 The most common case is an enumeration, such as employment status2 For this, Persistent pro!ides a helper Template Haskell #unction)
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> Te$plate*askell> ,%erloaded9trin!s> G)1Ts> ?lexibleContexts 4"7 i$port 1atabase/Persist i$port 1atabase/Persist/9Elite i$port 1atabase/Persist/T* data -$ploy$ent = -$ployed | 6ne$ployed | Ietired deri%in! @9how> Iead> -EA deri%ePersist?ield 8-$ploy$ent8 share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person na$e 9trin! e$ploy$ent -$ploy$ent |; $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration $i!rate)ll insert + Person 8Bruce Cayne8 Ietired insert + Person 8Peter Parker8 6ne$ployed insert + Person 8Michael8 -$ployed deri%ePersist?ield stores the data in the database using a string #ield, and per#orms marshaling using the 9how and Iead instances o# the datatype2 This may not be as e##icient

as storing !ia an integer, but it is much more #uture proo#) e!en i# you add e(tra constructors in the #uture, your data -ill still be !alid2

Persistent= %a3 S"L

The Persistent package pro!ides a type sa#e inter#ace to data stores2 It tries to be backend1 agnostic, such as not relying on relational #eatures o# SG52 &y e(perience has been you can easily per#orm BAV o# -hat you need to do -ith the high1le!el inter#ace2 4In #act, most o# my -eb apps use the high le!el inter#ace e(clusi!ely26 But occassionally you%ll -ant to use a #eature that%s speci#ic to a backend2 /ne #eature I%!e used in the past is #ull te(t search2 In this case, -e%ll use the SG5 I5IP"I operator, -hich is not modeled in Persistent2 We%ll get all people -ith the last name ISnoymanI and print the records out2

*H:

ctually, you can e(press a 5IP" operator directly in the normal synta( due to a #eature added in Persistent H2D, -hich allo-s backend1speci#ic operators2 But this is still a good e(ample, so let%s roll -ith it2
3"4 5) G6)G- ,%erloaded9trin!s> Te$plate*askell> FuasiFuotes> Type?a$ilies 4"7 3"4 5) G6)G- Generali'ed ewtype1eri%in!> G)1Ts> ?lexibleContexts 4"7 i$port 1atabase/Persist/9Elite @with9EliteConnA i$port 1atabase/Persist/T* @$kPersist> persist> share> $kMi!rate> sEl9ettin!sA i$port 1atabase/Persist/Generic9El @run9ElConn> runMi!ration> 9ElPersistA i$port 1atabase/Persist/Generic9El/Iaw @with9t$tA i$port 1ata/Text @TextA i$port 1atabase/Persist i$port 1atabase/Persist/9tore @PersistValueA i$port Control/Monad/I,/Class @li#tI,A i$port Euali#ied 1ata/Conduit as C i$port Euali#ied 1ata/Conduit/5ist as C5 share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person na$e Text |; $ain == I, @A $ain = with9EliteConn 8=$e$ory=8 + run9ElConn + do runMi!ration $i!rate)ll insert + Person 8Michael 9noy$an8 insert + Person 8Miria$ 9noy$an8 insert + Person 8-lie'er 9noy$an8 insert + Person 8Ga%riella 9noy$an8 insert + Person 8Gre! Ceber8 insert + Person 8Iick Iichardson8 the "" Persistent does not pro%ide the 5IX- keyword> but wePd like to !et "" whole 9noy$an #a$ily/// let sEl = 89-5-CT na$e ?I,M Person C*-I- na$e 5IX- P09noy$anP8 C/runIesourceT + with9t$t sEl :; C/++ C5/$apMB + li#tI, / print

There is also higher1le!el support that allo-s #or automated data marshaling2 Please see the Haddock PI docs #or more details2

Integration 3ith Yesod

So you%!e been con!inced o# the po-er o# Persistent2 Ho- do you integrate it -ith your Yesod application= I# you use the sca##olding, most o# the -ork is done #or you already2 But as -e normally do, -e%ll build up e!erything manually here to point out ho- it -orks under the sur#ace2 The yesod1persistent package pro!ides the meeting point bet-een Persistent and Yesod2 It pro!ides the HesodPersist typeclass, -hich standardi'es access to the database !ia the run1B method2 5et%s see this in action2
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> Generali'ed ewtype1eri%in!> ?lexibleContexts 4"7

*H>

3"4 5) G6)G- Te$plate*askell> ,%erloaded9trin!s> G)1Ts> MultiPara$TypeClasses 4"7 i$port Hesod i$port 1atabase/Persist/9Elite "" 1e#ine our entities as usual share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| Person #irst a$e 9trin! last a$e 9trin! a!e Int Gt 1esc deri%in! 9how |; "" Ce keep our connection pool in the #oundation/ )t pro!ra$ initiali'ation> we "" create our initial pool> and each ti$e we need to per#or$ an action we check "" out a sin!le connection #ro$ the pool/ data PersistTest = PersistTest ConnectionPool "" CePll create a sin!le route> to access a person/ ItPs a %ery co$$on "" occurrence to use an Id type in routes/ $kHesod 8PersistTest8 :parseIoutes| .person.4PersonId PersonI G-T |; "" othin! special here instance Hesod PersistTest "" ow we need to de#ine a HesodPersist instance> which will keep track o# "" which backend wePre usin! and how to run an action/ instance HesodPersist PersistTest where type HesodPersistBackend PersistTest = 9ElPersist run1B action = do PersistTest pool G" !etHesod run9ElPool action pool "" CePll Sust return the show %alue o# a person> or a LKL i# the Person doesnPt "" exist/ !etPersonI == PersonId "< *andler IepPlain !etPersonI personId = do person G" run1B + !etLKL personId return + IepPlain + toContent + show person openConnectionCount == Int openConnectionCount = (K $ain == I, @A $ain = with9ElitePool 8test/db&8 openConnectionCount + 2pool "< do run9ElPool @runMi!ration $i!rate)llA pool run9ElPool @insert + Person 8Michael8 89noy$an8 NVA pool warp1ebu! &KKK + PersistTest pool

There are t-o important pieces here #or general use2 run1B is used to run a DB action #rom -ithin a *andler2 Within the run1B, you can use any o# the #unctions -e%!e spoken about so #ar, such as insert and select5ist2

*H@

The type o# run1B is run1B == Hesod1B sub $aster a "< G*andler sub $aster a2 Hesod1B is de#ined as)
type Hesod1B sub $aster = HesodPersistBackend $aster @G*andler sub $asterA Since it is built on top o# the HesodPersistBackend associated type, it uses the appropriate

database backend based on the current site2 The other ne- #eature is !etLKL2 It -orks 9ust like !et, but instead o# returning a othin! -hen a result can%t be #ound, it returns a @H@ message page2 The !etPersonI #unction is a !ery common approach used in real1-orld Yesod applications) !etLKL a !alue and then return a response based on it2

Summary
Persistent brings the type sa#ety o# Haskell to your data access layer2 Instead o# -riting error1 prone, untyped data access, or manually -riting boilerplate marshal code, you can rely on Persistent to automate the process #or you2 The goal is to pro!ide e!erything you need, most o# the time2 For the times -hen you need something a bit more po-er#ul, Persistent gi!es you direct access to the underlying data store, so you can -rite -hate!er A1-ay 9oins you -ant2 Persistent integrates directly into the general Yesod -ork#lo-2 0ot only do helper packages like yesod"persistent pro!ide a nice layer, but packages like yesod"#or$ and yesod"auth also le!erage Persistent%s #eatures as -ell2

(eploying your $e-app

I can%t speak #or others, but I personally pre#er programming to system administration2 But the #act is that, e!entually, you need to ser!e your app someho-, and odds are that you%ll need to be the one to set it up2 There are some promising initiati!es in the Haskell -eb community to-ards making deployment easier2 In the #uture, -e may e!en ha!e a ser!ice that allo-s you to deploy your app -ith a single command2 But -e%re not there yet2 nd e!en i# -e -ere, such a solution -ill ne!er -ork #or e!eryone2 This chapter co!ers the di##erent options you ha!e #or deployment, and gi!es some general recommendations on -hat you should choose in di##erent situations2

Compiling
First things #irst) ho- do you build your production application= I# you%re using the sca##olded site, it%s as simple as cabal build2 I also recommend cleaning be#orehand to make sure there is no cached in#ormation, so a simple combination to build your e(ecutable is)
cabal clean TT cabal con#i!ure TT cabal build

*HA

$arp
s -e ha!e mentioned be#ore, Yesod is built on the Web pplication Inter#ace 4W I6, allo-ing it to run on any W I backend2 t the time o# -riting, the #ollo-ing backends are a!ailable)

Warp Fast$;I S$;I $;I Webkit De!elopment ser!er

The last t-o are not intended #or production deployments2 /# the remaining #our, all can be used #or production deployment in theory2 In practice, a $;I backend -ill likely be horribly ine##icient, since a ne- process must be spa-ned #or each connection2 nd S$;I is not nearly as -ell supported by #rontend -eb ser!ers as Warp 4!ia re!erse pro(ying6 or Fast$;I2 So bet-een the t-o remaining choices, Warp gets a !ery strong recommendation because)

It is signi#icantly #aster2 5ike Fast$;I, it can run behind a #rontend ser!er like 0gin(, using re!erse HTTP pro(y2 In addition, it is a #ully capable ser!er o# its o-n accord, and can there#ore be used -ithout any #rontend ser!er2

So that lea!es one last 3uestion) should Warp run on its o-n, or !ia re!erse pro(y behind a #rontend ser!er= For most use cases, I recommend the latter, because)

s #ast as Warp is, it is still optimi'ed as an application ser!er, not a static #ile ser!er2 8sing 0gin(, you can set up !irtual hosting to ser!e your static contents #rom a separate domain2 4It%s possible to do this -ith Warp, but a bit more in!ol!ed62 You can use 0gin( as either a load balancer or a SS5 pro(y2 4Though -ith -arp1tls it%s entirely possible to run an https site on Warp alone26

So my #inal recommendation is) set up 0gin( to re!erse pro(y to Warp2 number o# people in the Yesod community disagree -ith me here, and belie!e that the increased per#ormance and decreased comple(ity o# skipping the 0gin( step make standalone Warp a better choice2 Feel #ree to #ollo- either approach, they are both per#ectly !alid2

Configuration
In general, 0gin( -ill listen on port +H and your Yesod<Warp app -ill listen on some unpri!ileged port 4lets say @>:*62 You -ill then need to pro!ide a ngin(2con# #ile, such as)
dae$on o##R 4 1onPt run n!inx in the back!round> !ood #or $onitorin! apps e%ents 3

*HD

workerBconnections LK\VR

http 3 ser%er 3 listen DKR 4 Inco$in! port #or !inx ser%erBna$e www/$yser%er/co$R location . 3 proxyBpass http=..(N[/K/K/(=L&N(R 4 Ie%erse proxy to your Hesod app 7 7 7

You can add as many ser!er blocks as you like2 common addition is to ensure users al-ays access your pages -ith the --- pre#i( on the domain name, ensuring the R"ST#ul principle o# canonical 8R5s2 4You could 9ust as easily do the opposite and al-ays strip the ---, 9ust make sure that your choice is re#lected in both the ngin( con#ig and the approot o# your site26 In this case, -e -ould add the block)
ser%er 3 listen DKR ser%erBna$e $yser%er/co$R rewrite Q.@/WA http=..www/$yser%er/co$.+( per$anentR 7

highly recommended optimi'ation is to ser!e static #iles #rom a separate domain name, there#ore bypassing the cookie trans#er o!erhead2 ssuming that our static #iles are stored in the static #older -ithin our site #older, and the site #older is located at .ho$e.$ichael.sites.$ysite, this -ould look like)
ser%er 3 listen DKR ser%erBna$e static/$yser%er/co$R root .ho$e.$ichael.sites.$ysite.staticR 4 9ince yesod"static appends a content hash in the Euery strin!> 4 we are #ree to set expiration dates #ar in the #uture without 4 concerns o# stale content/ expires $axR 7

In order #or this to -ork, your site must properly re-rite static 8R5s to this alternate domain name2 The sca##olded site is set up to make this #airly simple !ia the 9ettin!s/staticIoot #unction and the de#inition o# urlIender,%erride2 Ho-e!er, i# you 9ust -ant to get the bene#it o# ngin(%s #aster static #ile ser!ing -ithout dealing -ith separate domain names, you can instead modi#y your original ser!er block like so)
ser%er 3 listen DKR 4 Inco$in! port #or !inx ser%erBna$e www/$yser%er/co$R location . 3 proxyBpass http=..(N[/K/K/(=L&N(R 4 Ie%erse proxy to your Hesod app 7 location .static 3 root .ho$e.$ichael.sites.$ysiteR 4 otice that we do WnotW include .static

*HF

7 7

expires $axR

Ser er Process
&any people are #amiliar -ith an pache<modWphp or 5ighttpd<Fast$;I kind o# setup, -here the -eb ser!er automatically spa-ns the -eb application2 With ngin(, either #or re!erse pro(ying or Fast$;I, this is not the case) you are responsible to run your o-n process2 I strongly recommend a monitoring utility -hich -ill automatically restart your application in case it crashes2 There are many great options out there, such as angel or daemontools2 To gi!e a concrete e(ample, here is an 8pstart con#ig #ile2 The #ile must be placed in .etc.init.$ysite/con#)
description 8My aweso$e Hesod application8 start on runle%el :N&LM;R stop on runle%el :JN&LM;R respawn chdir .ho$e.$ichael.sites.$ysite exec .ho$e.$ichael.sites.$ysite.dist.build.$ysite.$ysite

/nce this is in place, bringing up your application is as simple as sudo start $ysite2

!astC0I
Some people may pre#er using Fast$;I #or deployment2 In this case, you%ll need to add an e(tra tool to the mi(2 Fast$;I -orks by recei!ing ne- connection #rom a #ile descriptor2 The $ library assumes that this #ile descriptor -ill be H 4standard input6, so you need to use the spa-n1#cgi program to bind your application%s standard input to the correct socket2 It can be !ery con!enient to use 8ni( named sockets #or this instead o# binding to a port, especially -hen hosting multiple applications on a single host2 possible script to load up your app could be)
spawn"#c!i 2 "d .ho$e.$ichael.sites.$ysite 2 "s .t$p.$ysite/socket 2 "n 2 "M M(( 2 "u $ichael 2 "" .ho$e.$ichael.sites.$ysite.dist.build.$ysite"#astc!i.$ysite"#astc!i

You -ill also need to con#igure your #rontend ser!er to speak to your app o!er Fast$;I2 This is relati!ely painless in 0gin()
ser%er 3 listen DKR ser%erBna$e www/$yser%er/co$R location . 3 #astc!iBpass unix=.t$p.$ysite/socketR 7

*H+

That should look pretty #amiliar #rom abo!e2 The only last trick is that, -ith 0gin(, you need to manually speci#y all o# the Fast$;I !ariables2 It is recommended to store these in a separate #ile 4say, #astcgi2con#6 and then add include #astc!i/con#R to the end o# your http block2 The contents o# the #ile, to -ork -ith W I, should be)
#astc!iBpara$ #astc!iBpara$ #astc!iBpara$ #astc!iBpara$ #astc!iBpara$ #astc!iBpara$ #astc!iBpara$ #astc!iBpara$ #astc!iBpara$ #astc!iBpara$ #astc!iBpara$ #astc!iBpara$ F6-IHB9TII G I-F6-9TBM-T*,1 C, T- TBTHPC, T- TB5- GT* P)T*BI ?, 9-IV-IBPI,T,C,5 G)T-C)HBI T-I?)C9-IV-IB9,?TC)II-M,T-B)11I 9-IV-IB)11I 9-IV-IBP,IT 9-IV-IB )M+EueryBstrin!R +reEuestB$ethodR +contentBtypeR +contentBlen!thR +#astc!iBscriptBna$eR +ser%erBprotocolR CGI.(/(R n!inx.+n!inxB%ersionR +re$oteBaddrR +ser%erBaddrR +ser%erBportR +ser%erBna$eR

(esktop
nother ni#ty backend is -ai1handler1-ebkit2 This backend combines Warp and GtWebkit to create an e(ecutable that a user simply double1clicks2 This can be a con!enient -ay to pro!ide an o##line !ersion o# your application2 /ne o# the !ery nice con!eniences o# Yesod #or this is that your templates are all compiled into the e(ecutable, and thus do not need to be distributed -ith your application2 Static #iles do, ho-e!er2 There%s actually support #or embedding your static #iles directly in the e(ecutable as -ell, see the yesod1static docs #or more details2 similar approach, -ithout re3uiring the GtWebkit library, is -ai1handler1launch, -hich launches a Warp ser!er and then opens up the user%s de#ault -eb bro-ser2 There%s a little trickery in!ol!ed here) in order to kno- that the user is still using the site, wai"handler" launch inserts a IpingI .a!ascript snippet to e!ery HT&5 page it ser!es2 It wai"handler" launch doesn%t recei!e a ping #or t-o minutes, it shuts do-n2

C0I on Apache
$;I and Fast$;I -ork almost identically on pache, so it should be #airly straight1#or-ard to port this con#iguration2 You essentially need to accomplish t-o goals) *2 ;et the ser!er to ser!e your #ile as 4Fast6$;I2 :2 Re-rite all re3uests to your site to go through the 4Fast6$;I e(ecutable2 Here is a con#iguration #ile #or ser!ing a blog application, -ith an e(ecutable named Ibloggy2cgiI, li!ing in a sub#older named IblogI o# the document root2 This e(ample -as taken #rom an application li!ing in the path .#M.snoy$an.public.blo!2

*HB

,ptions U-xecCGI )dd*andler c!i"script /c!i ,ptions U?ollow9y$links Iewrite-n!ine ,n IewriteIule Q.#M.snoy$an.public.blo!+ .blo!. :I=&K(>9=(; IewriteCond +( JQblo!!y/c!i IewriteCond +( JQstatic. IewriteIule Q@/WA blo!!y/c!i.+( :5;

The #irst Re-riteRule is to deal -ith sub#olders2 In particular, it redirects a re3uest #or .blo! to .blo!.2 The #irst Re-rite$ond pre!ents directly re3uesting the e(ecutable, the second allo-s pache to ser!e the static #iles, and the last line does the actual re-riting2

!astC0I on lighttpd
For this e(ample, I%!e le#t o## some o# the basic Fast$;I settings like mime1types2 I also ha!e a more comple( #ile in production that prepends I---2I -hen absent and ser!es static #iles #rom a separate domain2 Ho-e!er, this should ser!e to sho- the basics2 Here, I<home<michael<#astcgiI is the #astcgi application2 The idea is to re-rite all re3uests to start -ith I<appI, and then ser!e e!erything beginning -ith I<appI !ia the Fast$;I e(ecutable2
ser%er/port = &KKK ser%er/docu$ent"root = 8.ho$e.$ichael8 ser%er/$odules = @8$odB#astc!i8> 8$odBrewrite8A url/rewrite"once = @ 8@/WA8 =< 8.app.+(8 A #astc!i/ser%er = @ 8.app8 =< @@ 8socket8 =< 8.t$p.test/#astc!i/socket8> 8check"local8 =< 8disable8> 8bin"path8 =< 8.ho$e.$ichael.#astc!i8> 4 #ull path to executable 8$in"procs8 =< (> 8$ax"procs8 =< &K> 8idle"ti$eout8 =< &K AA A

C0I on lighttpd
This is basically the same as the Fast$;I !ersion, but tells lighttpd to run a #ile ending in I2cgiI as a $;I e(ecutable2 In this case, the #ile li!es at I<home<michael<myapp2cgiI2
ser%er/port = &KKK ser%er/docu$ent"root = 8.ho$e.$ichael8 ser%er/$odules = @8$odBc!i8> 8$odBrewrite8A url/rewrite"once = @ 8@/WA8 =< 8.$yapp/c!i.+(8

**H

A c!i/assi!n = @8/c!i8 =< 88A

%/STful Content
/ne o# the stories #rom the early days o# the -eb is ho- search engines -iped out entire -ebsites2 When dynamic -eb sites -ere still a ne- concept, de!elopers didn%t appreciate the di##erence bet-een a G-T and P,9T re3uest2 s a result, they created pages1 accessed -ith the G-T method1 that -ould delete pages2 When search engines started cra-ling these sites, they could -ipe out all the content2 I# these -eb de!elopers had #ollo-ed the HTTP spec properly, this -ould not ha!e happened2 G-T re3uest is supposed to cause no side e##ects 4you kno-, like -iping out a site62 Recently, there has been a mo!e in -eb de!elopment to properly embrace Representational State Trans#er, also kno-n as R"ST2 This chapter describes the R"ST#ul #eatures in Yesod and ho- you can use them to create more robust -eb applications2

%e.uest methods
In many -eb #rame-orks, you -rite one handler #unction per resource2 In Yesod, the de#ault is to ha!e a separate handler #unction #or each re3uest method2 The t-o most common re3uest methods you -ill deal -ith in creating -eb sites are ;"T and P/ST2 These are the most -ell1supported methods in HT&5, since they are the only ones supported by -eb #orms2 Ho-e!er, -hen creating R"ST#ul PIs, the other methods are !ery use#ul2 Technically speaking, you can create -hiche!er re3uest methods you like, but it is strongly recommended to stick to the ones spelled out in the HTTP spec2 The most common o# these are) ;"T Read1only re3uests2 ssuming no other changes occur on the ser!er, calling a G-T re3uest multiple times should result in the same response, barring such things as Icurrent timeI or randomly assigned results2 P/ST general mutating re3uest2 P,9T re3uest should ne!er be submitted t-ice by the user2 common e(ample o# this -ould be to trans#er #unds #rom one bank account to another2 P8T $reate a ne- resource on the ser!er, or replace an e(isting one2 This method is sa#e to be called multiple times2 D"5"T" .ust like it sounds) -ipe out a resource on the ser!er2 $alling multiple times should be /P2 To a certain e(tent, this #its in !ery -ell -ith Haskell philosophy) a G-T re3uest is similar to a pure #unction, -hich cannot ha!e side e##ects2 In practice, your G-T #unctions -ill probably per#orm I,, such as reading in#ormation #rom a database, logging user actions, and so on2

***

See the routing and handlers chapter chapter #or more in#ormation on the synta( o# de#ining handler #unctions #or each re3uest method2

%epresentations
Suppose -e ha!e a Haskell datatype and !alue)
data Person = Person 3 na$e == 9trin!> a!e == Int 7 $ichael = Person 8Michael8 NM

We could represent that data as HT&5)

Gtable< Gtr< Gth< a$eG.th< Gtd<MichaelG.td< G.tr< Gtr< Gth<)!eG.th< Gtd<NMG.td< G.tr< G.table<

or -e could represent it as .S/0)

38na$e8=8Michael8>8a!e8=NM7

or as C&5)
Gperson< Gna$e<MichaelG.na$e< Ga!e<NMG.a!e< G.person<

/#ten times, -eb applications -ill use a di##erent 8R5 to get each o# these representations7 perhaps .person.$ichael/ht$l, .person.$ichael/Sson, etc2 Yesod #ollo-s the R"ST#ul principle o# a single 8R5 #or each resource2 So in Yesod, all o# these -ould be accessed #rom .person.$ichael2 Then the 3uestion becomes ho- do -e determine which representation to ser!e2 The ans-er is the HTTP )ccept header) it gi!es a prioriti'ed list o# content types the client is e(pecting2 Yesod -ill automatically determine -hich representation to ser!e based upon this header2 5et%s make that last sentence a bit more concrete -ith some code)
type ChooseIep = :ContentType; "< I, @ContentType> ContentA class *asIeps a where chooseIep == a "< ChooseIep

The chooseIep #unction takes t-o arguments) the !alue -e are getting representations #or, and a list o# content types that the client -ill accept2 We determine this by reading the )ccept

**:

re3uest header2 chooseIep returns a tuple containing the content type o# our response and the actual content2 This typeclass is the core o# Yesod%s R"ST#ul approach to representations2 "!ery handler #unction must return an instance o# *asIeps2 When Yesod generates the dispatch #unction, it automatically applies chooseIep to each handler, essentially gi!ing all #unctions the type *andler ChooseIep2 #ter running the *andler and obtaining the ChooseIep result, it is applied to the list o# content types parsed #rom the )ccept header2 Yesod pro!ides a number o# instances o# *asIeps out o# the bo(2 When -e use de#ault5ayout, #or e(ample, the return type is Iep*t$l, -hich looks like)
newtype Iep*t$l = Iep*t$l Content instance *asIeps Iep*t$l where chooseIep @Iep*t$l contentA B = return @8text.ht$l8> contentA

0otice that -e ignore entirely the list o# e(pected content types2 number o# the built in representations 4Iep*t$l, IepPlain, IepJson, IepY$l6 in #act only support a single representation, and there#ore -hat the client re3uests in the )ccept header is irrele!ant2

%epHtml+son
n e(ample to the contrary is Iep*t$lJson, -hich pro!ides either an HT&5 or .S/0 representation2 This instance helps greatly in programming . C applications that degrade nicely2 Here is an e(ample that returns either HT&5 or .S/0 data, depending on -hat the client -ants2
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> ,%erloaded9trin!s 4"7 3"4 5) G6)G- MultiPara$TypeClasses> Te$plate*askell 4"7 i$port Hesod data I = I $kHesod 8I8 :parseIoutes| . IootI G-T .49trin! a$eI G-T |; instance Hesod I !etIootI = de#ault5ayout + do setTitle 8*o$epa!e8 add9criptIe$ote 8http=..aSax/!oo!leapis/co$.aSax.libs.SEuery.(/L.SEuery/$in/Ss8 toCid!et :Sulius| +@#unction@A3 +@84aSax a8A/click@#unction@A3 SFuery/!etJ9, @+@thisA/attr@8hre#8A> #unction@oA3 +@8di%8A/text@o/na$eAR 7AR return #alseR 7AR 7AR |; let na$es = words 85arry Moe Curly8 :wha$let| GhN<)J)Y Version

**>

Gdi% 4results< )J)Y results will be placed here when you click 4 the na$es below/ Gul 4aSax< +#orall na$e G" na$es Gli< Ga hre#=O3 a$eI na$e7<43na$e7 GhN<*TM5 Version Gp< Clickin! the na$es below will redirect the pa!e 4 to an *TM5 %ersion/ Gul 4ht$l< +#orall na$e G" na$es Gli< Ga hre#=O3 a$eI na$e7<43na$e7 |; !et a$eI na$e = do let wid!et = do setTitle + to*t$l na$e :wha$let|5ooks like you ha%e Ja%ascript o##/ let Sson = obSect :8na$e8 /= na$e; de#ault5ayoutJson wid!et Sson $ain = warp1ebu! LKKK I

a$e= 43na$e7|;

/ur !etIootI handler creates a page -ith three links and some .a!ascript -hich intercept clicks on the links and per#orms asynchronous re3uests2 I# the user has .a!ascript enabled, clicking on the link -ill cause a re3uest to be sent -ith an )ccept header o# application.Sson2 In that case, !et a$eI -ill return the .S/0 representation instead2 I# the user disables .a!ascript, clicking on the link -ill send the user to the appropriate 8R52 -eb bro-ser places priority on an HT&5 representation o# the data, and there#ore the page de#ined by the -idget -ill be returned2 We can o# course e(tend this to -ork -ith C&5, tom #eeds, or e!en binary representations o# the data2 #un e(ercise could be -riting a -eb application that ser!es data simply using the de#ault 9how instances o# datatypes, and then -riting a -eb client that parses the results using the de#ault Iead instances2 You might be concerned about e##iciency here2 Doesn%t this approach mean -e ha!e to generate both an HT&5 and .S/0 response #or each re3uest= Thanks to la'iness, that%s not the case2 In !et a$eI, neither wid!et nor Sson -ill be e!aluated until the appropriate response type has been selected, and there#ore only one o# them -ill e!er be run2

<e3s !eeds
great, practical e(ample o# multiple representations i# the yesod1ne-s#eed package2 There are t-o ma9or #ormats #or ne-s #eeds on the -eb) RSS and tom2 They contain almost e(actly the same in#ormation, but are 9ust packaged up di##erently2

**@

The yesod"news#eed package de#ines a ?eed datatype -hich contains in#ormation like title, description, and last updated time2 It then pro!ides t-o separate sets o# #unctions #or displaying this data) one #or RSS, one #or tom2 They each de#ine their o-n representation datatypes)
newtype Iep)to$ = Iep)to$ Content instance *asIeps Iep)to$ where chooseIep @Iep)to$ cA B = return @type)to$> cA newtype IepIss = IepIss Content instance *asIeps IepIss where chooseIep @IepIss cA B = return @typeIss> cA

But there%s a third module -hich de#ines another datatype)

data Iep)to$Iss = Iep)to$Iss Iep)to$ IepIss instance *asIeps Iep)to$Iss where chooseIep @Iep)to$Iss @Iep)to$ aA @IepIss rAA = chooseIep : @type)to$> aA > @typeIss> rA ;

This datatype -ill automatically ser!e -hiche!er representation the client pre#ers, de#aulting to tom2 I# a client connects that only understands RSS, assuming it pro!ides the correct HTTP headers, Yesod -ill pro!ide RSS output2

Other re.uest headers

There are a great deal o# other re3uest headers a!ailable2 Some o# them only a##ect the trans#er o# data bet-een the ser!er and client, and should not a##ect the application at all2 For e(ample, )ccept"-ncodin! in#orms the ser!er -hich compression schemes the client understands, and *ost in#orms the ser!er -hich !irtual host to ser!e up2 /ther headers do a##ect the application, but are automatically read by Yesod2 For e(ample, the )ccept"5an!ua!e header speci#ies -hich human language 4"nglish, Spanish, ;erman, S-iss1;erman6 the client pre#ers2 See the i*+n chapter #or details on ho- this header is used2

Stateless
I%!e sa!ed this section #or the last, not because it is less important, but rather because there are no speci#ic #eatures in Yesod to en#orce this2 HTTP is a stateless protocol) each re3uest is to be seen as the beginning o# a con!ersation2 This means, #or instance, it doesn%t matter to the ser!er i# you re3uested #i!e pages pre!iously, it -ill treat your si(th re3uest as i# it%s your #irst one2 /n the other hand, some #eatures on -ebsites -on%t -ork -ithout some kind o# state2 For e(ample, ho- can you implement a shopping cart -ithout sa!ing in#ormation about items in bet-een re3uests=

**A

The solution to this is cookies, and built on top o# this, sessions2 We ha!e a -hole section addressing the sessions #eatures in Yesod2 Ho-e!er, I cannot stress enough that this should be used sparingly2 5et me gi!e you an e(ample2 There%s a popular bug tracking system that I deal -ith on a daily basis -hich horribly abuses sessions2 There%s a little drop1do-n on e!ery page to select the current pro9ect2 Seems harmless, right= What that dropdo-n does is set the current pro9ect in your session2 The result o# all this is that clicking on the I!ie- issuesI link is entirely dependent on the last pro9ect you selected2 There%s no -ay to create a bookmark to your IYesodI issues and a separate link #or your IHamletI issues2 The proper R"ST#ul approach to this is to ha!e one resource #or all o# the Yesod issues and a separate one #or all the Hamlet issues2 In Yesod, this is easily done -ith a route de#inition like)
. ProSectsI G-T .proSects.4ProSectI1 ProSectIssuesI G-T .issues.4IssueI1 IssueI G-T

Be nice to your users) proper stateless architecture means that basic #eatures like bookmarks, permalinks and the back<#or-ard button -ill al-ays -ork2

Summary
Yesod adheres to the #ollo-ing tenets o# R"ST)

8se the correct re3uest method2 "ach resource should ha!e precisely one 8R52 llo- multiple representations o# data on the same 8R52 Inspect re3uest headers to determine e(tra in#ormation about -hat the client -ants2

This makes it easy to use Yesod not 9ust #or building -ebsites, but #or building PIs2 In #act, using techni3ues such as RepHtml.son, you can ser!e both a user1#riendly, HT&5 page and a machine1#riendly, .S/0 page #rom the same 8R52

Yesod1s Monads
s you%!e read through this book, there ha!e been a number o# monads -hich ha!e appeared) and Hesod1B 4#or Persistent62 s -ith most monads, each one pro!ides some speci#ic #unctionality) *andler gi!es access to the re3uest and allo-s you to send responses, a Cid!et contains HT&5, $SS, and .a!ascript, and Hesod1B let%s you make database 3ueries2 In &odel1Eie-1$ontroller 4&E$6 terms, -e could consider Hesod1B to be the model, Cid!et to be the !ie-, and *andler to be the controller2
*andler, Cid!et

**D

So #ar, -e%!e presented some !ery straight1#or-ard -ays to use these monads) your main handler -ill run in *andler, using run1B to e(ecute a Hesod1B 3uery, and de#ault5ayout to return a Cid!et, -hich in turn -as created by calls to toCid!et2 Ho-e!er, i# -e ha!e a deeper understanding o# these types, -e can achie!e some #ancier results2

Monad Transformers
&onads are like onions2 &onads are not like cakes2Shrek, more or less Be#ore -e get into the heart o# Yesod%s monads, -e need to understand a bit about monad trans#ormers2 4I# you already kno- all about monad trans#ormers, you can likely skip this section26 Di##erent monads pro!ide di##erent #unctionality) Ieader allo-s read1only access to some piece o# data throughout a computation, -rror allo-s you to short1circuit computations, and so on2 /#ten times, ho-e!er, you -ould like to be able to combine a #e- o# these #eatures together2 #ter all, -hy not ha!e a computation -ith read1only access to some settings !ariable, that could error out at any time= /ne approach to this -ould be to -rite a ne- monad like Ieader-rror, but this has the ob!ious do-nside o# e(ponential comple(ity) you%ll need to -rite a ne- monad #or e!ery single possible combination2 Instead, -e ha!e monad trans#ormers2 In addition to Ieader, -e ha!e IeaderT, -hich adds reader #unctionality to any other monad2 So -e could represent our Ieader-rror as 4conceptually6)
type Ieader-rror = IeaderT -rror

In order to access our settings !ariable, -e can use the ask #unction2 But -hat about short1 circuiting a computation= We%d like to use throw-rror, but that -on%t e(actly -ork2 Instead, -e need to li#t our call into the ne(t monad up2 In other -ords)
throw-rror == errValue "< -rror li#t / throw-rror == errValue "< IeaderT -rror

There are a #e- things you should pick up here)

trans#ormer can be used to add #unctionality to an e(isting monad2 trans#ormer must al-ays -rap around an e(isting monad2 The #unctionality a!ailable in a -rapped monad -ill be dependent not only on the monad trans#ormer, but also on the inner monad that is being -rapped2

great e(ample o# that last point is the I, monad2 0o matter ho- many layers o# trans#ormers you ha!e around an I,, there%s still an I, at the core, meaning you can per#orm I</ in any o# these monad trans#ormer stacks2 You%ll o#ten see code that looks like li#tI, + put9tr5n 8*ello ThereJ82

**F

The Three Transformers

We%!e already discussed t-o o# our trans#ormers pre!iously) *andler and Cid!et2 .ust to recap, there are t-o special things about these trans#ormers) *2 In order to simpli#y error messages, they are not actual trans#ormers2 Instead, they are ne-types that hard1code their inner monads2Remember, this is -hy Yesod pro!ides a speciali'ed li#t #unction, -hich -orks #or *andler and Cid!et2 :2 In reality they ha!e e(tra type parameters #or the sub and master site2 s a result, the Yesod libraries pro!ide G*andler sub $aster a and GCid!et sub $aster a, and each site gets a pair o# type synonyms type *andler = G*andler My)pp My)pp and type Cid!et = GCid!et My)pp My )pp @A2 In persistent, -e ha!e a typeclass called Persist9tore2 This typeclass de#ines all o# the primiti!e operations you can per#orm on a database, like !et2 This typeclass essentially looks like class @Monad @b $AA =< Persist9tore b $2 b is the backend itsel#, and is in #act a monad trans#ormer, -hile $ is the inner monad that b -raps around2 Both SG5 and &ongoDB ha!e their o-n instances7 in the case o# SG5, it looks like)
instance MonadBaseControl I, $ =< PersistBackend 9ElPersist $

This means that you can run a SG5 database -ith any underlying monad, so long as that underlying monad supports MonadBaseControl I,, -hich allo-s you to properly deal -ith e(ceptions in a monad stack2 That basically means any trans#ormer stack built around I, 4besides e(ceptional cases like ContT62 Fortunately #or us, that includes both *andler and Cid!et2 The takea-ay here is that -e can layer our Persistent trans#ormer on top o# *andler or Cid!et2 This -asn%t al-ays the case2 Be#ore Yesod H2*H, Yesod -as built on top o# enumerators, -hich do not support MonadBaseControl2 In Yesod H2*H, -e mo!ed o!er to conduit, -hich greatly simpli#ied e!erything -e%re discussing here2 In order to make it simpler to re#er to the rele!ant Persistent trans#ormer, the yesod1persistent package de#ines the HesodPersistBackend associated type2 For e(ample, i# I ha!e a site called My)pp and it uses SG5, I -ould de#ine something like type instance HesodPersistBackend My)pp = 9ElPersist2 When -e -ant to run our database actions, -e%ll ha!e a 9ElPersist -rapped around a *andler or Cid!et2 We can then use the standard Persistent un-rap #unctions 4like run9ElPool6 to run the action and get back a normal *andler<Cid!et2 To automate this, -e pro!ide the run1B #unction2 Putting it all together, -e can no- run database actions inside our handlers and -idgets2 &ost o# the time in Yesod code, and especially thus #ar in this book, -idgets ha!e been treated as actionless containers that simply combine together HT&5, $SS and .a!ascript2 But i# you look at that last paragraph again, you%ll reali'e that%s not the -ay things ha!e to be2 Since a -idget is a trans#ormer on top o# a handler, anything you do in a handler can be done in a -idget, including database actions2 ll you ha!e to do is li#t2

**+

/,ample= (ata-ase&dri en na -ar

5et%s put some o# this ne- kno-ledge into action2 We -ant to create a Cid!et that generates its output based on the contents o# the database2 Pre!iously, our approach -ould ha!e been to load up the data in a *andler, and then pass that data into a Cid!et2 0o-, -e%ll do the loading o# data in the Cid!et itsel#2 This is a boon #or modularity, as this Cid!et can be used in any *andler -e -ant, -ithout any need to pass in the database contents2
3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> Te$plate*askell> ?lexibleContexts> FuasiFuotes> Type?a$ilies> MultiPara$TypeClasses> G)1Ts 4"7 i$port Hesod i$port 1atabase/Persist/9Elite i$port 1ata/Text @TextA i$port 1ata/Ti$e share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| 5ink title Text url Text added 6TCTi$e |; data 5inks-xa$ple = 5inks-xa$ple ConnectionPool $kHesod 85inks-xa$ple8 :parseIoutes| . IootI G-T .add"link )dd5inkI P,9T |; instance Hesod 5inks-xa$ple instance IenderMessa!e 5inks-xa$ple ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e instance HesodPersist 5inks-xa$ple where type HesodPersistBackend 5inks-xa$ple = 9ElPersist run1B db = do 5inks-xa$ple pool G" !etHesod run9ElPool db pool !etIootI == *andler Iep*t$l !etIootI = de#ault5ayout :wha$let| G#or$ $ethod=post action=O3)dd5inkI7< Gp< )dd a new link to 4 Ginput type=url na$e=url %alue=http=..< 2 titled 4 Ginput type=text na$e=title< 2 4 Ginput type=sub$it %alue=8)dd link8< GhN<-xistin! links Q3existin!5inks7 |; existin!5inks == Cid!et existin!5inks = do links G" li#t + run1B + select5ist :; :5i$itTo M> 1esc 5ink)dded;

**B

Gul<

:wha$let| +#orall -ntity B link G" links Gli< Ga hre#=43link6rl link7<43linkTitle link7

|;

post)dd5inkI == *andler @A post)dd5inkI = do url G" runInputPost + ireE url?ield 8url8 title G" runInputPost + ireE text?ield 8title8 now G" li#tI, !etCurrentTi$e run1B + insert + 5ink title url now setMessa!e 85ink added8 redirect IootI $ain == I, @A $ain = with9ElitePool 8links/db&8 (K + 2pool "< do run9ElPool @runMi!ration $i!rate)llA pool warp1ebu! &KKK + 5inks-xa$ple pool

Pay attention in particular to the existin!5inks #unction2 0otice ho- all -e needed to do -as apply li#t to a normal database action2 nd #rom -ithin !etIootI, -e treated existin!5inks like any ordinary Cid!et, no special parameters at all2 See the #igure #or the output o# this app2

Screenshot of the na -ar

/,ample= %e.uest information

5ike-ise, you can get re3uest in#ormation inside a Cid!et2 Here -e can determine the sort order o# a list based on a ;"T parameter2
3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> Te$plate*askell> FuasiFuotes> Type?a$ilies> MultiPara$TypeClasses> G)1Ts 4"7 i$port Hesod i$port 1ata/Text @TextA i$port 1ata/5ist @sortByA i$port 1ata/,rd @co$parin!A data Person = Person

*:H

3 person a$e == Text > person)!e == Int 7 people == :Person; people = : Person 8Miria$8 NM > Person 8-lie'er8 & > Person 8Michael8 NV > Person 8Ga%riella8 ( ; data People = People $kHesod 8People8 :parseIoutes| . IootI G-T |; instance Hesod People instance IenderMessa!e People ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e !etIootI == *andler Iep*t$l !etIootI = de#ault5ayout :wha$let| Gp< Ga hre#=8]sort=na$e8<9ort by na$e 2 | 4 Ga hre#=8]sort=a!e8<9ort by a!e 2 | 4 Ga hre#=8]8< o sort Q3showPeople7 |; showPeople == Cid!et showPeople = do $sort G" li#t + runInputGet + iopt text?ield 8sort8 let peopleP = case $sort o# Just 8na$e8 "< sortBy @co$parin! person a$eA people Just 8a!e8 "< sortBy @co$parin! person)!eA people B "< people :wha$let| Gdl< +#orall person G" peopleP Gdt<43person a$e person7 Gdd<43show + person)!e person7 |; $ain == I, @A $ain = warp1ebu! &KKK People

/nce again, all -e need to do is li#t our normal *andler code 4in this case, runInputGet6 to ha!e it run in our Cid!et2

*:*

Summary
I# you completely ignore this chapter, you%ll still be able to use Yesod to great bene#it2 The ad!antage o# understanding ho- Yesod%s monads interact is to be able to produce cleaner, more modular code2 Being able to per#orm arbitrary actions in a Cid!et can be a po-er#ul tool, and understanding ho- Persistent and your *andler code interact can help you make more in#ormed design decisions in your app2

Authentication and Authori6ation

uthentication and authori'ation are t-o !ery related, and yet separate, concepts2 While the #ormer deals -ith identi#ying a user, the latter determines -hat a user is allo-ed to do2 8n#ortunately, since both terms are o#ten abbre!iated as Iauth,I the concepts are o#ten con#lated2 Yesod pro!ides built1in support #or a number o# third1party authentication systems, such as /penID, Bro-serID and / uth2 These are systems -here your application trusts some e(ternal system #or !alidating a user%s credentials2 dditionally, there is support #or more commonly used username<pass-ord and email<pass-ord systems2 The #ormer route ensures simplicity #or users 4no ne- pass-ords to remember6 and implementors 4no need to deal -ith an entire security architecture6, -hile the latter gi!es the de!eloper more control2 /n the authori'ation side, -e are able to take ad!antage o# R"ST and type1sa#e 8R5s to create simple, declarati!e systems2 dditionally, since all authori'ation code is -ritten in Haskell, you ha!e the #ull #le(ibility o# the language at your disposal2 This chapter -ill co!er ho- to set up an IauthI solution in Yesod and discuss some trade1o##s in the di##erent authentication options2

O er ie3
The yesod1auth package pro!ides a uni#ied inter#ace #or a number o# di##erent authentication plugins2 The only real re3uirement #or these backends is that they identi#y a user based on some uni3ue string2 In /penID, #or instance, this -ould be the actual /penID !alue2 In Bro-serID, it%s the email address2 For HashDB 4-hich uses a database o# hashed pass-ords6, it%s the username2 "ach authentication plugin pro!ides its o-n system #or logging in, -hether it be !ia passing tokens -ith an e(ternal site or a email<pass-ord #orm2 #ter a success#ul login, the plugin sets a !alue in the user%s session to indicate his<her )uthId2 This )uthId is usually a Persistent ID #rom a table used #or keeping track o# users2 There are a #e- #unctions a!ailable #or 3uerying a user%s )uthId, most commonly $aybe)uthId, reEuire)uthId, $aybe)uth and reEuire)uth2 The re3uire !ersions -ill redirect to a login page i# the user is not logged in, -hile the second set o# #unctions 4the ones not ending in Id6 gi!e both the table ID and entity !alue2

*::

Since all o# the storage o# )uthId is built on top o# sessions, all o# the rules #rom there apply2 In particular, the data is stored in an encrypted, H& $ed client cookie, -hich automatically times out a#ter a certain con#igurable period o# inacti!ity2 dditionally, since there is no ser!er1side component to sessions, logging out simply deletes the data #rom the session cookie7 i# a user reuses an older cookie !alue, the session -ill still be !alid2 There are plans to add in a ser!er1side component to sessions -hich -ould allo- #orced logout2 This -ill almost certainly be implemented be#ore a *2H release o# Yesod2 /n the #lip side, authori'ation is handled by a #e- methods inside the Hesod typeclass2 For e!ery re3uest, these methods are run to determine i# access should be allo-ed, denied, or i# the user needs to be authenticated2 By de#ault, these methods allo- access #or e!ery re3uest2 lternati!ely, you can implement authori'ation in a more ad1hoc -ay by adding calls to reEuire)uth and the like -ithin indi!idual handler #unctions, though this undermines many o# the bene#its o# a declarati!e authori'ation system2

Authenticate Me
5et%s 9ump right in -ith an e(ample o# authentication2
3"4 5) G6)G- ,%erloaded9trin!s> Te$plate*askell> Type?a$ilies> MultiPara$TypeClasses> FuasiFuotes 4"7 i$port Hesod i$port Hesod/)uth i$port Hesod/)uth/BrowserId i$port Hesod/)uth/Goo!le-$ail i$port 1ata/Text @TextA i$port etwork/*TTP/Conduit @Mana!er> newMana!er> de#A data My)uth9ite = My)uth9ite 3 httpMana!er == Mana!er 7 $kHesod 8My)uth9ite8 :parseIoutes| . IootI G-T .auth )uthI )uth !et)uth |; instance Hesod My)uth9ite where "" ote= In order to lo! in with BrowserI1> you $ust correctly "" set your hostna$e here/ approot = )pproot9tatic 8http=..localhost=&KKK8 instance Hesod)uth My)uth9ite where type )uthId My)uth9ite = Text !et)uthId = return / Just / credsIdent lo!in1est B = IootI lo!out1est B = IootI authPlu!ins B = : authBrowserId > authGoo!le-$ail ;

*:>

auth*ttpMana!er = httpMana!er instance IenderMessa!e My)uth9ite ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e !etIootI == *andler Iep*t$l !etIootI = do $aid G" $aybe)uthId de#ault5ayout :wha$let| Gp<Hour current auth I1= 43show $aid7 +$aybe B G" $aid Gp< Ga hre#=O3)uthI 5o!outI7<5o!out +nothin! Gp< Ga hre#=O3)uthI 5o!inI7<Go to the lo!in pa!e |; $ain == I, @A $ain = do $an G" newMana!er de# warp1ebu! &KKK + My)uth9ite $an

We%ll start -ith the route declarations2 First -e declare our standard IootI route, and then -e set up the authentication subsite2 Remember that a subsite needs #our parameters) the path to the subsite, the route name, the subsite name, and a #unction to get the subsite !alue2 In other -ords, based on the line)
.auth )uthI )uth !et)uth

We need to ha!e !et)uth == My)uth9ite "< )uth2 While -e ha!en%t -ritten that #unction oursel!es, yesod1auth pro!ides it automatically2 With other subsites 4like static #iles6, -e pro!ide con#iguration settings in the subsite !alue, and there#ore need to speci#y the get #unction2 In the auth subsite, -e speci#y these settings in a separate typeclass, Hesod)uth2 Why not use the subsite !alue= There are a number o# settings -e -ould like to gi!e #or an auth subsite, and doing so #rom a record type -ould be incon!enient2 lso, since -e -ant to ha!e an )uthId associated type, a typeclass is more natural2 /n the #lip side, -hy not use a typeclass #or all subsites= It comes -ith a do-nside) you can then only ha!e a single instance per site, disallo-ing ser!ing di##erent sets o# static #iles #rom di##erent routes2 lso, the subsite !alue -orks better -hen -e -ant to load data at app initiali'ation2 So -hat e(actly goes in this Yesod uth instance= There are si( re3uired declarations)

is an associated type2 This is the !alue yesod"auth -ill gi!e you -hen you ask i# a user is logged in 4!ia $aybe)uthId or reEuire)uthId62 In our case, -e%re simply using Text, to store the ra- identi#ier1 email address in our case, as -e%ll soon see2 !et)uthId gets the actual )uthId #rom the Creds 4credentials6 data type2 This type has three pieces o# in#ormation) the authentication backend used 4bro-serid or googleemail in our case6, the actual identi#ier, and an associated list o# arbitrary e(tra
)uthId

*:@

in#ormation2 "ach backend pro!ides di##erent e(tra in#ormation7 see their docs #or more in#ormation2 lo!in1est gi!es the route to redirect to a#ter a success#ul login2 5ike-ise, lo!out1est gi!es the route to redirect to a#ter a logout2 authPlu!ins is a list o# indi!idual authentication backends to use2 In our e(ample, -e%re using Bro-serID, -hich logs in !ia &o'illa%s Bro-serID system, and ;oogle "mail, -hich authenticates a user%s email address using their ;oogle account2 The nice thing about these t-o backends is) o They re3uire no set up, as opposed to Facebook or / uth, -hich re3uire setting up credentials2 o They use email addresses as identi#iers, -hich people are com#ortable -ith, as opposed to /penID, -hich uses a 8R52 auth*ttpMana!er gets an HTTP connection manager #rom the #oundation type2 This allo- authentication backends -hich use HTTP connections 4i2e2, almost all third1 party login systems6 to share connections, a!oiding the cost o# restarting a T$P connection #or each re3uest2

In our IootI handler, -e ha!e some simple links to the login and logout pages, depending on -hether or not the user is logged in2 0otice ho- -e construct these subsite links) #irst -e gi!e the subsite route name 4)uthI6, #ollo-ed by the route -ithin the subsite 45o!inI and 5o!outI62 The #igures belo- sho- -hat the login process looks like #rom a user perspecti!e2

Initial page load

*:A

#ro3serI( login screen

Homepage after logging in

/mail
For many use cases, third1party authentication o# email -ill be su##icient2 /ccassionally, you%ll -ant users to actual create pass-ords on your site2 The sca##olded site does not include this setup, because)

In order to securely accept pass-ords, you need to be running o!er SS52 &any users are not ser!ing their sites o!er SS52 While the email backend properly salts and hashes pass-ords, a compromised database could still be problematic2 gain, -e make no assumptions that Yesod users are #ollo-ing secure deployment practices2 You need to ha!e a -orking system #or sending email2 &any -eb ser!ers these days are not e3uipped to deal -ith all o# the spam protection measures used by mail ser!ers2The e(ample belo- -ill use the system%s built1in sendmail e(ecutable2 I# you -ould like to a!oid the hassle o# dealing -ith an email ser!er yoursel#, you can use ma'on S"S2 There is a package called mime1mail1ses -hich pro!ides a drop1in replacement #or the sendmail code used belo-2 This is the approach -e use on the Haskellers2com site2

*:D

But assuming you are able to meet these demands, and you -ant to ha!e a separate pass-ord login speci#ically #or your site, Yesod o##ers a built1in backend2 It re3uires 3uite a bit o# code to set up, since it needs to store pass-ords securely in the database and send a number o# di##erent emails to users 4!eri#y account, pass-ord retrie!al, etc262 5et%s ha!e a look at a site that pro!ides email authentication, storing pass-ords in a Persistent SG5ite database2
3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> FuasiFuotes> G)1Ts> Te$plate*askell> MultiPara$TypeClasses> ?lexibleContexts 4"7 i$port Hesod i$port Hesod/)uth i$port Hesod/)uth/-$ail i$port 1atabase/Persist/9Elite i$port 1atabase/Persist/T* i$port 1ata/Text @TextA i$port etwork/Mail/Mi$e i$port Euali#ied 1ata/Text/5a'y/-ncodin! i$port Text/9hakespeare/Text @stextA i$port Text/Bla'e/Ienderer/6t#D @render*t$lA i$port Text/*a$let @sha$letA i$port 1ata/Maybe @isJustA i$port Control/Monad @SoinA share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| 6ser e$ail Text password Text Maybe "" Password $ay not be set yet %erkey Text Maybe "" 6sed #or resettin! passwords %eri#ied Bool 6niEue6ser e$ail |; data My-$ail)pp = My-$ail)pp Connection $kHesod 8My-$ail)pp8 :parseIoutes| . IootI G-T .auth )uthI )uth !et)uth |; instance Hesod My-$ail)pp where "" -$ails will include links> so be sure to include an approot so that "" the links are %alidJ approot = )pproot9tatic 8http=..localhost=&KKK8 instance IenderMessa!e My-$ail)pp ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e "" 9et up Persistent instance HesodPersist My-$ail)pp where type HesodPersistBackend My-$ail)pp = 9ElPersist run1B # = do My-$ail)pp conn G" !etHesod run9ElConn # conn instance Hesod)uth My-$ail)pp where type )uthId My-$ail)pp = 6serId lo!in1est B = IootI

*:F

lo!out1est B = IootI authPlu!ins B = :auth-$ail; "" eed to #ind the 6serId #or the !i%en e$ail address/ !et)uthId creds = run1B + do x G" insertBy + 6ser @credsIdent credsA othin! othin! ?alse return + Just + case x o# 5e#t @-ntity userid BA "< userid "" newly added user Ii!ht userid "< userid "" existin! user auth*ttpMana!er = error 8-$ail doesnPt need an *TTP $ana!er8 "" *erePs all o# the e$ail"speci#ic code instance Hesod)uth-$ail My-$ail)pp where type )uth-$ailId My-$ail)pp = 6serId add6n%eri#ied e$ail %erkey = run1B + insert + 6ser e$ail othin! @Just %erkeyA ?alse

sendVeri#y-$ail e$ail B %erurl = li#tI, + render9endMail @e$ptyMail + )ddress othin! 8noreply8A 3 $ailTo = :)ddress othin! e$ail; > $ail*eaders = : @89ubSect8> 8Veri#y your e$ail address8A ; > $ailParts = ::textPart> ht$lPart;; 7 where textPart = Part 3 partType = 8text.plainR charset=ut#"D8 > part-ncodin! = one > part?ilena$e = othin! > partContent = 1ata/Text/5a'y/-ncodin!/encode6t#D :stext| Please con#ir$ your e$ail address by clickin! on the link below/ 243%erurl7 Thank you |; > part*eaders = :; 7 ht$lPart = Part 3 partType = 8text.ht$lR charset=ut#"D8 > part-ncodin! = one > part?ilena$e = othin! > partContent = render*t$l :sha$let| Gp<Please con#ir$ your e$ail address by clickin! on the link below/ Gp< Ga hre#=43%erurl7<43%erurl7 Gp<Thank you |; > part*eaders = :; 7 !etVeri#yXey = run1B / #$ap @Soin / #$ap userVerkeyA / !et setVeri#yXey uid key = run1B + update uid :6serVerkey =/ Just key; %eri#y)ccount uid = run1B + do $u G" !et uid case $u o# othin! "< return othin! Just u "< do

*:+

update uid :6serVeri#ied =/ True; return + Just uid !etPassword = run1B / #$ap @Soin / #$ap userPasswordA / !et setPassword uid pass = run1B + update uid :6serPassword =/ Just pass; !et-$ailCreds e$ail = run1B + do $u G" !etBy + 6niEue6ser e$ail case $u o# othin! "< return othin! Just @-ntity uid uA "< return + Just -$ailCreds 3 e$ailCredsId = uid > e$ailCreds)uthId = Just uid > e$ailCreds9tatus = isJust + userPassword u > e$ailCredsVerkey = userVerkey u 7 !et-$ail = run1B / #$ap @#$ap user-$ailA / !et !etIootI == *andler Iep*t$l !etIootI = do $aid G" $aybe)uthId de#ault5ayout :wha$let| Gp<Hour current auth I1= 43show $aid7 +$aybe B G" $aid Gp< Ga hre#=O3)uthI 5o!outI7<5o!out +nothin! Gp< Ga hre#=O3)uthI 5o!inI7<Go to the lo!in pa!e |; $ain == I, @A $ain = with9EliteConn 8e$ail/db&8 + 2conn "< do run9ElConn @runMi!ration $i!rate)llA conn warp1ebu! &KKK + My-$ail)pp conn

Authori6ation
/nce you can authenticate your users, you can use their credentials to authorize re3uests2 uthori'ation in Yesod is simple and declarati!e) most o# the time, you 9ust need to add the authIoute and is)uthori'ed methods to your Yesod typeclass instance2 5et%s see an e(ample2
3"4 5) G6)G- ,%erloaded9trin!s> Te$plate*askell> Type?a$ilies> MultiPara$TypeClasses> FuasiFuotes 4"7 i$port Hesod i$port Hesod/)uth i$port Hesod/)uth/1u$$y "" Sust #or testin!> donPt use in real li#eJJJ i$port 1ata/Text @TextA i$port etwork/*TTP/Conduit @Mana!er> newMana!er> de#A data My)uth9ite = My)uth9ite 3 httpMana!er == Mana!er 7 $kHesod 8My)uth9ite8 :parseIoutes| . IootI G-T P,9T .ad$in )d$inI G-T .auth )uthI )uth !et)uth |;

*:B

instance Hesod My)uth9ite where authIoute B = Just + )uthI 5o!inI "" route na$e> then a boolean indicatin! i# itPs a write reEuest is)uthori'ed IootI True = is)d$in is)uthori'ed )d$inI B = is)d$in "" anyone can access other pa!es is)uthori'ed B B = return )uthori'ed is)d$in = do $u G" $aybe)uthId return + case $u o# othin! "< )uthenticationIeEuired Just 8ad$in8 "< )uthori'ed Just B "< 6nauthori'ed 8Hou $ust be an ad$in8 instance Hesod)uth My)uth9ite where type )uthId My)uth9ite = Text !et)uthId = return / Just / credsIdent lo!in1est B = IootI lo!out1est B = IootI authPlu!ins B = :auth1u$$y; auth*ttpMana!er = httpMana!er instance IenderMessa!e My)uth9ite ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e !etIootI == *andler Iep*t$l !etIootI = do $aid G" $aybe)uthId de#ault5ayout :wha$let| Gp< ote= 5o! in as 8ad$in8 to be an ad$inistrator/ Gp<Hour current auth I1= 43show $aid7 +$aybe B G" $aid Gp< Ga hre#=O3)uthI 5o!outI7<5o!out Gp< Ga hre#=O3)d$inI7<Go to ad$in pa!e G#or$ $ethod=post< Make a chan!e @ad$ins onlyA 2 4 Ginput type=sub$it< |; postIootI == *andler @A postIootI = do setMessa!e 8Hou $ade so$e chan!e to the pa!e8 redirect IootI !et)d$inI == *andler Iep*t$l !et)d$inI = de#ault5ayout :wha$let| Gp<I !uess youPre an ad$inJ Gp< Ga hre#=O3IootI7<Ieturn to ho$epa!e |;

*>H

$ain == I, @A $ain = do $ana!er G" newMana!er de# warp1ebu! &KKK + My)uth9ite $ana!er

should be your login page, almost al-ays )uthI 5o!inI2 is)uthori'ed is a #unction that takes t-o parameters) the re3uested route, and -hether or not the re3uest -as a I-riteI re3uest2 You can actually change the meaning o# -hat a -rite re3uest is using the isCriteIeEuest method, but the out1o#1the1bo( !ersion #ollo-s R"ST#ul principles) anything but a G-T, *-)1, ,PTI, 9 or TI)C- re3uest is a -rite re3uest2
authIoute

What%s con!enient about the body o# is)uthori'ed is that you can run any *andler code you -ant2 This means you can)

ccess the #ilesystem 4normal I/6 5ookup !alues in the database Pull any session or re3uest !alues you -ant

8sing these techni3ues, you can de!elop as sophisticated an authori'ation system as you like, or e!en tie into e(isting systems used by your organi'ation2

Conclusion
This chapter co!ered the basics o# setting up user authentication, as -ell as ho- the built1in authori'ation #unctions pro!ide a simple, declarati!e approach #or users2 While these are complicated concepts, -ith many approaches, Yesod should pro!ide you -ith the building blocks you need to create your o-n customi'ed auth solution2

Scaffolding and the Site Template

So you%re tired o# running small e(amples, and ready to -rite a real site= Then you%re at the right chapter2 "!en -ith the entire Yesod library at your #ingertips, there are still a lot o# steps you need to go through to get a production13uality site setup)

$on#ig #ile parsing Signal handling 4Tni(6 &ore e##icient static #ile ser!ing good #ile layout

The sca##olded site is a combination o# many Yesoders% best practices combined together into a ready1to1use skeleton #or your sites2 It is highly recommended #or all sites2 This chapter -ill e(plain the o!erall structure o# the sca##olding, ho- to use it, and some o# its less1than1 ob!ious #eatures2 For the most part, this chapter -ill not contain code samples2 It is recommended that you #ollo- along -ith an actual sca##olded site2

*>*

Due to the nature o# the sca##olded site, it is the most #luid component o# Yesod, and can change #rom !ersion to !ersion2 It is possible that the in#ormation in this chapter is slightly outdated2

Ho3 to Scaffold
The yesod package installs both a library and an e(ecutable 4con!eniently named yesod as -ell62 This e(ecutable pro!ides a #e- commands 4run yesod by itsel# to get a list62 In order to generate a sca##olding, the command is yesod init2 This -ill start a 3uestion1and1ans-er process -here you get to pro!ide basic details 4your name, the pro9ect name, etc62 #ter ans-ering the 3uestions, you -ill ha!e a site template in a sub#older -ith the name o# your pro9ect2 The most important o# these 3uestions is the database backend2 You get #our choices here) SG5ite, PostgreSG5, &ongoDB, and tiny2 tiny is not a database backend7 instead, it is speci#ying that you do not -ant to use any database2 This option also turns o## a #e- e(tra dependencies, gi!ing you a leaner o!erall site2 The remainder o# this chapter -ill #ocus on the sca##oldings #or one o# the database backends2 There -ill be minor di##erences #or the tiny backend2 #ter creating your #iles, the sca##older -ill print a message about getting started2 It gi!es t-o sets o# options #or commands) one using cabal, and the other using cabal1de!2 cabal1de! is basically a -rapper around cabal -hich causes all dependencies to be built in a sandbo(2 8sing it is a good -ay to ensure that installing other packages -ill not break your site setup2 It is strongly recommended2 I# you don%t ha!e cabal1de!, you can install it by running cabal install cabal1de!2 0ote that you really do need to use the cabal install 11only1dependencies 4or cabal1de! install 11only1dependencies6 command2 &ost likely, you do not yet ha!e all the dependencies in place needed by your site2 For e(ample, none o# the database backends, nor the .a!ascript mini#ier 4h9smin6 are installed -hen installing the yesod package2 Finally, to launch your de!elopment site, you -ould use yesod de!el 4or yesod 11de! de!el62 This site -ill automatically rebuild and reload -hene!er you change your code2

!ile Structure
The sca##olded site is built as a #ully cabali'ed Haskell package2 In addition to source #iles, con#ig #iles, templates, and static #iles are produced as -ell2

Ca-al file
Whether directly using cabal, or indirectly using yesod de!el, building your code -ill al-ays go through the cabal #ile2 I# you open the #ile, you%ll see that there are both library and e(ecutable blocks2 /nly one o# these is built at a time, depending on the !alue o# the library"only #lag2 I# library"only is turned on, then the library is built, -hich is hoyesod de!el calls your app2 /ther-ise, the e(ecutable is built2 *>:

The library"only #lag should only be used by yesod de!el7 you should ne!er be e(plicitly passing it into cabal2 There is an additional #lag, de%, that allo-s cabal to build an e(ecutable, but turns on some o# the same #eatures as the library1only #lag, i2e2, no optimi'ations and reload !ersions o# the Shakespearean template #unctions2 In general, you -ill build as #ollo-s)

When de!eloping, use yesod de!el e(clusi!ely2 When building a production build, per#orm cabal clean XX cabal con#igure XX cabal build2 This -ill produce an optimi'ed e(ecutable in your dist #older2In the past -e had a "#production #lag2 I# you produced a sca##olded site in the past, you may ha!e to use this #lag to get a production build2

You%ll also notice that -e speci#y all language e(tensions in the cabal #ile2 The e(tensions are speci#ied twice) once #or the e(ecutable, and once #or the library2 I# you add any e(tensions to the list, add it to both places2 You might be surprised to see the oI$plicitPrelude e(tension2 We turn this on since the site includes its o-n module, I$port, -ith a #e- changes to the Prelude that make -orking -ith Yesod a little more con!enient2 The last thing to note is the e(ported1modules list2 I# you add any modules to your application, you must update this list to get yesod de!el to -ork correctly2 8n#ortunately, neither $abal nor ;H$ -ill gi!e you a -arning i# you #orgot to make this update, and instead you%ll get a !ery scary1looking error message #rom yesod de!el2 /ne o# our planned impro!ements to yesod de!el is to check i# there are any missing modules2

%outes and entities

&ultiple times in this book, you%!e seen a comment like IWe%re declaring our routes<entities -ith 3uasi3uotes #or con!enience2 In a production site, you should use an e(ternal #ile2I The sca##olding uses such an e(ternal #ile2 Routes are de#ined in con#ig<routes, and entities in con#ig<models2 They ha!e the e(act same synta( as the 3uasi3uoting you%!e seen throughout the book, and yesod de!el kno-s to automatically recompile the appropriate modules -hen these #iles change2 The models #iles is re#erenced by Model/hs2 You are #ree to declare -hate!er you like in this #ile, but here are some guidelines)

ny data types used in entities must be imported<declared in &odel2hs, abo!e the persist?ile call2 Helper utilities should either be declared in I$port/hs or, i# !ery model1centric, in a #ile -ithin the Model #older and imported into Import2hs2

*>>

!oundation and Application modules

The $kHesod #unction -hich -e ha!e used throughout the book declares a #e- things)

Route type Route render #unction Dispatch #unction

The dispatch #unction re#ers to all o# the handler #unctions, and there#ore all o# those must either be de#ined in the same #ile as the dispatch #unction, or be imported by the dispatch #unction2 &ean-hile, the handler #unctions -ill almost certainly re#er to the route type2 There#ore, they must be either in the same #ile -here the route type is de#ined, or must import that #ile2 I# you #ollo- the logic here, your entire application must essentially li!e in a single #ile? $learly this isn%t -hat -e -ant2 So instead o# using $kHesod, the sca##olding site uses a decomposed !ersion o# the #unction2 ?oundation calls $kHesod1ata, -hich declares the route type and render #unction2 Since it does not declare the dispatch #unction, the handler #unctions need not be in scope2 I$port/hs imports ?oundation/hs, and all the handler modules import I$port/hs2 In )pplication/hs, -e call $kHesod1ispatch, -hich creates our dispatch #unction2 For this to -ork, all handler #unctions must be in scope, so be sure to add an import statement #or any ne- handler modules you create2 /ther than that, pplication2hs is pretty simple2 It pro!ides t-o #unctions) with1e%el)ppPort is used by yesod de!el to launch your app, and !et)pplication is used by the e(ecutable to launch2 Foundation2hs is much more e(citing2 It)

Declares your #oundation datatype Declares a number o# instances, such as Hesod, Hesod)uth, and HesodPersist Imports the messages #iles2 I# you look #or the line starting -ith $kMessa!e, you -ill see that it speci#ies the #older containing the messages 4messages6 and the de#ault language 4en, #or "nglish62

This is the right #ile #or adding e(tra instances #or your #oundation, such as Hesod)uth-$ail or HesodBreadcru$bs2 We%ll be re#erring back to this #ile later, as -e discussed some o# the special implementations o# Hesod typeclass methods2

Import
The I$port module -as born out o# a #e- commonly recurring patterns2

*>@

I -ant to de#ine some helper #unctions 4maybe the G< = $append operator6 to be used by all handlers2 I%m al-ays adding the same #i!e import statements 41ata/Text, Control/)pplicati%e, etc6 to e!ery handler module2 I -ant to make sure I ne!er use some e!il #unction 4head, read?ile, 2226 #rom Prelude2Yes, e!il is hyperbole2 I# you%re -ondering -hy I listed those #unctions as bad) head is partial, and thro-s e(ceptions on an empty list, and read?ile uses la'y I</, -hich doesn%t close #ile handles 3uickly enough2 lso, read?ile uses 9trin! instead o# Text2

The solution is to turn on the oI$plicitPrelude language e(tension, re1e(port the parts o# Prelude -e -ant, add in all the other stu## -e -ant, de#ine our o-n #unctions as -ell, and then import this #ile in all handlers2

Handler modules
Handler modules should go inside the Handler #older2 The site template includes one module) Handler<Home2hs2 Ho- you split up your handler #unctions into indi!idual modules is your decision, but a good rule o# thumb is)

Di##erent methods #or the same route should go in the same #ile, e2g2 !etBlo!I and postBlo!I2 Related routes can also usually go in the same #ile, e2g2, !etPeopleI and !etPersonI2

/# course, it%s entirely up to you2 When you add a ne- handler #ile, make sure you do the #ollo-ing)

dd it to !ersion control 4you are using !ersion control, right=62 dd it to the cabal #ile2 dd it to the pplication2hs #ile2 Put a module statement at the top, and an i$port I$port line belo- it2

/ne o# the planned impro!ements to the yesod e(ecutable is to automate these #our steps2

3idget!ile
It%s !ery common to -ant to include $SS and .a!ascript speci#ic to a page2 You don%t -ant to ha!e to remember to include those 5ucius and .ulius #iles manually e!ery time you re#er to a Hamlet #ile2 For this, the site template pro!ides the wid!et?ile #unction2 I# you ha!e a handler #unction)
!et*o$eI = de#ault5ayout +@wid!et?ile 8ho$epa!e8A

, Yesod -ill look #or the #ollo-ing #iles)

*>A

templates<homepage2hamlet templates<homepage2lucius templates<homepage2cassius templates<homepage29ulius

I# any o# those #iles are present, they -ill be automatically included in the output2 Due to the nature o# ho- this -orks, i# you launch your app -ith yesod de!el, and then create a ne- #ile 4e2g2, templates<homepage29ulius6, the contents -ill not be included until the #ile calling wid!et?ile is recompiled2 In such a case, you may need to #orce a sa!e o# that #ile to get yesod de!el to recompile2

defaultLayout
/ne o# the #irst things you%re going to -ant to customi'e is the look o# your site2 The layout is actually broken up into t-o #iles)

templates<de#ault1layout1-rapper2hamlet contains 9ust the basic shell o# a page2 This #ile is interpreted as plain Hamlet, not as a Widget, and there#ore cannot re#er to other -idgets, embed i*+n strings, or add e(tra $SS<.S2 templates<de#ault1layout2hamlet is -here you -ould put the bulk o# your page2 You must remember to include the wid!et !alue in the page, as that contains the per1page contents2 This #ile is interpreted as a Widget2

lso, since de#ault1layout is included !ia the wid!et?ile #unction, any 5ucius, $assius, or .ulius #iles named de#ault1layout2T -ill automatically be included as -ell2

Static files
The sca##olded site automatically includes the static #ile subsite, optimi'ed #or ser!ing #iles that -ill not change o!er the li#etime o# the current build2 What this means is that)

When your static #ile identi#iers are generated 4e2g2, static<mylogo2png becomes $ylo!oBpn!6, a 3uery1string parameter is added to it -ith a hash o# the contents o# the #ile2 ll o# this happens at compile time2 When yesod"static ser!es your static #iles, it sets e(piration headers #ar in the #uture, and incldues an etag based on a hash o# your content2 Whene!er you embed a link to $ylo!oBpn!, the rendering includes the 3uery1string parameter2 I# you change the logo, recompile, and launch your ne- app, the 3uery string -ill ha!e changed, causing users to ignore the cached copy and do-nload a ne- !ersion2

dditionally, you can set a speci#ic static root in your Settings2hs #ile to ser!e #rom a di##erent domain name2 This has the ad!antage o# not re3uiring transmission o# cookies #or static #ile re3uests, and also lets you o##load static #ile hosting to a $D0 or a ser!ice like ma'on S>2 See the comments in the #ile #or more details2

*>D

nother optimi'ation is that $SS and .a!ascript included in your -idgets -ill not be included inside your HT&52 Instead, their contents -ill be -ritten to an e(ternal #ile, and a link gi!en2 This #ile -ill be named based on a hash o# the contents as -ell, meaning) *2 $aching -orks properly2 :2 Yesod can a!oid an e(pensi!e disk -rite o# the $SS<.a!ascript #ile contents i# a #ile -ith the same hash already e(ists2 Finally, all o# your .a!ascript is automatically mini#ied !ia h9smin2

Conclusion
The purpose o# this chapter -as not to e(plain e!ery line that e(ists in the sca##olded site, but instead to gi!e a general o!er!ie- to ho- it -orks2 The best -ay to become more #amiliar -ith it is to 9ump right in and start -riting a Yesod site -ith it2

Internationali6ation
8sers e(pect our so#t-are to speak their language2 8n#ortunately #or us, there -ill likely be more than one language in!ol!ed2 While doing simple string replacement isn%t too in!ol!ed, correctly dealing -ith all the grammar issues can be tricky2 #ter all, -ho -ants to see I5ist * #ile4s6I #rom a program output= But a real i*+n solution needs to do more than 9ust pro!ide a means o# achie!ing the correct output2 It needs to make this process easy #or both the programmer and the translator and relati!ely error1proo#2 Yesod%s ans-er to the problem gi!es you)

Intelligent guessing o# the user%s desired language based on re3uest headers, -ith the ability to o!erride2 simple synta( #or gi!ing translations -hich re3uires no Haskell kno-ledge2 4 #ter all, most translators aren%t programmers26 The ability to bring in the #ull po-er o# Haskell #or tricky grammar issues as necessary, along -ith a de#ault selection o# helper #unctions to co!er most needs2 bsolutely no issues at all -ith -ord order2

Synopsis
"" O$essa!es.en/$s! *ello= *ello -nterIte$Count= I would like to buy= Purchase= Purchase Ite$Count countOInt= Hou ha%e purchased 43showInt count7 43plural count 8ite$8 8ite$s87/ 9witch5an!ua!e= 9witch lan!ua!e to= 9witch= 9witch "" O$essa!es.he/$s! *ello= ^_`a -nterIte$Count= b_cd` ef_g hci= Purchase= ecd

*>F

Ite$Count count= bhcd 43showInt count7 43plural count 8^hgjk8 8gjk87/ 9witch5an!ua!e= ` ela m`ne= 9witch= m`ne "" Oi(Dn"synopsis/hs 3"4 5) G6)G- ,%erloaded9trin!s> FuasiFuotes> Te$plate*askell> Type?a$ilies> MultiPara$TypeClasses 4"7 i$port Hesod data I(D = I(D

$kMessa!e 8I(D 8 8$essa!es8 8en8 plural == Int "< 9trin! "< 9trin! "< 9trin! plural ( x B = x plural B B y = y showInt == Int "< 9trin! showInt = show instance Hesod I(D instance IenderMessa!e I(D ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e $kHesod 8I(D 8 :parseIoutes| . IootI G-T .buy BuyI G-T .lan! 5an!I P,9T |; !etIootI == *andler Iep*t$l !etIootI = de#ault5ayout :wha$let| Gh(<B3Ms!*ello7 G#or$ action=O3BuyI7< B3Ms!-nterIte$Count7 Ginput type=text na$e=count< Ginput type=sub$it %alue=B3Ms!Purchase7< G#or$ action=O35an!I7 $ethod=post< B3Ms!9witch5an!ua!e7 Gselect na$e=lan!< Goption %alue=en<-n!lish Goption %alue=he<*ebrew Ginput type=sub$it %alue=B3Ms!9witch7< |; !etBuyI == *andler Iep*t$l !etBuyI = do count G" runInputGet + ireE int?ield 8count8 de#ault5ayout :wha$let| Gp<B3Ms!Ite$Count count7 |; post5an!I == *andler @A post5an!I = do lan! G" runInputPost + ireE text?ield 8lan!8 set5an!ua!e lan! redirect IootI $ain == I, @A $ain = warp1ebu! &KKK I(D

*>+

O er ie3
&ost e(isting i*+n solutions out there, like gette(t or .a!a message bundles, -ork on the principle o# string lookups2 8sually some #orm o# print#1interpolation is used to interpolate !ariables into the strings2 In Yesod, as you might guess, -e instead rely on types2 This gi!es us all o# our normal ad!antages, such as the compiler automatically catching mistakes2 5et%s take a concrete e(ample2 Suppose our application has t-o things it -ants to say to a user) say hello, and state ho- many users are logged into the system2 This can be modeled -ith a sum type)
data MyMessa!e = Ms!*ello | Ms!6sers5o!!edIn Int

I can also -rite a #unction to turn this datatype into an "nglish representation)
to-n!lish to-n!lish to-n!lish to-n!lish in/8 == MyMessa!e "< 9trin! Ms!*ello = 8*ello thereJ8 @Ms!6sers5o!!edIn (A = 8There is ( user lo!!ed in/8 @Ms!6sers5o!!edIn iA = 8There are 8 UU show i UU 8 users lo!!ed

We can also -rite similar #unctions #or other languages2 The ad!antage to this inside1Haskell approach is that -e ha!e the #ull po-er o# Haskell #or addressing tricky grammar issues, especially plurali'ation2 You may think plurali'ation isn%t so complicated) you ha!e one !ersion #or * item, and another #or any other count2 That might be true in "nglish, but it%s not true #or e!ery language2 Russian, #or e(ample, has si( di##erent #orms, and you need to use some modulus logic to determine -hich one to use2 The do-nside, ho-e!er, is that you ha!e to -rite all o# this inside o# Haskell, -hich -on%t be !ery translator1#riendly2 To sol!e this, Yesod introduces the concept o# message #iles2 We%ll co!er that in a little bit2 ssuming -e ha!e this #ull set o# translation #unctions, ho- do -e go about using them= What -e need is a ne- #unction to -rap them all up together, and then choose the appropriate translation #unction based on the user%s selected language2 /nce -e ha!e that, Yesod can automatically choose the most rele!ant render #unction and call it on the !alues you pro!ide2 In order to simpli#y things a bit, Hamlet has a special interpolation synta(, B3///7, -hich handles all the calls to the render #unctions2 nd in order to associate a render #unction -ith your application, you use the HesodMessa!e typeclass2

Message files
The simplest approach to creating translations is !ia message #iles2 The setup is simple) there is a single #older containing all o# your translation #iles, -ith a single #ile #or each language2 "ach #ile is named based on its language code, e2g2 en2msg2 nd each line in a #ile handles one phrase, -hich correlates to a single constructor in your message data type2 *>B

The sca##olded site already includes a #ully con#igured message #older2 So #irstly, a -ord about language codes2 There are really t-o choices a!ailable) using a t-o1 letter language code, or a language15/$ 5" code2 For e(ample, -hen I load up a page in my -eb bro-ser, it sends t-o language codes) en18S and en2 What my bro-ser is saying is Ii# you ha!e merican "nglish, I like that the most2 I# you ha!e "nglish, I%ll take that instead2I So -hich #ormat should you use in your application= &ost likely t-o1letter codes, unless you are actually creating separate translations by locale2 This ensures that someone asking #or $anadian "nglish -ill still see your "nglish2 Behind the scenes, Yesod -ill add the t-o1letter codes -here rele!ant2 For e(ample, suppose a user has the #ollo-ing language list)
pt"BI> es> he

What this means is II like Bra'ilian Porteguese, then Spanish, and then Hebre-2I Suppose your application pro!ides the languages pt 4general Porteguese6 and "nglish, -ith "nglish as the de#ault2 Strictly #ollo-ing the user%s language list -ould result in the user being ser!ed "nglish2 Instead, Yesod translates that list into)
pt"BI> es> he> pt

In other -ords) unless you%re gi!ing di##erent translations based on locale, 9ust stick to the t-o1letter language codes2 0o- -hat about these message #iles= The synta( should be !ery #amiliar a#ter your -ork -ith Hamlet and Persistent2 The line starts o## -ith the name o# the message2 Since this is a data constructor, it must start -ith a capital letter2 0e(t, you can ha!e indi!idual parameters, -hich must be gi!en as lo-er case2 These -ill be arguments to the data constructor2 The argument list is terminated by a colon, and then #ollo-ed by the translated string, -hich allo-s usage o# our typical !ariable interpolation synta( 43$yVar72 By re#erring to the parameters de#ined be#ore the colon, and using translation helper #unctions to deal -ith issues like plurali'ation, you can create all the translated messages you need2

Specifying types
Since -e -ill be creating a datatype out o# our message speci#ications, each parameter to a data constructor must be gi!en a data type2 We use a N1synta( #or this2 For e(ample, to create the datatype data MyMessa!e = Ms!*ello | Ms!9ay)!e Int, -e -ould -rite)
*ello= *i thereJ 9ay)!e a!eOInt= Hour a!e is= 43show a!e7

But there are t-o problems -ith this) *2 It%s not !ery DRY 4don%t repeat yoursel#6 to ha!e to speci#y this datatype in e!ery #ile2 :2 Translators -ill be con#used ha!ing to speci#y these datatypes2 So instead, the type speci#ication is only re3uired in the main language #ile2 This is speci#ied as the third argument in the $kMessa!e #unction2 This also speci#ies -hat the backup language -ill be, to be used -hen none o# the languages pro!ided by your application match the user%s language list2 *@H

%enderMessage typeclass
Your call to $kMessa!e creates an instance o# the IenderMessa!e typeclass, -hich is the core o# Yesod%s i*+n2 It is de#ined as)
class IenderMessa!e $aster $essa!e where renderMessa!e == $aster "< :Text; "" Q lan!ua!es "< $essa!e "< Text

0otice that there are t-o parameters to the IenderMessa!e class) the master site and the message type2 In theory, -e could skip the master type here, but that -ould mean that e!ery site -ould need to ha!e the same set o# translations #or each message type2 When it comes to shared libraries like #orms, that -ould not be a -orkable solution2 The renderMessa!e #unction takes a parameter #or each o# the class%s type parameters) master and message2 The e(tra parameter is a list o# languages the user -ill accept, in descending order o# priority2 The method then returns a user1ready Text that can be displayed2 simple instance o# IenderMessa!e may in!ol!e no actual translation o# strings7 instead, it -ill 9ust display the same !alue #or e!ery language2 For e(ample)
data MyMessa!e = *ello | Greet Text instance IenderMessa!e My)pp MyMessa!e where renderMessa!e B B *ello = 8*ello8 renderMessa!e B B @Greet na$eA = 8Celco$e> 8 G< na$e G< 8J8

0otice ho- -e ignore the #irst t-o parameters to renderMessa!e2 We can no- e(tend this to support multiple languages)
render-n *ello = 8*ello8 render-n @Greet na$eA = 8Celco$e> 8 G< na$e G< 8J8 render*e *ello = 8^_`a8 render*e @Greet na$eA = 8^hije ^ho_gj> 8 G< na$e G< 8J8 instance IenderMessa!e My)pp MyMessa!e where renderMessa!e B @8en8=BA = render-n renderMessa!e B @8he8=BA = render*e renderMessa!e $aster @B=lan!sA = renderMessa!e $aster lan!s renderMessa!e B :; = render-n

The idea here is #airly straight1#or-ard) -e de#ine helper #unctions to support each language2 We then add a clause to catch each o# those languages in the render&essage de#inition2 We then ha!e t-o #inal cases) i# no languages matched, continue checking -ith the ne(t language in the user%s priority list2 I# -e%!e e(hausted all languages the user speci#ied, then use the de#ault language 4in our case, "nglish62 But odds are that you -ill ne!er need to -orry about -riting this stu## manually, as the message #ile inter#ace does all this #or you2 But it%s al-ays a good idea to ha!e an understanding o# -hat%s going on under the sur#ace2

*@*

Interpolation
/ne -ay to use your ne- IenderMessa!e instance -ould be to directly call the renderMessa!e #unction2 This -ould -ork, but it%s a bit tedious) you need to pass in the #oundation !alue and the language list manually2 Instead, Hamlet pro!ides a speciali'ed i*+n interpolation, -hich looks like B3///72 Why the underscore= 8nderscore is already a -ell1established character #or i*+n, as it is used in the gette(t library2 Hamlet -ill then automatically translate that to a call to renderMessa!e2 /nce Hamlet gets the output Text !alue, it uses the to*t$l #unction to produce an *t$l !alue, meaning that any special characters 4Y, X, Z6 -ill be automatically escaped2

Phrases; not 3ords

s a #inal note, I%d 9ust like to gi!e some general i*+n ad!ice2 5et%s say you ha!e an application #or selling turtles2 You%re going to use the -ord IturtleI in multiple places, like IYou ha!e added @ turtles to your cart2I and IYou ha!e purchased @ turtles, congratulations?I s a programmer, you%ll immediately notice the code reuse potential) -e ha!e the phrase I@ turtlesI t-ice2 So you might structure your message #ile as)
)dd9tart= Hou ha%e added )dd-nd= to your cart/ Purchase9tart= Hou ha%e purchased Purchase-nd= > con!ratulationsJ Turtles countOInt= 43show count7 43plural count 8turtle8 8turtles87

ST/P RI;HT TH"R"? This is all -ell and good #rom a programming perspecti!e, but translations are not programming2 There are a many things that could go -rong -ith this, such as)

Some languages might put Ito your cartI be#ore IYou ha!e added2I &aybe IaddedI -ill be constructed di##erently depending -hether you added * or more turtles2 There are a bunch o# -hitespace issues as -ell2

So the general rule is) translate entire phrases, not 9ust -ords2

Creating a Su-site
Ho- many sites pro!ide authentication systems= /r need to pro!ide $R8D 4$R8D6 management o# some ob9ects= /r a blog= /r a -iki= The theme here is that many -ebsites include common components that can be reused throughout multiple sites2 Ho-e!er, it is o#ten 3uite di##icult to get code to be modular enough to be truly plug1and1play) a component -ill re3uire hooks into the routing system,

*@:

usually #or multiple routes, and -ill need some -ay o# sharing styling in#ormation -ith the master site2 In Yesod, the solution is subsites2 subsite is a collection o# routes and their handlers that can be easily inserted into a master site2 By using type classes, it is easy to ensure that the master site pro!ides certain capabilities, and to access the de#ault site layout2 nd -ith type1 sa#e 8R5s, it%s easy to link #rom the master site to subsites2

Hello $orld
Writing subsites is a little bit tricky, in!ol!ing a number o# di##erent types2 5et%s start o## -ith a simple Hello World subsite)
3"4 5) G6)G- FuasiFuotes> Type?a$ilies> MultiPara$TypeClasses 4"7 3"4 5) G6)G- Te$plate*askell> ?lexibleInstances> ,%erloaded9trin!s 4"7 i$port Hesod "" 9ubsites ha%e #oundations Sust like $aster sites/ data *ello9ub = *ello9ub "" Ce ha%e a #a$iliar analo!ue #ro$ $kHesod> with Sust one extra para$eter/ "" CePll discuss that later/ $kHesod9ub 8*ello9ub8 :; :parseIoutes| . 9ubIootI G-T |; "" )nd wePll spell out the handler type si!nature/ !et9ubIootI == Hesod $aster =< G*andler *ello9ub $aster Iep*t$l !et9ubIootI = de#ault5ayout :wha$let|Celco$e to the subsiteJ|; "" )nd letPs create a $aster site that calls it/ data Master = Master 3 !et*ello9ub == *ello9ub 7 $kHesod 8Master8 :parseIoutes| . IootI G-T .subsite 9ubsiteI *ello9ub !et*ello9ub |; instance Hesod Master "" 9pellin! out type si!nature a!ain/ !etIootI == G*andler sub Master Iep*t$l "" could also replace sub with Master !etIootI = de#ault5ayout :wha$let| Gh(<Celco$e to the ho$epa!e Gp< ?eel #ree to %isit the 4 Ga hre#=O39ubsiteI 9ubIootI7<subsite 2 as well/ |; $ain = warp1ebu! &KKK + Master *ello9ub

*@>

This !ery simple e(ample actually sho-s most o# the complications in!ol!ed in creating a subsite2 5ike a normal Yesod application, e!erything in a subsite is centered around a #oundation datatype, *ello9ub in our case2 We then use $kHesod9ub, in much the same -ay that -e use $kHesod, to create the route datatype and the dispatch<render #unctions2 4We%ll come back to that e(tra parameter in a second26 What%s interesting is the type signature o# !et9ubIootI2 8p until no-, -e ha!e tried to ignore the G*andler datatype, or if -e need to ackno-ledge its e(istence, pretend like the #irst t-o type arguments are al-ays the same2 0o- -e get to #inally ackno-ledge the truth about this #unny datatype2 handler #unction al-ays has t-o #oundation types associated -ith it) the subsite and the master site2 When you -rite a normal application, those t3o datatypes are the same2 Ho-e!er, -hen you are -orking in a subsite, they -ill necessarily be di##erent2 So the type signature #or !et9ubIootI uses *ello9ub #or the #irst argument and $aster #or the second2 The de#ault5ayout #unction is part o# the Yesod typeclass2 There#ore, in order to call it, the $aster type argument must be an instance o# Hesod2 The ad!antage o# this approach is that any modi#ications to the master site%s de#ault5ayout method -ill automatically be re#lected in subsites2 When -e embed a subsite in our master site route de#inition, -e need to speci#y #our pieces o# in#ormation) the route to use as the base o# the subsite 4in this case, .subsite6, the constructor #or the subsite routes 49ubsiteI6, the subsite #oundation data type 4*ello9ub6 and a #unction that takes a master #oundation !alue and returns a subsite #oundation !alue 4!et*ello9ub62 In the de#inition o# getRootR, -e can see ho- the route constructor gets used2 In a sense, 9ubsiteI promotes any subsite route to a master site route, making it possible to sa#ely link to it #rom any master site template2

*@@

/>AMPL/S
#log= i9:n; authentication; authori6ation; and data-ase
This is a simple blog app2 It allo-s an admin to add blog posts !ia a rich te(t editor 4nicedit6, allo-s logged1in users to comment, and has #ull i*+n support2 It is also a good e(ample o# using a Persistent database, le!eraging Yesod%s authori'ation system, and templates2

While in general -e recommend placing templates, Persist entity de#initions, and routing in separate #iles, -e%ll keep it all in one #ile here #or con!enience2 The one e(ception you%ll see belo- -ill be i*+n messages2

We%ll start o## -ith our language e(tensions2 In sca##olded code, the language e(tensions are speci#ied in the cabal #ile, so you -on%t need to put this in your indi!idual Haskell #iles2

3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> FuasiFuotes> Te$plate*askell> G)1Ts> ?lexibleContexts> MultiPara$TypeClasses 4"7

0o- our imports2

i$port i$port i$port i$port i$port i$port

Hesod Hesod/)uth Hesod/?or$/ ic @Hesod ic> nic*t$l?ieldA Hesod/)uth/BrowserId @authBrowserIdA 1ata/Text @TextA etwork/*TTP/Conduit @Mana!er> newMana!er> de#A

*@A

i$port 1atabase/Persist/9Elite @ ConnectionPool> 9ElPersist> run9ElPool> runMi!ration > create9ElitePool A i$port 1ata/Ti$e @6TCTi$e> !etCurrentTi$eA i$port Control/)pplicati%e @@G+<A> @GW<A> pureA

First -e%ll set up our Persistent entities2 We%re going to both create our data types 4!ia mkPersist6 and create a migration #unction, -hich -ill automatically create and update our SG5 schema2 I# you -ere using the &ongoDB backend, migration -ould not be needed2

share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist5owerCase|

Peeps track o# users2 In a more robust application, -e -ould also keep account creation date, display name, etc2

6ser e$ail Text 6niEue6ser e$ail

n indi!idual blog entry 4I%!e a!oided using the -ord IpostI due to the con#usion -ith the re3uest method P/ST62

-ntry title Text posted 6TCTi$e content *t$l

We need to tack on this Ideri!ingI line since Html doesn%t speci#y instances #or Read, Sho- or "32 I# you get an error message about Icannot deri!eI in your

*@D

o-n code, try adding the deri!ing statement2

deri%in!

nd a comment on the blog post2

Co$$ent entry -ntryId posted 6TCTi$e user 6serId na$e Text text Textarea |;

"!ery site has a #oundation datatype2 This !alue is initiali'ed be#ore launching your application, and is a!ailable throughout2 We%ll store a database connection pool and HTTP connection manager in ours2 See the !ery end o# this #ile #or ho- those are initiali'ed2

data Blo! = Blo! 3 connPool == ConnectionPool > httpMana!er == Mana!er 7

To make i*+n easy and translator #riendly, -e ha!e a special #ile #ormat #or translated messages2 There is a single #ile #or each language, and each #ile is named based on the language code 4e2g2, en, es, de1D"6 and placed in that #older2 We also speci#y the main language #ile 4here, IenI6 as a de#ault language2

$kMessa!e 8Blo!8 8//.$essa!es"blo!8 8en8

*@F

/ur en message #ile contains the #ollo-ing content)

0ot n dmin) You must be an administrator to access this page2

WelcomeHomepage) Welcome to the homepage See rchi!e) See the archi!e

0o"ntries) There are no entries in the blog 5oginToPost) dmins can login to post 0e-"ntry) Post to blog 0e-"ntryTitle) Title 0e-"ntry$ontent) $ontent

Please$orrect"ntry) Your submitted entry had some errors, please correct and try again2 "ntry$reated titleNTe(t) Your ne- blog post, ORtitleS, has been created

"ntryTitle titleNTe(t) Blog post) ORtitleS $ommentsHeading) $omments 0o$omments) There are no comments dd$ommentHeading) dd a $omment 5oginTo$omment) You must be logged in to comment dd$ommentButton) dd comment

$omment0ame) Your display name

*@+

$ommentTe(t) $omment $omment dded) Your comment has been added Please$orrect$omment) Your submitted comment had some errors, please correct and try again2

HomepageTitle) Yesod Blog Demo Blog rchi!eTitle) Blog rchi!e

0o- -e%re going to set up our routing table2 We ha!e #our entries) a homepage, an entry list page 4BlogR6, an indi!idual entry page 4"ntryR6 and our authentication subsite2 0ote that BlogR and "ntryR both accept ;"T and P/ST methods2 The P/ST methods are #or adding a ne- blog post and adding a necomment, respecti!ely2

$kHesod 8Blo!8 :parseIoutes| . IootI G-T .blo! Blo!I G-T P,9T .blo!.4-ntryId -ntryI G-T P,9T .auth )uthI )uth !et)uth |;

"!ery #oundation needs to be an instance o# the Yesod typeclass2 This is -here -e con#igure !arious settings2

instance Hesod Blo! where

The base o# our application2 0ote that in order to make Bro-serID -ork properly, this must be a !alid 8R52

*@B

approot = )pproot9tatic 8http=..localhost=&KKK8

/ur authori'ation scheme2 We -ant to ha!e the #ollo-ing rules)

T /nly admins can add a ne- entry2 T /nly logged in users can add a ne- comment2 T ll other pages can be accessed by anyone2

We set up our routes in a R"ST#ul -ay, -here the actions that could make changes are al-ays using a P/ST method2 s a result, -e can simply check #or -hether or not a re3uest is a -rite re3uest, gi!en by the True in the second #ield2

First, -e%ll authori'e re3uests to add a ne- entry2

is)uthori'ed Blo!I True = do $auth G" $aybe)uth case $auth o# othin! "< return )uthenticationIeEuired Just @-ntity B userA | is)d$in user "< return )uthori'ed | otherwise "< unauthori'edI Ms! ot)n)d$in

0o- -e%ll authori'e re3uests to add a ne- comment2

is)uthori'ed @-ntryI BA True = do $auth G" $aybe)uth case $auth o# othin! "< return )uthenticationIeEuired Just B "< return )uthori'ed

*AH

nd #or all other re3uests, the result is al-ays authori'ed2

is)uthori'ed B B = return )uthori'ed

Where a user should be redirected to i# they get an uthenticationRe3uired2

authIoute B = Just @)uthI 5o!inIA

This is -here -e de#ine our site look1and1#eel2 The #unction is gi!en the content #or the indi!idual page, and -raps it up -ith a standard template2

de#ault5ayout inside = do

Yesod encourages the get1#ollo-ing1post pattern, -here a#ter a P/ST, the user is redirected to another page2 In order to allo- the P/ST page to gi!e the user some kind o# #eedback, -e ha!e the get&essage and set&essage #unctions2 It%s a good idea to al-ays check #or pending messages in your de#ault5ayout #unction2

$$s! G" !etMessa!e

We use -idgets to compose together HT&5, $SS and .a!ascript2 t the end o# the day, -e need to un-rap all o# that into simple HT&52 That%s -hat the -idgetToPage$ontent #unction is #or2 We%re going to gi!e it a -idget consisting o# the content -e recei!ed #rom the indi!idual page 4inside6, plus a standard $SS #or all pages2 We%ll use the 5ucius template language to create the latter2

*A*

pc G" wid!etToPa!eContent + do toCid!et :lucius| body 3 width= [VKpxR $ar!in= (e$ autoR #ont"#a$ily= sans"seri#R 7 textarea 3 width= LKKpxR hei!ht= NKKpxR 7 4$essa!e 3 color= 4\KKR 7 |; inside

nd #inally -e%ll use a ne- Hamlet template to -rap up the indi!idual components 4title, head data and body data6 into the #inal output2

ha$letToIep*t$l :ha$let| +doctype M Ght$l< Ghead< Gtitle<43pa!eTitle pc7 Q3pa!e*ead pc7 Gbody< +$aybe $s! G" $$s! Gdi% 4$essa!e<43$s!7 Q3pa!eBody pc7 |;

This is a simple #unction to check i# a user is the admin2 In a real application, -e -ould likely store the admin bit in the database itsel#, or check -ith some e(ternal system2 For no-, I%!e 9ust hard1coded my o-n email address2

is)d$in == 6ser "< Bool is)d$in user = user-$ail user == 8$ichaelOsnoy$an/co$8

*A:

In order to access the database, -e need to create a YesodPersist instance, -hich says -hich backend -e%re using and ho- to run an action2

instance HesodPersist Blo! where type HesodPersistBackend Blo! = 9ElPersist run1B # = do $aster G" !etHesod let pool = connPool $aster run9ElPool # pool

This is a con!enience synonym2 It is de#ined automatically #or you in the sca##olding2

type ?or$ x = *t$l "< M?or$ Blo! Blo! @?or$Iesult x> Cid!etA

In order to use yesod1#orm and yesod1auth, -e need an instance o# Render&essage #or Form&essage2 This allo-s us to control the i*+n o# indi!idual #orm messages2

instance IenderMessa!e Blo! ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e

In order to use the built1in nic HT&5 editor, -e need this instance2 We 9ust take the de#ault !alues, -hich use a $D01hosted !ersion o# 0ic2

instance Hesod ic Blo!

In order to use yesod1auth, -e need a Yesod uth instance2

*A>

instance Hesod)uth Blo! where type )uthId Blo! = 6serId lo!in1est B = IootI lo!out1est B = IootI auth*ttpMana!er = httpMana!er

We%ll use [Bro-serID\4https)<<bro-serid2org<6, -hich is a third1party system using email addresses as your identi#ier2 This makes it easy to s-itch to other systems in the #uture, locally authenticated email addresses 4also included -ith yesod1auth62

authPlu!ins B = :authBrowserId;

This #unction takes someone%s login credentials 4i2e2, his<her email address6 and gi!es back a 8serId2

!et)uthId creds = do let e$ail = credsIdent creds user = 6ser e$ail res G" run1B + insertBy user return + Just + either entityXey id res

Homepage handler2 The one important detail here is our usage o# ]setTitleI], -hich allo-s us to use i*+n messages #or the title2 We also use this message -ith a ]WR&sg222S] interpolation in Hamlet2

!etIootI == *andler Iep*t$l !etIootI = de#ault5ayout + do setTitleI Ms!*o$epa!eTitle :wha$let| Gp<B3Ms!Celco$e*o$epa!e7 Gp< Ga hre#=O3Blo!I7<B3Ms!9ee)rchi%e7 |;

*A@

De#ine a #orm #or adding ne- entries2 We -ant the user to pro!ide the title and content, and then #ill in the post date automatically !ia ]get$urrentTime]2

entry?or$ == ?or$ -ntry entry?or$ = render1i%s + -ntry G+< areE text?ield @#ield9ettin!s5abel Ms! ew-ntryTitleA othin! GW< a#or$M @li#tI, !etCurrentTi$eA GW< areE nic*t$l?ield @#ield9ettin!s5abel Ms! ew-ntryContentA othin!

;et the list o# all blog entries, and present an admin -ith a #orm to create a ne- entry2

!etBlo!I == *andler Iep*t$l !etBlo!I = do $user G" $aybe)uth entries G" run1B + select5ist :; :1esc -ntryPosted; @entryCid!et> enctypeA G" !enerate?or$Post entry?or$ de#ault5ayout + do setTitleI Ms!Blo!)rchi%eTitle :wha$let| +i# null entries Gp<B3Ms! o-ntries7 +else Gul< +#orall -ntity entryId entry G" entries Gli< Ga hre#=O3-ntryI entryId7<43entryTitle entry7

We ha!e three possibilities) the user is logged in as an admin, the user is logged in and is not an admin, and the user is not logged in2 In the #irst case, -e should display the entry #orm2 In the second, -e%ll do nothing2 In the third, -e%ll pro!ide a login link2

+$aybe -ntity B user G" $user +i# is)d$in user G#or$ $ethod=post enctype=43enctype7< Q3entryCid!et7

*AA

Gdi%< Ginput type=sub$it %alue=B3Ms! ew-ntry7< +nothin! Gp< |; Ga hre#=O3)uthI 5o!inI7<B3Ms!5o!inToPost7

Process an incoming entry addition2 We don%t do any permissions checking, since is uthori'ed handles it #or us2 I# the #orm submission -as !alid, -e add the entry to the database and redirect to the ne- entry2 /ther-ise, -e ask the user to try again2

postBlo!I == *andler Iep*t$l postBlo!I = do @@res> entryCid!etA> enctypeA G" run?or$Post entry?or$ case res o# ?or$9uccess entry "< do entryId G" run1B + insert entry setMessa!eI + Ms!-ntryCreated + entryTitle entry redirect + -ntryI entryId B "< de#ault5ayout + do setTitleI Ms!PleaseCorrect-ntry :wha$let| G#or$ $ethod=post enctype=43enctype7< Q3entryCid!et7 Gdi%< Ginput type=sub$it %alue=B3Ms! ew-ntry7< |;

#orm #or comments, !ery similar to our entryForm abo!e2 It takes the "ntryId o# the entry the comment is attached to2 By using pure, -e embed this !alue in the resulting $omment output, -ithout ha!ing it appear in the generated HT&52

co$$ent?or$ == -ntryId "< ?or$ Co$$ent co$$ent?or$ entryId = render1i%s + Co$$ent G+< pure entryId GW< a#or$M @li#tI, !etCurrentTi$eA GW< a#or$M reEuire)uthId GW< areE text?ield @#ield9ettin!s5abel Ms!Co$$ent a$eA othin! GW< areE textarea?ield @#ield9ettin!s5abel Ms!Co$$entTextA othin!

*AD

Sho- an indi!idual entry, comments, and an add comment #orm i# the user is logged in2

!et-ntryI == -ntryId "< *andler Iep*t$l !et-ntryI entryId = do @entry> co$$entsA G" run1B + do entry G" !etLKL entryId co$$ents G" select5ist :; :)sc Co$$entPosted; return @entry> $ap entityVal co$$entsA $user G" $aybe)uth @co$$entCid!et> enctypeA G" !enerate?or$Post @co$$ent?or$ entryIdA de#ault5ayout + do setTitleI + Ms!-ntryTitle + entryTitle entry :wha$let| Gh(<43entryTitle entry7 Garticle<43entryContent entry7 Gsection /co$$ents< Gh(<B3Ms!Co$$ents*eadin!7 +i# null co$$ents Gp<B3Ms! oCo$$ents7 +else +#orall Co$$ent Bentry posted Buser na$e text G" co$$ents Gdi% /co$$ent< Gspan /by<43na$e7 Gspan /at<43show posted7 Gdi% /content<43text7 Gsection< Gh(<B3Ms!)ddCo$$ent*eadin!7 +$aybe B G" $user G#or$ $ethod=post enctype=43enctype7< Q3co$$entCid!et7 Gdi%< Ginput type=sub$it %alue=B3Ms!)ddCo$$entButton7< +nothin! Gp< Ga hre#=O3)uthI 5o!inI7<B3Ms!5o!inToCo$$ent7 |;

Recei!e an incoming comment submission2

post-ntryI == -ntryId "< *andler Iep*t$l post-ntryI entryId = do @@res> co$$entCid!etA> enctypeA G" run?or$Post @co$$ent?or$ entryIdA case res o# ?or$9uccess co$$ent "< do B G" run1B + insert co$$ent setMessa!eI Ms!Co$$ent)dded

*AF

redirect + -ntryI entryId B "< de#ault5ayout + do setTitleI Ms!PleaseCorrectCo$$ent :wha$let| G#or$ $ethod=post enctype=43enctype7< Q3co$$entCid!et7 Gdi%< Ginput type=sub$it %alue=B3Ms!)ddCo$$entButton7< |;

Finally our main #unction2

$ain == I, @A $ain = do pool G" create9ElitePool 8blo!/db&8 (K "" create a new pool "" per#or$ any necessary $i!ration run9ElPool @runMi!ration $i!rate)llA pool $ana!er G" newMana!er de# "" create a new *TTP $ana!er warp1ebu! &KKK + Blo! pool $ana!er "" start our ser%er

$iki= markdo3n; chat su-site; e ent source

This e(ample -ill tie together a #e- di##erent ideas2 We%ll start -ith a chat subsite, -hich allo-s us to embed a chat -idget on any page2 We%ll use the HT&5 A e!ent source PI to handle sending e!ents #rom the ser!er to the client2
"" OChat/hs 3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> FuasiFuotes> Te$plate*askell> ?lexibleInstances> MultiPara$TypeClasses> ?lexibleContexts 4"7 "" | This $odules de#ines a subsite that allows you to insert a chat box on "" any pa!e o# your site/ It uses e%entsource #or sendin! the $essa!es #ro$ "" the ser%er to the browser/ $odule Chat where i$port i$port i$port i$port i$port i$port i$port Hesod Control/Concurrent/Chan @Chan> dupChan> writeChanA 1ata/Text @TextA etwork/Cai/-%ent9ource @9er%er-%ent @//A> e%ent9ource)ppChanA 5an!ua!e/*askell/T*/9yntax @Type @VarTA> Pred @ClassPA> $k a$eA Bla'e/Byte9trin!/Builder/Char/6t#D @#ro$TextA 1ata/Monoid @$appendA

"" | ,ur subsite #oundation/ Ce keep a channel o# e%ents that all connections "" will share/ data Chat = Chat @Chan 9er%er-%entA "" | Ce need to know how to check i# a user is lo!!ed in and how to !et "" his.her userna$e @#or printin! $essa!esA/ class @Hesod $aster> IenderMessa!e $aster ?or$Messa!eA =< HesodChat $aster where

*A+

!et6ser a$e == G*andler sub $aster Text is5o!!edIn == G*andler sub $aster Bool "" ow we set up our subsite/ The #irst ar!u$ent is the subsite> %ery si$ilar "" to how weP%e used $kHesod in the past/ The second ar!u$ent is speci#ic to "" subsites/ Chat it $eans here is 8the $aster site $ust be an instance o# "" HesodChat8/ "" "" Ce de#ine two routes= a route #or sendin! $essa!es #ro$ the client to the "" ser%er> and one #or openin! up the e%ent strea$ to recei%e $essa!es #ro$ "" the ser%er/ $kHesod9ub 8Chat8 : ClassP PPHesodChat :VarT + $k a$e 8$aster8; ; :parseIoutes| .send 9endI P,9T .rec% Iecei%eI G-T |; "" | Get a $essa!e #ro$ the user and send it to all listeners/ post9endI == HesodChat $aster =< G*andler Chat $aster @A post9endI = do #ro$ G" !et6ser a$e "" ote that wePre usin! G-T para$eters #or si$plicity o# the )Sax code/ "" This could easily be switched to P,9T/ onetheless> our o%erall "" approach is still I-9T#ul since this route can only be accessed %ia a "" P,9T reEuest/ body G" runInputGet + ireE text?ield 8$essa!e8 "" Get the channel Chat chan G" !etHesod9ub "" 9end an e%ent to all listeners with the userPs na$e and $essa!e/ li#tI, + writeChan chan + 9er%er-%ent othin! othin! + return + #ro$Text #ro$ Z$appendZ #ro$Text 8= 8 Z$appendZ #ro$Text body "" | 9end an e%entstrea$ response with all $essa!es strea$ed in/ !etIecei%eI == G*andler Chat $aster @A !etIecei%eI = do "" ?irst we !et the $ain channel Chat chanK G" !etHesod9ub "" Ce duplicated the channel> which allows us to create broadcast "" channels/ chan G" li#tI, + dupChan chanK "" ow we use the e%ent source )PI/ e%ent9ource)ppChan takes two para$eters= "" the channel o# e%ents to read #ro$> and the C)I reEuest/ It returns a "" C)I response> which we can return with sendCaiIesponse/ reE G" waiIeEuest res G" li#t + e%ent9ource)ppChan chan reE sendCaiIesponse res "" | Pro%ide a wid!et that the $aster site can e$bed on any pa!e/

*AB

chatCid!et == HesodChat $aster =< @Ioute Chat "< Ioute $asterA "< GCid!et sub $aster @A "" This toMaster ar!u$ent tells us how to con%ert a Ioute Chat into a $aster "" route/ Hou $i!ht think this is redundant in#or$ation> but takin! this "" approach $eans we can ha%e $ultiple chat subsites in a sin!le site/ chatCid!et toMaster = do "" Get so$e uniEue identi#iers to help in creatin! our *TM5.C99/ Ie$e$ber> "" we ha%e no idea what the $aster sitePs *TM5 will look like> so we "" should not assu$e we can $ake up identi#iers that wonPt be reused/ "" )lso> itPs possible that $ultiple chatCid!ets could be e$bedded in the "" sa$e pa!e/ chat G" li#t newIdent "" the containin! di% output G" li#t newIdent "" the box containin! the $essa!es input G" li#t newIdent "" input #ield #ro$ the user ili G" li#t is5o!!edIn "" check i# wePre already lo!!ed in i# ili then do "" 5o!!ed in= show the wid!et :wha$let| Gdi% 443chat7< GhN<Chat Gdi% 443output7< Ginput 443input7 type=text placeholder=8-nter Messa!e8< |; "" Just so$e C99 toCid!et :lucius| 443chat7 3 position= absoluteR top= Ne$R ri!ht= Ne$R 7 443output7 3 width= NKKpxR hei!ht= &KKpxR border= (px solid 4\\\R o%er#low= autoR 7 |; "" )nd now that Ja%ascript toCid!etBody :Sulius| .. 9et up the recei%in! end %ar output = docu$ent/!et-le$entById@843output78AR %ar src = new -%ent9ource@8O3toMaster Iecei%eI78AR src/on$essa!e = #unction@$s!A 3 .. This #unction will be called #or each new $essa!e/ %ar p = docu$ent/create-le$ent@8p8AR p/appendChild@docu$ent/createText ode@$s!/dataAAR output/appendChild@pAR .. )nd now scroll down within the output di% so the $ost recent $essa!e .. is displayed/ output/scrollTop = output/scroll*ei!htR

7R

.. 9et up the sendin! end= send a $essa!e %ia )Sax whene%er the user hits .. enter/

*DH

%ar input = docu$ent/!et-le$entById@843input78AR input/onkeyup = #unction@e%entA 3 %ar keycode = @e%ent/keyCode ] e%ent/keyCode = e%ent/whichAR i# @keycode == P(&PA 3 %ar xhr = new YM5*ttpIeEuest@AR %ar %al = input/%alueR input/%alue = 88R %ar para$s = 8]$essa!e=8 U encode6II@%alAR xhr/open@8P,9T8> 8O3toMaster 9endI78 U para$sAR xhr/send@nullAR 7 7 |; else do "" 6ser isnPt lo!!ed in> !i%e a not"lo!!ed"in $essa!e/ $aster G" li#t !etHesod :wha$let| Gp< Hou $ust be 4 +$aybe ar G" authIoute $aster Ga hre#=O3ar7<lo!!ed in +nothin! lo!!ed in 2 to chat/ |;

This module stands on its o-n, and can be used in any application2 0e(t -e%ll pro!ide such a dri!er application) a -iki2 /ur -iki -ill ha!e a hard1coded homepage, and then a -iki section o# the site2 We%ll be using multiple dynamic pieces to allo- an arbitrary hierarchy o# pages -ithin the Wiki2 For storage, -e%ll 9ust use a mutable re#erence to a Map2 For a production application, this should be replaced -ith a proper database2 The content -ill be stored and ser!ed as &arkdo-n2 yesod"auth%s dummy plugin -ill pro!ide us -ith 4#ake6 authentication2
3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> FuasiFuotes> Te$plate*askell> ?lexibleInstances> MultiPara$TypeClasses> ?lexibleContexts 4"7 i$port Hesod i$port Hesod/)uth i$port Hesod/)uth/1u$$y @auth1u$$yA i$port Chat i$port Control/Concurrent/Chan @Chan> newChanA i$port etwork/Cai/*andler/Carp @runA i$port 1ata/Text @TextA i$port Euali#ied 1ata/Text/5a'y as T5 i$port Euali#ied 1ata/I,Ie# as I i$port Euali#ied 1ata/Map as Map i$port Text/Markdown @$arkdown> de#A "" | ,ur #oundation type has both the chat subsite and a $utable re#erence to "" a $ap o# all our wiki contents/ ote that the key is a list o# Texts> since "" a wiki can ha%e an arbitrary hierarchy/ "" "" In a real application> we would want to store this in#or$ation in a "" database o# so$e sort/

*D*

data Ciki = Ciki 3 !etChat == Chat > wikiContent == I/I,Ie# @Map/Map :Text; TextA 7 "" 9et up our routes as usual/ $kHesod 8Ciki8 :parseIoutes| . IootI G-T "" .wiki.WTexts CikiI G-T P,9T "" .chat ChatI Chat !etChat "" .auth )uthI )uth !et)uth "" |;

the ho$epa!e note the $ultipiece #or the wiki hierarchy the chat subsite the auth subsite

instance Hesod Ciki where authIoute B = Just + )uthI 5o!inI "" !et a workin! lo!in link "" ,ur custo$ de#ault5ayout will add the chat wid!et to e%ery pa!e/ "" CePll also add lo!in and lo!out links to the top/ de#ault5ayout wid!et = do pc G" wid!etToPa!eContent + wid!et << chatCid!et ChatI $$s! G" !etMessa!e ha$letToIep*t$l :ha$let| +doctype M Ght$l< Ghead< Gtitle<43pa!eTitle pc7 Q3pa!e*ead pc7 Gbody< +$aybe $s! G" $$s! Gdi% /$essa!e<43$s!7 Gna%< Ga hre#=O3)uthI 5o!inI7<5o!in 2 | 4 Ga hre#=O3)uthI 5o!outI7<5o!out Q3pa!eBody pc7 |; "" ?airly standard Hesod)uth instance/ CePll use the du$$y plu!in so that you "" can create any na$e you want> and store the lo!in na$e as the )uthId/ instance Hesod)uth Ciki where type )uthId Ciki = Text authPlu!ins B = :auth1u$$y; lo!in1est B = IootI lo!out1est B = IootI !et)uthId = return / Just / credsIdent auth*ttpMana!er = error 8auth*ttpMana!er8 "" not used by auth1u$$y "" Just i$ple$ent authentication based on our yesod"auth usa!e/ instance HesodChat Ciki where !et6ser a$e = reEuire)uthId is5o!!edIn = do $a G" $aybe)uthId return + $aybe ?alse @const TrueA $a instance IenderMessa!e Ciki ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e "" othin! special here> Sust !i%in! a link to the root o# the wiki/ !etIootI == *andler Iep*t$l !etIootI = de#ault5ayout :wha$let|

*D:

Gp<Celco$e to the CikiJ Gp< Ga hre#=O3wikiIoot7<Ciki root |; where wikiIoot = CikiI :; "" ) #or$ #or !ettin! wiki content wiki?or$ $text = render1i%s + areE textarea?ield 8Pa!e body8 $text "" 9how a wiki pa!e and an edit #or$ !etCikiI == :Text; "< *andler Iep*t$l !etCikiI pa!e = do "" Get the re#erence to the contents $ap icontent G" #$ap wikiContent !etHesod "" )nd read the $ap #ro$ inside the re#erence content G" li#tI, + I/readI,Ie# icontent "" 5ookup the contents o# the current pa!e> i# a%ailable let $text = Map/lookup pa!e content "" Generate a #or$ with the current contents as the de#ault %alue/ "" ote that we use the Textarea wrapper to !et a Gtextarea</ @#or$> BA G" !enerate?or$Post + wiki?or$ + #$ap Textarea $text de#ault5ayout + do case $text o# "" CePre treatin! the input as $arkdown/ The $arkdown packa!e "" auto$atically handles Y99 protection #or us/ Just text "< toCid!et + $arkdown de# + T5/#ro$9trict text othin! "< :wha$let|Gp<Pa!e does not yet exist|; :wha$let| GhN<-dit pa!e G#or$ $ethod=post< Q3#or$7 Gdi%< Ginput type=sub$it< |; "" Get a sub$itted wiki pa!e and updated the contents/ postCikiI == :Text; "< *andler Iep*t$l postCikiI pa!e = do icontent G" #$ap wikiContent !etHesod content G" li#tI, + I/readI,Ie# icontent let $text = Map/lookup pa!e content @@res> #or$A> BA G" run?or$Post + wiki?or$ + #$ap Textarea $text case res o# ?or$9uccess @Textarea tA "< do li#tI, + I/ato$icModi#yI,Ie# icontent + 2$ "< @Map/insert pa!e t $> @AA setMessa!e 8Pa!e updated8 redirect + CikiI pa!e B "< de#ault5ayout :wha$let| G#or$ $ethod=post< Q3#or$7 Gdi%< Ginput type=sub$it< |; $ain == I, @A $ain = do

*D>

"" Create our ser%er e%ent channel chan G" newChan "" Initially ha%e a blank database o# wiki pa!es icontent G" I/newI,Ie# Map/e$pty "" Iun our app warp1ebu! &KKK + Ciki @Chat chanA icontent

+SO< $e- Ser ice

5et%s create a !ery simple -eb ser!ice) it takes a .S/0 re3uest and returns a .S/0 response2 We%re going to -rite the ser!er in W I<Warp, and the client in http1conduit2 We%ll be using aeson #or .S/0 parsing and rendering2 We could also -rite the ser!er in Yesod itsel#, but #or such a simple e(ample, the e(tra #eatures o# Yesod don%t add much2

Ser er
W I uses the conduit package to handle streaming re3uest bodies, and e##iciently generates responses using bla'e1builder2 aeson uses attoparsec #or parsing7 by using attoparsec1conduit -e get easy interoperability -ith W I2 This plays out as)
3"4 5) i$port i$port i$port i$port i$port i$port i$port i$port i$port i$port i$port G6)G- ,%erloaded9trin!s 4"7 etwork/Cai @Iesponse> response5B9> )pplication> reEuestBodyA etwork/*TTP/Types @statusNKK> statusLKKA etwork/Cai/*andler/Carp @runA 1ata/)eson/Parser @SsonA 1ata/Conduit/)ttoparsec @sinkParserA Control/Monad/I,/Class @li#tI,A 1ata/)eson @Value> encode> obSect> @/=AA Control/-xception @9o$e-xceptionA 1ata/Byte9trin! @Byte9trin!A 1ata/Conduit @IesourceT> @++AA Control/-xception/5i#ted @handleA

$ain == I, @A $ain = run &KKK app app == )pplication app reE = handle in%alidJson + do %alue G" reEuestBody reE ++ sinkParser Sson newValue G" li#tI, + $odValue %alue return + response5B9 statusNKK :@8Content"Type8> 8application.Sson8A; + encode newValue in%alidJson == 9o$e-xception "< IesourceT I, Iesponse in%alidJson ex = return + response5B9 statusLKK :@8Content"Type8> 8application.Sson8A; + encode + obSect : @8$essa!e8 /= show exA ;

*D@

"" )pplication"speci#ic lo!ic would !o here/ $odValue == Value "< I, Value $odValue = return

Client
http1conduit -as -ritten as a companion to W I2 It too uses conduit and bla'e"builder per!asi!ely, meaning -e once again get easy interop -ith aeson2 #e- e(tra comments #or those not #amiliar -ith http"conduit)
Mana!er is present to keep track o# open connections, so that multiple re3uests to the same ser!er use the same connection2 You usually -ant to use the withMana!er #unction to create and clean up this Mana!er, since it is e(ception sa#e2 We need to kno- the si'e o# our re3uest body, -hich can%t be determined directly #rom a Builder2 Instead, -e con!ert the Builder into a la'y Byte9trin! and take the si'e #rom there2 There are a number o# di##erent #unctions #or initiating a re3uest2 We use http, -hich allo-s us to directly access the data stream2 There are other higher le!el #unctions 4such as http5bs6 that let you ignore the issues o# sources and get the entire body directly2

3"4 5) G6)G- ,%erloaded9trin!s 4"7 i$port etwork/*TTP/Conduit @ http> parse6rl> withMana!er> IeEuestBody @IeEuestBody5B9A > reEuestBody> $ethod> Iesponse @//A A i$port 1ata/)eson @Value @,bSect> 9trin!AA i$port 1ata/)eson/Parser @SsonA i$port 1ata/Conduit @@++AA i$port 1ata/Conduit/)ttoparsec @sinkParserA i$port Control/Monad/I,/Class @li#tI,A i$port 1ata/)eson @encode> @/=A> obSectA $ain == I, @A $ain = withMana!er + 2$ana!er "< do %alue G" li#tI, $akeValue "" Ce need to know the si'e o# the reEuest body> so we con%ert to a "" Byte9trin! let %alueB9 = encode %alue reEP G" li#tI, + parse6rl 8http=..localhost=&KKK.8 let reE = reEP 3 $ethod = 8P,9T8> reEuestBody = IeEuestBody5B9 %alueB9 7 Iesponse status %ersion headers body G" http reE $ana!er resValue G" body ++ sinkParser Sson li#tI, + handleIesponse resValue "" )pplication"speci#ic #unction to $ake the reEuest %alue $akeValue == I, Value $akeValue = return + obSect : @8#oo8 /= @8bar8 == 9trin!AA ; "" )pplication"speci#ic #unction to handle the response #ro$ the ser%er handleIesponse == Value "< I, @A handleIesponse = print

*DA

Case Study= Sphin,&-ased Search

Sphin( is a search ser!er, and po-ers the search #eature on many sites, including Yesod%s o-n site2 While the actual code necessary to integrate Yesod -ith Sphin( is relati!ely short, it touches on a number o# complicated topics, and is there#ore a great case study in ho- to play -ith some o# the under1the1sur#ace details o# Yesod2 There are essentially three di##erent pieces at play here)

Storing the content -e -ish to search2 This is #airly straight1#or-ard Persistent code, and -e -on%t d-ell on it much in this chapter2 ccessing Sphin( search results #rom inside Yesod2 Thanks to the sphin( package, this is actually !ery easy2 Pro!iding the document content to Sphin(2 This is -here the interesting stu## happens, and -ill sho- ho- to deal -ith streaming content #rom a database directly to C&5, -hich gets sent directly o!er the -ire to the client2

Sphin, Setup
8nlike many o# our other e(amples, to start -ith here -e%ll need to actually con#igure and run our e(ternal Sphin( ser!er2 I%m not going to go into all the details o# Sphin(, partly because it%s not rele!ant to our point here, and mostly because I%m not an e(pert on Sphin(2 Sphin( pro!ides three main command line utilities) searchd is the actual search daemon that recei!es re3uests #rom the client 4in this case, our -eb app6 and returns the search results2 indexer parses the set o# documents and creates the search inde(2 search is a debugging utility that -ill run simple 3ueries against Sphin(2 There are t-o important settings) the source and the inde(2 The source tells Sphin( -here to read document in#ormation #rom2 It has direct support #or &ySG5 and PostgreSG5, as -ell as a more general C&5 #ormat kno-n as (mlpipe:2 We%re going to use the last one2 This not only -ill gi!e us more #le(ibility -ith choosing Persistent backends, but -ill also demonstrate some more po-er#ul Yesod concepts2 The second setting is the inde(2 Sphin( can handle multiple indices simultaneously, -hich allo-s it to pro!ide search #or multiple ser!ices at once2 "ach inde( -ill ha!e a source it pulls #rom2 In our case, -e%re going to pro!ide a 8R5 #rom our application 4<search<(mlpipe6 that pro!ides the C&5 #ile re3uired by Sphin(, and then pipe that through to the inde(er2 So -e%ll add the #ollo-ing to our Sphin( con#ig #ile)
source searcherBsrc 3 type = x$lpipeN

*DD

x$lpipeBco$$and = curl http=..localhost=&KKK.search.x$lpipe

index searcher 3 source = searcherBsrc path = .%ar.data.searcher docin#o = extern charsetBtype = ut#"D 7

In order to build your search inde(, you -ould run indexer searcher2 /b!iously this -on%t -ork until you ha!e your -eb app running2 For a production site, it -ould make sense to run this command !ia a crontab script so the inde( is regularly updated2

#asic Yesod Setup

5et%s get our basic Yesod setup going2 We%re going to ha!e a single table in the database #or holding documents, -hich consist o# a title and content2 We%ll store this in a SG5ite database, and pro!ide routes #or searching, adding documents, !ie-ing documents and pro!iding the (mlpipe #ile to Sphin(2
share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| 1oc title Text content Textarea |; data 9earcher = 9earcher ConnectionPool $kHesod 89earcher8 :parseIoutes| . IootI G-T .doc.41ocId 1ocI G-T .add"doc )dd1ocI P,9T .search 9earchI G-T .search.x$lpipe Y$lpipeI G-T |; instance Hesod 9earcher instance HesodPersist 9earcher where type HesodPersistBackend 9earcher = 9ElPersist run1B action = do 9earcher pool G" !etHesod run9ElPool action pool instance IenderMessa!e 9earcher ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e add1oc?or$ == *t$l "< M?or$ 9earcher 9earcher @?or$Iesult 1oc> Cid!etA add1oc?or$ = renderTable + 1oc G+< areE text?ield 8Title8 othin! GW< areE textarea?ield 8Contents8 othin! search?or$ == *t$l "< M?or$ 9earcher 9earcher @?or$Iesult Text> Cid!etA search?or$ = render1i%s + areE @search?ield TrueA 8Fuery8 othin!

*DF

!etIootI == *andler Iep*t$l !etIootI = do docCount G" run1B + count @:; == :?ilter 1oc;A @@B> docCid!etA> BA G" run?or$Post add1oc?or$ @@B> searchCid!etA> BA G" run?or$Get search?or$ let docs = i# docCount == ( then 8There is currently ( docu$ent/8 else 8There are currently 8 UU show docCount UU 8 docu$ents/8 de#ault5ayout :wha$let| Gp<Celco$e to the search application/ 43docs7 G#or$ $ethod=post action=O3)dd1ocI7< Gtable< Q3docCid!et7 Gtr< Gtd colspan=&< Ginput type=sub$it %alue=8)dd docu$ent8< G#or$ $ethod=!et action=O39earchI7< Q3searchCid!et7 Ginput type=sub$it %alue=9earch< |; post)dd1ocI == *andler Iep*t$l post)dd1ocI = do @@res> docCid!etA> BA G" run?or$Post add1oc?or$ case res o# ?or$9uccess doc "< do docid G" run1B + insert doc setMessa!e 81ocu$ent added8 redirect + 1ocI docid B "< de#ault5ayout :wha$let| G#or$ $ethod=post action=O3)dd1ocI7< Gtable< Q3docCid!et7 Gtr< Gtd colspan=&< Ginput type=sub$it %alue=8)dd docu$ent8< |; !et1ocI == 1ocId "< *andler Iep*t$l !et1ocI docid = do doc G" run1B + !etLKL docid de#ault5ayout + :wha$let| Gh(<43docTitle doc7 Gdi% /content<43docContent doc7 |; data Iesult = Iesult 3 resultId == 1ocId > resultTitle == Text > result-xcerpt == *t$l 7 !etIesult == 1ocId "< 1oc "< Text "< I, Iesult !etIesult docid doc Estrin! = do excerptP G" 9/build-xcerpts excerptCon#i! :T/unpack + escape + docContent doc; 8searcher8 @unpack Estrin!A

*D+

let excerpt = case excerptP o# 9T/,k bss "< pre-scaped5a'yText + decode6t#DCith i!nore + 5/concat bss B "< return @A return Iesult 3 resultId = docid > resultTitle = docTitle doc > result-xcerpt = excerpt 7 where excerptCon#i! = -/altCon#i! 3 -/port = \&(N 7 escape == Textarea "< Text escape = T/concatMap escapeChar / unTextarea where escapeChar PGP = 8TltR8 escapeChar P<P = 8T!tR8 escapeChar PTP = 8Ta$pR8 escapeChar c = T/sin!leton c !etIesults == Text "< *andler :Iesult; !etIesults Estrin! = do sphinxIesP G" li#tI, + 9/Euery con#i! 8searcher8 @unpack Estrin!A case sphinxIesP o# 9T/,k sphinxIes "< do let docids = $ap @Xey / PersistIntVL / 9T/docu$entIdA + 9T/$atches sphinxIes #$ap catMaybes + run1B + #orM docids + 2docid "< do $doc G" !et docid case $doc o# othin! "< return othin! Just doc "< li#tI, + Just G+< !etIesult docid doc Estrin! B "< error + show sphinxIesP where con#i! = 9/de#aultCon#i! 3 9/port = \&(N > 9/$ode = 9T/)ny 7 !et9earchI == *andler Iep*t$l !et9earchI = do @@#or$Ies> searchCid!etA> BA G" run?or$Get search?or$ searchIesults G" case #or$Ies o# ?or$9uccess Estrin! "< !etIesults Estrin! B "< return :; de#ault5ayout + do toCid!et :lucius| /excerpt 3 color= !reenR #ont"style= italic 7 /$atch 3 back!round"color= yellowR 7 |; :wha$let| G#or$ $ethod=!et action=O39earchI7< Q3searchCid!et7

*DB

Ginput type=sub$it %alue=9earch< +i# not + null searchIesults Gh(<Iesults +#orall result G" searchIesults Gdi% /result< Ga hre#=O31ocI + resultId result7<43resultTitle result7 Gdi% /excerpt<43result-xcerpt result7 |; !etY$lpipeI == *andler IepY$l !etY$lpipeI = do 9earcher pool G" !etHesod let headers = :@8Content"Type8> 8text.x$l8A; $ana!edConn G" li#t + takeIesource pool let conn = $rValue $ana!edConn li#t + $rIeuse $ana!edConn True let source = #ull1oc9ource conn C/+= renderBuilder de# #lush9ource = C/$ap,utput C/Chunk source sendCaiIesponse + Iesponse9ource statusNKK headers #lush9ource entityTo-%ents == @-ntity 1ocA "< :Y/-%ent; entityTo-%ents @-ntity docid docA = : Y/-%entBe!in-le$ent docu$ent :@8id8> :Y/ContentText + toPathPiece docid;A; > Y/-%entBe!in-le$ent content :; > Y/-%entContent + Y/ContentText + unTextarea + docContent doc > Y/-%ent-nd-le$ent content > Y/-%ent-nd-le$ent docu$ent ; #ull1oc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent #ull1oc9ource conn = $concat : C5/source5ist start-%ents > doc9ource conn > C5/source5ist end-%ents ; doc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent doc9ource conn = select9ourceConn conn :; :; C/+= C5/concatMap entityTo-%ents to a$e == Text "< Y/ a$e to a$e x = Y/ a$e x @Just 8http=..sphinxsearch/co$.8A @Just 8sphinx8A docset> sche$a> #ield> docu$ent> content == Y/ a$e docset = to a$e 8docset8 sche$a = to a$e 8sche$a8 #ield = to a$e 8#ield8 docu$ent = to a$e 8docu$ent8 content = 8content8 "" no pre#ix start-%ents> end-%ents == :Y/-%ent; start-%ents = : Y/-%entBe!in1ocu$ent > Y/-%entBe!in-le$ent docset :; > Y/-%entBe!in-le$ent sche$a :; > Y/-%entBe!in-le$ent #ield :@8na$e8> :Y/ContentText 8content8;A; > Y/-%ent-nd-le$ent #ield > Y/-%ent-nd-le$ent sche$a ;

*FH

end-%ents = : Y/-%ent-nd-le$ent docset ; $ain == I, @A $ain = with9ElitePool 8searcher/db&8 (K + 2pool "< do run9ElPool @runMi!ration $i!rate)llA pool warp1ebu! &KKK + 9earcher pool

Hope#ully all o# this looks pretty #amiliar by no-2 0e(t -e%ll de#ine some #orms) one #or creating documents, and one #or searching)
add1oc?or$ == *t$l "< M?or$ 9earcher 9earcher @?or$Iesult 1oc> Cid!etA add1oc?or$ = renderTable + 1oc G+< areE text?ield 8Title8 othin! GW< areE textarea?ield 8Contents8 othin! search?or$ == *t$l "< M?or$ 9earcher 9earcher @?or$Iesult Text> Cid!etA search?or$ = render1i%s + areE @search?ield TrueA 8Fuery8 othin! !etIootI == *andler Iep*t$l !etIootI = do docCount G" run1B + count @:; == :?ilter 1oc;A @@B> docCid!etA> BA G" run?or$Post add1oc?or$ @@B> searchCid!etA> BA G" run?or$Get search?or$ let docs = i# docCount == ( then 8There is currently ( docu$ent/8 else 8There are currently 8 UU show docCount UU 8 docu$ents/8 de#ault5ayout :wha$let| Gp<Celco$e to the search application/ 43docs7 G#or$ $ethod=post action=O3)dd1ocI7< Gtable< Q3docCid!et7 Gtr< Gtd colspan=&< Ginput type=sub$it %alue=8)dd docu$ent8< G#or$ $ethod=!et action=O39earchI7< Q3searchCid!et7 Ginput type=sub$it %alue=9earch< |; post)dd1ocI == *andler Iep*t$l post)dd1ocI = do @@res> docCid!etA> BA G" run?or$Post add1oc?or$ case res o# ?or$9uccess doc "< do docid G" run1B + insert doc setMessa!e 81ocu$ent added8 redirect + 1ocI docid B "< de#ault5ayout :wha$let| G#or$ $ethod=post action=O3)dd1ocI7< Gtable< Q3docCid!et7 Gtr< Gtd colspan=&< Ginput type=sub$it %alue=8)dd docu$ent8< |; !et1ocI == 1ocId "< *andler Iep*t$l !et1ocI docid = do

*F*

doc G" run1B + !etLKL docid de#ault5ayout + :wha$let| Gh(<43docTitle doc7 Gdi% /content<43docContent doc7 |; data Iesult = Iesult 3 resultId == 1ocId > resultTitle == Text > result-xcerpt == *t$l 7 !etIesult == 1ocId "< 1oc "< Text "< I, Iesult !etIesult docid doc Estrin! = do excerptP G" 9/build-xcerpts excerptCon#i! :T/unpack + escape + docContent doc; 8searcher8 @unpack Estrin!A let excerpt = case excerptP o# 9T/,k bss "< pre-scaped5a'yText + decode6t#DCith i!nore + 5/concat bss B "< return @A return Iesult 3 resultId = docid > resultTitle = docTitle doc > result-xcerpt = excerpt 7 where excerptCon#i! = -/altCon#i! 3 -/port = \&(N 7 escape == Textarea "< Text escape = T/concatMap escapeChar / unTextarea where escapeChar PGP = 8TltR8 escapeChar P<P = 8T!tR8 escapeChar PTP = 8Ta$pR8 escapeChar c = T/sin!leton c !etIesults == Text "< *andler :Iesult; !etIesults Estrin! = do sphinxIesP G" li#tI, + 9/Euery con#i! 8searcher8 @unpack Estrin!A case sphinxIesP o# 9T/,k sphinxIes "< do let docids = $ap @Xey / PersistIntVL / 9T/docu$entIdA + 9T/$atches sphinxIes #$ap catMaybes + run1B + #orM docids + 2docid "< do $doc G" !et docid case $doc o# othin! "< return othin! Just doc "< li#tI, + Just G+< !etIesult docid doc Estrin! B "< error + show sphinxIesP where con#i! = 9/de#aultCon#i! 3 9/port = \&(N > 9/$ode = 9T/)ny 7

*F:

!et9earchI == *andler Iep*t$l !et9earchI = do @@#or$Ies> searchCid!etA> BA G" run?or$Get search?or$ searchIesults G" case #or$Ies o# ?or$9uccess Estrin! "< !etIesults Estrin! B "< return :; de#ault5ayout + do toCid!et :lucius| /excerpt 3 color= !reenR #ont"style= italic 7 /$atch 3 back!round"color= yellowR 7 |; :wha$let| G#or$ $ethod=!et action=O39earchI7< Q3searchCid!et7 Ginput type=sub$it %alue=9earch< +i# not + null searchIesults Gh(<Iesults +#orall result G" searchIesults Gdi% /result< Ga hre#=O31ocI + resultId result7<43resultTitle result7 Gdi% /excerpt<43result-xcerpt result7 |; !etY$lpipeI == *andler IepY$l !etY$lpipeI = do 9earcher pool G" !etHesod let headers = :@8Content"Type8> 8text.x$l8A; $ana!edConn G" li#t + takeIesource pool let conn = $rValue $ana!edConn li#t + $rIeuse $ana!edConn True let source = #ull1oc9ource conn C/+= renderBuilder de# #lush9ource = C/$ap,utput C/Chunk source sendCaiIesponse + Iesponse9ource statusNKK headers #lush9ource entityTo-%ents == @-ntity 1ocA "< :Y/-%ent; entityTo-%ents @-ntity docid docA = : Y/-%entBe!in-le$ent docu$ent :@8id8> :Y/ContentText + toPathPiece docid;A; > Y/-%entBe!in-le$ent content :; > Y/-%entContent + Y/ContentText + unTextarea + docContent doc > Y/-%ent-nd-le$ent content > Y/-%ent-nd-le$ent docu$ent ; #ull1oc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent #ull1oc9ource conn = $concat : C5/source5ist start-%ents > doc9ource conn > C5/source5ist end-%ents ; doc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent doc9ource conn = select9ourceConn conn :; :; C/+= C5/concatMap entityTo-%ents

*F>

to a$e == Text "< Y/ a$e to a$e x = Y/ a$e x @Just 8http=..sphinxsearch/co$.8A @Just 8sphinx8A docset> sche$a> #ield> docu$ent> content == Y/ a$e docset = to a$e 8docset8 sche$a = to a$e 8sche$a8 #ield = to a$e 8#ield8 docu$ent = to a$e 8docu$ent8 content = 8content8 "" no pre#ix start-%ents> end-%ents == :Y/-%ent; start-%ents = : Y/-%entBe!in1ocu$ent > Y/-%entBe!in-le$ent docset :; > Y/-%entBe!in-le$ent sche$a :; > Y/-%entBe!in-le$ent #ield :@8na$e8> :Y/ContentText 8content8;A; > Y/-%ent-nd-le$ent #ield > Y/-%ent-nd-le$ent sche$a ; end-%ents = : Y/-%ent-nd-le$ent docset ; $ain == I, @A $ain = with9ElitePool 8searcher/db&8 (K + 2pool "< do run9ElPool @runMi!ration $i!rate)llA pool warp1ebu! &KKK + 9earcher pool

The True parameter to searchField makes the #ield auto1#ocus on page load2 Finally, -e ha!e some standard handlers #or the homepage 4sho-s the add document #orm and the search #orm6, the document display, and adding a document2
!etIootI == *andler Iep*t$l !etIootI = do docCount G" run1B + count @:; == :?ilter 1oc;A @@B> docCid!etA> BA G" run?or$Post add1oc?or$ @@B> searchCid!etA> BA G" run?or$Get search?or$ let docs = i# docCount == ( then 8There is currently ( docu$ent/8 else 8There are currently 8 UU show docCount UU 8 docu$ents/8 de#ault5ayout :wha$let| Gp<Celco$e to the search application/ 43docs7 G#or$ $ethod=post action=O3)dd1ocI7< Gtable< Q3docCid!et7 Gtr< Gtd colspan=&< Ginput type=sub$it %alue=8)dd docu$ent8< G#or$ $ethod=!et action=O39earchI7< Q3searchCid!et7 Ginput type=sub$it %alue=9earch< |; post)dd1ocI == *andler Iep*t$l post)dd1ocI = do @@res> docCid!etA> BA G" run?or$Post add1oc?or$ case res o# ?or$9uccess doc "< do

*F@

docid G" run1B + insert doc setMessa!e 81ocu$ent added8 redirect + 1ocI docid B "< de#ault5ayout :wha$let| G#or$ $ethod=post action=O3)dd1ocI7< Gtable< Q3docCid!et7 Gtr< Gtd colspan=&< Ginput type=sub$it %alue=8)dd docu$ent8< |; !et1ocI == 1ocId "< *andler Iep*t$l !et1ocI docid = do doc G" run1B + !etLKL docid de#ault5ayout + :wha$let| Gh(<43docTitle doc7 Gdi% /content<43docContent doc7 |; data Iesult = Iesult 3 resultId == 1ocId > resultTitle == Text > result-xcerpt == *t$l 7 !etIesult == 1ocId "< 1oc "< Text "< I, Iesult !etIesult docid doc Estrin! = do excerptP G" 9/build-xcerpts excerptCon#i! :T/unpack + escape + docContent doc; 8searcher8 @unpack Estrin!A let excerpt = case excerptP o# 9T/,k bss "< pre-scaped5a'yText + decode6t#DCith i!nore + 5/concat bss B "< return @A return Iesult 3 resultId = docid > resultTitle = docTitle doc > result-xcerpt = excerpt 7 where excerptCon#i! = -/altCon#i! 3 -/port = \&(N 7 escape == Textarea "< Text escape = T/concatMap escapeChar / unTextarea where escapeChar PGP = 8TltR8 escapeChar P<P = 8T!tR8 escapeChar PTP = 8Ta$pR8 escapeChar c = T/sin!leton c !etIesults == Text "< *andler :Iesult; !etIesults Estrin! = do sphinxIesP G" li#tI, + 9/Euery con#i! 8searcher8 @unpack Estrin!A case sphinxIesP o# 9T/,k sphinxIes "< do

*FA

let docids = $ap @Xey / PersistIntVL / 9T/docu$entIdA + 9T/$atches sphinxIes #$ap catMaybes + run1B + #orM docids + 2docid "< do $doc G" !et docid case $doc o# othin! "< return othin! Just doc "< li#tI, + Just G+< !etIesult docid doc Estrin! B "< error + show sphinxIesP where con#i! = 9/de#aultCon#i! 3 9/port = \&(N > 9/$ode = 9T/)ny 7 !et9earchI == *andler Iep*t$l !et9earchI = do @@#or$Ies> searchCid!etA> BA G" run?or$Get search?or$ searchIesults G" case #or$Ies o# ?or$9uccess Estrin! "< !etIesults Estrin! B "< return :; de#ault5ayout + do toCid!et :lucius| /excerpt 3 color= !reenR #ont"style= italic 7 /$atch 3 back!round"color= yellowR 7 |; :wha$let| G#or$ $ethod=!et action=O39earchI7< Q3searchCid!et7 Ginput type=sub$it %alue=9earch< +i# not + null searchIesults Gh(<Iesults +#orall result G" searchIesults Gdi% /result< Ga hre#=O31ocI + resultId result7<43resultTitle result7 Gdi% /excerpt<43result-xcerpt result7 |; !etY$lpipeI == *andler IepY$l !etY$lpipeI = do 9earcher pool G" !etHesod let headers = :@8Content"Type8> 8text.x$l8A; $ana!edConn G" li#t + takeIesource pool let conn = $rValue $ana!edConn li#t + $rIeuse $ana!edConn True let source = #ull1oc9ource conn C/+= renderBuilder de# #lush9ource = C/$ap,utput C/Chunk source sendCaiIesponse + Iesponse9ource statusNKK headers #lush9ource entityTo-%ents == @-ntity 1ocA "< :Y/-%ent; entityTo-%ents @-ntity docid docA = : Y/-%entBe!in-le$ent docu$ent :@8id8> :Y/ContentText + toPathPiece docid;A; > Y/-%entBe!in-le$ent content :; > Y/-%entContent + Y/ContentText + unTextarea + docContent doc > Y/-%ent-nd-le$ent content

*FD

> Y/-%ent-nd-le$ent docu$ent ; #ull1oc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent #ull1oc9ource conn = $concat : C5/source5ist start-%ents > doc9ource conn > C5/source5ist end-%ents ; doc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent doc9ource conn = select9ourceConn conn :; :; C/+= C5/concatMap entityTo-%ents to a$e == Text "< Y/ a$e to a$e x = Y/ a$e x @Just 8http=..sphinxsearch/co$.8A @Just 8sphinx8A docset> sche$a> #ield> docu$ent> content == Y/ a$e docset = to a$e 8docset8 sche$a = to a$e 8sche$a8 #ield = to a$e 8#ield8 docu$ent = to a$e 8docu$ent8 content = 8content8 "" no pre#ix start-%ents> end-%ents == :Y/-%ent; start-%ents = : Y/-%entBe!in1ocu$ent > Y/-%entBe!in-le$ent docset :; > Y/-%entBe!in-le$ent sche$a :; > Y/-%entBe!in-le$ent #ield :@8na$e8> :Y/ContentText 8content8;A; > Y/-%ent-nd-le$ent #ield > Y/-%ent-nd-le$ent sche$a ; end-%ents = : Y/-%ent-nd-le$ent docset ; $ain == I, @A $ain = with9ElitePool 8searcher/db&8 (K + 2pool "< do run9ElPool @runMi!ration $i!rate)llA pool warp1ebu! &KKK + 9earcher pool

Searching
0o- that -e%!e got the boring stu## out o# the -ay, let%s 9ump into the actual searching2 We%re going to need three pieces o# in#ormation #or displaying a result) the document ID it comes #rom, the title o# that document, and the e(cerpts2 "(cerpts are the highlighted portions o# the document -hich contain the search term2

*FF

Search %esult

So let%s start o## by de#ining a Result datatype)

data Iesult = Iesult 3 resultId == 1ocId > resultTitle == Text > result-xcerpt == *t$l 7

0e(t -e%ll look at the search handler)

!et9earchI == *andler Iep*t$l !et9earchI = do @@#or$Ies> searchCid!etA> BA G" run?or$Get search?or$ searchIesults G" case #or$Ies o# ?or$9uccess Estrin! "< !etIesults Estrin! B "< return :; de#ault5ayout + do toCid!et :lucius| /excerpt 3 color= !reenR #ont"style= italic 7 /$atch 3 back!round"color= yellowR 7 |; :wha$let|

*F+

G#or$ $ethod=!et action=O39earchI7< Q3searchCid!et7 Ginput type=sub$it %alue=9earch< +i# not + null searchIesults Gh(<Iesults +#orall result G" searchIesults Gdi% /result< Ga hre#=O31ocI + resultId result7<43resultTitle result7 Gdi% /excerpt<43result-xcerpt result7 |; !etY$lpipeI == *andler IepY$l !etY$lpipeI = do 9earcher pool G" !etHesod let headers = :@8Content"Type8> 8text.x$l8A; $ana!edConn G" li#t + takeIesource pool let conn = $rValue $ana!edConn li#t + $rIeuse $ana!edConn True let source = #ull1oc9ource conn C/+= renderBuilder de# #lush9ource = C/$ap,utput C/Chunk source sendCaiIesponse + Iesponse9ource statusNKK headers #lush9ource entityTo-%ents == @-ntity 1ocA "< :Y/-%ent; entityTo-%ents @-ntity docid docA = : Y/-%entBe!in-le$ent docu$ent :@8id8> :Y/ContentText + toPathPiece docid;A; > Y/-%entBe!in-le$ent content :; > Y/-%entContent + Y/ContentText + unTextarea + docContent doc > Y/-%ent-nd-le$ent content > Y/-%ent-nd-le$ent docu$ent ; #ull1oc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent #ull1oc9ource conn = $concat : C5/source5ist start-%ents > doc9ource conn > C5/source5ist end-%ents ; doc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent doc9ource conn = select9ourceConn conn :; :; C/+= C5/concatMap entityTo-%ents to a$e == Text "< Y/ a$e to a$e x = Y/ a$e x @Just 8http=..sphinxsearch/co$.8A @Just 8sphinx8A docset> sche$a> #ield> docu$ent> content == Y/ a$e docset = to a$e 8docset8 sche$a = to a$e 8sche$a8 #ield = to a$e 8#ield8 docu$ent = to a$e 8docu$ent8 content = 8content8 "" no pre#ix start-%ents> end-%ents == :Y/-%ent; start-%ents = : Y/-%entBe!in1ocu$ent > Y/-%entBe!in-le$ent docset :; > Y/-%entBe!in-le$ent sche$a :; > Y/-%entBe!in-le$ent #ield :@8na$e8> :Y/ContentText 8content8;A; > Y/-%ent-nd-le$ent #ield > Y/-%ent-nd-le$ent sche$a

*FB

; end-%ents = : Y/-%ent-nd-le$ent docset ; $ain == I, @A $ain = with9ElitePool 8searcher/db&8 (K + 2pool "< do run9ElPool @runMi!ration $i!rate)llA pool warp1ebu! &KKK + 9earcher pool

0othing magical here, -e%re 9ust relying on the search?or$ de#ined abo!e, and the !etIesults #unction -hich hasn%t been de#ined yet2 This #unction 9ust takes a search string, and returns a list o# results2 This is -here -e #irst interact -ith the Sphin( PI2 We%ll be using t-o #unctions) Euery -ill return a list o# matches, and build-xcerpts -ill return the highlighted e(cerpts2 5et%s #irst look at Euery)
!etIesults == Text "< *andler :Iesult; !etIesults Estrin! = do sphinxIesP G" li#tI, + 9/Euery con#i! 8searcher8 @unpack Estrin!A case sphinxIesP o# 9T/,k sphinxIes "< do let docids = $ap @Xey / PersistIntVL / 9T/docu$entIdA + 9T/$atches sphinxIes #$ap catMaybes + run1B + #orM docids + 2docid "< do $doc G" !et docid case $doc o# othin! "< return othin! Just doc "< li#tI, + Just G+< !etIesult docid doc Estrin! B "< error + show sphinxIesP where con#i! = 9/de#aultCon#i! 3 9/port = \&(N > 9/$ode = 9T/)ny 7 !et9earchI == *andler Iep*t$l !et9earchI = do @@#or$Ies> searchCid!etA> BA G" run?or$Get search?or$ searchIesults G" case #or$Ies o# ?or$9uccess Estrin! "< !etIesults Estrin! B "< return :; de#ault5ayout + do toCid!et :lucius| /excerpt 3 color= !reenR #ont"style= italic 7 /$atch 3 back!round"color= yellowR 7 |; :wha$let| G#or$ $ethod=!et action=O39earchI7< Q3searchCid!et7 Ginput type=sub$it %alue=9earch< +i# not + null searchIesults Gh(<Iesults

*+H

+#orall result G" searchIesults Gdi% /result< Ga hre#=O31ocI + resultId result7<43resultTitle result7 Gdi% /excerpt<43result-xcerpt result7 |; !etY$lpipeI == *andler IepY$l !etY$lpipeI = do 9earcher pool G" !etHesod let headers = :@8Content"Type8> 8text.x$l8A; $ana!edConn G" li#t + takeIesource pool let conn = $rValue $ana!edConn li#t + $rIeuse $ana!edConn True let source = #ull1oc9ource conn C/+= renderBuilder de# #lush9ource = C/$ap,utput C/Chunk source sendCaiIesponse + Iesponse9ource statusNKK headers #lush9ource entityTo-%ents == @-ntity 1ocA "< :Y/-%ent; entityTo-%ents @-ntity docid docA = : Y/-%entBe!in-le$ent docu$ent :@8id8> :Y/ContentText + toPathPiece docid;A; > Y/-%entBe!in-le$ent content :; > Y/-%entContent + Y/ContentText + unTextarea + docContent doc > Y/-%ent-nd-le$ent content > Y/-%ent-nd-le$ent docu$ent ; #ull1oc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent #ull1oc9ource conn = $concat : C5/source5ist start-%ents > doc9ource conn > C5/source5ist end-%ents ; doc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent doc9ource conn = select9ourceConn conn :; :; C/+= C5/concatMap entityTo-%ents to a$e == Text "< Y/ a$e to a$e x = Y/ a$e x @Just 8http=..sphinxsearch/co$.8A @Just 8sphinx8A docset> sche$a> #ield> docu$ent> content == Y/ a$e docset = to a$e 8docset8 sche$a = to a$e 8sche$a8 #ield = to a$e 8#ield8 docu$ent = to a$e 8docu$ent8 content = 8content8 "" no pre#ix start-%ents> end-%ents == :Y/-%ent; start-%ents = : Y/-%entBe!in1ocu$ent > Y/-%entBe!in-le$ent docset :; > Y/-%entBe!in-le$ent sche$a :; > Y/-%entBe!in-le$ent #ield :@8na$e8> :Y/ContentText 8content8;A; > Y/-%ent-nd-le$ent #ield > Y/-%ent-nd-le$ent sche$a ; end-%ents = : Y/-%ent-nd-le$ent docset ;

*+*

$ain == I, @A $ain = with9ElitePool 8searcher/db&8 (K + 2pool "< do run9ElPool @runMi!ration $i!rate)llA pool warp1ebu! &KKK + 9earcher pool

takes three parameters) the con#iguration options, the inde( to search against 4searcher in this case6 and the search string2 It returns a list o# document IDs that contain the search string2 The tricky bit here is that those documents are returned as IntVL !alues, -hereas -e need 1ocIds2 We%re taking ad!antage o# the #act that the SG5 Persistent backends use a PersistIntVL constructor #or their IDs, and simply -rap up the !alues appropriately2
Euery

I# you%re dealing -ith a backend that has non1numeric IDs, like &ongoDB, you%ll need to -ork out something a bit more cle!er than this2 We then loop o!er the resulting IDs to get a :Maybe Iesult; !alue, and use catMaybes to turn it into a :Iesult;2 In the -here clause, -e de#ine our local settings, -hich o!erride the de#ault port and set up the search to -ork -hen any term matches the document2 5et%s #inally look at the !etIesult #unction)
!etIesult == 1ocId "< 1oc "< Text "< I, Iesult !etIesult docid doc Estrin! = do excerptP G" 9/build-xcerpts excerptCon#i! :T/unpack + escape + docContent doc; 8searcher8 @unpack Estrin!A let excerpt = case excerptP o# 9T/,k bss "< pre-scaped5a'yText + decode6t#DCith i!nore + 5/concat bss B "< 88 return Iesult 3 resultId = docid > resultTitle = docTitle doc > result-xcerpt = excerpt 7 where excerptCon#i! = -/altCon#i! 3 -/port = \&(N 7 escape == Textarea "< Text escape = T/concatMap escapeChar / unTextarea where escapeChar PGP = 8TltR8 escapeChar P<P = 8T!tR8 escapeChar PTP = 8Ta$pR8 escapeChar c = T/sin!leton c

takes #our parameters) the con#iguration options, the te(tual contents o# the document, the search inde( and the search term2 The interesting bit is that -e entity escape the te(t content2 Sphin( -on%t automatically escape these #or us, so -e must do it e(plicitly2
build-xcerpts

Similarly, the result #rom Sphin( is a list o# la'y ByteStrings2 But o# course, -e%d rather ha!e Html2 So -e concat that list into a single la'y ByteString, decode it to a la'y te(t 4ignoring *+:

in!alid 8TF1+ character se3uences6, and use pre"scaped5a'yTe(t to make sure that the tags inserted #or matches are not escaped2 sample o# this HT&5 is)
T4DN&KR 1epart$ents/ The President shall ha%e Gspan class=P$atchP<PowerG.span< to #ill up all Vacancies T4DN&KR people/ )$end$ent (( The Judicial Gspan class=P$atchP<powerG.span< o# the 6nited 9tates shall T4DN&KR Surisdiction/ N/ Con!ress shall ha%e Gspan class=P$atchP<powerG.span< to en#orce this article by T4DN&KR M/ The Con!ress shall ha%e Gspan class=P$atchP<powerG.span< to en#orce> by appropriate le!islation T4DN&KR

Streaming ,mlpipe output

We%!e sa!ed the best #or last2 For the ma9ority o# Yesod handlers, the recommended approach is to load up the database results into memory and then produce the output document based on that2 It%s simpler to -ork -ith, but more importantly it%s more resilient to e(ceptions2 I# there%s a problem loading the data #rom the database, the user -ill get a proper AHH response code2 What do I mean by Iproper AHH response code=I I# you start streaming a response to a client, and encounter an e(ception hal#-ay through, there%s no -ay to change the status code7 the user -ill see a :HH response that simply stops in the middle2 0ot only can this partial content be con#using, but it%s an in!alid usage o# the HTTP spec2 Ho-e!er, generating the (mlpipe output is a per#ect e(ample o# the alternati!e2 There are potentially a huge number o# documents 4the yesod-eb2com code handles tens o# thousands o# these6, and documents could easily be se!eral hundred kilobytes2 I# -e take a non1 streaming approach, this can lead to huge memory usage and slo- response times2 So ho- e(actly do -e create a streaming response= s -e co!er in the W I chapter, -e ha!e a Iesponse9ource constructor that uses a stream o# bla'e1builder Builders2 From the Yesod side, -e can a!oid the normal Yesod response procedure and send a W I response directly using the sendCaiIesponse #unction2 So there are at least t-o o# the pieces o# this pu''le2 0o- -e kno- -e -ant to create a stream o# Builders #rom some C&5 content2 Fortunately, the (ml1conduit package pro!ides this inter#ace directly2 x$l"conduit pro!ides some high1 le!el inter#aces #or dealing -ith documents as a -hole, but in our case, -e%re going to need to use the lo-1le!el "!ent inter#ace to ensure minimal memory impact2 So the #unction -e%re interested in is)
renderBuilder == Iesource $ =< Iender9ettin!s "< Conduit -%ent $ Builder b

In plain "nglish, that means renderBuilder takes some settings 4-e%ll 9ust use the de#aults6, and -ill then con!ert a stream o# -%ents to a stream o# Builders2 This is looking pretty good, all -e need no- is a stream o# -%ents2 Speaking o# -hich, -hat should our C&5 document actually look like= It%s pretty simple, -e ha!e a sphinx=docset root element, a sphinx=sche$a element containing a single

*+>

4-hich de#ines the content #ield6, and then a sphinx=docu$ent #or each document in our database2 That last element -ill ha!e an id attribute and a child content element2
sphinx=#ield

Sample ,mlpipe document

Gsphinx=docset x$lns=sphinx=8http=..sphinxsearch/co$.8< Gsphinx=sche$a< Gsphinx=#ield na$e=8content8.< G.sphinx=sche$a< Gsphinx=docu$ent id=8(8< Gcontent<barG.content< G.sphinx=docu$ent< Gsphinx=docu$ent id=8N8< Gcontent<#oo bar ba'G.content< G.sphinx=docu$ent< G.sphinx=docset<

"!ery document is going to start o## -ith the same e!ents 4start the docset, start the schema, etc6 and end -ith the same e!ent 4end the docset62 We%ll start o## by de#ining those)
to a$e == Text "< Y/ a$e to a$e x = Y/ a$e x @Just 8http=..sphinxsearch/co$.8A @Just 8sphinx8A docset> sche$a> #ield> docu$ent> content == Y/ a$e docset = to a$e 8docset8 sche$a = to a$e 8sche$a8 #ield = to a$e 8#ield8 docu$ent = to a$e 8docu$ent8 content = 8content8 "" no pre#ix start-%ents> end-%ents == :Y/-%ent; start-%ents = : Y/-%entBe!in1ocu$ent > Y/-%entBe!in-le$ent docset :; > Y/-%entBe!in-le$ent sche$a :; > Y/-%entBe!in-le$ent #ield :@8na$e8> :Y/ContentText 8content8;A; > Y/-%ent-nd-le$ent #ield > Y/-%ent-nd-le$ent sche$a ; end-%ents = : Y/-%ent-nd-le$ent docset ;

0o- that -e ha!e the shell o# our document, -e need to get the -%ents #or each indi!idual document2 This is actually a #airly simple #unction)
entityTo-%ents == @-ntity 1ocA "< :Y/-%ent; entityTo-%ents @-ntity docid docA = : Y/-%entBe!in-le$ent docu$ent :@8id8> :Y/ContentText + toPathPiece docid;A; > Y/-%entBe!in-le$ent content :; > Y/-%entContent + Y/ContentText + unTextarea + docContent doc > Y/-%ent-nd-le$ent content > Y/-%ent-nd-le$ent docu$ent ;

*+@

We start the document element -ith an id attribute, start the content, insert the content, and then close both elements2 We use toPathPiece to con!ert a 1ocId into a Text !alue2 0e(t, -e need to be able to con!ert a stream o# these entities into a stream o# e!ents2 For this, -e can use the built1in concatMap #unction #rom 1ata/Conduit/5ist) C5/concatMap entityTo-%ents2 But -hat -e really -ant is to stream those e!ents directly #rom the database2 For most o# this book, -e%!e used the select5ist #unction, but Persistent also pro!ides the 4more po-er#ul6 select9ourceConn #unction2 So -e end up -ith the #unction)
doc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent doc9ource conn = select9ourceConn conn :; :; C/+= C5/concatMap entityTo-%ents

The ^_ operator 9oins together a source and a conduit into a ne- source2 0o- that -e ha!e our -%ent source, all -e need to do is surround it -ith the document start and end e!ents2 With 9ource%s Monoid instance, this is a piece o# cake)
#ull1oc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent #ull1oc9ource conn = $concat : C5/source5ist start-%ents > doc9ource conn > C5/source5ist end-%ents ;

We%re almost there, no- -e 9ust need to tie it together in !etY$lpipeI2 We need to get a database connection to be used2 0ormally, database connections are taken and returned automatically !ia the run1B #unction2 In our case, -e -ant to check out a connection and keep it a!ailable until the response body is completely sent2 To do this, -e use the takeIesource #unction, -hich registers a cleanup action -ith the IesourceT monad2 ll W I applications li!e in a IesourceT trans#ormer2 You can get more in#ormation on IesourceT in the conduit appendi(2 By de#ault, a resource -ill not be returned to the pool2 This has to do -ith proper e(ception handling, but is not rele!ant #or our use case2 There#ore, -e need to #orce the connection to be returned to the pool2
!etY$lpipeI == *andler IepY$l !etY$lpipeI = do 9earcher pool G" !etHesod let headers = :@8Content"Type8> 8text.x$l8A; $ana!edConn G" li#t + takeIesource pool let conn = $rValue $ana!edConn li#t + $rIeuse $ana!edConn True let source = #ull1oc9ource conn C/+= renderBuilder de# sendCaiIesponse + Iesponse9ource statusNKK headers source

We get our connection pool #rom the #oundation !ariable, then send a W I response2 We use the Iesponse9ource constructor, and pro!ide it the status code, response headers, and body2

!ull code
*+A

3"4 5) G6)G- ,%erloaded9trin!s> Type?a$ilies> Te$plate*askell> FuasiFuotes> MultiPara$TypeClasses> G)1Ts> ?lexibleContexts 4"7 i$port Hesod i$port 1ata/Text @Text> unpackA i$port Control/)pplicati%e @@G+<A> @GW<AA i$port 1atabase/Persist/9Elite i$port 1atabase/Persist/Fuery/Generic9El @select9ourceConnA i$port 1atabase/Persist/9tore @PersistValue @PersistIntVLAA i$port Euali#ied Text/9earch/9phinx as 9 i$port Euali#ied Text/9earch/9phinx/Types as 9T i$port Euali#ied Text/9earch/9phinx/-xcerptCon#i!uration as i$port Euali#ied 1ata/Byte9trin!/5a'y as 5 i$port 1ata/Text/5a'y/-ncodin! @decode6t#DCithA i$port 1ata/Text/-ncodin!/-rror @i!noreA i$port 1ata/Maybe @catMaybesA i$port Control/Monad @#orMA i$port Euali#ied 1ata/Text as T i$port Text/Bla'e @pre-scaped5a'yTextA i$port Euali#ied 1ata/Conduit as C i$port Euali#ied 1ata/Conduit/5ist as C5 i$port Euali#ied 1ata/YM5/Types as Y i$port etwork/Cai @Iesponse @Iesponse9ourceAA i$port etwork/*TTP/Types @statusNKKA i$port Text/YM5/9trea$/Iender @renderBuilder> de#A i$port 1ata/Monoid @$concatA i$port 1ata/Conduit/Pool @takeIesource> $rValue> $rIeuseA share :$kPersist sEl9ettin!s> $kMi!rate 8$i!rate)ll8; :persist| 1oc title Text content Textarea |; data 9earcher = 9earcher ConnectionPool $kHesod 89earcher8 :parseIoutes| . IootI G-T .doc.41ocId 1ocI G-T .add"doc )dd1ocI P,9T .search 9earchI G-T .search.x$lpipe Y$lpipeI G-T |; instance Hesod 9earcher instance HesodPersist 9earcher where type HesodPersistBackend 9earcher = 9ElPersist run1B action = do 9earcher pool G" !etHesod run9ElPool action pool instance IenderMessa!e 9earcher ?or$Messa!e where renderMessa!e B B = de#ault?or$Messa!e add1oc?or$ == *t$l "< M?or$ 9earcher 9earcher @?or$Iesult 1oc> Cid!etA add1oc?or$ = renderTable + 1oc G+< areE text?ield 8Title8 othin! GW< areE textarea?ield 8Contents8 othin!

*+D

search?or$ == *t$l "< M?or$ 9earcher 9earcher @?or$Iesult Text> Cid!etA search?or$ = render1i%s + areE @search?ield TrueA 8Fuery8 othin! !etIootI == *andler Iep*t$l !etIootI = do docCount G" run1B + count @:; == :?ilter 1oc;A @@B> docCid!etA> BA G" run?or$Post add1oc?or$ @@B> searchCid!etA> BA G" run?or$Get search?or$ let docs = i# docCount == ( then 8There is currently ( docu$ent/8 else 8There are currently 8 UU show docCount UU 8 docu$ents/8 de#ault5ayout :wha$let| Gp<Celco$e to the search application/ 43docs7 G#or$ $ethod=post action=O3)dd1ocI7< Gtable< Q3docCid!et7 Gtr< Gtd colspan=&< Ginput type=sub$it %alue=8)dd docu$ent8< G#or$ $ethod=!et action=O39earchI7< Q3searchCid!et7 Ginput type=sub$it %alue=9earch< |; post)dd1ocI == *andler Iep*t$l post)dd1ocI = do @@res> docCid!etA> BA G" run?or$Post add1oc?or$ case res o# ?or$9uccess doc "< do docid G" run1B + insert doc setMessa!e 81ocu$ent added8 redirect + 1ocI docid B "< de#ault5ayout :wha$let| G#or$ $ethod=post action=O3)dd1ocI7< Gtable< Q3docCid!et7 Gtr< Gtd colspan=&< Ginput type=sub$it %alue=8)dd docu$ent8< |; !et1ocI == 1ocId "< *andler Iep*t$l !et1ocI docid = do doc G" run1B + !etLKL docid de#ault5ayout + :wha$let| Gh(<43docTitle doc7 Gdi% /content<43docContent doc7 |; data Iesult = Iesult 3 resultId == 1ocId > resultTitle == Text > result-xcerpt == *t$l 7 !etIesult == 1ocId "< 1oc "< Text "< I, Iesult !etIesult docid doc Estrin! = do excerptP G" 9/build-xcerpts excerptCon#i!

*+F

:T/unpack + escape + docContent doc; 8searcher8 @unpack Estrin!A let excerpt = case excerptP o# 9T/,k bss "< pre-scaped5a'yText + decode6t#DCith i!nore + 5/concat bss B "< 88 return Iesult 3 resultId = docid > resultTitle = docTitle doc > result-xcerpt = excerpt 7 where excerptCon#i! = -/altCon#i! 3 -/port = \&(N 7 escape == Textarea "< Text escape = T/concatMap escapeChar / unTextarea where escapeChar PGP = 8TltR8 escapeChar P<P = 8T!tR8 escapeChar PTP = 8Ta$pR8 escapeChar c = T/sin!leton c !etIesults == Text "< *andler :Iesult; !etIesults Estrin! = do sphinxIesP G" li#tI, + 9/Euery con#i! 8searcher8 @unpack Estrin!A case sphinxIesP o# 9T/,k sphinxIes "< do let docids = $ap @Xey / PersistIntVL / 9T/docu$entIdA + 9T/$atches sphinxIes #$ap catMaybes + run1B + #orM docids + 2docid "< do $doc G" !et docid case $doc o# othin! "< return othin! Just doc "< li#tI, + Just G+< !etIesult docid doc Estrin! B "< error + show sphinxIesP where con#i! = 9/de#aultCon#i! 3 9/port = \&(N > 9/$ode = 9T/)ny 7 !et9earchI == *andler Iep*t$l !et9earchI = do @@#or$Ies> searchCid!etA> BA G" run?or$Get search?or$ searchIesults G" case #or$Ies o# ?or$9uccess Estrin! "< !etIesults Estrin! B "< return :; de#ault5ayout + do toCid!et :lucius| /excerpt 3 color= !reenR #ont"style= italic 7 /$atch 3 back!round"color= yellowR 7 |;

*++

:wha$let| G#or$ $ethod=!et action=O39earchI7< Q3searchCid!et7 Ginput type=sub$it %alue=9earch< +i# not + null searchIesults Gh(<Iesults +#orall result G" searchIesults Gdi% /result< Ga hre#=O31ocI + resultId result7<43resultTitle result7 Gdi% /excerpt<43result-xcerpt result7 |; !etY$lpipeI == *andler IepY$l !etY$lpipeI = do 9earcher pool G" !etHesod let headers = :@8Content"Type8> 8text.x$l8A; $ana!edConn G" li#t + takeIesource pool let conn = $rValue $ana!edConn li#t + $rIeuse $ana!edConn True let source = #ull1oc9ource conn C/+= renderBuilder de# #lush9ource = C/$ap,utput C/Chunk source sendCaiIesponse + Iesponse9ource statusNKK headers #lush9ource entityTo-%ents == @-ntity 1ocA "< :Y/-%ent; entityTo-%ents @-ntity docid docA = : Y/-%entBe!in-le$ent docu$ent :@8id8> :Y/ContentText + toPathPiece docid;A; > Y/-%entBe!in-le$ent content :; > Y/-%entContent + Y/ContentText + unTextarea + docContent doc > Y/-%ent-nd-le$ent content > Y/-%ent-nd-le$ent docu$ent ; #ull1oc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent #ull1oc9ource conn = $concat : C5/source5ist start-%ents > doc9ource conn > C5/source5ist end-%ents ; doc9ource == Connection "< C/9ource @C/IesourceT I,A Y/-%ent doc9ource conn = select9ourceConn conn :; :; C/+= C5/concatMap entityTo-%ents to a$e == Text "< Y/ a$e to a$e x = Y/ a$e x @Just 8http=..sphinxsearch/co$.8A @Just 8sphinx8A docset> sche$a> #ield> docu$ent> content == Y/ a$e docset = to a$e 8docset8 sche$a = to a$e 8sche$a8 #ield = to a$e 8#ield8 docu$ent = to a$e 8docu$ent8 content = 8content8 "" no pre#ix start-%ents> end-%ents == :Y/-%ent; start-%ents = : Y/-%entBe!in1ocu$ent > Y/-%entBe!in-le$ent docset :; > Y/-%entBe!in-le$ent sche$a :; > Y/-%entBe!in-le$ent #ield :@8na$e8> :Y/ContentText 8content8;A; > Y/-%ent-nd-le$ent #ield

*+B

> Y/-%ent-nd-le$ent sche$a ; end-%ents = : Y/-%ent-nd-le$ent docset ; $ain == I, @A $ain = with9ElitePool 8searcher/db&8 (K + 2pool "< do run9ElPool @runMi!ration $i!rate)llA pool warp1ebu! &KKK + 9earcher pool

*BH

Appendices
monad&control
monad1control is used in a #e- places -ithin Yesod, most notably to ensure proper e(ception handling -ithin Persistent2 It is a general purpose package to e(tend standard #unctionality in monad trans#ormers2

O er ie3
/ne o# the po-er#ul, and sometimes con#using, #eatures in Haskell is monad trans#ormers2 They allo- you to take di##erent pieces o# #unctionality1 such as mutable state, error handling, or logging1 and compose them together easily2 Though I s-ore I%d ne!er -rite a monad tutorial, I%m going to employ a pain#ul analogy here) monads are like onions2 4&onads are not like cakes26 By that, I mean layers2 We ha!e the core monad1 also kno-n as the innermost or bottom monad2 /n top o# this core, -e add layers, each adding a ne- #eature and spreading out-ard<up-ard2 s a moti!ating e(ample, let%s consider an "rror monad stacked on top o# the I/ monad)
newtype -rrorT e $ a = -rrorT 3 run-rrorT == $ @-ither e aA 7 type My9tack = -rrorT My-rror I,

0o- pay close attention here) "rrorT is 9ust a simple ne-type around an "ither -rapped in a monad2 ;etting rid o# the ne-type, -e ha!e)
type -rrorT6nwrapped e $ a = $ @-ither e aA

t some point, -e%ll need to actually per#orm some I/ inside our &yStack2 I# -e -ent -ith the un-rapped approach, it -ould be tri!ial, since there -ould be no "rrorT constructor in the -ay2 Ho-e!er, -e need that ne-type -rapper #or a -hole bunch o# type reasons I -on%t go into here 4this isn%t a monad trans#ormer tutorial a#ter all62 So the solution is the &onadTrans typeclass)
class MonadTrans t where li#t == Monad $ =< $ a "< t $ a

I%ll admit, the #irst time I sa- that type signature, my response -as stunned con#usion, and incredulity that it actually meant anything2 But looking at an instance helps a bit)
instance @-rror eA =< MonadTrans @-rrorT eA where li#t $ = -rrorT + do a G" $ return @Ii!ht aA

ll -e%re doing is -rapping the inside o# the I/ -ith a Right !alue, and then applying our ne-type -rapper2 This allo-s us to take an action that li!es in I/, and Ili#tI it to the outer<upper monad2 But no- to the point at hand2 This -orks !ery -ell #or simple #unctions2 For e(ample) *B*

say*i == I, @A say*i = put9tr5n 8*ello8 say*i-rror == -rrorT My-rror I, @A say*i-rror = li#t + put9tr5n 8*ello8

But let%s take something slightly more complicated, like a callback)

withMy?ile == @*andle "< I, aA "< I, a withMy?ile = with?ile 8test/txt8 CriteMode say*i == *andle "< I, @A say*i handle = hPut9tr5n handle 8*i there8 useMy?ile == I, @A useMy?ile = withMy?ile say*i

So #ar so good, right= 0o- let%s say that -e need a !ersion o# sayHi that has access to the "rror monad)
say*i-rror == *andle "< -rrorT My-rror I, @A say*i-rror handle = do li#t + hPut9tr5n handle 8*i there> errorJ8 throw-rror My-rror

We -ould like to -rite a #unction that combines -ith&yFile and sayHi"rror2 8n#ortunately, ;H$ doesn%t like this !ery much)
useMy?ile-rrorBad == -rrorT My-rror I, @A useMy?ile-rrorBad = withMy?ile say*i-rror CouldnPt $atch expected type Z-rrorT My-rror I, @AP with actual type ZI, @AP

Why does this happen, and ho- can -e -ork around it=

Intuition
5et%s try and de!elop an e(ternal intuition o# -hat%s happening here2 The "rrorT monad trans#ormer adds e(tra #unctionality to the I/ monad2 We%!e de#ined a -ay to Itack onI that e(tra #unctionality to normal I/ actions) -e add that Right constructor and -rap it all in "rrorT2 Wrapping in Right is our -ay o# saying Iit -ent /P,I there -asn%t anything -rong -ith this action2 0o- this intuiti!ely makes sense) since the I/ monad doesn%t ha!e the concept o# returning a &y"rror -hen something goes -rong, it -ill al-ays succeed in the li#ting phase2 40ote) This has nothing to do -ith runtime e(ceptions, don%t e!en think about them26 What -e ha!e is a guaranteed one1directional translation up the monad stack2 5et%s take another e(ample) the Reader monad2 Reader has access to some e(tra piece o# data #loating around2 Whate!er is running in the inner monad doesn%t kno- about that e(tra piece o# in#ormation2 So ho- -ould you do a li#t= You 9ust ignore that e(tra in#ormation2 The Writer monad= Don%t -rite anything2 State= Don%t change anything2 I%m seeing a pattern here2

*B:

But no- let%s try and go in the opposite direction) I ha!e something in a Reader, and I%d like to run it in the base monad 4e2g2, I/62 Well222 that%s not going to -ork, is it= I need that e(tra piece o# in#ormation, I%m relying on it, and it%s not there2 There%s simply no -ay to go in the opposite direction -ithout pro!iding that e(tra !alue2 /r is there= I# you remember, -e%d pointed out earlier that "rrorT is 9ust a simple -rapper around the inner monad2 In other -ords, i# I ha!e errorValue == -rrorT My-rror I, MyValue, I can apply run-rrorT and get a !alue o# type I, @-ither My-rror MyValueA2 The looks 3uite a bit like bi1directional translation, doesn%t it= Well, not 3uite2 We originally had an -rrorT My-rror I, monad, -ith a !alue o# type MyValue2 0o- -e ha!e a monad o# type I, -ith a !alue o# type -ither My-rror MyValue2 So this process has in #act changed the !alue, -hile the li#ting process lea!es it the same2 But still, -ith a little #ancy #oot-ork -e can un-rap the "rrorT, do some processing, and then -rap it back up again2
useMy?ile-rror( == -rrorT My-rror I, @A useMy?ile-rror( = let unwrapped == *andle "< I, @-ither My-rror @AA unwrapped handle = run-rrorT + say*i-rror handle applied == I, @-ither My-rror @AA applied = withMy?ile unwrapped rewrapped == -rrorT My-rror I, @A rewrapped = -rrorT applied in rewrapped

This is the crucial point o# this -hole article, so look closely2 We #irst un-rap our monad2 This means that, to the outside -orld, it%s no- 9ust a plain old I/ !alue2 Internally, -e%!e stored all the in#ormation #rom our "rrorT trans#ormer2 0o- that -e ha!e a plain old I/, -e can easily pass it o## to -ith&yFile2 -ith&yFile takes in the internal state and passes it back out unchanged2 Finally, -e -rap e!erything back up into our original "rrorT2 This is the entire pattern o# monad1control) -e embed the e(tra #eatures o# our monad trans#ormer inside the !alue2 /nce in the !alue, the type system ignores it and #ocuses on the inner monad2 When -e%re done playing around -ith that inner monad, -e can pull our state back out and reconstruct our original monad stack2

Types
I purposely started -ith the "rrorT trans#ormer, as it is one o# the simplest #or this in!ersion mechanism2 8n#ortunately, others are a bit more complicated2 Take #or instance ReaderT2 It is de#ined as newtype IeaderT r $ a = IeaderT 3 runIeaderT == r "< $ a 72 I# -e apply runIeaderT to it, -e get a #unction that returns a monadic !alue2 So -e%re going to need some e(tra machinery to deal -ith all that stu##2 nd this is -hen -e lea!e Pansas behind2 There are a #e- approaches to sol!ing these problems2 In the past, I implemented a solution using type #amilies in the neither package2 nders Paseorg implemented a much more

*B>

straight1#or-ard solution in monad1peel2 nd #or e##iciency, in monad1control, Bas !an Di9k uses $PS 4continuation passing style6 and e(istential types2 The code taken #rom monad1control actually applies to !ersion H2:2 H2> changed things 9ust a bit, by making the state e(plicit -ith an associated type, and generali'ing MonadControlI, to MonadBaseControl, but the concepts are still the same2 The #irst type -e%re going to look at is)
type Iun t = #orall n o b/ @Monad n> Monad o> Monad @t oAA =< t n b "< n @t o bA

That%s incredibly dense, let%s talk it out2 The only IinputI datatype to this thing is t, a monad trans#ormer2 Run is a #unction that -ill then -ork -ith any combination o# types n, o and b 4that%s -hat the #orall means62 n and o are both monads, -hile b is a simple !alue contained by them2 The le#t hand side o# the Run #unction, t n b, is our monad trans#ormer -rapped around the n monad and holding a b !alue2 So #or e(ample, that could be a MyTrans ?irstMonad MyValue2 It then returns a !alue -ith the trans#ormer IpoppedI inside, -ith a brand nemonad at its core2 In other -ords, ?irstMonad @MyTrans ewMonad MyValueA2 That might sound pretty scary at #irst, but it actually isn%t as #oreign as you%d think) this is essentially -hat -e did -ith "rrorT2 We started -ith "rrorT on the outside, -rapping around I/, and ended up -ith an I/ by itsel# containing an "ither2 Well guess -hat) another -ay to represent an "ither is -rrorT My-rror Identity2 So essentially, -e pulled the I/ to the outside and plunked an Identity in its place2 We%re doing the same thing in a Run) pulling the First&onad outside and replacing it -ith a 0e-&onad2 0o- might be a good time to get a beer2 lright, no- -e%re getting some-here2 I# -e had access to one o# those Run #unctions, -e could use it to peel o## the "rrorT on our sayHi"rror #unction and pass it to -ith&yFile2 With the magic o# unde#ined, -e can play such a game)
errorIun == Iun @-rrorT My-rrorA errorIun = unde#ined useMy?ile-rrorN == I, @-rrorT My-rror Identity @AA useMy?ile-rrorN = let a#terIun == *andle "< I, @-rrorT My-rror Identity @AA a#terIun handle = errorIun + say*i-rror handle applied == I, @-rrorT My-rror Identity @AA applied = withMy?ile a#terIun in applied

This looks eerily similar to our pre!ious e(ample2 In #act, errorRun is acting almost identically to run"rrorT2 Ho-e!er, -e%re still le#t -ith t-o problems) -e don%t kno- -here to get that errorRun !alue #rom, and -e still need to restructure the original "rrorT a#ter -e%re done2

*B@

MonadTransControl
/b!iously in the speci#ic case -e ha!e be#ore us, -e could use our kno-ledge o# the "rrorT trans#ormer to beat the types into submission and create our Run #unction manually2 But -hat -e really -ant is a general solution #or many trans#ormers2 t this point, you kno- -e need a typeclass2 So let%s re!ie- -hat -e need) access to a Run #unction, and some -ay to restructure our original trans#ormer a#ter the #act2 nd thus -as born &onadTrans$ontrol, -ith its single method li#t$ontrol)
class MonadTrans t =< MonadTransControl t where li#tControl == Monad $ =< @Iun t "< $ aA "< t $ a

5et%s look at this closely2 li#t$ontrol takes a #unction 4the one -e%ll be -riting62 That #unction is pro!ided -ith a Run #unction, and must return a !alue in some monad 4m62 li#t$ontrol -ill then take the result o# that #unction and reinstate the original trans#ormer on top o# e!erything2
useMy?ile-rror& == Monad $ =< -rrorT My-rror I, @-rrorT My-rror $ @AA useMy?ile-rror& = li#tControl inside where inside == Monad $ =< Iun @-rrorT My-rrorA "< I, @-rrorT My-rror $ @AA inside run = withMy?ile + helper run helper == Monad $ =< Iun @-rrorT My-rrorA "< *andle "< I, @-rrorT My-rror $ @AA helper run handle = run @say*i-rror handle == -rrorT My-rror I, @AA

$lose, but not e(actly -hat I had in mind2 What%s up -ith the double monads= Well, let%s start at the end) sayHi"rror handle returns a !alue o# type -rrorT My-rror I, @A2 This -e knealready, no surprises2 What might be a little surprising 4it got me, at least6 is the ne(t t-o steps2 First -e apply run to that !alue2 5ike -e%d discussed be#ore, the result is that the I/ inner monad is popped to the outside, to be replaced by some arbitrary monad 4represented by m here62 So -e end up -ith an I/ 4"rrorT &y"rror m 4662 /k222 We then get the same result a#ter applying -ith&yFile2 0ot surprising2 The last step took me a long time to understand correctly2 Remember ho- -e said that -e reconstruct the original trans#ormer= Well, so -e do) by plopping it right on top o# e!erything else -e ha!e2 So our end result is the pre!ious type1 I, @-rrorT My-rror $ @AA1 -ith a -rrorT My-rror stuck on the #ront2 Well, that seems 9ust about utterly -orthless, right= Well, almost2 But don%t #orget, that ImI can be any monad, including I/2 I# -e treat it that -ay, -e get -rrorT My-rror I, @-rrorT My-rror I, @AA2 That looks a lot like $ @$ aA, and -e -ant 9ust plain old $ a2 Fortunately, no- -e%re in luck)
useMy?ile-rrorL == -rrorT My-rror I, @A useMy?ile-rrorL = Soin useMy?ile-rror&

*BA

nd it turns out that this usage is so common, that Bas had mercy on us and de#ined a helper #unction)
control == @Monad $> Monad @t $A> MonadTransControl tA =< @Iun t "< $ @t $ aAA "< t $ a control = Soin / li#tControl

So all -e need to -rite is)

useMy?ile-rrorM == -rrorT My-rror I, @A useMy?ile-rrorM = control inside where inside == Monad $ =< Iun @-rrorT My-rrorA "< I, @-rrorT My-rror $ @AA inside run = withMy?ile + helper run helper == Monad $ =< Iun @-rrorT My-rrorA "< *andle "< I, @-rrorT My-rror $ @AA helper run handle = run @say*i-rror handle == -rrorT My-rror I, @AA

nd 9ust to make it a little shorter)

useMy?ile-rrorV == -rrorT My-rror I, @A useMy?ile-rrorV = control + 2run "< withMy?ile + run / say*i-rror

MonadControlIO
The &onadTrans class pro!ides the li#t method, -hich allo-s you to li#t an action one le!el in the stack2 There is also the &onadI/ class that pro!ides li#tI/, -hich li#ts an I/ action as #ar in the stack as desired2 We ha!e the same breakdo-n in monad1control2 But #irst, -e need a corrolary to Run)
type IunInBase $ base = #orall b/ $ b "< base @$ bA

Instead o# dealing -ith a trans#ormer, -e%re dealing -ith t-o monads2 base is the underlying monad, and m is a stack built on top o# it2 RunInBase is a #unction that takes a !alue o# the entire stack, pops out that base, and puts in on the outside2 8nlike in the Run type, -e don%t replace it -ith an arbitrary monad, but -ith the original one2 To use some more concrete types)
IunInBase @-rrorT My-rror I,A I, = #orall b/ -rrorT My-rror I, b "< I, @-rrorT My-rror I, bA

This should look #airly similar to -hat -e%!e been looking at so #ar, the only di##erence is that -e -ant to deal -ith a speci#ic inner monad2 /ur &onad$ontrolI/ class is really 9ust an e(tension o# &onad$ontrolTrans using this RunInBase2
class MonadI, $ =< MonadControlI, $ where li#tControlI, == @IunInBase $ I, "< I, aA "< $ a

Simply put, li#t$ontrolI/ takes a #unction -hich recei!es a RunInBase2 That RunInBase can be used to strip do-n our monad to 9ust an I/, and then li#t$ontrolI/ builds e!erything back up again2 nd like &onad$ontrolTrans, it comes -ith a helper #unction
controlI, == MonadControlI, $ =< @IunInBase $ I, "< I, @$ aAA "< $ a controlI, = Soin / li#tControlI,

We can easily re-rite our pre!ious e(ample -ith it)

*BD

useMy?ile-rror[ == -rrorT My-rror I, @A useMy?ile-rror[ = controlI, + 2run "< withMy?ile + run / say*i-rror

nd as an ad!antage, it easily scales to multiple trans#ormers)

say*iCra'y == *andle "< IeaderT Int @9tateT 1ouble @-rrorT My-rror I,AA @A say*iCra'y handle = li#tI, + hPut9tr5n handle 8MadnessJ8 useMy?ileCra'y == IeaderT Int @9tateT 1ouble @-rrorT My-rror I,AA @A useMy?ileCra'y = controlI, + 2run "< withMy?ile + run / say*iCra'y

%eal Life /,amples

5et%s sol!e some real1li#e problems -ith this code2 Probably the biggest moti!ating use case is e(ception handling in a trans#ormer stack2 For e(ample, let%s say that -e -ant to automatically run some cleanup code -hen an e(ception is thro-n2 I# this -ere normal I/ code, -e%d use)
on-xception == I, a "< I, b "< I, a

But i# -e%re in the "rrorT monad, -e can%t pass in either the action or the cleanup2 In comes controlI/ to the rescue)
on-xception-rror == -rrorT My-rror I, a "< -rrorT My-rror I, b "< -rrorT My-rror I, a on-xception-rror action a#ter = controlI, + 2run "< run action Zon-xceptionZ run a#ter

5et%s say -e need to allocate some memory to store a Double in2 In the I/ monad, -e could 9ust use the alloca #unction2 /nce again, our solution is simple)
alloca-rror == @Ptr 1ouble "< -rrorT My-rror I, bA "< -rrorT My-rror I, b alloca-rror # = controlI, + 2run "< alloca + run / #

Lost State
5et%s re-ind a bit to our on"(ception"rror2 It uses on"(ception under the sur#ace, -hich has a type signature) I, a "< I, b "< I, a2 5et me ask you something) -hat happened to the b in the output= Well, it -as thoroughly ignored2 But that seems to cause us a bit o# a problem2 #ter all, -e store our trans#ormer state in#ormation in the !alue o# the inner monad2 I# -e ignore it, -e%re essentially ignoring the monadic side e##ects as -ell? nd the ans-er is that, yes, this does happen -ith monad1control2 $ertain #unctions -ill drop some o# the monadic side e##ects2 This is put best by Bas, in the comments on the rele!ant #unctions) 0ote, any monadic side e##ects in m o# the IreleaseI computation -ill be discarded7 it is run only #or its side e##ects in I/2 In practice, monad1control -ill usually be doing the right thing #or you, but you need to be a-are that some side e##ects may disappear2

*BF

More Complicated Cases

In order to make our tricks -ork so #ar, -e%!e needed to ha!e #unctions that gi!e us #ull access to play around -ith their !alues2 Sometimes, this isn%t the case2 Take, #or instance)
addMVar?inali'er == MVar a "< I, @A "< I, @A

In this case, -e are re3uired to ha!e no !alue inside our #inali'er #unction2 Intuiti!ely, the #irst thing -e should notice is that there -ill be no -ay to capture our monadic side e##ects2 So ho- do -e get something like this to compile= Well, -e need to e(plicitly tell it to drop all o# its state1holding in#ormation)
addMVar?inali'er-rror == MVar a "< -rrorT My-rror I, @A "< -rrorT My-rror I, @A addMVar?inali'er-rror $%ar # = controlI, + 2run "< return + li#tI, + addMVar?inali'er $%ar @run # << return @AA

nother case #rom the same module is)

$odi#yMVar == MVar a "< @a "< I, @a> bAA "< I, b

Here, -e ha!e a restriction on the return type in the second argument) it must be a tuple o# the !alue passed to that #unction and the #inal return !alue2 8n#ortunately, I can%t see a -ay o# -riting a little -rapper around modi#y&Ear to make it -ork #or "rrorT2 Instead, in this case, I copied the de#inition o# modi#y&Ear and modi#ied it)
$odi#yMVar == MVar a "< @a "< -rrorT My-rror I, @a> bAA "< -rrorT My-rror I, b $odi#yMVar $ io = Control/-xception/Control/$ask + 2restore "< do a G" li#tI, + takeMVar $ @aP>bA G" restore @io aA Zon-xception-rrorZ li#tI, @putMVar $ aA li#tI, + putMVar $ aP return b

Conduit
$onduits are a solution to the streaming data problem2 /#ten times, la'iness allo-s us to process large amounts o# data -ithout pulling all !alues into memory2 Ho-e!er, doing so in the presence o# I</ re3uires us to use la'y I</2 The main do-nside to la'y I</ is non1 determinism) -e ha!e no guarantees o# -hen our resource #inali'ers -ill be run2 For small application, this may be acceptable, but #or a high1load ser!er, -e could 3uickly run out o# scarce resources, such as #ile handles2 $onduits allo- us to process large streams o# data -hile still retaining deterministic resource handling2 They pro!ide a uni#ied inter#ace #or data streams, -hether they come #rom #iles, sockets, or memory2 nd -hen combined -ith IesourceT, -e can sa#ely allocate resources, kno-ing that they -ill al-ays be reclaimed1 e!en in the presence o# e(ceptions2 This appendi( co!ers !ersion H2: o# the conduit package2

*B+

Conduits in !i e Minutes
While a good understanding o# the lo-er1le!el mechanics o# conduits is ad!isable, you can get !ery #ar -ithout it2 5et%s start o## -ith some high1le!el e(amples2 Don%t -orry i# some o# the details seem a bit magical right no-2 We%ll co!er e!erything in the course o# this appendi(2 5et%s start -ith the terminology, and then some sample code2 Source producer o# data2 The data could be in a #ile, coming #rom a socket, or in memory as a list2 To access this data, -e pull #rom the source2 Sink consumer o# data2 Basic e(amples -ould be a sum #unction 4adding up a stream o# numbers #ed in6, a #ile sink 4-hich -rites all incoming bytes to a #ile6, or a socket2 We push data into a sink2 When the sink #inishes processing 4-e%ll e(plain that later6, it returns some !alue2 $onduit trans#ormer o# data2 The simplest e(ample is a map #unction, though there are many others2 5ike a sink, -e push data into a conduit2 But instead o# returning a single !alue at the end, a conduit can return multiple outputs e!ery time it is pushed to2 Fuse 4Thanks to Da!id &a'ieres #or the term26 conduit can be #used -ith a source to produce a ne-, modi#ied source 4the += operator62 For e(ample, you could ha!e a source that reads bytes #rom a #ile, and a conduit that decodes bytes into te(t2 I# you #use them together, you -ould no- ha!e a source that reads te(t #rom a #ile2 5ike-ise, a conduit and a sink can #use into a ne- sink 4=+6, and t-o conduits can #use into a ne- conduit 4=+=62 $onnect You can connect a source to a sink using the ++ operator2 Doing so -ill pull data #rom the source and push it to the sink, until either the source or sink signals that they are Idone2I 5et%s see some e(amples o# conduit code2
3"4 5) i$port i$port i$port i$port i$port i$port i$port i$port G6)G- ,%erloaded9trin!s 4"7 1ata/Conduit "" the core library Euali#ied 1ata/Conduit/5ist as C5 "" so$e list"like #unctions Euali#ied 1ata/Conduit/Binary as CB "" bytes Euali#ied 1ata/Conduit/Text as CT 1ata/Byte9trin! @Byte9trin!A 1ata/Text @TextA Euali#ied 1ata/Text as T Control/Monad/9T @run9TA

"" 5etPs start with the basics= connectin! a source to a sink/ CePll use the "" built in #ile #unctions to i$ple$entin! e##icient> constant"$e$ory> "" resource"#riendly #ile copyin!/ "" "" Two thin!s to note= we use ++ to connect our source to our sink> and then "" use runIesourceT/

*BB

copy?ile == ?ilePath "< ?ilePath "< I, @A copy?ile src dest = runIesourceT + CB/source?ile src ++ CB/sink?ile dest "" The 1ata/Conduit/5ist $odule pro%ides a nu$ber o# helper #unctions #or "" creatin! sources> sinks> and conduits/ 5etPs look at a typical #old= su$$in! "" nu$bers/ su$9ink == Iesource $ =< 9ink Int $ Int su$9ink = C5/#old @UA K "" I# we want to !o a little $ore low"le%el> we can code our sink with the "" sink9tate #unction/ This #unction takes three para$eters= an initial state> "" a push #unction @recei%e so$e $ore dataA> and a close #unction/ su$9inkN == Iesource $ =< 9ink Int $ Int su$9inkN = sink9tate K "" initial %alue "" update the state with the new input and "" indicate that we want $ore input @2accu$ i "< return + 9tateProcessin! @accu$ U iAA @2accu$ "< return accu$A "" return the current accu$ %alue on close "" )nother co$$on helper #unction is source5ist/ 5etPs see how we can co$bine "" that #unction with our su$9ink to rei$ple$ent the built"in su$ #unction/ su$P == :Int; "< Int su$P input = run9T + runIesourceT + C5/source5ist input ++ su$9ink "" 9ince this is *askell> letPs write a source to !enerate all o# the "" ?ibonacci nu$bers/ CePll use source9tate/ The state will contain the next "" two nu$bers in the seEuence/ Ce also need to pro%ide a pull #unction> which "" will return the next nu$ber and update the state/ #ibs == Iesource $ =< 9ource $ Int #ibs = source9tate @K> (A "" initial state @2@x> yA "< return + 9tate,pen @y> x U yA xA "" 9uppose we want to !et the su$ o# the #irst (K ?ibonacci nu$bers/ Ce can use "" the isolate conduit to $ake sure the su$ sink only consu$es (K %alues/ su$Ten?ibs == Int su$Ten?ibs = run9T "" runs #ine in pure code + runIesourceT + #ibs += C5/isolate (K "" #use the source and conduit into a source ++ su$9ink "" Ce can also #use the conduit into the sink instead> we Sust swap a #ew "" operators/ su$Ten?ibsN == Int su$Ten?ibsN = run9T + runIesourceT + #ibs ++ C5/isolate (K =+ su$9ink

:HH

"" )lri!ht> letPs $ake so$e conduits/ 5etPs turn our nu$bers into text/ 9ounds "" like a Sob #or a $ap/// intToText == Int "< Text "" Sust a helper #unction intToText = T/pack / show texti#y == Iesource $ =< Conduit Int $ Text texti#y = C5/$ap intToText "" 5ike pre%iously> we can use a conduit9tate helper #unction/ But here> we "" donPt e%en need state> so we pro%ide a du$$y state %alue/ texti#yN == Iesource $ =< Conduit Int $ Text texti#yN = conduit9tate @A @2@A input "< return + 9tateProducin! @A :intToText input;A @2@A "< return :;A "" 5etPs $ake the unlines conduit> that puts a newline on the end o# each piece "" o# input/ CePll Sust use C5/$apR #eel #ree to write it with conduit9tate as "" well #or practice/ unlinesP == Iesource $ =< Conduit Text $ Text unlinesP = C5/$ap + 2t "< t ZT/appendZ 82n8 "" )nd letPs write a #unction that prints the #irst "" use 6T?D encodin!/ write?ibs == Int "< ?ilePath "< I, @A write?ibs count dest = runIesourceT + #ibs += C5/isolate count += texti#y += unlinesP += CT/encode CT/ut#D ++ CB/sink?ile dest "" Ce used the += operator to a "" sin!le source/ Ce can also sink/ Ce can e%en co$bine the write?ibsN == Int "< ?ilePath write?ibsN count dest = runIesourceT + #ibs += C5/isolate count += texti#y ++ unlinesP =+ CT/encode CT/ut#D =+ CB/sink?ile dest #ibs to a #ile/ CePll

#use the conduits into the sources> producin! do the opposite= #use the conduits into the two/ "< I, @A

"" ,r we could #use all those inner conduits into a sin!le conduit/// so$eInt5ines == IesourceThrow $ "" encodin! can throw an exception =< Int "< Conduit Int $ Byte9trin! so$eInt5ines count = C5/isolate count =+= texti#y =+= unlinesP

:H*

=+= CT/encode CT/ut#D "" and then use that conduit write?ibs& == Int "< ?ilePath "< I, @A write?ibs& count dest = runIesourceT + #ibs += so$eInt5ines count ++ CB/sink?ile dest $ain == I, @A $ain = do put9tr5n + 8?irst ten #ibs= 8 UU show su$Ten?ibs write?ibs NK 8#ibs/txt8 copy?ile 8#ibs/txt8 8#ibsN/txt8

Structure of this Chapter

The remainder o# this chapter co!ers #i!e ma9or topics in conduits)
IesourceT,

the underlying techni3ue that allo-s us to ha!e guaranteed resource deallocation2 Sources, our data producers Sinks, our data consumers $onduits, our data trans#ormers Bu##ering, -hich allo-s us to a!oid In!ersion o# $ontrol

The %esource monad transformer

The Resource trans#ormer 4IesourceT6 plays a !ital role in proper resource management in the conduit pro9ect2 It is included -ithin the conduit package itsel#2 We%ll e(plaining IesourceT as its o-n entity2 While some o# the design decisions clearly are biased to-ards conduits, IesourceT should remain a usable tool in its o-n right2

0oals
What%s -rong -ith the #ollo-ing code=
i$port 9yste$/I, $ain = do output G" open?ile 8output/txt8 CriteMode input G" open?ile 8input/txt8 IeadMode hGetContents input <<= hPut9tr output hClose input hClose output

I# the #ile input2t(t does not e(ist, then an e(ception -ill be thro-n -hen trying to open it2 s a result, hClose output -ill ne!er be called, and -e%ll ha!e leaked a scarce resource 4a #ile descriptor62 In our tiny program, this isn%t a big deal, but clearly -e can%t a##ord such -aste in a long running, highly acti!e ser!er process2 :H:

Fortunately, sol!ing the problem is easy)

i$port 9yste$/I, $ain = with?ile 8output/txt8 CriteMode + 2output "< with?ile 8input/txt8 IeadMode + 2input "< hGetContents input <<= hPut9tr output

makes sure that the *andle is al-ays closed, e!en in the presence o# e(ceptions2 It also handles asynchronous e(ceptions2 /!erall, it%s a great approach to use222 -hen you can use it2 While o#ten with?ile is easy to use, sometimes it can re3uire restructuring our programs2 nd this restructuring can range #rom mildly tedious to -ildly ine##icient2
with?ile

5et%s take enumerators #or e(ample2 I# you look in the documentation, there is an enu$?ile #unction 4#or reading contents #rom a #ile6, but no iter?ile 4#or -riting contents to a #ile62 That%s because the #lo- o# control in an iteratee doesn%t allo- proper allocation o# the Handle2 Instead, in order to -rite to a #ile, you need to allocate the Handle be#ore entering the Iteratee, e2g2)
i$port 9yste$/I, i$port 1ata/-nu$erator i$port 1ata/-nu$erator/Binary $ain = with?ile 8output/txt8 CriteMode + 2output "< runB + enu$?ile 8input/txt8 ++ iter*andle output

This code -orks #ine, but imagine that, instead o# simply piping data directly to the #ile, there -as a huge amount o# computation that occurred be#ore -e need to use the output handle2 We -ill ha!e allocated a #ile descriptor long be#ore -e needed it, and thereby locked up a scarce resource in our application2 Besides this, there are times -hen -e can't allocate the #ile be#ore hand, such as -hen -e -on%t kno- -hich #ile to open until -e%!e read #rom the input #ile2 /ne o# the stated goals o# conduits is to sol!e this problem, and it does so !ia IesourceT2 s a result, the abo!e program can be -ritten in conduit as)
3"4 5) G6)G- ,%erloaded9trin!s 4"7 i$port 1ata/Conduit i$port 1ata/Conduit/Binary $ain = runIesourceT + source?ile 8input/txt8 ++ sink?ile 8output/txt8

Ho3 it $orks
There are essentially three base #unctions on IesourceT, and then a bunch o# con!eniences thro-n on top2 The #irst #unction is)
re!ister == I, @A "< IesourceT I, IeleaseXey

:H>

This #unction, and the others belo-, are actually more polymorphic than implied here, allo-ing other monads besides I,2 In #act, almost any trans#ormer on top o# I,, as -ell as any 9T stacks, -ork2 We%ll co!er the details o# that later2 This #unction registers a piece o# code that it asserts must be run2 It gi!es back a IeleaseXey, -hich is used by the ne(t #unction)
release == IeleaseXey "< IesourceT I, @A

$alling release on a IeleaseXey immediately per#orms the action you pre!iously registered2 You may call release on the same IeleaseXey as many times as you like7 the #irst time it is called, it unregisters the action2 This means you can sa#ely register an action like a memory #ree, and ha!e no concerns that it -ill be called t-ice2 "!entually, -e%ll -ant to e(it our special IesourceT2 To do so, -e use)
runIesourceT == IesourceT I, a "< I, a

This seemingly innocuous #unction is -here all the magic happens2 It runs through all o# the registered cleanup actions and per#orms them2 It is #ully e(ception sa#e, meaning the cleanups -ill be per#ormed in the presence o# both synchronous and asynchronous e(ceptions2 nd as mentioned be#ore, calling release -ill unregister an action, so there is no concern o# double1 #reeing2 Finally, as a con!enience, -e pro!ide one more #unction #or the common case o# allocating a resource and registering a release action)
with == I, a "" Q allocate "< @a "< I, @AA "" Q #ree resource "< IesourceT I, @IeleaseXey> aA

So, to re-ork our #irst buggy e(ample to use IesourceT, -e -ould -rite)
i$port 9yste$/I, i$port Control/Monad/Trans/Iesource i$port Control/Monad/Trans/Class @li#tA $ain = runIesourceT + do @release,> outputA G" with @open?ile 8output/txt8 CriteModeA hClose @releaseI> inputA G" with @open?ile 8input/txt8 IeadModeA hClose li#t + hGetContents input <<= hPut9tr output release releaseI release release,

0o- there is no concern o# any e(ceptions pre!enting the releasing o# resources2 We could skip the release calls i# -e -ant to, and in an e(ample this small, it -ould not make any di##erence2 But #or larger applications, -here -e -ant processing to continue, this ensures that the *andles are #reed as early as possible, keeping our scarce resource usage to a minimum2

Some Type Magic

:H@

s alluded to, there%s a bit more to IesourceT than simply running in I,2 5et%s co!er some o# the things -e need #rom this underlying Monad2

&utable re#erences to keep track o# the registered release actions2 You might think -e could 9ust use a 9tateT trans#ormer, but then our state -ouldn%t sur!i!e e(ceptions2 We only -ant to register actions in the base monad2 For e(ample, i# -e ha!e a IesourceT @CriterT :Int; I,A stack, -e only -ant to register I, actions2 This makes it easy to li#t our stacks around 4i2e2, add an e(tra trans#ormer to the middle o# an e(isting stack6, and a!oids con#using issues about the threading o# other monadic side1e##ects2 Some -ay to guarantee an action is per#ormed, e!en in the presence o# e(ceptions2 This boils do-n to needing a bracket1like #unction2

For the #irst point, -e de#ine a ne- typeclass to represent monads that ha!e mutable re#erences)
class Monad $ =< *asIe# $ where type Ie# $ == W "< W newIe#P == a "< $ @Ie# $ aA readIe#P == Ie# $ a "< $ a writeIe#P == Ie# $ a "< a "< $ @A $odi#yIe#P == Ie# $ a "< @a "< @a> bAA "< $ b $ask == @@#orall a/ $ a "< $ aA "< $ bA "< $ b $askB == $ a "< $ a try == $ a "< $ @-ither 9o$e-xception aA

We ha!e an associated type to signi#y -hat the re#erence type should be2 4For #ans o# #undeps, you%ll see in the ne(t section that this has to be an associated type26 Then -e pro!ide a number o# basic re#erence operations2 Finally, there are some #unctions to help -ith e(ceptions, -hich are needed to sa#ely implement the #unctions described in the last section2 The instance #or I, is !ery straight1#or-ard)
instance *asIe# I, where type Ie# I, = I/I,Ie# newIe#P = I/newI,Ie# $odi#yIe#P = I/ato$icModi#yI,Ie# readIe#P = I/readI,Ie# writeIe#P = I/writeI,Ie# $ask = -/$ask $askB = -/$askB try = -/try

Ho-e!er, -e ha!e a problem -hen it comes to implementing the 9T instance) there is no -ay to deal -ith e(ceptions in the 9T monad2 s a result, $ask, $askB and try are gi!en de#ault implementations that do no e(ception checking2 This gi!es rise to the #irst -ord o# -arning) operations in the ST monad are not e,ception safe2 You should not be allocating scarce resources in ST -hen using IesourceT2 You might be -ondering -hy bother -ith IesourceT at all then #or 9T2 The ans-er is that there is a lot you can do -ith conduits -ithout allocating scarce resources, and 9T is a great -ay to do this in a pure -ay2 But more on this later2 0o- onto point :) -e need some -ay to deal -ith this base monad concept2 gain, -e use an associated type 4again e(plained in the ne(t section62 /ur solution looks something like) :HA

class @*asIe# @Base $A> Monad $A =< Iesource $ where type Base $ == W "< W resource5i#tBase == Base $ a "< $ a

But -e #orgot about point >) some bracket1like #unction2 So -e need one more method in this typeclass)
resourceBracketB == Base $ a "< Base $ b "< $ c "< $ c

The reason the #irst t-o arguments to resourceBracketB 4allocation and cleanup6 li!e in Base $ instead o# $ is that, in IesourceT, all allocation and cleanup li!es in the base monad2 So on top o# our *asIe# instance #or I,, -e no- need a Iesource instance as -ell2 This is similarly straight1#or-ard)
instance Iesource I, where type Base I, = I, resource5i#tBase = id resourceBracketB = -/bracketB

We ha!e similar 9T instances, -ith resourceBracketB ha!ing no e(ception sa#ety2 The #inal step is dealing -ith monad trans#ormers2 We don%t need to pro!ide a *asIe# instance, but -e do need a Iesource instance2 The tricky part is pro!iding a !alid implementation o# resourceBracketB2 For this, -e use some #unctions #rom monad1control)
instance @MonadTransControl t> Iesource $> Monad @t $AA =< Iesource @t $A where type Base @t $A = Base $ resource5i#tBase = li#t / resource5i#tBase resourceBracketB a b c = controlP + 2run "< resourceBracketB a b @run cA where controlP # = li#tCith # <<= restoreT / return

For any trans#ormer, its base is the base o# its inner monad2 Similarly, -e li#t to the base by li#ting to the inner monad and then li#ting to the base #rom there2 The tricky part is the implemetnation o# resourceBracketB2 I -ill not go into a detailed e(planation, as I -ould simply make a #ool o# mysel#2

(efinition of %esourceT
We no- ha!e enough in#ormation to understand the de#inition o# IesourceT)
newtype IeleaseXey = IeleaseXey Int type Ie#Count = Int type extXey = Int data IeleaseMap base = IeleaseMap J extXey JIe#Count J@IntMap @base @AAA

:HD

newtype IesourceT $ a = IesourceT @Ie# @Base $A @IeleaseMap @Base $AA "< $ aA

We see that IeleaseXey is simply an Int2 I# you skip a #e- lines do-n, this -ill make sense, since -e%re using an IntMap to keep track o# the registered actions2 We also de#ine t-o type synonyms) Ie#Count and extXey2 extXey keeps track o# the most recently assigned !alue #or a key, and is incremented each time re!ister is called2 We%ll touch on Ie#Count later2 The IeleaseMap is three pieces o# in#ormation) the ne(t key and the re#erence count, and then the map o# all registered actions2 0otice that IeleaseMap takes a type parameter base, -hich states -hich monad release actions must li!e in2 Finally, a IesourceT is essentially a IeaderT that keeps a mutable re#erence to a IeleaseMap2 The re#erence type is determined by the base o# the monad in 3uestion, as is the cleanup monad2 This is -hy -e need to use associated types2 The ma9ority o# the rest o# the code in the Control/Monad/Trans/Iesource module is 9ust pro!iding instances #or the IesourceT type2

Other Typeclasses
There are three other typeclasses pro!ided by the module) Resource8nsa#eI/ ny monad -hich can li#t I, actions into it, but that this may be considered unsa#e2 The prime candidate here is 9T2 $are should be taken to only li#t actions -hich do not ac3uire scarce resources and -hich don%t I#ire the missiles2I In other -ords, all the normal -arnings o# unsa#eI,To9T apply2 ResourceThroFor actions that can thro- e(ceptions2 This automatically applies to all I,1based monads2 For 9T1based monads, you can use the supplied -xceptionT trans#ormer to pro!ide e(ception1thro-ing capabilities2 Some #unctions in conduit, #or e(ample, -ill re3uire this 4e2g2, te(t decoding62 ResourceI/ con!enience class tying together a bunch o# other classes, included the t-o mentioned abo!e2 This is purely #or con!enience7 you could achie!e the same e##ect -ithout this type class, you%d 9ust ha!e to do a lot more typing2

!orking
It -ould seem that #orking a thread -ould be inherently unsa#e -ith IesourceT, since the parent thread may call runIesourceT -hile the child thread is still accessing some o# the allocated resources2 This is indeed true, if you use the normal #orkI, #unction2 You can%t actually use the standard #orkI,, since it only operates in the I, monad, but you could use the #ork #unction #rom li#ted1base2 In #act, due to this issue, the regions package does not pro!ide a MonadBaseControl instance #or its trans#ormer 4-hich is !ery similar to IesourceT62 Ho-e!er, our goal in IesourceT is not to make it impossible #or programmers :HF

to mess up, only to make it easier to do the right thing2 There#ore, -e still pro!ide the instance, e!en though it could be abused2 In order to sol!e this, IesourceT includes re#erence counting2 When you #ork a ne- thread !ia resource?orkI,, the Ie#Count !alue o# the IeleaseMap is incremented2 "!ery time runIesourceT is called, the !alue is decremented2 /nly -hen the !alue hits H are all the release actions called2

Con enience /,ports

In addition to -hat%s been listed so #ar, there are a #e- e(tra #unctions e(ported 4mostly6 #or con!enience2
newIe#, writeIe#,

and readIe# -rap up the *asIe# !ersions o# the #unctions and allo- them to run in any IesourceT2 withI, is essentially a type1restricted !ersion o# with, but -orking around some o# the nastiness -ith types you -ould other-ise run into2 In general) you%ll -ant to use withI, -hen -riting I, code2 transIesourceT let%s you modi#y -hich monad your ResourceT is running in, assuming it keeps the same base2
transIesourceT == @Base $ p =< @$ a "< n "< IesourceT "< IesourceT transIesourceT # @IesourceT Base nA aA $ a n a $xA = IesourceT @2r "< # @$x rAA

Source
I think it%s simplest to understand sources by looking at the types)
data 9ourceIesult $ a = ,pen @9ource $ aA a | Closed data 9ource $ a = 9ource 3 sourcePull == IesourceT $ @9ourceIesult $ aA > sourceClose == IesourceT $ @A 7

source has 9ust t-o operations on it) you can pull data #rom it, and you can close it 4think o# closing a #ile handle62 When you pull, you either get some data and the a ne- 9ource 4the source is still open6, or nothing 4the source is closed62 5et%s look at some o# the simplest sources)
i$port Prelude hidin! @repeatA i$port 1ata/Conduit "" | e%er !i%e any data eo# == Monad $ =< 9ource $ a eo# = 9ource 3 sourcePull = return Closed > sourceClose = return @A 7

:H+

"" | )lways !i%e the sa$e %alue repeat == Monad $ =< a "< 9ource $ a repeat a = 9ource 3 sourcePull = return + ,pen @repeat aA a > sourceClose = return @A 7 $ain == I, @A $ain = return @A

These sources are !ery straight1#or-ard, since they al-ays return the same results2 dditionally, their close records don%t do anything2 You might think that this is a bug) shouldn%t a call to sourcePull return Closed a#ter it%s been closed= This isn%t re3uired, since one o# the rules o# sources is that they can ne!er be reused2 In other -ords)

I# a 9ource returns ,pen, it has pro!ided you -ith a ne- 9ource -hich you should use in place o# the original one2 I# it returns Closed, then you cannot per#orm any more operations on it2

Don%t -orry too much about the in!ariant2 In practice, you -ill almost ne!er call sourcePull or sourceClose yoursel#2 In #act, you hardly e!en -rite them yoursel# either 4that%s -hat source9tate and sourceI, are #or62 The point is that -e can make some assumptions -hen -e implement our sources2

State
There is something similar about the t-o sources mentioned abo!e) they ne!er change2 They always return the same !alue2 In other -ords, they ha!e no state2 For almost all serious sources, -e%ll need some kind o# state2 The state might actually be de#ined outside o# our program2 For e(ample, i# -e -rite a source that reads data #rom a *andle, -e don%t need to manually speci#y any state, since the *andle itsel# already has2 The -ay -e store state in a source is by updating the returned 9ource !alue in the ,pen constructor2 This is best seen -ith an e(ample2
i$port 1ata/Conduit i$port Control/Monad/Trans/Iesource "" | Pro%ide data #ro$ the list> one ele$ent at a ti$e/ source5ist == Iesource $ =< :a; "< 9ource $ a source5ist list = 9ource 3 sourcePull = case list o# :; "< return Closed "" no $ore data "" This is where we store our state= by return a new "" 9ource with the rest o# the list x=xs "< return + ,pen @source5ist xsA x > sourceClose = return @A 7 $ain == I, @A $ain = return @A

:HB

"ach time -e pull #rom the source, it checks the input list2 I# the list is empty, pulling returns Closed, -hich makes sense2 I# the list is not empty, pulling returns ,pen -ith both the ne(t !alue in the list, and a ne- 9ource !alue containing the rest o# the input list2

sourceState and sourceIO

In addition to being able to manually create 9ources, -e also ha!e a #e- con!enience #unctions that allo- us to create most sources in a more high1le!el #ashion2 source9tate let%s you -rite code similar to ho- you -ould use the 9tate monad2 You pro!ide an initial state, your pull #unction is pro!ided -ith the current state, and it returns a ne- state and a return !alue2 5et%s use this to reimplement source5ist2
i$port 1ata/Conduit i$port Control/Monad/Trans/Iesource "" | Pro%ide data #ro$ the list> one ele$ent at a ti$e/ source5ist == Iesource $ =< :a; "< 9ource $ a source5ist stateK = source9tate stateK pull where pull :; = return 9tateClosed pull @x=xsA = return + 9tate,pen xs x $ain == I, @A $ain = return @A

0otice the usage o# the 9tateClosed and 9tate,pen constructors2 These are !ery similar to Closed and ,pen, e(cept that instead o# speci#ying the ne(t 9ource to be used, you pro!ide the ne(t state 4here, the remainder o# the list62 The other common acti!ity is to per#orm some I</ allocation 4like opening a #ile6, registering some cleanup action 4closing that #ile6, and ha!ing a #unction #or pulling data #rom that resource2 conduit comes built1in -ith a source?ile #unction that gi!es a stream o# Byte9trin!s2 5et%s -rite a -ildly ine##icient alternati!e that returns a stream o# characters2
i$port i$port i$port i$port 1ata/Conduit Control/Monad/Trans/Iesource 9yste$/I, Control/Monad/I,/Class @li#tI,A

source?ile == IesourceI, $ =< ?ilePath "< 9ource $ Char source?ile #p = sourceI, @open?ile #p IeadModeA hClose @2h "< li#tI, + do eo# G" hIs-,? h i# eo# then return I,Closed else #$ap I,,pen + hGetChar hA $ain == I, @A $ain = return @A

5ike source9tate, it uses a !ariant on the ,pen and Closed constructors2 sourceI, does a number o# things #or us) :*H

It registers the cleanup #unction -ith the IesourceT trans#ormer, ensuring it gets called e!en in the presence o# e(ceptions2 It sets up the sourceClose record to release the resource immediately2 s soon as you return I,Closed, it -ill release the resource2

Sinks
sink consumes a stream o# data, and produces a result2 sink must al-ays produce a result, and must al-ays produce a single result2 This is encoded in the types themsel!es2 There is a Monad instance #or sink, making it simple to compose multiple sinks together into a larger sink2 You can also use the built1in sink #unctions to per#orm most o# your -ork2 5ike sources, you%ll rarely need to di!e into the inner -orkings2 5et%s start o## -ith an e(ample) getting lines #rom a stream o# Chars 4-e%ll assume 8ni( line endings #or simplicity62
i$port 1ata/Conduit i$port Euali#ied 1ata/Conduit/5ist as C5 "" Get a sin!le line #ro$ the strea$/ sink5ine == Iesource $ =< 9ink Char $ 9trin! sink5ine = sink9tate id "" initial state> nothin! at the be!innin! o# the line push close where "" ,n a new line> return the contents up until here push #ront P2nP = return + 9tate1one othin! + #ront :; "" Just another character> add it to the #ront and keep !oin! push #ront char = return + 9tateProcessin! + #ront / @char=A "" Got an -,? be#ore hittin! a newline> Sust !i%e what we ha%e so #ar close #ront = return + #ront :; "" Get all the lines #ro$ the strea$> until we hit a blank line or -,?/ sink5ines == Iesource $ =< 9ink Char $ :9trin!; sink5ines = do line G" sink5ine i# null line then return :; else do lines G" sink5ines return + line = lines content == 9trin! content = unlines : 8This is the #irst line/8 > 8*erePs the second/8 > 88 > 8)#ter the blank/8 ; $ain == I, @A $ain = do

:**

lines G" runIesourceT + C5/source5ist content ++ sink5ines $apMB put9tr5n lines

Running this sample produces the e(pected output)

This is the #irst line/ *erePs the second/ sink5ine demonstrates usage o# the sink9tate #unction, -hich is !ery similar to the source9tate #unction -e 9ust sa-2 It takes three arguments) an initial state, a push #unction

4takes the current state and ne(t input, and returns a ne- state and result6 and a close #unction 4takes the current state and returns an output62 s opposed to source9tate1 -hich doesn%t need a close #unction1 a sink is re3uired to al-ays return a result2 /ur push #unction has t-o clauses2 When it gets a ne-line character, it indicates that processing is complete !ia 9tate1one2 The othin! indicates that there is no le#to!er input 4-e%ll discuss that later62 It also gi!es an output o# all the characters it has recei!ed2 The second clause simply appends the ne- character to the e(isting state and indicates that -e are still -orking !ia 9tateProcessin!2 The close #unction returns all characters2
sink5ines sho-s ho- -e can use the monadic inter#ace to produce ne- sinks2 I# you replace sink5ine -ith !et5ine, this -ould look like standard code to pull lines #rom standard input2

This #amiliar inter#ace should make it easy to get up and running 3uickly2

Types
The types #or sinks are 9ust a bit more in!ol!ed than sources2 5et%s ha!e a look)
type 9inkPush input $ output = input "< IesourceT $ @9inkIesult input $ outputA type 9inkClose $ output = IesourceT $ output data 9inkIesult input $ output = Processin! @9inkPush input $ outputA @9inkClose $ outputA | 1one @Maybe inputA output data 9ink input $ output = 9ink o1ata output | 9ink1ata 3 sinkPush == 9inkPush input $ output > sinkClose == 9inkClose $ output 7 | 9ink5i#t @IesourceT $ @9ink input $ outputAA

Whene!er a sink is pushed to, it can either say it needs more data 4Processin!6 or say it%s all done2 When still processing, it must pro!ided updated push and close #unction7 -hen done, it returns any le#to!er inut and the output2 Fairly straight1#or-ard2 The #irst real IgotchaI is the three constructors #or 9ink2 Why do -e need 9ink o1ata) aren%t sinks all about consuming data= The ans-er is that -e need it to e##iciently implement our Monad instance2 When -e use return, -e%re gi!ing back a !alue that re3uires no data in

:*:

order to compute it2 We could model this -ith the 9ink1ata constructor, -ith something like)
$yIeturn a = 9ink1ata @2input "< return @1one @Just inputA aAA @return aA

But doing so -ould #orce reading in an e(tra bit o# input that -e don%t need right no-, and possibly -ill ne!er need2 4Ha!e a look again at the sink5ines e(ample26 So instead, -e ha!e an e(tra constructor to indicate that no input is re3uired2 5ike-ise, 9ink5i#t is pro!ided in order to implement an e##icient MonadTrans instance2

Sinks= no helpers
5et%s try to implement some sinks on the Ibare metalI, -ithout any helper #unctions2
i$port i$port i$port i$port 1ata/Conduit 9yste$/I, Control/Monad/Trans/Iesource Control/Monad/I,/Class @li#tI,A

"" Consu$e all input and discard it/ sink ull == Iesource $ =< 9ink a $ @A sink ull = 9ink1ata push close where push Bi!nored = return + Processin! push close close = return @A "" 5etPs strea$ characters to a #ile/ *ere we do need so$e kind o# "" initiali'ation/ Ce do this by initiali'in! in a push #unction> "" and then returnin! a di##erent push #unction #or subseEuent "" calls/ By usin! withI,> we know that the handle will be closed e%en "" i# therePs an exception/ sink?ile == IesourceI, $ =< ?ilePath "< 9ink Char $ @A sink?ile #p = 9ink1ata pushInit closeInit where pushInit char = do @releaseXey> handleA G" withI, @open?ile #p CriteModeA hClose push releaseXey handle char closeInit = do "" e%er opened a #ile> so nothin! to do here return @A push releaseXey handle char = do li#tI, + hPutChar handle char return + Processin! @push releaseXey handleA @close releaseXey handleA close releaseXey B = do "" Close the #ile handle as soon as possible/ return @A "" )nd wePll count how $any %alues were in the strea$/ count == Iesource $ =< 9ink a $ Int count = 9ink1ata @push KA @close KA where

:*>

push count Bi!nored = return + Processin! @push countPA @close countPA where countP = count U ( close count = return count $ain == I, @A $ain = return @A

0othing is particularly complicated to implement2 You should notice a common pattern here) declaring your push and close #unctions in a where clause, and then using them t-ice) once #or the initial 9ink1ata, and once #or the Processin! constructor2 This can become a bit tedious7 that%s -hy -e ha!e helper #unctions2

Sinks= 3ith helpers

5et%s re-rite sink?ile and count to take ad!antage o# the helper #unctions sinkI, and sink9tate, respecti!ely2
i$port 1ata/Conduit i$port 9yste$/I, i$port Control/Monad/I,/Class @li#tI,A "" Ce ne%er ha%e to touch the release key directly> sinkI, auto$atically "" releases our resource as soon as we return I,1one #ro$ our push #unction> "" or sinkClose is called/ sink?ile == IesourceI, $ =< ?ilePath "< 9ink Char $ @A sink?ile #p = sinkI, @open?ile #p CriteModeA hClose "" push= notice that we are !i%en the handle and the input @2handle char "< do li#tI, + hPutChar handle char return I,Processin!A "" close= wePre also !i%en the handle> but we donPt use it @2Bhandle "< return @AA "" )nd wePll count how $any %alues were in the strea$/ count == Iesource $ =< 9ink a $ Int count = sink9tate K "" The push #unction !ets both the current state and the next input/// @2state Bi!nored "< "" and it returns the new state return + 9tateProcessin! + state U (A "" The close #unction !ets the #inal state and returns the output/ @2state "< return stateA $ain == I, @A $ain = return @A

0othing dramatic, 9ust slightly shorter, less error1prone code2 8sing these t-o helper #unctions is highly recommended, as it ensures proper resource management and state updating2

:*@

List functions
s easy as it is to -rite your o-n sinks, you%ll likely -ant to take ad!antage o# the built1in sinks a!ailable in the Data2$onduit25ist module2 These pro!ide analogues to common list #unctions, like #olding2 4The module also has some Conduits, like map26 I# you%re looking #or some -ay to practice -ith conduits, reimplementing the #unctions in the 5ist module1 both -ith and -ithout the helper #unctions1 -ould be a good start2 5et%s look at some simple things -e can make out o# the built1in sinks2
i$port 1ata/Conduit i$port Euali#ied 1ata/Conduit/5ist as C5 i$port Control/Monad/I,/Class @li#tI,A "" ) su$ #unction/ su$P == Iesource $ =< 9ink Int $ Int su$P = C5/#old @UA K "" Print e%ery input %alue to standard output/ printer == @9how a> IesourceI, $A =< 9ink a $ @A printer = C5/$apMB @li#tI, / printA "" 9u$ up all the %alues in a strea$ a#ter the #irst #i%e/ su$9kip?i%e == Iesource $ =< 9ink Int $ Int su$9kip?i%e = do C5/drop M C5/#old @UA K "" Print each input nu$ber and su$ the total print9u$ == IesourceI, $ =< 9ink Int $ Int print9u$ = do total G" C5/#oldM !o K li#tI, + put9tr5n + 89u$= 8 UU show total return total where !o accu$ int = do li#tI, + put9tr5n + 8 ew input= 8 UU show int return + accu$ U int $ain == I, @A $ain = return @A

Connecting
t the end o# the day, -e%re actually going to -ant to use our sinks2 While -e could manually call sinkPush and sinkClose, it%s tedious2 For e(ample)
$ain == I, @A $ain = runIesourceT + do res G" case print9u$ o#

:*A

9ink1ata push close "< loop :(//(K; push close 9ink o1ata res "< return res li#tI, + put9tr5n + 8Got a result= 8 UU show res where loop :; Bpush close = close loop @x=xsA push close = do $res G" push x case $res o# 1one Ble#to%er res "< return res Processin! pushP closeP "< loop xs pushP closeP

Instead, the recommended approach is to connect your sink to a source2 0ot only is this simpler, it%s less error prone, and means you ha!e a lot o# #le(ibility in -here your data is coming #rom2 To re-rite the e(ample abo!e)
$ain == I, @A $ain = runIesourceT + do res G" C5/source5ist :(//(K; ++ print9u$ li#tI, + put9tr5n + 8Got a result= 8 UU show res

$onnecting takes care o# testing #or the sink constructor 49ink1ata !ersus 9ink o1ata !ersus 9ink5i#t6, pulling #rom the source, and pushing to<closing the sink2 Ho-e!er, there is one thing I -anted to point out #rom the long1-inded e(ample2 /n the second to last line, -e ignore the le#to!er !alue o# 1one2 This brings up the issue o# data loss2 This is an important topic that has had a lot o# thought put into it2 8n#ortunately, -e can%t #ully co!er it yet, as -e ha!en%t discussed the main culprit in the drama) Conduits 4the type, not the package62 But as a 3uick note here, the le#to!er !alue #rom the 1one constructor is not al-ays ignored2 The Monad instance, #or e(ample, uses it to pass data #rom one sink to the ne(t in a binding2 nd in #act, the real connect operator doesn't al-ays thro- a-ay the le#to!ers2 When -e co!er resumable sources later, -e%ll see that the le#to!er !alue is put back on the bu##er to allo- later sinks reusing an e(isting source to pull the !alue2

Conduit
This section co!ers the #inal ma9or datatype in our package, conduits2 While sources produce a stream o# data and sinks consume a stream, conduits trans#orm a stream2

Types
s -e did pre!iously, let%s start o## by looking at the types in!ol!ed2
data ConduitIesult input $ output = Producin! @Conduit input $ outputA :output; | ?inished @Maybe inputA :output; data Conduit input $ output = Conduit 3 conduitPush == input "< IesourceT $ @ConduitIesult input $ outputA > conduitClose == IesourceT $ :output; 7

:*D

This should look !ery similar to -hat -e%!e seen -ith sinks2 conduit can be pushed to, in -hich case it returns a result2 result either indicates that it is still producing data, or that it is #inished2 When a conduit is closed, it returns some more output2 But let%s e(amine the idiosyncracies a bit2 5ike sinks, -e can only push one piece o# input at a time, and le#to!er data may be H or * pieces2 Ho-e!er, there are a #e- changes)

When producing 4the e3ui!alent o# processing #or a sink6, -e can return output2 This is because a conduit -ill product a ne- stream o# output instead o# producing a single output !alue at the end o# processing2 sink al-ays returns a single output !alue, -hile a conduit returns H or more outputs 4a list62 To understand -hy, consider conduits such as concatMap 4produces multiple outputs #or one input6 and #ilter 4returns H or * output #or each input62 We ha!e no special constructor like 9ink o1ata2 That%s because -e pro!ide no Monad instance #or conduits2 We%ll see later ho- you can still use a #amiliar &onadic approach to creating conduits2

/!erall conduits should seem !ery similar to -hat -e%!e co!ered so #ar2

Simple conduits
We%ll start o## by de#ining some simple conduits that don%t ha!e any state2
i$port Prelude hidin! @$ap> concatMapA i$port 1ata/Conduit "" ) si$ple conduit that Sust passes on the data as"is/ passThrou!h == Monad $ =< Conduit input $ input passThrou!h = Conduit 3 conduitPush = 2input "< return + Producin! passThrou!h :input; > conduitClose = return :; 7 "" $ap %alues in a strea$ $ap == Monad $ =< @input "< outputA "< Conduit input $ output $ap # = Conduit 3 conduitPush = 2input "< return + Producin! @$ap #A :# input; > conduitClose = return :; 7 "" $ap and concatenate concatMap == Monad $ =< @input "< :output;A "< Conduit input $ output concatMap # = Conduit 3 conduitPush = 2input "< return + Producin! @concatMap #A + # input > conduitClose = return :; 7 $ain == I, @A $ain = return @A

:*F

Stateful conduits
/# course, not all conduits can be declared -ithout state2 Doing so on the bare metal is not too di##icult2
i$port i$port i$port i$port Prelude hidin! @re%erseA Euali#ied 1ata/5ist 1ata/Conduit Control/Monad/Trans/Iesource

"" Ie%erse the ele$ents in the strea$/ ote that this has the sa$e downside as "" the standard re%erse #unction= you ha%e to read the entire strea$ into "" $e$ory be#ore producin! any output/ re%erse == Iesource $ =< Conduit input $ input re%erse = $kConduit :; where $kConduit state = Conduit @push stateA @close stateA push state input = return + Producin! @$kConduit + input = stateA :; close state = return state "" 9a$e thin! with sort= it will pull e%erythin! into $e$ory sort == @,rd input> Iesource $A =< Conduit input $ input sort = $kConduit :; where $kConduit state = Conduit @push stateA @close stateA push state input = return + Producin! @$kConduit + input = stateA :; close state = return + 1ata/5ist/sort state $ain == I, @A $ain = return @A

But -e can do better2 .ust like source9tate and sink9tate, -e ha!e conduit9tate to simpli#y things2
i$port Prelude hidin! @re%erseA i$port Euali#ied 1ata/5ist i$port 1ata/Conduit "" Ie%erse the ele$ents in the strea$/ ote that this has the sa$e downside as "" the standard re%erse #unction= you ha%e to read the entire strea$ into "" $e$ory be#ore producin! any output/ re%erse == Iesource $ =< Conduit input $ input re%erse = conduit9tate :; push close where push state input = return + 9tateProducin! @input = stateA :; close state = return state "" 9a$e thin! with sort= it will pull e%erythin! into $e$ory sort == @,rd input> Iesource $A =< Conduit input $ input sort = conduit9tate :; push close where push state input = return + 9tateProducin! @input = stateA :; close state = return + 1ata/5ist/sort state

:*+

$ain == I, @A $ain = return @A

'sing conduits
The -ay Conduits interact -ith the rest o# the package is !ia #using2 conduit can be #used into a source, producing a ne- source, #used into a sink to produce a ne- sink, or #used -ith another conduit to produce a ne- conduit2 It%s best to 9ust look at the #usion operators2
"" 5e#t #usion= source U conduit = source @+=A == @Iesource $> Is9ource srcA =< src $ a "< Conduit a $ b "< 9ource $ b "" Ii!ht #usion= conduit U sink = sink @=+A == Iesource $ =< Conduit a $ b "< 9ink b $ c "< 9ink a $ c "" Middle #usion= conduit U conduit = conduit @=+=A == Iesource $ =< Conduit a $ b "< Conduit b $ c "< Conduit a $ c

8sing these operators is straight#or-ard2

useConduits = do runIesourceT + C5/source5ist :(//(K; += re%erse += C5/$ap show ++ C5/consu$e "" eEui%alent to runIesourceT + C5/source5ist :(//(K; ++ re%erse =+ C5/$ap show =+ C5/consu$e "" and eEui%alent to runIesourceT + C5/source5ist :(//(K; ++ @re%erse =+= C5/$ap showA =+ C5/consu$e

There is in #act one last -ay o# e(pressing the same idea2 I%ll lea!e it as an e(ercise to the reader to disco!er it2 It may seem like all these di##erent approaches are redundant2 While occassionally you can in #act choose -hiche!er approach you #eel like using, in many cases you -ill need a speci#ic approach2 For e(ample)

I# you ha!e a stream o# numbers, and you -ant to apply a conduit 4e2g2, $ap show6 to only some o# the stream that -ill be passed to a speci#ic sink, you%ll -ant to use the right #usion operator2 I# you%re reading a #ile, and -ant to parse the entire #ile as te(tual data, you%ll -ant to use le#t #usion to con!ert the entire stream2

:*B

I# you -ant to create reusable conduits that combine together indi!idual, smaller conduits, you%ll use middle #usion2

(ata loss
5et%s #orget about conduits #or a moment2 Instead, suppose -e -ant to -rite a program1 using plain old lists1 that -ill take a list o# numbers, apply some kind o# trans#ormation to them, take the #irst #i!e trans#ormed !alues and do something -ith them, and then do something else -ith the remaining non&transformed !alues2 For e(ample, -e -ant something like)
$ain = do let list = :(//(K; trans#or$ed = $ap show list @be!in> endA = split)t M trans#or$ed untrans#or$ed = $ap read end $apMB put9tr5n be!in print + su$ untrans#or$ed

But clearly this isn%t a good general solution, since -e don%t -ant to ha!e to trans#orm and then untrans#orm e!ery element in the list2 For one thing, -e may not al-ays ha!e an in!erse #unction2 nother issue is e##iciency2 In this case, -e can -rite something more e##icient)
$ain = do let list = :(//(K; @be!in> endA = split)t M list trans#or$ed = $ap show be!in $apMB put9tr5n trans#or$ed print + su$ end

0ote the change) -e per#orm our split be#ore trans#orming any elements2 This -orks because, -ith $ap, -e ha!e a *1to1* correspondence bet-een the input and output elements2 So splitting at A be#ore or a#ter mapping show is the same thing2 But -hat happens i# -e replace $ap show -ith something more de!ious2
de%iousTrans#or$ = concatMap !o where !o ( = :show (; !o N = :show N> 8two8; !o & = replicate M 8three8 !o x = :show x;

We no longer ha!e the *1to1* correspondence2 s a result, -e can%t use the second method2 But it%s e!en -orse) -e can%t use the #irst method either, since there%s no in!erse o# our de%iousTrans#or$2 There%s only one solution to the problem that I%m a-are o#) trans#orm elements one at a time2 The #inal program looks like this)
de%iousTrans#or$ de%iousTrans#or$ de%iousTrans#or$ de%iousTrans#or$ ( N & x = = = = :show (; :show N> 8two8; replicate M 8three8 :show x;

::H

trans#or$M == :Int; "< @:9trin!;> :Int;A trans#or$M list = !o :; list where !o output @x=xsA | new5en <= M = @take M outputP> xsA | otherwise = !o outputP xs where outputP = output UU de%iousTrans#or$ x new5en = len!th outputP "" 1e!enerate case= not enou!h input to $ake M outputs !o output :; = @output> :;A $ain = do let list = :(//(K; @be!in> endA = trans#or$M list $apMB put9tr5n be!in print + su$ end

The #inal output o# this program is

( N two three three L\

What%s important to note is that the number > is con!erted into #i!e copies o# the -ord IthreeI, yet only t-o o# them sho- up in the output2 The rest are discarded in the take M call2 This -hole e(ercise is 9ust to demonstrate the issue o# data loss in conduits2 By #orcing conduits to accept only one input at a time, -e a!oid the issue o# trans#orming too many elements at once2 That doesn%t mean -e don%t lose any data) i# a conduit produces too much output #or the recei!ing sink to handle, some o# it may be lost2 To put all this another -ay) conduits a!oid chunking to get a-ay #rom data loss2 This is not an issue uni3ue to conduits2 I# you look in the implementation o# concatMapM #or enumerator, you%ll see that it #orces elements to be handled one at a time2 In conduits, -e opted to #orce the issue at the type le!el2

Se.uencedSink
Suppose -e -ant to be able to combine up e(isting conduits and sinks to produce a ne-, more po-er#ul conduit2 For e(ample, -e -ant to -rite a conduit that takes a stream o# numbers and sums up e!ery #i!e2 In other -ords, #or the input :(//MK;, it should result in the se3uence :(M>LK>VM>\K>((M>(LK>(VM>(\K>N(M>NLK;2 We can de#initely do this -ith the lo-1le!el conduit inter#ace2
su$MIaw == Iesource $ =< Conduit Int $ Int su$MIaw = conduit9tate @K> KA push close where

::*

push @total> countA input | newCount == M = return + 9tateProducin! @K> KA :newTotal; | otherwise = return + 9tateProducin! @newTotal> newCountA :; where newTotal = total U input newCount = count U ( close @total> countA | count == K = return :; | otherwise = return :total; $ain == I, @A $ain = return @A

But this is #rustrating, since -e already ha!e all the tools -e need to do this at a high le!el? There%s the #old sink #or adding up the numbers, and the isolate conduit -hich -ill only allo- up to a certain number o# elements to be passed to a sink2 $an%t -e combine these someho-= The ans-er is a 9eEuenced9ink2 The idea is to create a normal 9ink, e(cept it returns a special output called a 9eEuenced9inkIesponse2 This !alue can emit ne- output, stop processing data, or trans#er control to a ne- conduit2 4See the Haddocks #or more in#ormation26 Then -e can turn this into a Conduit using the seEuence9ink #unction2 This #unction also takes some state !alue that gets passed through to the sink2 So -e can re-rite su$MIaw in a much more high1le!el manner2
su$M == Iesource $ =< Conduit Int $ Int su$M = seEuence9ink @A + 2@A "< do next9u$ G" C5/isolate M =+ C5/#old @UA K return + -$it @A :next9u$; $ain = return @A

ll o# the @A in there are simply the unused state !ariable being passed around, they can be ignored2 /ther-ise, -e%re doing e(actly -hat -e -ant2 We #use isolate to #old to get the sum o# the ne(t #i!e elements #rom the stream2 We then emit that !alue, and start all o!er again2 5et%s say -e -ant to modi#y this slightly2 We -ant to get the #irst + sums, and then pass through the remaining !alues, multiplied by :2 We can keep track o# ho- many !alues -e%!e returned in our state, and then use the 9tartConduit constructor to pass control to the multiply1by1: conduit ne(t2
su$MPass == Iesource $ =< Conduit Int $ Int su$MPass = seEuence9ink K + 2count "< do i# count == D then return + 9tartConduit + C5/$ap @W NA else do next9u$ G" C5/isolate M =+ C5/#old @UA K return + -$it @count U (A :next9u$; $ain = return @A

These are ob!iously !ery contri!ed e(amples, but I hope it makes clear the po-er and simplicity a!ailable #rom this approach2

:::

#uffering
Bu##ering is one o# the uni3ue #eatures o# conduits2 With bu##ering, conduits no longer need to control the #lo- o# your application2 In some cases, this can lead to simpler code2

In ersion of Control
Bu##ering -as actually one o# the main moti!ations in the creation o# the conduit package2 To see its importance, -e need to consider the approach -e%!e seen so #ar, -hich -e%ll call in!ersion o# control, or Io$2 In!ersion o# control can mean di##erent things in di##erent circles2 I# you ob9ect to its usage here, go ahead replace it -ith some other phrase like I-arm, #u''y thing2I I -on%t be o##ended2 Suppose you -ant to count ho- many ne-line characters there are in a #ile2 In the standard imperati!e approach, you -ould do someting like) *2 :2 >2 @2 A2 /pen the #ile Pull some data into a bu##er 5oop o!er the !alues in the bu##er, incrementing a counter on each ne-line character Return to : $lose the #ile

0otice that your code is e(plicitly calling out to other code and that code is returning control back to your code2 You ha!e retained #ull control o# the #lo- o# e(ecution o# your program2 The conduit approach -e%!e seen so #ar does not -ork this -ay2 Instead, you -ould) *2 Write a sink that counts ne-lines and adds the result to an accumulator2 :2 $onnect the sink to a source There%s no doubt in my mind that this is an easier approach2 You don%t ha!e to -orry about opening and closing #iles or pulling data #rom the #ile2 Instead, the data you need to process is simply presented to you2 This is the ad!antage o# Io$) you can #ocus on speci#ically your piece o# the code2 We use this Io$ approach all o!er Haskell) #or e(ample, instead o# readMVar and putMVar, you can use withMVar2 Don%t bother -ith open?ile and close?ile, 9ust use with?ile and pass in a #unction that uses the *andle2 "!en $ has a !ersion o# this) -hy $alloc and #ree -hen you could 9ust alloca= ctually, that last one is a huge red herring2 /# course you can%t 9ust use alloca #or e!erything2 alloca only allocates memory locally on the stack, not dynamically on the heap2 There%s no -ay to return your allocated memory outside the current #unction2 But actually, the same restriction applies to the -hole #amily o# with #unctions) you can ne!er return an allocated resource outside o# the IblockI2 8sually this -orks out 9ust #ine, but -e need to recogni'e that this is a change in ho- -e structure our programs2 /#ten times, ::>

-ith simple e(amples, this is a minor change2 Ho-e!er, in larger settings this can become !ery di##icult to manage, bordering on impossible at times2

A 3e- ser er
5et%s say -e%re going to -rite a -eb ser!er2 We%re going to use the #ollo-ing lo-1le!el operations)
data 9ocket rec% == 9ocket "< Int "< I, Byte9trin! "" returns e$pty when the socket is closed send)ll == 9ocket "< Byte9trin! "< I, @A

We%re up to the part -here -e need to implement the #unction handleConn that handles an indi!idual connection2 It -ill look something like this)
data IeEuest "" reEuest headers> *TTP %ersion> etc data Iesponse "" status code> response headers> resposne body type )pplication = IeEuest "< I, Iesponse handleConn == )pplication "< 9ocket "< I, @A

What does our handleConn need to do= In broad strokes) *2 :2 >2 @2 A2 Parse the re3uest line Parse the re3uest headers $onstruct the IeEuest !alue Pass IeEuest to the )pplication and get back a Iesponse Send the Iesponse o!er the 9ocket

We start o## by -riting steps * and : manually, -ithout using conduits2 We%ll do this !ery simply and 9ust assume three space1separated strings2 We end up -ith something that looks like)
data IeEuest5ine = IeEuest5ine Byte9trin! Byte9trin! Byte9trin! parseIeEuest5ine == 9ocket "< I, IeEuest5ine parseIeEuest5ine socket = do bs G" rec% socket LK\V let @$ethod=path=%ersion=i!noredA = 9D/words bs return + IeEuest5ine $ethod path %ersion

There are t-o issues here) it doesn%t handle the case -here there are less than three -ords in the chunk o# data, and it thro-s a-ay any e(tra data2 We can de#initely sol!e both o# these issues manually, but it%s !ery tedious2 It%s much easier to implement this in terms o# conduits2
i$port i$port i$port i$port i$port 1ata/Byte9trin! @Byte9trin!A Euali#ied 1ata/Byte9trin! as 9 1ata/Conduit Euali#ied 1ata/Conduit/Binary as CB Euali#ied 1ata/Conduit/5ist as C5

data IeEuest5ine = IeEuest5ine Byte9trin! Byte9trin! Byte9trin!

::@

parseIeEuest5ine == 9ink Byte9trin! I, IeEuest5ine parseIeEuest5ine = do let space = to-nu$ + #ro$-nu$ P P let !etCord = do CB/dropChile @== spaceA bss G" CB/takeChile @.= spaceA =+ C5/consu$e return + 9/concat bss $ethod G" !etCord path G" !etCord %ersion G" !etCord return + IeEuest5ine $ethod path %ersion $ain = return @A

This means that our code -ill automatically be supplied -ith more data as it comes in, and any e(tra data -ill automatically be bu##ered in the 9ource, ready #or the ne(t time it%s used2 0o- -e can easily structure our program together, demonstrating the po-er o# the conduits approach)
i$port i$port i$port i$port i$port data type data data type 1ata/Byte9trin! @Byte9trin!A 1ata/Conduit 1ata/Conduit/ etwork @source9ocketA Control/Monad/I,/Class @li#tI,A etwork/9ocket @9ocketA

IeEuest5ine = IeEuest5ine Byte9trin! Byte9trin! Byte9trin! *eaders = :@Byte9trin!> Byte9trin!A; IeEuest = IeEuest IeEuest5ine *eaders Iesponse = Iesponse )pplication = IeEuest "< I, Iesponse

parseIeEuest*eaders == 9ink Byte9trin! I, *eaders parseIeEuest*eaders = unde#ined parseIeEuest5ine == 9ink Byte9trin! I, IeEuest5ine parseIeEuest5ine = unde#ined sendIesponse == 9ocket "< Iesponse "< I, @A sendIesponse = unde#ined handleConn == )pplication "< 9ocket "< I, @A handleConn app socket = do reE G" runIesourceT + source9ocket socket ++ do reEuest5ine G" parseIeEuest5ine headers G" parseIeEuest*eaders return + IeEuest reEuest5ine headers res G" li#tI, + app reE li#tI, + sendIesponse socket res $ain = return @A

$hither the re.uest -ody2

This is all great, until -e reali'e we can't read the request body2 The )pplication is simply gi!en the IeEuest, and li!es in the I, monad2 It has no access -hatsoe!er to the incoming stream o# data2

::A

There%s an easy #i( #or this actually) ha!e the )pplication li!e in the 9ink monad2 This is the !ery approach -e took -ith enumerator1based W I H2@2 Ho-e!er, there are t-o problems)

People #ind it con#using2 What people e pect is that the IeEuest !alue -ould ha!e a reEuestBody !alue o# type 9ource2 This makes certain kinds o# usage incredibly di##icult2 For e(ample, trying to -rite an HTTP pro(y combining W I and http"enu$erator pro!ed to be almost impossible2

This is the do-nside o# in!ersion o# control2 /ur code -ants to be in control2 It -ants to be gi!en something to pull #rom, something to push to, and run -ith it2 We need some solution to the problem2 I# you think that the situation I described -ith the pro(y isn%t so bad, it%s because I%!e gone easy on the details2 We also need to take into account streaming the response body, and the streaming needs to happen on both the client and ser!er side2 The simplest solution -ould be to 9ust create a ne- 9ource and pass that to the )pplication2 8n#ortunately, this -ill cause problems -ith our bu##ering2 You see, -hen -e connect our source to the parseIeEuest5ine and parseIeEuest*eaders sinks, it made a call to rec%2 I# the data it recei!ed -as not enough to co!er all o# the headers, it -ould issue another call2 When it had enough data, it -ould stop2 Ho-e!er, odds are that it didn%t stop e actly at the end o# the headers2 It likely consumed a bit o# the re3uest body as -ell2 I# -e 9ust create a ne- source and pass that to the re3uest, it -ill be missing the beginning o# the re3uest body2 We need some -ay to pass that bu##ered data along2

#ufferedSource
nd so -e #inally get to introduce the last data type in conduits) Bu##ered9ource2 This is an abstract data type, but all it really does is keep a mutable re#erence to a bu##er and an underlying 9ource2 In order to create one o# these, you use the bu##er9ource #unction2
bu##er9ource ==Iesource $ =< 9ource $ a "< IesourceT $ @Bu##ered9ource $ aA

This one little change is -hat allo-s us to easily sol!e our -eb ser!er dilemna2 Instead o# connecting a 9ource to our parsing 9inks, -e use a Bu##ered9ource2 t the end o# each connection, any le#to!er data is put back on the bu##er2 For our -eb ser!er case, -e can nocreate a Bu##ered9ource, use that to read the re3uest line and headers, and then pass that same Bu##ered9ource to the application #or reading the re3uest body2

Typeclass
We -ant to be able to connect a bu##ered source to a sink, 9ust like -e -ould a regular source2 We -ould also like to be able to #use it to a conduit2 In order to make this con!enient, conduit has a typeclass, Is9ource2 There are instances pro!ided #or both 9ource and Bu##ered9ource2 Both the connect 4++6 and le#t1#use 4+=6 operators use this typeclass2

::D

There%s one IgotchaI in the Bu##ered9ource instance o# this typeclass, so let%s e(plain it2 Suppose -e -ant to -rite a #ile copy #unction, -ithout any bu##ering2 This is a #airly standard usage o# conduits)
source?ile input ++ sink?ile output

When this line is run, both the input and output #iles are opened, the data is copied, and then both #iles are closed2 5et%s change this e(ample slightly to use bu##ering)
bsrc G" bu##er9ource + source?ile input bsrc ++ isolate MK =+ sink?ile output( bsrc ++ sink?ile outputN

When is the input #ile opened and closed= The opening occurs on the #irst line, -hen bu##ering the source2 nd i# -e #ollo- the normal rules #rom sources, the #ile should be closed a#ter the second line2 Ho-e!er, i# -e did that, -e couldn%t reuse bsrc #or line >? So instead, ++ does not close the #ile2 s a result, you can pass a bu##ered source to as many actions as you -ant, -ithout concerns that the #ile handle has been closed out #rom under you2 I# you remember #rom earlier, the in!ariant o# a source is that it cannot be pulled #rom a#ter it returns a Closed response2 In order to allo- you to -ork more easily -ith a Bu##ered9ource, this in!ariant is rela(ed2 It is the responsibility o# the Bu##er9ource implementation to ensure that a#ter the underlying 9ource is closed, it is ne!er used again2 This presents one ca!eat) -hen you%re #inished -ith a bu##ered source, you should manually call bsourceClose on it2 Ho-e!er, as usual, this is merely an optimi'ation, as the source -ill automatically be closed -hen runIesourceT is called2

%ecapping the 3e- ser er

So -hat e(actly does our -eb ser!er look like no-=
i$port i$port i$port i$port i$port data type data data type 1ata/Byte9trin! @Byte9trin!A 1ata/Conduit 1ata/Conduit/ etwork @source9ocketA Control/Monad/I,/Class @li#tI,A etwork/9ocket @9ocketA

IeEuest5ine = IeEuest5ine Byte9trin! Byte9trin! Byte9trin! *eaders = :@Byte9trin!> Byte9trin!A; IeEuest = IeEuest IeEuest5ine *eaders @Bu##ered9ource I, Byte9trin!A Iesponse = Iesponse )pplication = IeEuest "< IesourceT I, Iesponse

parseIeEuest*eaders == 9ink Byte9trin! I, *eaders parseIeEuest*eaders = unde#ined parseIeEuest5ine == 9ink Byte9trin! I, IeEuest5ine parseIeEuest5ine = unde#ined sendIesponse == 9ocket "< Iesponse "< I, @A

::F

sendIesponse = unde#ined handleConn == )pplication "< 9ocket "< I, @A handleConn app socket = runIesourceT + do bsrc G" bu##er9ource + source9ocket socket reEuest5ine G" bsrc ++ parseIeEuest5ine headers G" bsrc ++ parseIeEuest*eaders let reE = IeEuest reEuest5ine headers bsrc res G" app reE li#tI, + sendIesponse socket res $ain = return @A

We%!e made a #e- minor changes2 Firstly, the )pplication no- li!es in the IesourceT I, monad2 This isn%t strictly necessary, but it%s !ery con!enient) the application can no- register cleanup actions that -ill only take place a#ter the response has been #ully sent to the client2 But the ma9or changes are in the handleConn #unction2 We no- start o## by bu##ering our source2 This bu##ered source is then used t-ice in our #unction, and then passed o## to the application2

$e- Application Interface

It is a problem almost e!ery language used #or -eb de!elopment has dealt -ith) the lo- le!el inter#ace bet-een the -eb ser!er and the application2 The earliest e(ample o# a solution is the !enerable and battle1-orn $;I 4$;I6, pro!iding a language1agnostic inter#ace using only standard input, standard output and en!ironment !ariables2 Back -hen Perl -as becoming the de #acto -eb programming language, a ma9or shortcoming o# $;I became apparent) the process needed to be started ane- #or each re3uest2 When dealing -ith an interpretted language and application re3uiring database connection, this o!erhead became unbearable2 Fast$;I 4and later S$;I6 arose as a successor to $;I, but it seems that much o# the programming -orld -ent in a di##erent direction2 "ach language began creating its o-n standard #or inter#acing -ith ser!ers2 modWperl2 modWpython2 modWphp2 modWruby2 Within the same language, multiple inter#aces arose2 In some cases, -e e!en had inter#aces on top o# inter#aces2 nd all o# this led to much duplicated e##ort) a Python application designed to -ork -ith Fast$;I -ouldn%t -ork -ith modWpython7 modWpython only e(ists #or certain -ebser!ers7 and these programming language speci#ic -eb ser!er e(tensions need to be -ritten #or each programming language2 Haskell has its o-n history2 We originally had the cgi package, -hich pro!ided a monadic inter#ace2 The #astcgi package then pro!ided the same inter#ace2 &ean-hile, it seemed that the ma9ority o# Haskell -eb de!elopment #ocused on the standalone ser!er2 The problem is that each ser!er comes -ith its o-n inter#ace, meaning that you need to target a speci#ic backend2 This means that it is impossible to share common #eatures, like ;`IP encoding, de!elopment ser!ers, and testing #rame-orks2 W I attempts to sol!e this, by pro!iding a generic and e##icient inter#ace bet-een -eb ser!ers and applications2 ny handler supporting the inter#ace can ser!e any W I application, -hile any application using the inter#ace can run on any handler2

::+

t the time o# -riting, there are !arious backends, including Warp, Fast$;I, and de!elopment ser!er2 There are e!en more esoteric backends like -ai1handler1-ebkit #or creating desktop apps2 -ai1e(tra pro!ides many common middle-are components like ;`IP, .S/01P and !irtual hosting2 -ai1test makes it easy to -rite unit tests, and -ai1handler1de!el lets you de!elop your applications -ithout -orrying about stopping to compile2 Yesod targets W I, and Happstack is in the process o# con!erting o!er as -ell2 It%s also used by some applications that skip the #rame-ork entirely, including the ne- Hoogle2 Yesod pro!ides an alternate approach #or a de!el ser!er, kno-n as yesod de!el2 The di##erence #rom -ai1handler1de!el is that yesod de!el actually compiles your code each time, respecting all settings in your cabal #ile2 This is the recommended aproach #or general Yesod de!elopment2

The Interface
The inter#ace itsel# is !ery straight1#or-ard) an application takes a re3uest and returns a response2 response is an HTTP status, a list o# headers and a response body2 re3uest contains !arious in#ormation) the re3uested path, 3uery string, re3uest body, HTTP !ersion, and so on2

%esponse #ody
Haskell has a datatype kno-n as a la'y bytestring2 By utili'ing la'iness, you can create large !alues -ithout e(hausting memory2 8sing la'y I</, you can do such tricks as ha!ing a !alue -hich represents the entire contents o# a #ile, yet only occupies a small memory #ootprint2 In theory, a la'y bytestring is the only representation necessary #or a response body2 In practice, -hile la'y byte strings are -onder#ul #or generating IpureI !alues, the la'y I</ necessary to read a #ile introduces some non1determinism into our programs2 When ser!ing thousands o# small #iles a second, the limiting #actor is not memory, but #ile handles2 8sing la'y I</, #ile handles may not be #reed immediately, leading to resource e(haustion2 To deal -ith this, W I uses conduits2 Eersions o# W I be#ore *2H used enumerators in place o# conduits2 While both conduits and enumerators sol!e the same basic problem, e(perience sho-ed that enumerators -ere too constricting in their in!ersion o# control approach, making it di##icult to structure more complicated systems like a streaming pro(y ser!er2 $onduits -ere designed -ith the e(press purpose o# making a better W I2 The data type rele!ant to us no- is a source2 source produces a stream o# data, producing a single chunk at a time2 In the case o# W I, the re3uest body -ould be a source passed to the application, and the response body -ould be a source returned #rom the application2 There are t-o #urther optimi'ations) many systems pro!ide a send#ile system call, -hich sends a #ile directly to a socket, bypassing a lot o# the memory copying inherent in more general I</ system calls2 dditionally, there is a datatype in Haskell called Builder -hich allo-s e##icient copying o# bytes into bu##ers2

::B

The W I response body there#ore has three constructors) one #or pure builders 4IesponseBuilder6, one #or a source o# builders 4Iesponse9ource6 and one #or #iles 4Iesponse?ile62

%e.uest #ody
In order to a!oid the need to load the entire re3uest body into memory, -e use sources here as -ell2 Since the purpose o# these !alues are #or reading 4not -riting6, -e use Byte9trin!s in place o# Builders2 There is a record inside IeEuest called reEuestBody, -ith type Bu##ered9ource I, Byte9trin!2 We can use all o# the standard conduit #unctions to interact -ith this source2 The re3uest body could in theory contain any type o# data, but the most common are 8R5 encoded and multipart #orm data2 The -ai1e(tra package contains built1in support #or parsing these in a memory1e##icient manner2

Hello $orld
To demonstrate the simplicity o# W I, let%s look at a hello -orld e(ample2 In this e(ample, -e%re going to use the /!erloadedStrings language e(tension to a!oid e(plicitly packing string !alues into bytestrings2
3"4 5) G6)G- ,%erloaded9trin!s 4"7 i$port etwork/Cai i$port etwork/*TTP/Types @statusNKKA i$port etwork/Cai/*andler/Carp @runA application B = return + response5B9 statusNKK :@8Content"Type8> 8text.plain8A; 8*ello Corld8 $ain = run &KKK application

5ines : through @ per#orm our imports2 Warp is pro!ided by the -arp package, and is the premiere W I backend2 W I is also built on top o# the http1types package, -hich pro!ides a number o# datatypes and con!enience !alues, including statusNKK2 First -e de#ine our application2 Since -e don%t care about the speci#ic re3uest parameters, -e ignore the argument to the #unction2 For any re3uest, -e are returning a response -ith status code :HH 4I/PI6, and te(t<plain content type and a body containing the -ords IHello WorldI2 Pretty straight1#or-ard2

Middle3are
In addition to allo-ing our applications to run on multiple backends -ithout code changes, the W I allo-s us another bene#its) middle-are2 &iddle-are is essentially an application trans#ormer, taking one application and returning another one2

:>H

&iddle-are components can be used to pro!ide lots o# ser!ices) cleaning up 8R5s, authentication, caching, .S/01P re3uests2 But perhaps the most use#ul and most intuiti!e middle-are is g'ip compression2 The middle-are -orks !ery simply) it parses the re3uest headers to determine i# a client supports compression, and i# so compresses the response body and adds the appropriate response header2 The great thing about middle-ares is that they are unobtrusi!e2 5et%s see ho- -e -ould apply the g'ip middle-are to our hello -orld application2
3"4 5) G6)G- ,%erloaded9trin!s 4"7 i$port etwork/Cai i$port etwork/Cai/*andler/Carp @runA i$port etwork/Cai/Middleware/G'ip @!'ip> de#A i$port etwork/*TTP/Types @statusNKKA application B = return + response5B9 statusNKK :@8Content"Type8> 8text.plain8A; 8*ello Corld8 $ain = run &KKK + !'ip de# application

We added an import line to actually ha!e access to the middle-are, and then simply applied g'ip to our application2 You can also chain together multiple middle-ares) a line such as !'ip ?alse + Ssonp + other$iddleware + $yapplication is per#ectly !alid2 /ne -ord o# -arning) the order the middle-are is applied can be important2 For e(ample, 9sonp needs to -ork on uncompressed data, so i# you apply it a#ter you apply g'ip, you%ll ha!e trouble2

Settings Types
5et%s say you%re -riting a -ebser!er2 You -ant the ser!er to take a port to listen on, and an application to run2 So you create the #ollo-ing #unction)
run == Int "< )pplication "< I, @A

But suddenly you reali'e that some people -ill -ant to customi'e their timeout durations2 So you modi#y your PI)
run == Int "< Int "< )pplication "< I, @A

So, -hich Int is the timeout, and -hich is the port= Well, you could create some type aliases, or comment your code2 But there%s another problem creeping into our code) this run #unction is getting unmanageable2 Soon -e%ll need to take an e(tra parameter to indicate hoe(ceptions should be handled, and then another one to control -hich host to bind to, and so on2 So a more e(tensible solution is to introduce a settings datatype)
data 9ettin!s = 9ettin!s 3 settin!sPort == Int > settin!s*ost == 9trin! > settin!sTi$eout == Int 7

:>*

nd this makes the calling code almost sel#1documenting)

run 9ettin!s 3 settin!sPort = DKDK > settin!s*ost = 8(N[/K/K/(8 > settin!sTi$eout = &K 7 $y)pp

;reat, couldn%t be clearer, right= True, but -hat happens -hen you ha!e AH settings to your -ebser!er2 Do you really -ant to ha!e to speci#y all o# those each time= /# course not2 So instead, the -ebser!er should pro!ide a set o# de#aults)
de#ault9ettin!s = 9ettin!s &KKK 8(N[/K/K/(8 &K

nd no-, instead o# needing to -rite that long bit o# code abo!e, -e can get a-ay -ith)
run de#ault9ettin!s 3 settin!sPort = DKDK 7 $y)pp "" @(A

This is great, e(cept #or one minor hitch2 5et%s say -e no- decide to add an e(tra record to 9ettin!s2 ny code out in the -ild looking like this)
run @9ettin!s DKDK 8(N[/K/K/(8 &KA $y)pp "" @NA -ill be broken, since the 9ettin!s constructor no- takes @

arguments2 The proper thing to do -ould be to bump the ma9or !ersion number so that dependent packages don%t get broken2 But ha!ing to change ma9or !ersions #or e!ery minor setting you add is a nuisance2 The solution= Don%t e(port the 9ettin!s constructor)
$odule My9er%er @ 9ettin!s > settin!sPort > settin!s*ost > settin!sTi$eout > run > de#ault9ettin!s A where

With this approach, no one can -rite code like 4:6, so you can #reely add ne- records -ithout any #ear o# code breaking2 The one do-nside o# this approach is that it%s not immediately ob!ious #rom the Haddocks that you can actually change the settings !ia record synta(2 That%s the point o# this chapter) to clari#y -hat%s going on in the libraries that use this techni3ue2 I personally use this techni3ue in a #e- places, #eel #ree to ha!e a look at the Haddocks to see -hat I mean2

Warp) Settings http1conduit) Re3uest and &anagerSettings (ml1conduit o Parsing) ParseSettings o Rendering) RenderSettings

:>:

s a tangential issue, http"conduit and x$l"conduit actually create instances o# the De#ault typeclass instead o# declaring a brand ne- identi#ier2 This means you can 9ust type de# instead o# de#aultParser9ettin!s2

http&conduit
&ost o# Yesod is about ser!ing content o!er HTTP2 But that%s only hal# the story) someone has to recei!e it2 nd e!en -hen you%re -riting a -eb app, sometimes that someone -ill be you2 I# you -ant to consume content #rom other ser!ices or interact -ith R"ST#ul PIs, you%ll need to -rite client code2 nd the recommended approach #or that is http1conduit2 This chapter is not directly connected to Yesod, and -ill be generally use#ul #or anyone -anting to make HTTP re3uests2

Synopsis
3"4 5) G6)G- ,%erloaded9trin!s 4"7 i$port etwork/*TTP/Conduit "" the $ain $odule "" The strea$in! inter#ace uses conduits i$port 1ata/Conduit i$port 1ata/Conduit/Binary @sink?ileA i$port Euali#ied 1ata/Byte9trin!/5a'y as 5 i$port Control/Monad/I,/Class @li#tI,A $ain == I, @A $ain = do "" 9i$plest Euery= Sust download the in#or$ation #ro$ the !i%en 6I5 as a "" la'y Byte9trin!/ si$ple*ttp 8http=..www/exa$ple/co$.#oo/txt8 <<= 5/write?ile 8#oo/txt8 "" 6se the strea$in! inter#ace instead/ Ce need to run all o# this inside a "" IesourceT> to ensure that all our connections !et properly cleaned up in "" the case o# an exception/ runIesourceT + do "" Ce need a Mana!er> which keeps track o# open connections/ si$ple*ttp "" creates a new $ana!er on each run @i/e/> it ne%er reuses "" connectionsA/ $ana!er G" li#tI, + newMana!er de# "" ) $ore e##icient %ersion o# the si$ple*ttp Euery abo%e/ ?irst we "" parse the 6I5 to a reEuest/ reE G" li#tI, + parse6rl 8http=..www/exa$ple/co$.#oo/txt8 "" ow !et the response res G" http reE $ana!er "" )nd #inally strea$ the %alue to a #ile responseBody res ++ sink?ile 8#oo/txt8

:>>

"" Make it a P,9T reEuest> donPt #ollow redirects> and accept any "" status code/ let reEN = reE 3 $ethod = 8P,9T8 > redirectCount = K > check9tatus = 2B B "< othin! 7 resN G" http reEN $ana!er responseBody resN ++ sink?ile 8post"#oo/txt8

Concepts
The simplest -ay to make a re3uest in http"conduit is -ith the si$ple*ttp #unction2 This #unction takes a 9trin! gi!ing a 8R5 and returns a Byte9trin! -ith the contents o# that 8R52 But under the sur#ace, there are a #e- more steps)

ne- connection Mana!er is allocated2 The 8R5 is parsed to a IeEuest2 I# the 8R5 is in!alid, then an e(ception is thro-n2 The HTTP re3uest is made, #ollo-ing any redirects #rom the ser!er2 I# the response has a status code outside the :HH1range, an e(ception is thro-n2 The response body is read into memory and returned2 runIesourceT is called, -hich -ill #ree up any resources 4e2g2, the open socket to the ser!er62

I# you -ant more control o# -hat%s going on, then you can con#igure any o# the steps abo!e 4plus a #e- more6 by e(plicitly creating a IeEuest !alue, allocating your Mana!er manually, and using the http and http5bs #unctions2

%e.uest
The easiest -ay to creating a IeEuest is -ith the parse6rl #unction2 This #unction -ill return a !alue in any ?ailure monad, such as Maybe or I,2 The last o# those is the most commonly used, and results in a runtime e(ception -hene!er an in!alid 8R5 is pro!ided2 Ho-e!er, you can use a di##erent monad i#, #or e(ample, you -ant to !alidate user input2
i$port etwork/*TTP/Conduit i$port 9yste$/-n%iron$ent @!et)r!sA i$port Euali#ied 1ata/Byte9trin!/5a'y as 5 i$port Control/Monad/I,/Class @li#tI,A $ain == I, @A $ain = do ar!s G" !et)r!s case ar!s o# :url9trin!; "< case parse6rl url9trin! o# othin! "< put9tr5n 89orry> in%alid 6I58 Just reE "< withMana!er + 2$ana!er "< do Iesponse B B B lbs G" http5bs reE $ana!er li#tI, + 5/put9tr lbs B "< put9tr5n 89orry> please pro%ide exactly one 6I58

:>@

The IeEuest type is abstract so that http"conduit can add ne- settings in the #uture -ithout breaking the PI 4see the Settings Type chapter #or more in#ormation62 In order to make changes to indi!idual records, you use record notation2 For e(ample, a modi#ication to our program that issues *-)1 re3uests and prints the response headers -ould be)
3"4 5) i$port i$port i$port i$port G6)G- ,%erloaded9trin!s 4"7 etwork/*TTP/Conduit 9yste$/-n%iron$ent @!et)r!sA Euali#ied 1ata/Byte9trin!/5a'y as 5 Control/Monad/I,/Class @li#tI,A

$ain == I, @A $ain = do ar!s G" !et)r!s case ar!s o# :url9trin!; "< case parse6rl url9trin! o# othin! "< put9tr5n 89orry> in%alid 6I58 Just reE "< withMana!er + 2$ana!er "< do let reE*ead = reE 3 $ethod = 8*-)18 7 Iesponse status B headers B G" http reE*ead $ana!er li#tI, + do print status $apMB print headers B "< put9tr5n 89orry> please pro%ide exa$ple one 6I58

There are a number o# di##erent con#iguration settings in the PI, some note-orthy ones are) pro(y llo-s you to pass the re3uest through the gi!en pro(y ser!er2 redirect$ount Indicate ho- many redirects to #ollo-2 De#ault is *H2 checkStatus $heck the status code o# the return !alue2 By de#ault, gi!es an e(ception #or any non1 :CC response2 re3uestBody The re3uest body to be sent2 Be sure to also update the $ethod2 For the common case o# url1encoded data, you can use the url-ncodedBody #unction2

Manager
The connection manager allo-s you to reuse connections2 When making multiple 3ueries to a single ser!er 4e2g2, accessing ma'on S>6, this can be critical #or creating e##icient code2 manager -ill keep track o# multiple connections to a gi!en ser!er 4taking into account port and SS5 as -ell6, automatically reaping unused connections as needed2 When you make a re3uest, http"conduit #irst tries to check out an e(isting connection2 When you%re #inished -ith the connection 4i# the ser!er allo-s keep1ali!e6, the connection is returned to the manager2 I# anything goes -rong, the connection is closed2 To keep our code e(ception1sa#e, -e use the IesourceT monad trans#ormer2 ll this means #or you is that your code needs to be -rapped inside a call to runIesourceT, either implicitly

:>A

or e(plicitly, and that code inside that block -ill need to li#tI, to per#orm normal I/ actions2 There are t-o -ays you can get ahold o# a manager2 newMana!er -ill return a manager that -ill not be automatically closed 4you can use closeMana!er to do so manually6, -hile withMana!er -ill start a ne- IesourceT block, allo- you to use the manager, and then automatically close the IesourceT -hen you%re done2 I# you -ant to use a IesourceT #or an entire application, and ha!e no need to close it, you should probably use newMana!er2 /ne other thing to point out) you ob!iously don%t -ant to create a ne- manager #or each and e!ery re3uest7 that -ould de#eat the -hole purpose2 You should create your Mana!er early and then share it2

%esponse
The Iesponse datatype has three pieces o# in#ormation) the status code, the response headers, and the response body2 The #irst t-o are straight1#or-ard7 let%s discuss the body2 The Iesponse type has a type !ariable to allo- the response body to be o# multiple types2 I# you -ant to use http"conduit%s streaming inter#ace, you -ant this to be a 9ource2 For the simple inter#ace, it -ill be a la'y Byte9trin!2 /ne thing to note is that, e!en though -e use a la'y Byte9trin!, the entire response is held in memory2 In other -ords, -e per#orm no la'y I</ in this package2 The conduit package does pro!ide a la'y module -hich -ould allo- you to read this !alue in la'ily, but like any la'y I</, it%s a bit unsa#e, and de#initely non1deterministic2 I# you need it though, you can use it2

http and httpL-s

So let%s tie it together2 The http #unction gi!es you access to the streaming inter#ace 4i2e2, it returns a Iesponse using a Bu##ered9ource6 -hile http5bs returns a la'y Byte9trin!2 Both o# these return !alues in the IesourceT trans#ormer so that they can access the Mana!er and ha!e connections handled properly in the case o# e(ceptions2 I# you -ant to ignore the remainder o# a large response body, you can connect to the sink ull sink2 The underlying connection -ill automatically be closed, pre!enting you #rom ha!ing to read a large response body o!er the net-ork2

,ml&conduit
&any de!elopers cringe at the thought o# dealing -ith C&5 #iles2 C&5 has the reputation o# ha!ing a complicated data model, -ith ob#uscated libraries and huge layers o# comple(ity sitting bet-een you and your goal2 I%d like to posit that a lot o# that pain is actually a language and library issue, not inherent to C&52

:>D

/nce again, Haskell%s type system allo-s us to easily break do-n the problem to its most basic #orm2 The (ml1types package neatly deconstructs the C&5 data model 4both a streaming and D/&1based approach6 into some simple DTs2 Haskell%s standard immutable data structures make it easier to apply trans#orms to documents, and a simple set o# #unctions makes parsing and rendering a bree'e2 We%re going to be co!ering the (ml1conduit package2 8nder the sur#ace, this package uses a lot o# the approaches Yesod in general does #or high per#ormance) bla'e1builder, te(t, conduit and attoparsec2 But #rom a user perspecti!e, it pro!ides e!erything #rom the simplest PIs 4read?ile<write?ile6 through #ull control o# C&5 e!ent streams2 In addition to x$l"conduit, there are a #e- related packages that come into play, like (ml1 hamlet and (ml:html2 We%ll co!er both ho- to use all these packages, and -hen they should be used2

Synopsis Input >ML file

Gdocu$ent title=8My Title8< Gpara<This is a para!raph/ It has Ge$<e$phasi'edG.e$< and Gstron!<stron!G.stron!< words/G.para< Gi$a!e hre#=8$yi$a!e/pn!8.< G.docu$ent<

Haskell code
3"4 5) 3"4 5) i$port i$port i$port G6)G- FuasiFuotes 4"7 G6)G- ,%erloaded9trin!s 4"7 Prelude hidin! @read?ile> write?ileA Text/YM5 Text/*a$let/YM5

$ain == I, @A $ain = do "" read?ile will throw any parse errors as runti$e exceptions "" de# uses the de#ault settin!s 1ocu$ent prolo!ue root epilo!ue G" read?ile de# 8input/x$l8 "" root is the root ele$ent o# the docu$ent> letPs $odi#y it let rootP = trans#or$ root "" )nd now we write out/ 5etPs indent our output write?ile de# 3 rsPretty = True 7 8output/ht$l8 + 1ocu$ent prolo!ue rootP epilo!ue "" CePll turn out Gdocu$ent< into an Y*TM5 docu$ent trans#or$ == -le$ent "< -le$ent trans#or$ @-le$ent Bna$e attrs childrenA = -le$ent 8ht$l8 :; :x$l| Ghead< Gtitle<

:>F

+$aybe title G" lookup 8title8 attrs 243title7 +nothin! 6ntitled 1ocu$ent Gbody< +#orall child G" children Q3!o ode child7 |; !o !o !o !o !o ode ode ode ode ode == ode "< : ode; @ ode-le$ent eA = : ode-le$ent + !o-le$ e; @ odeContent tA = : odeContent t; @ odeCo$$ent BA = :; "" hide co$$ents @ odeInstruction BA = :; "" and hide processin! instructions too

"" con%ert each source ele$ent to its Y*TM5 eEui%alent !o-le$ == -le$ent "< -le$ent !o-le$ @-le$ent 8para8 attrs childrenA = -le$ent 8p8 attrs + concatMap !o ode children !o-le$ @-le$ent 8e$8 attrs childrenA = -le$ent 8i8 attrs + concatMap !o ode children !o-le$ @-le$ent 8stron!8 attrs childrenA = -le$ent 8b8 attrs + concatMap !o ode children !o-le$ @-le$ent 8i$a!e8 attrs BchildrenA = -le$ent 8i$!8 @$ap #ix)ttr attrsA :; "" i$a!es canPt ha%e children where #ix)ttr @8hre#8> %alueA = @8src8> %alueA #ix)ttr x = x !o-le$ @-le$ent na$e attrs childrenA = "" donPt know what to do> Sust pass it throu!h/// -le$ent na$e attrs + concatMap !o ode children

Output >HTML
G]x$l %ersion=8(/K8 encodin!=86T?"D8]< Ght$l< Ghead< Gtitle< My Title G.title< G.head< Gbody< Gp< This is a para!raph/ It has Gi< e$phasi'ed G.i< and Gb< stron! G.b< words/ G.p< Gi$! src=8$yi$a!e/pn!8.< G.body< G.ht$l<

:>+

Types
5et%s take a bottom1up approach to analy'ing types2 This section -ill also ser!e as a primer on the C&5 data model itsel#, so don%t -orry i# you%re not completely #amiliar -ith it2 I think the #irst place -here Haskell really sho-s its strength is -ith the 0ame datatype2 &any languages 4like .a!a6 struggle -ith properly e(pressing names2 The issue is that there are in #act three components to a name) its local name, its namespace 4optional6, and its pre#i( 4also optional62 5et%s look at some C&5 to e(plain)
Gno"na$espace.< Gno"pre#ix x$lns=8#irst"na$espace8 #irst"attr=8%alue(8.< G#oo=with"pre#ix x$lns=#oo=8second"na$espace8 #oo=second"attr=8%alueN8.<

The #irst tag has a local name o# no"na$espace, and no namespace or pre#i(2 The second tag 4local name) no"pre#ix6 also has no pre#i(, but it does ha!e a namespace 4#irst" na$espace62 #irst"attr, ho-e!er, does not inherit that namespace) attribute namespaces must al-ays be e(plicitly set -ith a pre#i(2 0amespaces are almost al-ays 8RIs o# some sort, though there is nothing in any speci#ication re3uiring that it be so2 The third tag has a local name o# with"pre#ix, a pre#i( o# #oo and a namespace o# second" na$espace2 Its attribute has a second"attr local name and the same pre#i( and namespace2 The x$lns and x$lns=#oo attributes are part o# the namespace speci#ication, and are not considered attributes o# their respecti!e elements2 So let%s re!ie- -hat -e need #rom a name) e!ery name has a local name, and it can optionally ha!e a pre#i( and namespace2 Seems like a simple #it #or a record type)
data a$e = a$e 3 na$e5ocal a$e == Text > na$e a$espace == Maybe Text > na$ePre#ix == Maybe Text 7

ccording the the C&5 namespace standard, t-o names are considered e3ui!alent i# they ha!e the same localname and namespace2 In other -ords, the pre#i( is not important2 There#ore, x$l"types de#ines -E and ,rd instances that ignore the pre#i(2 The last class instance -orth mentioning is Is9trin!2 It -ould be !ery tedious to ha!e to manually type out a$e 8p8 othin! othin! e!ery time -e -ant a paragraph2 I# you turn on ,%erloaded9trin!s, 8p8 -ill resol!e to that all by itsel#? In addition, the Is9trin! instance recogni'es something called $lark notation, -hich allo-s you to pre#i( the namespace surrounded in curly brackets2 In other -ords)
83na$espace7ele$ent8 == a$e 8ele$ent8 @Just 8na$espace8A othin!

:>B

8ele$ent8 ==

a$e 8ele$ent8

othin!

othin!

The !our Types of <odes

C&5 documents are a tree o# nested nodes2 There are in #act #our di##erent types o# nodes allo-ed) elements, content 4i2e2, te(t6, comments, and processing instructions2 You may not be #amiliar -ith that last one, it%s less commonly used2 It is marked up as)
G]tar!et data]<

There are t-o surprising #acts about processing instructions 4PIs6) PIs don%t ha!e attributes2 While o#ten times you%ll see processing instructions that appear to ha!e attributes, there are in #act no rules about that data o# an instruction2 The G]x$l ///]< stu## at the beginning o# a document is not a processing instruction2 It is simply the beginning o# the document 4kno-n as the C&5 declaration6, and happens to look an a-#ul lot like a PI2 The di##erence though is that the G]x$l ///]< line -ill not appear in your parsed content2 Since processing instructions ha!e t-o pieces o# te(t associated -ith them 4the target and the data6, -e ha!e a simple data type)
data Instruction = Instruction 3 instructionTar!et == Text > instruction1ata == Text 7

$omments ha!e no special datatype, since they are 9ust te(t2 But content is an interesting one) it could contain either plain te(t or unresol!ed entities 4e2g2, Tcopyri!ht"state$entR62 (ml1 types keeps those unresol!ed entities in all the data types in order to completely match the spec2 Ho-e!er, in practice, it can be !ery tedious to program against those data types2 nd in most use cases, an unresol!ed entity is going to end up as an error any-ay2 So the Te(t2C&5 module de#ines its o-n set o# datatypes #or nodes, elements and documents that remo!es all unresol!ed entities2 I# you need to deal -ith unresol!ed entities instead, you should use the Te(t2C&528nresol!ed module2 From no- on, -e%ll be #ocusing only on the Text/YM5 data types, though they are almost identical to the x$l"types !ersions2 ny-ay, a#ter that detour) content is 9ust a piece o# te(t, and there#ore it too does not ha!e a special datatype2 The last node type is an element, -hich contains three pieces o# in#ormation) a name, a list o# attributes and a list o# children nodes2 n attribute has t-o pieces o# in#ormation) a name and a !alue2 4In x$l"types, this !alue could contain unresol!ed entities as -ell26 So our -le$ent is de#ined as)
data -le$ent = -le$ent 3 ele$ent a$e == a$e > ele$ent)ttributes == :@ a$e> TextA; > ele$ent odes == : ode; 7

Which o# course begs the 3uestion) -hat does a ode look like= This is -here Haskell really shines) its sum types model the C&5 data model per#ectly2

:@H

data

= | | |

ode ode-le$ent -le$ent odeInstruction Instruction odeContent Text odeCo$$ent Text

(ocuments
So no- -e ha!e elements and nodes, but -hat about an entire document= 5et%s 9ust lay out the datatypes)
data 1ocu$ent = 1ocu$ent 3 docu$entProlo!ue == Prolo!ue > docu$entIoot == -le$ent > docu$ent-pilo!ue == :Miscellaneous; 7 data Prolo!ue = Prolo!ue 3 prolo!ueBe#ore == :Miscellaneous; > prolo!ue1octype == Maybe 1octype > prolo!ue)#ter == :Miscellaneous; 7 data Miscellaneous = MiscInstruction Instruction | MiscCo$$ent Text data 1octype = 1octype 3 doctype a$e == Text > doctypeI1 == Maybe -xternalI1 7 data -xternalI1 = 9yste$I1 Text | PublicI1 Text Text

The C&5 spec says that a document has a single root element 4docu$entIoot62 It also has an optional doctype statement2 Be#ore and a#ter both the doctype and the root element, you are allo-ed to ha!e comments and processing instructions2 4You can also ha!e -hitespace, but that is ignored in the parsing26 So -hat%s up -ith the doctype= Well, it speci#ies the root element o# the document, and then optional public and system identi#iers2 These are used to re#er to DTD #iles, -hich gi!e more in#ormation about the #ile 4e2g2, !alidation rules, de#ault attributes, entity resolution62 5et%s see some e(amples)
GJ1,CTHP- root< GJ"" no external identi#ier ""< GJ1,CTHP- root 9H9T-M 8root/dtd8< GJ"" a syste$ identi#ier ""< GJ1,CTHP- root P6B5IC 8My Ioot Public Identi#ier8 8root/dtd8< GJ"" public identi#iers ha%e a syste$ I1 as well ""<

nd that, my #riends, is the entire C&5 data model2 For many parsing purposes, you%ll be able to simply ignore the entire 1ocu$ent datatype and go immediately to the docu$entIoot2

:@*

/ ents
In addition to the document PI, x$l"types de#ines an "!ent datatype2 This can be used #or constructing streaming tools, -hich can be much more memory e##icient #or certain kinds o# processing 4eg, adding an e(tra attribute to all elements62 We -ill not be co!ering the streaming PI currently, though it should look !ery #amiliar a#ter analy'ing the document PI2 You can see an e(ample o# the streaming PI in the Sphin( case study2

Te,t?>ML
The recommended entry point to (ml1conduit is the Te(t2C&5 module2 This module e(ports all o# the datatypes you%ll need to manipulate C&5 in a D/& #ashion, as -ell as a number o# di##erent approaches #or parsing and rendering C&5 content2 5et%s start -ith the simple ones)
read?ile == Parse9ettin!s "< ?ilePath "< I, 1ocu$ent write?ile == Iender9ettin!s "< ?ilePath "< 1ocu$ent "< I, @A This introduces the Parse9ettin!s and Iender9ettin!s datatypes2 You can

use these to modi#y the beha!ior o# the parser and renderer, such as adding character entities and turning on pretty 4i2e2, indented6 output2 Both these types are instances o# the De#ault typeclass, so you can simply use de# -hen these need to be supplied2 That is ho- -e -ill supply these !alues through the rest o# the chapter7 please see the PI docs #or more in#ormation2 It%s -orth pointing out that in addition to the #ile1based PI, there is also a te(t1 and bytestring1based PI2 The bytestring1po-ered #unctions all per#orm intelligent encoding detections, and support 8TF1+, 8TF1*D and 8TF1>:, in either big or little endian, -ith and -ithout a Byte1/rder &arker 4B/&62 ll output is generated in 8TF1+2 For comple( data lookups, -e recommend using the higher1le!el cursors PI2 The standard Text/YM5 PI not only #orms the basis #or that higher le!el, but is also a great PI #or simple C&5 trans#ormations and #or C&5 generation2 See the synopsis #or an e(ample2

A note a-out file paths

In the type signature abo!e, -e ha!e a type ?ilePath2 Ho-e!er, this isn1t Prelude.FilePath2 The standard Prelude de#ines a type synonym type ?ilePath = :Char;2 8n#ortunately, there are many limitations to using such an approach, including con#usion o# #ilename character encodings and di##erences in path separators2 Instead, x$l"conduit uses the system1#ilepath package, -hich de#ines an abstract ?ilePath type2 I%!e personally #ound this to be a much nicer approach to -ork -ith2 The package is #airly easy to #ollo-, so I -on%t go into details here2 But I do -ant to gi!e a #e- 3uick e(planations o# ho- to use it)

:@:

Since a ?ilePath is an instance o# Is9trin!, you can type in regular strings and they -ill be treated properly, as long as the ,%erloaded9trin!s e(tension is enabled2 4I highly recommend enabling it any-ay, as it makes dealing -ith Text !alues much more pleasant26 I# you need to e(plicitly con!ert to or #rom Prelude%s ?ilePath, you should use the encodeString and decodeString, respecti!ely2 This takes into account #ile path encodings2 Instead o# manually splicing together directory names and #ile names -ith e(tensions, use the operators in the ?ilesyste$/Path/Current,9 module, e2g2 $y#older G.< #ilena$e G/< extension2

Cursor
Suppose you -ant to pull the title out o# an CHT&5 document2 You could do so -ith the Text/YM5 inter#ace -e 9ust described, using standard pattern matching on the children o# elements2 But that -ould get !ery tedious, !ery 3uickly2 Probably the gold standard #or these kinds o# lookups is CPath, -here you -ould be able to -rite .ht$l.head.title2 nd that%s e(actly -hat inspired the design o# the Te(t2C&52$ursor combinators2 cursor is an C&5 node that kno-s its location in the tree7 it%s able to tra!erse up-ards, side-ays, and do-n-ards2 48nder the sur#ace, this is achie!ed by tying the knot26 There are t-o #unctions a!ailable #or creating cursors #rom Text/YM5 types) #ro$1ocu$ent and #ro$ ode2 We also ha!e the concept o# an (is, de#ined as type )xis = Cursor "< :Cursor;2 It%s easiest to get started by looking at e(ample a(es) child returns 'ero or more cursors that are the child o# the current one, parent returns the single parent cursor o# the input, or an empty list i# the input is the root element, and so on2 In addition, there are some a(es that take predicates2 ele$ent is a commonly used #unction that #ilters do-n to only elements -hich match the gi!en name2 For e(ample, ele$ent 8title8 -ill return the input element i# its name is ItitleI, or an empty list other-ise2 nother common #unction -hich isn%t 3uite an a(is is content == Cursor "< :Text;2 For all content nodes, it returns the contained te(t7 other-ise, it returns an empty list2 nd thanks to the monad instance #or lists, it%s easy to string all o# these together2 For e(ample, to do our title lookup, -e -ould -rite the #ollo-ing program)
3"4 5) i$port i$port i$port i$port G6)G- ,%erloaded9trin!s 4"7 Prelude hidin! @read?ileA Text/YM5 Text/YM5/Cursor Euali#ied 1ata/Text as T

$ain == I, @A $ain = do doc G" read?ile de# 8test/x$l8 let cursor = #ro$1ocu$ent doc print + T/concat + child cursor <<= ele$ent 8head8 <<= child

:@>

<<= ele$ent 8title8 <<= descendant <<= content

What this says is) *2 :2 >2 @2 A2 ;et me all the child nodes o# the root element Filter do-n to only the elements named IheadI ;et all the children o# all those head elements Filter do-n to only the elements named ItitleI ;et all the descendants o# all those title elements2 4 descendant is a child, or a descendant o# a child2 Yes, that -as a recursi!e de#inition26 D2 ;et only the te(t nodes2 So #or the input document)
Ght$l< Ghead< Gtitle<My Gb<TitleG.b<G.title< G.head< Gbody< Gp<?oo bar ba'G.p< G.body< G.ht$l<

We end up -ith the output My Title2 This is all -ell and good, but it%s much more !erbose than the CPath solution2 To combat this !erbosity, ristid Breitkreu' added a set o# operators to the $ursor module to handle many common cases2 So -e can re-rite our e(ample as)
3"4 5) i$port i$port i$port i$port G6)G- ,%erloaded9trin!s 4"7 Prelude hidin! @read?ileA Text/YM5 Text/YM5/Cursor Euali#ied 1ata/Text as T

$ain == I, @A $ain = do doc G" read?ile de# 8test/x$l8 let cursor = #ro$1ocu$ent doc print + T/concat + cursor +. ele$ent 8head8 T. ele$ent 8title8 T.. content

says to apply the a(is on the right to the children o# the cursor on the le#t2 T. is almost identical, but is instead used to combine t-o a(es together2 This is a general rule in Text/YM5/Cursor) operators beginning -ith ^ directly apply an a(is, -hile X -ill combine t-o together2 T.. is used #or applying an a(is to all descendants2
+.

5et%s go #or a more comple(, i# more contri!ed, e(ample2 We ha!e a document that looks like)
Ght$l< Ghead< Gtitle<*eadin!sG.title< G.head< Gbody< Gh!roup< Gh(<*eadin! ( #ooG.h(<

:@@

GhN class=8#oo8<*eadin! N #ooG.hN< G.h!roup< Gh!roup< Gh(<*eadin! ( barG.h(< GhN class=8bar8<*eadin! N barG.hN< G.h!roup< G.body< G.ht$l<

We -ant to get the content o# all the h( tags -hich precede an hN tag -ith a class attribute o# IbarI2 To per#orm this con!oluted lookup, -e can -rite)
3"4 5) i$port i$port i$port i$port G6)G- ,%erloaded9trin!s 4"7 Prelude hidin! @read?ileA Text/YM5 Text/YM5/Cursor Euali#ied 1ata/Text as T

$ain == I, @A $ain = do doc G" read?ile de# 8testN/x$l8 let cursor = #ro$1ocu$ent doc print + T/concat + cursor +.. ele$ent 8hN8 <=< attributeIs 8class8 8bar8 <=< precedin!9iblin! <=< ele$ent 8h(8 T.. content

5et%s step through that2 First -e get all h: elements in the document2 4+.. gets all descendants o# the root element26 Then -e #ilter out only those -ith class=bar2 That <=< operator is actually the standard operator #rom $ontrol2&onad7 yet another ad!antage o# the monad instance o# lists2 precedin!9iblin! #inds all nodes that come be#ore our node and share the same parent2 4There is also a precedin! a(is -hich takes all elements earlier in the tree26 We then take 9ust the h( elements, and then grab their content2 The e3ui!alent CPath, #or comparison, -ould be ..hN:Oclass = PbarP;.precedin!" siblin!==h(..text@A2 While the cursor PI isn%t 3uite as succinct as CPath, it has the ad!antages o# being standard Haskell code, and o# type sa#ety2

,ml&hamlet
Thanks to the simplicity o# Haskell%s data type system, creating C&5 content -ith the Text/YM5 )PI is easy, i# a bit !erbose2 The #ollo-ing code)
3"4 5) G6)G- ,%erloaded9trin!s 4"7 i$port Text/YM5 i$port Prelude hidin! @write?ileA $ain == I, @A $ain = write?ile de# 8test&/x$l8 + 1ocu$ent @Prolo!ue :;

othin! :;A root :;

:@A

where root = -le$ent 8ht$l8 :; : ode-le$ent + -le$ent 8head8 :; : ode-le$ent + -le$ent 8title8 :; : odeContent 8My 8 > ode-le$ent + -le$ent 8b8 :; : odeContent 8Title8 ; ; ; > ode-le$ent + -le$ent 8body8 :; : ode-le$ent + -le$ent 8p8 :; : odeContent 8#oo bar ba'8 ; ; ;

produces
G]x$l %ersion=8(/K8 encodin!=86T?"D8]< Ght$l<Ghead<Gtitle<My Gb<TitleG.b<G.title<G.head<Gbody<Gp<#oo bar ba'G.p<G.body<G.ht$l<

This is leaps and bounds easier than ha!ing to deal -ith an imperati!e, mutable1!alue1based PI 4cough, .a!a, cough6, but it%s #ar #rom pleasant, and obscures -hat -e%re really trying to achie!e2 To simpli#y things, -e ha!e the (ml1hamlet package, -hich using Guasi1Guotation to allo- you to type in your C&5 in a natural synta(2 For e(ample, the abo!e could be re-ritten as)
3"4 5) 3"4 5) i$port i$port i$port G6)G- ,%erloaded9trin!s 4"7 G6)G- FuasiFuotes 4"7 Text/YM5 Text/*a$let/YM5 Prelude hidin! @write?ileA

$ain == I, @A $ain = write?ile de# 8test&/x$l8 + 1ocu$ent @Prolo!ue :; where root = -le$ent 8ht$l8 :; :x$l| Ghead< Gtitle< My 4 Gb<Title Gbody< Gp<#oo bar ba' |;

othin! :;A root :;

5et%s make a #e- points)

The synta( is almost identical to normal Hamlet, e(cept 8R51interpolation 4NR222S6 has been remo!ed2 s such) o 0o close tags2 o Whitespace1sensiti!e2 o I# you -ant to ha!e -hitespace at the end o# a line, use a O at the end2 t the beginning, use a backslash2

:@D

n x$l interpolation -ill return a list o# odes2 So you still need to -rap up the output in all the normal 1ocu$ent and root -le$ent constructs2 There is no support #or the special /class and 4id attribute #orms2

nd like normal Hamlet, you can use !ariable interpolation and control structures2 So a slightly more comple( e(ample -ould be)
3"4 5) 3"4 5) i$port i$port i$port i$port G6)G- ,%erloaded9trin!s 4"7 G6)G- FuasiFuotes 4"7 Text/YM5 Text/*a$let/YM5 Prelude hidin! @write?ileA 1ata/Text @Text> packA

data Person = Person 3 person a$e == Text > person)!e == Int 7 people == :Person; people = : Person 8Michael8 NV > Person 8Miria$8 NM > Person 8-lie'er8 & > Person 8Ga%riella8 ( ; $ain == I, @A $ain = write?ile de# 8people/x$l8 + 1ocu$ent @Prolo!ue :; where root = -le$ent 8ht$l8 :; :x$l| Ghead< Gtitle<9o$e People Gbody< Gh(<9o$e People +i# null people Gp<There are no people/ +else Gdl< +#orall person G" people Q3person odes person7 |; person odes == Person "< : ode; person odes person = :x$l| Gdt<43person a$e person7 Gdd<43pack + show + person)!e person7 |;

othin! :;A root :;

#e- more notes)

The caret1interpolation 4QR222S6 takes a list o# nodes, and so can easily embed other x$l13uotations2 8nlike Hamlet, hash1interpolations 4OR222S6 are not polymorphic, and can only accept Text !alues2

:@F

,ml@html
So #ar in this chapter, our e(amples ha!e re!ol!ed around CHT&52 I%!e done that so #ar simply because it is likely to be the most #amiliar #orm o# C&5 #or most o# our readers2 But there%s an ugly side to all this that -e must ackno-ledge) not all CHT&5 -ill be correct HT&52 The #ollo-ing discrepancies e(ist)

There are some !oid tags 4e2g2, i$!, br6 in HT&5 -hich do not need to ha!e close tags, and in #act are not allo-ed to2 HT&5 does not understand sel#1closing tags, so Gscript<G.script< and Gscript.< mean !ery di##erent things2 $ombining the pre!ious t-o points) you are #ree to sel#1close !oid tags, though to a bro-ser it -on%t mean anything2 In order to a!oid 3uirks mode, you should start your HT&5 documents -ith a 1,CTHP- statement2 We do not -ant the C&5 declaration G]x$l ///]< at the top o# an HT&5 page We do not -ant any namespaces used in HT&5, -hile CHT&5 is #ully namespaced2 The contents o# Gstyle< and Gscript< tags should not be escaped2

That%s -here the (ml:html package comes into play2 It pro!ides a ToHtml instance #or odes, 1ocu$ents and -le$ents2 In order to use it, 9ust import the Te(t2C&52Cml:Html module2
3"4 5) i$port i$port i$port i$port i$port G6)G- ,%erloaded9trin!s> FuasiFuotes 4"7 Text/Bla'e @to*t$lA Text/Bla'e/Ienderer/9trin! @render*t$lA Text/YM5 Text/*a$let/YM5 Text/YM5/Y$lN*t$l @A othin! :;A

$ain == I, @A $ain = put9tr + render*t$l + to*t$l + 1ocu$ent @Prolo!ue :; root :; root == -le$ent root = -le$ent 8ht$l8 :; :x$l| Ghead< Gtitle<Test Gscript<i# @M G V || D < \A alert@8*ello CorldJ8AR Gstyle<body < h( 3 color= red 7 Gbody< Gh(<*ello CorldJ |;

/utputs) 4-hitespace added6

GJ1,CTHP- *TM5< Ght$l< Ghead< Gtitle<TestG.title< Gscript<i# @M G V || D < \A alert@8*ello CorldJ8ARG.script<

:@+

Gstyle<body < h( 3 color= red 7G.style< G.head< Gbody< Gh(<*ello CorldJG.h(< G.body< G.ht$l<

/numerator Package
Due to the #act that Yesod has mo!ed on to conduits, this chapter -ill no longer be included in the actual Yesod book2 It is kept on the site as a re#erence #or those -ho are still using it2 /ne o# the upcoming patterns in Haskell is enumerators2 They are designed to sol!e the problems o# producing, modi#ying and consuming streams o# data2 "numerators are considered a bit intimidating, possibly because)

There are multiple implementations, all -ith slightly di##erent approaches2 Some o# the implementations 4in my opinion6 use incredibly con#using naming2 The tutorials that get -ritten usually don%t directly target an e(isting implementation, and -ork more on building up intuition than gi!ing instructions on ho- to use the library2

The Yesod #rame-ork uses enumerators behind the scenes in a number o# places) in W I<Warp #or dealing -ith re3uest and response bodies, in Persistent #or recei!ing the results #rom a database 3uery, and in !arious side packages like http1enumerator and (ml1 enumerator2 We al-ays use the enumerator package2 That package is the topic o# this chapter2 This chapter is composed o# three main sections, each dealing -ith one o# the main concepts)

We start -ith iteratees, -hich are consumers2 They are #ed data and do something -ith it2 0e(t -e co!er enumerators -hich are producers2 They #eed data to an iteratee2 Finally, -e address enumeratees2 These are pipes -hich are #ed data #rom an enumerator and in turn #eed data to an iteratee2

You may be -ondering -hy -e bother -ith this, -hen most o# these problems can be addressed by la'y I</2 The basic problem -ith la'y I</ is its non1determinism2 For more in#ormation, it%s best to go to the original source on this topic) /leg2

Iteratees
n Iteratee is a data consumer2 It is the core type o# the enumerator package, on -hich the other types are based2

Intuition
5et%s say -e -ant to -rite a #unction that sums the numbers in a list2 Forgetting uninteresting details like space leaks, a per#ectly good implementation could be) :@B

su$( == :Int; "< Int su$( :; = K su$( @x=xsA = x U su$( xs

But let%s say that -e don%t ha!e a list o# numbers2 Instead, the user is typing numbers on the command line, and hitting I3I -hen done2 In other -ords, -e ha!e a #unction like)
!et u$ber == I, @Maybe IntA !et u$ber = do x G" read5ine i# x == 8E8 then return othin! else return + Just + read x

We could -rite our ne- sum #unction as)

su$N == I, Int su$N = do $aybe u$ G" !et u$ber case $aybe u$ o# othin! "< return K Just nu$ "< do rest G" su$N return + nu$ U rest

It%s #airly annoying to ha!e to -rite t-o completely separate sum #unctions 9ust because our data source changed2 Ideally, -e -ould like to generali'e things a bit2 5et%s start by noticing a similarity bet-een these t-o #unctions) they both only yield a !alue -hen they are in#ormed that there are no more numbers2 In the case o# sum*, -e check #or an empty list7 in sum:, -e check #or 0othing2

Stream (atatype
The #irst datatype de#ined in the enumerator package is)
data 9trea$ a = Chunks :a; | -,?

The "/F constructor indicates that no more data is a!ailable2 The $hunks constructor simply allo-s us to put multiple pieces o# data together #or e##iciency2 We could no- re-rite sum: to use this Stream datatype)
!et u$berN == I, @9trea$ IntA !et u$berN = do $aybe u$ G" !et u$ber "" usin! the ori!inal !et u$ber #unction case $aybe u$ o# othin! "< return -,? Just nu$ "< return + Chunks :nu$; su$& == I, Int su$& = do strea$ G" !et u$berN case strea$ o# -,? "< return K Chunks nu$s "< do let nu$sP = su$ nu$s

:AH

rest G" su$& return + nu$sP U rest

0ot that it%s much better than sum:, but at least it sho-s ho- to use the Stream datatype2 The problem here is that -e still re#er e(plicitly to the get0umber: #unction, hard1coding the data source2 /ne possible solution is to make the data source an argument to the sum #unction, ie)
su$L == I, @9trea$ IntA "< I, Int su$L !et u$ = do strea$ G" !et u$ case strea$ o# -,? "< return K Chunks nu$s "< do let nu$sP = su$ nu$s rest G" su$L !et u$ return + nu$sP U rest

That%s all -ell and good, but let%s pretend -e -ant to ha!e two datasources to sum o!er) !alues the user enters on the command line, and some numbers -e read o!er an HTTP connection, perhaps2 The problem here is one o# control) sum@ is running the sho- here by calling get0um2 This is a pull data model2 "numerators ha!e an in ersion of control5push model, putting the enumerator in charge2 This allo-s cool things like #eeding in multiple data sources, and also makes it easier to -rite enumerators that properly deal -ith resource allocation2

The Step datatype

So -e need a ne- datatype that -ill represent the state o# our summing operation2 We%re going to allo- our operations to be in one o# three states)

Waiting #or more data2 lready calculated a result2 For con!enience, -e also ha!e an error state2 This isn%t strictly necessary 4it could be modeled by choosing an "itherT kind o# monad, #or e(ample6, but it%s simpler2

s you could guess, these states -ill correspond to three constructors #or the Step datatype2 The error state is modeled by -rror 9o$e-xception, building on top o# Haskell%s e(tensible e(ception system2 The already calculated constructor is)
Hield b @9trea$ aA

Here, a is the input to our iteratee and b is the output2 This constructor allo-s us to simultaneously produce a result and sa!e any Ile#to!erI input #or another iteratee that may run a#ter us2 4This -on%t be the case -ith the sum #unction, -hich al-ays consumes all its input, but -e%ll see some other e(amples that do not consume all output26 0o- the 3uestion is ho- to represent the state o# an iteratee that%s -aiting #or more data2 You might at #irst -ant to declare some datatype to represent the internal state and pass that

:A*

around someho-2 That%s not ho- it -orks) instead, -e simply use a #unction 4!ery Haskell o# us, right=6)
Continue @9trea$ a "< Iteratee a $ bA

"ureka? We%!e #inally seen the Iteratee datatype? ctually, Iteratee is a !ery boring datatype that is only present to allo- us to declare cool instances 4eg, &onad6 #or our #unctions2 Iteratee is de#ined as)
newtype Iteratee a $ b = Iteratee @$ @9tep a $ bAA

nd the complete Step datatype is)

data 9tep a $ b = -rror 9o$e-xception | Hield b @9trea$ aA | Continue @9trea$ a "< Iteratee a $ bA

This is important) Iteratee is 4ust a ne3type 3rapper around a Step inside a monad2 .ust keep that in mind as you look at de#initions in the enumerator package2 So kno-ing this, -e can think o# the $ontinue constructor as)
Continue @9trea$ a "< $ @9tep a $ bAA

That%s much easier to approach) that #unction takes some input data and returns a ne- state o# the iteratee2 5et%s see -hat our sum #unction -ould look like using this Step datatype)
su$M == Monad $ =< 9tep Int $ Int "" Int input> any $onad> Int output su$M = Continue + !o K "" a co$$on pattern> you always start with a Continue where !o == Monad $ =< Int "< 9trea$ Int "< Iteratee Int $ Int "" )dd the new input to the runnin! su$ and create a new Continue !o runnin!9u$ @Chunks nu$sA = do let runnin!9u$P = runnin!9u$ U su$ nu$s "" This next line is Wu!lyW> !ood thin! there are so$e helper "" #unctions to clean it up/ More on that below/ Iteratee + return + Continue + !o runnin!9u$P "" Produce the #inal result !o runnin!9u$ -,? = Iteratee + return + Hield runnin!9u$ -,? In order to run this code, you can use runB + enu$5ist D :(//(K; su$M2 But this gets into

some o# the "numerator black magic -e -on%t discuss till later2 The #irst real line 4Continue + !o K6 initiali'es our iteratee to its starting state2 .ust like e!ery other sum #unction, -e need to e(plicitly state that -e are starting #rom H some-here2 The real -orkhorse is the go #unction2 0otice ho- -e are really passing the state o# the iteratee around as the #irst argument to go) this is also a !ery common pattern in iteratees2 We need to handle t-o di##erent cases) -hen handed an "/F, the go #unction must Yield a !alue2 4Well, it could also produce an "rror !alue, but it de#initely cannot $ontinue26 In that case, -e simply yield the running sum and say there -as no data le#t o!er2 When -e recei!e some input data !ia $hunks, -e simply add it to the running sum and create a ne- $ontinue based on the same go #unction2

:A:

0o- let%s -ork on making that #unction a little bit prettier by using some built1in helper #unctions2 The pattern Iteratee / return is common enough to -arrant a helper #unction, namely)
returnI == Monad $ =< 9tep a $ b "< Iteratee a $ b returnI = Iteratee / return

So #or e(ample,
!o runnin!9u$ -,? = Iteratee + return + Hield runnin!9u$ -,?

becomes
!o runnin!9u$ -,? = returnI + Hield runnin!9u$ -,?

But e!en that is common enough to -arrant a helper #unction)

yield == Monad $ =< b "< 9trea$ a "< Iteratee a $ b yield x chunk = returnI + Hield x chunk

so our line becomes

!o runnin!9u$ -,? = yield runnin!9u$ -,?

Similarly,
Iteratee + return + Continue + !o runnin!9u$P

becomes
continue + !o runnin!9u$P

Monad instance for Iteratee

This is all !ery nice) -e no- ha!e an iteratee that can be #ed numbers #rom any monad and sum them2 It can e!en take input #rom di##erent sources and sum them together2 4By the -ay, I ha!en%t actually sho-n you ho- to #eed those numbers in) that is in part : about enumerators26 But let%s be honest) sumA is an ugly #unction2 Isn%t there something easier= In #act, there is2 Remember ho- I said Iteratee really 9ust e(isted to #acilitate typeclass instances= This includes a monad instance2 Feel #ree to look at the code to see ho- that instance is de#ined, but here -e%ll 9ust look at ho- to use it)
su$V == Monad $ =< Iteratee Int $ Int su$V = do $aybe u$ G" head "" not head #ro$ PreludeJ case $aybe u$ o# othin! "< return K Just i "< do rest G" su$V return + i U rest

:A>

That head #unction is not #rom Prelude, it%s #rom the Data2"numerator module2 Its type signature is)
head == Monad $ =< Iteratee a $ @Maybe aA

-hich basically means gi!e me the ne(t piece o# input i# it%s there2 We%ll look at this #unction in more depth in a bit2 ;o compare the code #or sumD -ith sum:) they are ama'ingly similar2 You can o#ten build up more complicated iteratees by using some simple iteratees and the &onad instance o# Iteratee2

Interlea ed I5O
lright, let%s look at a totally di##erent problem2 We -ant to be #ed some strings and print them to the screen one line at a time2 /ne approach -ould be to use la'y I</)
la'yI, == I, @A la'yI, = do s G" lines Z#$apZ !etContents $apMB put9tr5n s

But this has t-o dra-backs)

It%s tied do-n to a single input source, stdin2 This could be -orked around -ith an argument gi!ing a datasource2 But let%s say the data source is some scarce resource 4think) #ile handles on a !ery busy -eb ser!er62 We ha!e no guarantees -ith la'y I</ o# -hen those #ile handles -ill be released2

5et%s look at ho- to -rite this in our ne- high1le!el monadic iteratee approach)
interlea%ed == MonadI, $ =< Iteratee 9trin! $ @A interlea%ed = do $aybe5ine G" head case $aybe5ine o# othin! "< return @A Just line "< do li#tI, + put9tr5n line interlea%ed

The li#tI/ #unction comes #rom the trans#ormers package, and simply promotes an action in the I/ monad to any arbitrary &onadI/ action2 0otice ho- -e don%t really track any state -ith this iteratee) -e don%t care about its result, only its side e##ects2

Implementing head
s a last e(ample, let%s actually implement the head #unction2
headP == Monad $ =< Iteratee a $ @Maybe aA

:A@

headP = continue !o where !o @Chunks :;A = continue !o !o @Chunks @x=xsAA = yield @Just xA @Chunks xsA !o -,? = yield othin! -,?

5ike our sumD #unction, this also -raps an inner IgoI #unction -ith a continue2 Ho-e!er, -e no- ha!e three clauses #or our go #unction2 The #irst handles the case o# Chunks :;2 To 3uote the enumerator docs) 4$hunks [\6 is used to indicate that a stream is still acti!e, but currently has no a!ailable data2 Iteratees should ignore empty chunks2 The second clause handles the case -here -e are gi!en some data2 In this case, -e yield the #irst element in the list, and return the rest as le#to!er data2 The third clause handles the end o# input by returning 0othing2

/,ercises
*2 Write an enumeratee -hich takes he( chars 4eg, ID" DB""FI6 to Word+s2 Its type signature should be -nu$eratee Char CordD $ b2 :2 Write the opposite enumeratee, eg -nu$eratee CordD Char $ b2 >2 $reate a 3uickcheck property that ensures that these t-o #unctions -ork correctly2

Summary

"numeratees are the pipes connecting enumerators to iteratees2 The strange type signature o# an "numeratee hides a lot o# possible po-er2 "specially notice ho- similar their type signatures are to "numerators2 You can merge an "numeratee into an Iteratee -ith SoinI + enu$eratee ++ iteratee2 Don%t #orget that you can use the &onad instance o# Iteratee -hen creating your o-n enumeratees2 You can al-ays compose multiple enumeratees together, such as in http1enumerator2

/numerators
Whereas an Iteratee is a consumer, an "numerator is a producer2 t a high le!el, an "numerator is in #act an !teratee transformer, mo!ing an Iteratee #rom one state to a ne- one2 5et%s look at ho- this -orks2

/,tracting a alue
So #ar, -e%!e -ritten a #e- iteratees, but -e still don%t kno- ho- to e(tract !alues #rom them2 To start, let%s remember that Iteratee is 9ust a ne-type -rapper around 9tep)

:AA

newtype Iteratee a $ b = Iteratee 3 runIteratee == $ @9tep a $ bA 7

First -e need to un-rap the Iteratee and deal -ith the Step !alue inside2 Remember also that Step has three constructors) $ontinue, Yield and "rror2 We%ll handle the "rror constructor by returning our result in an "ither2 Yield already pro!ides the data -e%re looking #or2 The tricky case is $ontinue) here, -e ha!e an iteratee that is still e(pecting more data2 This is -here the "/F constructor comes in handy) it%s our little -ay to tell the iteratee to #inish -hat it%s doing and get on -ith things2 I# you remember #rom abo!e, I said a -ell1beha!ing iteratee -ill ne!er return a $ontinue a#ter recei!ing an "/F7 no- -e%ll see -hy)
extract == Monad $ =< Iteratee a $ b "< $ @-ither 9o$e-xception bA extract @Iteratee $stepA = do step G" $step case step o# Continue k "< do let Iteratee $stepP = k -,? stepP G" $stepP case stepP o# Continue B "< error 8Misbeha%in! iteratee8 Hield b B "< return + Ii!ht b -rror e "< return + 5e#t e Hield b B "< return + Ii!ht b -rror e "< return + 5e#t e

Fortunately, you don%t need to rede#ine this yoursel#) enumerator includes both a run and runW #unction2 5et%s go ahead and use it on our sumD #unction)
$ain = runB su$V <<= print

I# you run this, the result -ill be H2 This emphasi'es an important point) an iteratee is not 9ust how to process incoming data, it is the state of the processing2 In this case, -e ha!en%t done anything to change the initial state o# sumD, so -e still ha!e the initial !alue o# H2 To gi!e an analogy) think o# an iteratee as a machine2 When you #eed it data, you modi#y the internal state but you can%t see any o# those changes on the outside2 When you are done #eeding the data, you press a button and it spits out the result2 I# you don%t #eed in any data, your result is the initial state2

Adding data
5et%s say that -e actually -ant to sum some numbers2 For e(ample, the numbers * to *H2 We need some -ay to #eed that into our sumD iteratee2 In order to approach this, -e%ll once again need to un-rap our Iteratee and deal -ith the Step !alue directly2 In our case, -e kno- -ith certainty that the Step constructor -e used is $ontinue, so it%s sa#e to -rite our #unction as)
su$[ == Monad $ =< Iteratee Int $ Int su$[ = Iteratee + do Continue k G" runIteratee su$V runIteratee + k + Chunks :(//(K;

:AD

But in general, -e -on%t kno- -hat constructor -ill be lying in -ait #or us2 We need to properly deal -ith $ontinue, Yield and "rror2 We%!e seen -hat to do -ith $ontinue) #eed it the data2 With Yield and "rror, the right action in general is to do nothing, since -e%!e already arri!ed at our #inal result 4either a success#ul Yield or an "rror62 So the IproperI -ay to -rite the abo!e #unction is)
su$D == Monad $ =< Iteratee Int $ Int su$D = Iteratee + do step G" runIteratee su$V case step o# Continue k "< runIteratee + k + Chunks :(//(K; B "< return step

/numerator type synonym

What -e%!e done -ith sumF and sum+ is per#orm a trans#ormation on the Iteratee2 But -e%!e done this in a !ery limited -ay) -e%!e hard1coded in the original Iteratee #unction 4sumD62 We could 9ust make this an argument to the #unction)
su$\ == Monad $ =< Iteratee Int $ Int "< Iteratee Int $ Int su$\ ori! = Iteratee + do step G" runIteratee ori! case step o# Continue k "< runIteratee + k + Chunks :(//(K; B "< return step

But since -e al-ays 9ust -ant to un-rap the Iteratee !alue any-ay, it turns out that it%s more natural to make the argument o# type Step, ie)
su$(K == Monad $ =< 9tep Int $ Int "< Iteratee Int $ Int su$(K @Continue kA = k + Chunks :(//(K; su$(K step = returnI step

This type signature 4take a Step, return an Iteratee6 turns out to be !ery common)
type -nu$erator a $ b = 9tep a $ b "< Iteratee a $ b

&eaning sum*H%s type signature could also be e(pressed as)

su$(K == Monad $ =< -nu$erator Int $ Int

/# course, -e need some helper #unction to connect an "numerator and an Iteratee)

apply-nu$ == Monad $ =< -nu$erator a $ b "< Iteratee a $ b "< Iteratee a $ b apply-nu$ enu$ iter = Iteratee + do step G" runIteratee iter runIteratee + enu$ step

5et me repeat the intuition here) the "numerator is trans#orming the Iteratee #rom its initial state to a ne- state by #eeding it more data2 In order to use this #unction, -e could -rite)
runB @apply-nu$ su$(K su$VA <<= print

:AF

This results in AA, e(actly as -e%d e(pect2 But no- -e can see one o# the bene#its o# enumerators) -e can use multiple data sources2 5et%s say -e ha!e another enumerator)
su$(( == Monad $ =< -nu$erator Int $ Int su$(( @Continue kA = k + Chunks :((//NK; su$(( step = returnI step

Then -e could simply apply both enumerators)

runB @apply-nu$ su$(( + apply-nu$ su$(K su$VA <<= print

nd -e -ould get the result :*H2 4Yes, 4* a :H6 T *H _ :*H26 But don%t -orry, you don%t need to -rite this apply"num #unction yoursel#) enumerator pro!ides a ^^ operator -hich does the same thing2 Its type signature is a bit scarier, since it%s a generali'ation o# apply"num, but it -orks the same, and e!en makes code more readable)
runB @su$(( ++ su$(K ++ su$VA <<= print

is a synonym #or __YY, -hich is simply #lip ZZ__2 I #ind ++ the most readable, but Y&&E 4Y&&E62
++

Some -uilt&in enumerators

/# course, -riting a -hole #unction 9ust to pass some numbers to our sum #unction seems a bit tedious2 We could easily make the list an argument to the #unction)
su$(N == Monad $ =< :Int; "< -nu$erator Int $ Int su$(N nu$s @Continue kA = k + Chunks nu$s su$(N B step = returnI step

But no- there%s not e!en anything Int1speci#ic in our #unction2 We could easily generali'e this to)
!eneric9u$(N == Monad $ =< :a; "< -nu$erator a $ b !eneric9u$(N nu$s @Continue kA = k + Chunks nu$s !eneric9u$(N B step = returnI step

nd in #act, enumerator comes built in -ith the enum5ist #unction -hich does this2 enum5ist also takes an Integer argument to indicate the ma(imum number o# elements to stick in a chunk2 For e(ample, -e could -rite)
runB @enu$5ist M :(//&K; ++ su$VA <<= print

4That produces @DA i# you%re counting26 The #irst argument to enum5ist should ne!er a##ect the result, though it may ha!e some per#ormance impact2 Data2"numerator includes t-o other enumerators) enum"/F simply passes an "/F to the iteratee2 concat"nums is slightly more interesting7 it combines multiple enumerators together2 For e(ample)
runB @concat-nu$s

:A+

: > > ;

enu$5ist enu$5ist enu$5ist ++ su$VA

( :(//(K; ( :((//NK; ( :N(//&K; <<= print

This also produces @DA2

Some non&pure input

"numerators are much more interesting -hen they aren%t simply dealing -ith pure !alues2 In the #irst part o# this tutorial, -e ga!e the e(ample o# the user entering numbers on the command line)
!et u$ber == I, @Maybe IntA !et u$ber = do x G" !et5ine i# x == 8E8 then return othin! else return + Just + read x su$N == I, Int su$N = do $aybe u$ G" !et u$ber case $aybe u$ o# othin! "< return K Just nu$ "< do rest G" su$N return + nu$ U rest

We re#erred to this as the pull1model) sum: pulled each !alue #rom get0umber2 5et%s see i# -e can re-rite get0umber to be a pusher instead o# a pullee2
!et u$ber-nu$ == MonadI, $ =< -nu$erator Int $ b !et u$ber-nu$ @Continue kA = do x G" li#tI, !et5ine i# x == 8E8 then continue k else k @Chunks :read x;A <<== !et u$ber-nu$ !et u$ber-nu$ step = returnI step

First, notice that -e check -hich constructor -as passed, and only per#orm any actions i# it -as $ontinue2 I# it -as $ontinue, -e get the line o# input #rom the user2 I# the line is I3I 4our indication to stop #eeding in !alues6, -e do nothing2 You might ha!e thought that -e should pass an "/F2 But i# -e did that, -e%d be pre!enting other data #rom being sent to this iteratee2 Instead, -e simply return the original Step !alue2 I# the line -as not I3I, -e con!ert it to an Int !ia read, create a Stream !alue -ith the $hunks datatype, and pass it to k2 4I# -e -anted to do things properly, -e%d check i# ( is really an Int and use the "rror constructor7 I lea!e that as an e(ercise to the reader26 t this point, let%s look at type signatures)
k @Chunks :read x;A == Iteratee Int $ b

:AB

I# -e simply le#t o## the rest o# the line, our program -ould typecheck2 Ho-e!er, it -ould only e!er read one !alue #rom the command line7 the <<== !et u$ber-nu$ causes our enumerator to loop2 /ne last thing to note about our #unction) notice the b in our type signature2
!et u$ber-nu$ == MonadI, $ =< -nu$erator Int $ b

This is saying that our "numerator can #eed Ints to any Iteratee accepting Ints, and it doesn%t matter -hat the #inal output type -ill be2 This is in general the -ay enumerators -ork2 This allo-s us to create drastically di##erent iteratees that -ork -ith the same enumerators)
intsTo9trin!s == @9how a> Monad $A =< Iteratee a $ 9trin! intsTo9trin!s = @unlines / $ap showA Z#$apZ consu$e

nd then both o# these lines -ork)

runB @!et u$ber-nu$ ++ su$VA <<= print runB @!et u$ber-nu$ ++ intsTo9trin!sA <<= print

/,ercises
*2 Write an enumeratee -hich takes he( chars 4eg, ID" DB""FI6 to Word+s2 Its type signature should be -nu$eratee Char CordD $ b2 :2 Write the opposite enumeratee, eg -nu$eratee CordD Char $ b2 >2 $reate a 3uickcheck property that ensures that these t-o #unctions -ork correctly2

Summary

"numeratees are the pipes connecting enumerators to iteratees2 The strange type signature o# an "numeratee hides a lot o# possible po-er2 "specially notice ho- similar their type signatures are to "numerators2 You can merge an "numeratee into an Iteratee -ith SoinI + enu$eratee ++ iteratee2 Don%t #orget that you can use the &onad instance o# Iteratee -hen creating your o-n enumeratees2 You can al-ays compose multiple enumeratees together, such as in http1enumerator2

/numeratees
The #inal piece in the pu''le is the "numeratee2 This is a combination o# an Iteratee 4it consumes some data6 and an "numerator 4is passes it other data on62 It can be thought o# as a stream transformer, or as a pipe connecting an "numerator to an Iteratee2

0enerali6ing get<um-er/num
:DH

"arlier, -e created a get0umber"num #unction -ith a type signature)

!et u$ber-nu$ == MonadI, $ =< -nu$erator Int $ b

I# you don%t remember, this means get0umber"num produces a stream o# Ints2 In particular, our get0umber"num #unction read lines #rom stdin, con!erted them to ints and #ed them into an iteratee2 It stopped reading lines -hen it sa- a I3I2 But this #unctionality seems like it could be use#ul outside the realm o# Ints2 We may like to deal -ith the original Strings, #or e(ample, or Bools, or a bunch o# other things2 We could easily de#ine a more generali'ed #unction -hich simply doesn%t do the String to Int con!ersion)
line-nu$ line-nu$ x G" i# x == MonadI, $ =< -nu$erator 9trin! $ b @Continue kA = do li#tI, !et5ine == 8E8 then continue k else k @Chunks :x;A <<== line-nu$ line-nu$ step = returnI step

$ool, let%s plug this into our sumIter #unction 4I%!e renamed the sumD #unction #rom the pre!ious t-o parts6)
line-nu$ ++ su$Iter

ctually, that doesn%t type check) line"num produces 9trin!s, and sumIter takes Ints2 We need to modi#y one o# them someho-2
su$Iter9trin! == Monad $ =< Iteratee 9trin! $ Int su$Iter9trin! = Iteratee + do inner9tep G" runIteratee su$Iter return + !o inner9tep where !o == Monad $ =< 9tep Int $ Int "< 9tep 9trin! $ Int !o @Hield res BA = Hield res -,? !o @-rror errA = -rror err !o @Continue kA = Continue + 2strin!s "< Iteratee + do let ints = #$ap read strin!s == 9trea$ Int step G" runIteratee + k ints return + !o step

What -e%!e done here is -rap around the original iteratee2 s usual, -e #irst need to un-rap the Iteratee constructor and the monad to get at the heart o# the Step !alue2 /nce -e ha!e that innerStep !alue, -e pass it to the go #unction, -hich simply trans#orms that !alues in the Stream !alue #rom Strings to Ints2

/ en more general
/# course, it -ould be nice i# -e could apply this trans#ormation to TanyT iteratee2 To start -ith, let%s 9ust pass the inner iteratee and the mapping #unction as parameters2

:D*

$apIter == Monad $ =< @a,ut "< aInA "< Iteratee aIn $ b "< Iteratee a,ut $ b $apIter # innerIter = Iteratee + do inner9tep G" runIteratee innerIter return + !o inner9tep where !o @Hield res BA = Hield res -,? !o @-rror errA = -rror err !o @Continue kA = Continue + 2strin!s "< Iteratee + do let ints = #$ap # strin!s step G" runIteratee + k ints return + !o step

We could call this like)

runB @line-nu$ ++ $apIter read su$IterA <<= print

0othing much to see here, it%s basically identical to the pre!ious !ersion2 What%s #unny is that enumerator comes built in -ith a $ap #unction to do 9ust this, but it has a signi#icantly di##erent type signature)
$ap == Monad $ =< @ao "< aiA "< -nu$eratee ao ai $ b

since)
type -nu$eratee a,ut aIn $ b = 9tep aIn $ b "< Iteratee a,ut $ @9tep aIn $ bA

that%s e3ui!alent to)

$ap == Monad $ =< @a,ut "< aInA "< 9tep aIn $ b "< Iteratee a,ut $ @9tep aIn $ bA

What%s -ith all this e(tra complication in type signature= Well, it%s not necessary #or map itsel#, but it is necessary #or a -hole bunch o# other similar #unctions2 But let%s #ocus on this map #or a second so -e don%t get lost) the #irst argument is the same old mapping #unction -e had be#ore2 The second argument is a Step !alue2 This isn%t really so surprising) in our mapIter, -e took an Iteratee -ith the same parameters, and -e internally 9ust un-rapped it to a Step2 But -hat%s happening -ith that return !alue= Remembering the meanings #or all these datatypes, it%s an Iteratee -hich -ill be #ed a stream o# a,uts and return a Step 4aka, a neiteratee, right=62 This kind o# makes intuiti!e sense) -e%!e introduced a middle man -hich accepts input #rom one source and trans#orms a Step to a ne-er state2 But no- perhaps the trickiest part o# the -hole thing) ho- do -e actually use this map #unction= It turns out that an "numeratee is close enough in type signature to an "numerator that -e can 9ust do)
$ap read ++ su$Iter

But the type signature on that turns out to be a little bit -eird)

:D:

Iteratee 9trin! $ @9tep Int $ IntA

Remembering that an Iteratee is 9ust a -rapped up Step, -hat -e%!e got here is an iteratee that takes Strings and returns an Iteratee, -hich in turn takes Ints and produces an Int2 Ha!ing this #ancy result allo-s us to do one o# our great tricks -ith iteratees) plug in data #rom multiple sources2 For e(ample, -e could plug some Strings into this -hole ugly thing, run it, get a new iteratee -hich takes Ints, #eed that some Ints and get an Int result2 4I# all that -ent o!er your head, don%t -orry2 I -on%t be talking about that kind o# stu## any more26 But o#ten times, -e don't need all o# that po-er2 We 9ust -ant to stick our enumeratee onto our iteratee and get a ne- iteratee2 In our case, -e -ant to attach our map onto the sumIter to produce a ne- iteratee that takes Strings and returns Ints2 In order to do that, -e need a #unction like this)
unnest == Monad $ =< Iteratee 9trin! $ @9tep Int $ IntA "< Iteratee 9trin! $ Int unnest outer = do "" usin! the Monad instance o# Iteratee inner G" outer "" inner == 9tep Int $ Int !o inner where !o @-rror eA = throw-rror e !o @Hield x BA = yield x -,? !o @Continue kA = k -,? <<== !o

We can then run our unholy mess -ith)

runB @line-nu$ ++ unnest + $ap read ++ su$IterA <<= print

nd actually, the unnest #unction is a!ailable in Data2"numerator, and it%s called 9oinI2 So -e should really -rite)
runB @line-nu$ ++ SoinI + $ap read ++ su$IterA <<= print

Skipping
5et%s -rite a slightly more interesting enumeratee) this one skips e!ery other input !alue2
skip == Monad $ =< -nu$eratee a a $ b skip @Continue kA = do x G" head B G" head "" the one wePre skippin! case x o# othin! "< return + Continue k Just y "< do new9tep G" li#t + runIteratee + k + Chunks :y; skip new9tep skip step = return step

What%s interesting about the approach here is ho- similar it looks to an "numerator2 We%re doing a lot o# the same things) checking i# the Step !alue is a $ontinue7 i# it%s not, then simply return it2 Then -e capitali'e on the Iteratee &onad instance, using the head #unction to pop :D>

t-o !alues out o# the stream2 I# there%s no more data, -e return the original $ontinue !alue) 9ust like -ith an "numerator, -e don%t gi!e an "/F so that -e can #eed more data into the iteratee later2 I# there is data, -e pass it o## to the iteratee, get our ne- step !alue and then loop2 nd -hat%s cool about enumeratees is -e can chain these all together)
runB @line-nu$ ++ SoinI + skip ++ SoinI + $ap read ++ su$IterA <<= print

Here, -e read lines, skip e!ery other input, con!ert the Strings to Ints and sum them2

%eal life e,amples= http&enumerator package

I started -orking on these tutorials as I -as -orking on the http1enumerator package2 I think the usage o# enumeratees there is a great e(planation o# the bene#its they can o##er in real li#e2 There are three di##erent -ays the response body can be broken up)

$hunked encoding2 In this case, the -eb ser!er gi!es a he( string speci#ying the length o# the ne(t chunk and then that chunk2 t the end, it sends a H to indicate the end o# that response2 $ontent length2 Here, the -eb ser!er sends a header be#ore any o# the body is sent speci#ying the total length o# the body2 0othing at all2 In this case, the response body lasts until an end1o#1#ile2

In addition, the body may or may not be ;`IP compressed2 We end up -ith the #ollo-ing enumeratees, each -ith type signature -nu$eratee Byte9trin! Byte9trin! $ b) chunked"ncoding, content5ength and ung'ip2 We then get to do something akin to)
let parseBody x = i# @8trans#er"encodin!8> 8chunked8A Zele$Z response*eaders then SoinI + chunked-ncodin! ++ x else case $len o# Just len "< SoinI + content5en!th len ++ x othin! "< x "" no enu$eratee applied at all let deco$press x = i# @8content"encodin!8> 8!'ip8A Zele$Z response*eaders then SoinI + un!'ip ++ x else x runB + socket-nu$erator ++ parseBody + deco$press + bodyIteratee

We create a chain) the data #rom the ser!er is #ed into the parseBody #unction2 In the case o# chunked encoding, the data is processed appropriately and then headers are #iltered out2 I# -e are dealing -ith content length, then only the speci#ied number o# bytes are read2 nd in the case o# neither o# those, parseBody is a no1op2 Whate!er the case may be, the ra- response body is then #ed into decompress2 I# the body is ;`IPed, then ung'ip in#lates it, other-ise decompress is a no1op2 Finally, the parsed and in#lated data is #ed into the user1supplied bodyIteratee #unction2 The user remains bliss#ully una-are o# any steps the data took to get to him<her2 :D@

/,ercises
*2 Write an enumeratee -hich takes he( chars 4eg, ID" DB""FI6 to Word+s2 Its type signature should be -nu$eratee Char CordD $ b2 :2 Write the opposite enumeratee, eg -nu$eratee CordD Char $ b2 >2 $reate a 3uickcheck property that ensures that these t-o #unctions -ork correctly2

Summary

"numeratees are the pipes connecting enumerators to iteratees2 The strange type signature o# an "numeratee hides a lot o# possible po-er2 "specially notice ho- similar their type signatures are to "numerators2 You can merge an "numeratee into an Iteratee -ith SoinI + enu$eratee ++ iteratee2 Don%t #orget that you can use the &onad instance o# Iteratee -hen creating your o-n enumeratees2 You can al-ays compose multiple enumeratees together, such as in http1enumerator2

&& /<( O! +O# &&

:DA