Techniques for Anaphora Resolution: A Survey

Tejaswini Deoskar CS 674 5/17/2 4

1 !ntro"uction
Anaphora resolution is one of the #ore prolific areas of research in the $%& co##unity' Anaphora is a u(iquitous pheno#enon in natural lan)ua)e an" is thus require" in al#ost every conceiva(le $%& application' There is a *i"e variety of *or+ in the area, (ase" on various theoretical approaches' Si#ply state", anaphora resolution is the pro(le# of fin"in) the reference of a noun phrase' This noun phrase can (e a fully specifie" $& -"efinite or in"efinite., a pronoun, a "e#onstrative or a refle/ive' Typically this pro(le# can (e "ivi"e" into t*o parts 0 -i. 1in"in) the co2reference of a full $& -co##only referre" to as co2reference resolution. -ii. 1in"in) the reference of a pronoun or refle/ive -co##only referre" to as anaphora resolution.' The secon" part of the pro(le# #ay (e thou)ht of as a su(set of the first' Thou)h there are si#ilarities in the t*o pro(le#s, there are si)nificant "ifferences in the function of pronouns an" that of full $&3s in natural lan)ua)e "iscourse' Thus si)nificant "ifference is seen in their "istri(ution too' 1or instance, a (roa" heuristic is that pronouns usually refer to entities that are not farther than 224 sentences, *hile "efinite $&3s can refer to entities that are quite far a*ay' !n this paper, ! e/a#ine in "etail various approaches in this area, *ith #ore focus on anaphora resolution than noun phrase co2reference resolution' ! loo+ at these approaches fro# the point of vie* of un"erstan"in) the state of art of the fiel" an" also fro# the vie* of un"erstan"in) the interaction (et*een $%& research in the co#putational lin)uistics co##unity an" theoretical lin)uistics' 5ue to the secon" )oal, ! have loo+e" at so#e classical results in the fiel" -such as 6o((s 1777. -even thou)h they are "ate"., since they *ere #otivate" #ainly (y lin)uistic consi"erations' ! also note that #ost +no*le")e sources in anaphora resolution research have "ra*n on structural in"ications of infor#ation pro#inence, (ut have not consi"ere" other sources such as tense an" aspect, *hich #ay prove to (e i#portant +no*le")e sources'

2 Relevance of this pro(le#

2.1 Relevance to NLP 1ro# the $%& point of vie*, anaphora resolution is require" in #ost pro(le#s such as question2ans*erin), infor#ation e/traction, te/t su##ari8ation, "ialo)ue interpretation syste#s, etc' Thus to a lar)e e/tent, successful en"2to2en" syste#s require a successful anaphora resolution #o"ule' This i#plies that the various for#s of preprocessin) require" in anaphora resolution syste#s, such as noun phrase i"entification, #orpholo)ical processin), se#antic class "eter#ination, etc' are equally relevant to the issue' 2.2 Relevance from the Linguistics point of view 9in"in) Theory is one of the #a:or results of the principles an" para#eters approach "evelope" in Cho#s+y -17;1. an" is one of the #ainstays of )enerative lin)uistics' The 9in"in) Theory "eals *ith the relations (et*een no#inal e/pressions an" possi(le antece"ents' !t atte#pts to provi"e a structural account of the co#ple#entarity of "istri(ution (et*een pronouns, refle/ives an" R2e/pressions1' Con"ition A: A refle/ive #ust (e (oun" in its )overnin) cate)ory2 Con"ition 9: A pronoun #ust (e free in its )overnin) cate)ory Con"ition C: An R2e/pression #ust (e free' 6o*ever, this for#ulation of the 9in"in) Theory runs into #a:or pro(le#s e#pirically' Currently, various #o"ifications to the stan"ar" 9in"in) Theory e/ist as also so#e co#pletely "ifferent fra#e*or+s -such as Reinhart an" Reulan" -1774.3s se#antic pre"icate (ase" theory. to e/plain (in"in) pheno#enon' 2.3 Dichotomy between Linguistic and NLP Research The 9in"in) Theory -an" its various for#ulations. "eals only *ith intrasentential anaphora, *hich is a very s#all su(set of the anaphoric pheno#enon that practical $%& syste#s are intereste" in resolvin)' A #uch lar)er set of anaphoric pheno#enon is the resolution of pronouns intersententially' This pro(le# is "ealt *ith (y 5iscourse Representation Theory an" #ore specifically (y Centerin) Theory -<ros8 et al', 1775.' Centerin) Theory, (ein) #ore co#putationally tracta(le than #ost lin)uistic theories, has (een a popular theoretical fra#e*or+ to "eter#ine the "iscourse pro#inence -an" hence ran+in). of a potential antece"ent'
1

An R2e/pression is a referrin) e/pression li+e =ohn, 9ill, The "o), etc' *hich i"entifies an entity int eh real *orl"' 2 !n 9in"in) Theory, "istance is #easure" usin) the notion of )overnin) cate)ory' 9 is a )overnin) cate)ory for A iff 9 is the #ini#al cate)ory containin) A, a )overnor of A an" a su(:ect accessi(le to A'

>any $%& (ase" approaches have incorporate" syntactic an" pra)#atic constraints fro# lin)uistic theory -(in"in) theory an" "iscourse theory. into their al)orith#s' 9ut lin)uistic research has lar)ely i)nore" the fin"in)s o(taine" fro# co#putational anaphora resolution' This is (ecause $%& research focuses on the "evelop#ent of (roa" covera)e syste#s, *hile lin)uistics research typically focuses on very narro* "o#ain pheno#enon' 1or e/a#ple, a structural theory li+e the stan"ar" 9in"in) Theory "oes not concern itself *ith lo)ophoric pronouns -pronouns that )et their reference fro# conte/t in the "iscourse., *hile for an $%& syste#, (oth types of pronouns -those that fall *ithin the "o#ain of 9T an" lo)ophoric pronouns. are equally i#portant an" are han"le" (y the sa#e syste#' Another aspect is that until the recent past an" possi(ly even no*, lin)uistic theory has not (een concerne" *ith its co#putational or i#ple#entational feasi(ility in practical syste#s, lea"in) to the theories (ein) lar)ely i)nore" (y the $%& co##unity' ?hile $%& syste#s rely on the stan"ar" 9in"in) Theory for intra2sentential syntactic constraints, current lin)uistic theories "ealin) *ith (in"in) have to account for #uch #ore co#ple/ (in"in) pheno#enon in lan)ua)es other than @n)lish' %an)ua)es *ith so2 calle" long distance an" short distance refle/ives are fairly co##on' @/a#ples are 5utch, !celan"ic, 1innish, >arathi, etc' %on) "istance refle/ives are those that cannot have a coreferrin) antece"ent in a local "o#ain -li+e a clause., (ut nee" an antece"ent in a lar)er "o#ain' Short2"istance refle/ives are those that nee" an antece"ent in the local "o#ain -li+e @n)lish hi#self/herself, etc'.' There is al#ost no $%& research on anaphoric pheno#enon in such lan)ua)es, *hich appears to (e a #uch #ore "ifficult pro(le# to tac+le' !n a""ition to a #uch #ore co#ple/ "istri(ution, lon) "istance refle/ives have no )en"er or nu#(er properties, crosslin)uistically' !t is also co##on that they overlap in for# *ith a personal pronoun in the lan)ua)e, #a+in) the pro(le# #uch #ore co#plicate"'

2 9ac+)roun"
Research in anaphora resolution falls into t*o (roa" cate)ories2 +no*le")e2rich approaches, an" +no*le")e2poor approaches' @arlier syste#s ten"e" to (e +no*le")e2 rich' 6o*ever, *ith the pressure to "evelop fully auto#ate" syste#s, the a"vent of cheaper an" #ore corpus (ase" $%& tools li+e part of speech ta))ers an" shallo* parsers, an" "evelop#ent of #achine learnin) an" statistical techniques, #o"ern syste#s have ten"e" to use a +no*le")e2poor approach'

2'1 Ano*le")e2rich Approaches

@arly research in anaphora resolution usually e#ploye" a rule (ase", al)orith#ic approach, an" *as )enerally +no*le")e2rich' !t *as (ase" on co##only o(serve" heuristics a(out anaphoric pheno#enon' !t usually assu#e" a full an" correctly parse" input' @valuation *as typically carrie" out (y han" on a s#all set of evaluation e/a#ples' 4

9ase" on the +in" of +no*le")e e#ploye", these approaches can (roa"ly (e "ivi"e" into t*o cate)ories' 2.1.1 ynta!"based approaches These approaches typically assu#e the e/istence of a fully parse" syntactic tree an" traverse the tree loo+in) for antece"ents an" applyin) appropriate syntactic an" #orpholo)ical constraints on the#' 6o((s 1777 is a classical result usin) this approach' #obb$s %lgorithm 6o((s 1777 *as one of the first results *hich o(taine" an i#pressive accuracy in pronoun resolution' 6e use" a naBve al)orith# that *or+s on the surface parse trees of the sentences in the te/t' !ntrasententially, the al)orith# perfor#e" a left2to2ri)ht (rea"th2 first search -every no"e of "epth n is visite" (efore any no"e of "epth nC1., )ivin) preference to closer antece"ents' !ntersententially too, a left2to2ri)ht (rea"th2first search, *hich i#ple#ents a preference for su(:ects to (e the antece"ents is i#ple#ente"4' The al)orith# collects possi(le antece"ents an" then chec+s for ones that #atch the pronoun in nu#(er an" )en"er' !t also ta+es syntactic constraints into consi"eration' The t*o #ost i#portant constraints, (ase" on Con"ition 9 of the 9in"in) Theory are 1' a non2refle/ive pronoun an" its antece"ent #ay not occur in the sa#e si#ple/ sentence' 2' The antece"ent of a pronoun #ust prece"e or co##an" the pronoun' 9oth these constraints are han"le" (y the al)orith# itself -Step 2,4 an" ; respectively, in 6o((s 1777. The al)orith# *as evaluate" on 4 e/a#ples of pronoun occurrences in three "ifferent te/ts' The pronouns covere" *ere he, she, it an" they' Dverall, the al)orith# resolve" ;;'4E of the cases correctly' To)ether *ith a fe* selectional constraints that the al)orith# i#ple#ente", it achieve" a perfor#ance of 71'7E' 6o*ever these results are for cases *here there *as no conflict -that is, there *ere no #ultiple antece"ents that ha" to (e chosen fro#.' !n cases *here there *as a choice of antece"ents to (e #a"e, the perfor#ance *as ;1';E' @ven thou)h these results are i#pressive, the naBve al)orith# fails on a nu#(er of +no*n cases, such as sentence prono#inali8ation as illustrate" in -1. -1. 1or" *as in trou(le, an" he +ne* it' 6o((s 1777

!t also cannot "eal *ith the class of picture noun e/a#ples, such as -2. (2) John saw a picture of him.
4

Ho

s !"##

The "etaile" al)orith# is )iven in 6o((s 1777 an" is not repeate" here for lac+ of space'

The al)orith# searches the tree to the ri)ht of the pronoun, in or"er to "eal *ith antece"ents *hich follo* the pronoun' 6o*ever, there is no search (elo* the S or $& no"es causin) it to fail on sentences li+e -4. -4. >aryi sac+e" out in his: apart#ent (efore Sa#: +ic+e" heri out' 6o((s 1777

The evaluation *as "one #anually, as also all preprocessin)' This #a+es it "ifficult to co#pare perfor#ance *ith other approaches "evelop later' 6o*ever, 6o((s al)orith# re#ains the #ain al)orith# that #any syntactically (ase" approaches still use, even thou)h they au)#ent it *ith other +no*le")e sources' 2.1.2 Discourse"&ased %pproaches Another tra"itional #etho" of o(tainin) the reference of pronouns is "iscourse (ase", calle" the Centerin) Theory' This theory #o"els the attentional salience of "iscourse entities, an" relates it to referential continuity' Centerin) Theory can (e su##ari8e" as )iven (elo* -(ase" on Ai((le 2 1. a' 1or each utterance in a "iscourse, there is only one entity that is the center of attention' (' The center of utterance is #ost li+ely to (e prono#inali8e" -Rule 1 of CT.' c' Consecutive utterances in a "iscourse ten" to #aintain the sa#e entity as the center of attention -Rule 2 of CT.' CT2(ase" approaches are attractive fro# the co#putational point of vie* (ecause the infor#ation they require can (e o(taine" fro# structural properties of utterances alone, as oppose" to costly se#antic infor#ation' The #ost *ell2+no* result usin) this approach is that of 9renan, 1rie"#an an" &ollar" 17;7 -91&.' 91& #a+es use of syntactic an" #orpholo)ical consi"erations li+e nu#(er an" )en"er a)ree#ent to eli#inate unsuita(le can"i"ates, (ut uses centerin) principles to ran+ potential can"i"ates' A #anual co#parison of 6o((3s naBve al)orith# -6o((s 1777. an" 91& , sho*e" that the t*o perfor#e" equally over a fictional "o#ain of 1 utterances' 6o*ever, the 6o((s al)orith# outperfor#e" the CT2(ase" al)orith# -;7E as co#pare" to 77E. in a "o#ain consistin) of ne*spaper articles -Tetreault 2 1.' The #ain "ra*(ac+ of the CT is its preference for intersentential references as oppose" to intrasentential' A #o"ification of the CT al)orith# *hich "iscar"s the notion of (ac+*ar" an" for*ar" loo+in) centers -lists *hich +eep trac+ of the centers of previous an" follo*in) utterances., (ut uses the i"ea of #o"elin) the attentional state of the current utterance is the S2list approach "escri(e" in Stru(e 177;' The S2list is "ifferent fro# the CT2(ase" approach in that it can inclu"e ele#ents fro# (oth previous an" current utterances *hile the CT2(ase" approach uses only the previous utterance' The S2 list ele#ents are also or"ere" not (y )ra##atical role -as in CT. (ut (y infor#ation status an" then (y surface or"er'

!n another #anual co#parison, the S2list approach outperfor#s the CT2(ase" approach (y ;5E as co#pare" to 76E -Tetreaut 2 1. Tetreault 2 1 presents a #o"ification of the CT2(ase" approach calle" the %eft2Ri)ht Centerin) approach -%RC.' &sycholin)uistic research clai#s that listeners try to resolve references as soon as they hear an anaphor' !f ne* infor#ation appears that contra"icts this choice of antece"ent, they reanaly8e an" fin" another antece"ent' This psycholin)uistic fact is #o"ele" in the %RC' The %RC *or+s (y first tryin) to fin" an antece"ent in the current utterance' !f this "oes not *or+, then antece"ents in previous utterances are consi"ere", )oin) fro# left2to2ri)ht *ithin an utterance' !n a further #o"ification -%RC21., infor#ation a(out the su(:ect of the utterance is also enco"e"' 2.1.3 #ybrid %pproaches These approaches #a+e use of a nu#(er of +no*le")e sources, inclu"in) syntactic, "iscourse, #orpholo)ical, se#antic, etc' to ran+ possi(le antece"ents' Dne of the #ost *ell +no*s syste#s usin) this approach is that of %appin an" %eass -1774.' They use a #o"el that calculates the "iscourse salience of a can"i"ate (ase" on "ifferent factors that are calculate" "yna#ically an" use this salience #easure to ran+ potential can"i"ates' They "o not use costly se#antic or real *orl" +no*le")e in evaluatin) antece"ents, other than )en"er an" nu#(er a)ree#ent' 9efore the salience #easures are applie", a syntactic an" #orpholo)ical constraint filter eli#inates can"i"ates that "o not satisfy syntactic constraints of the 9in"in) Theory or constraints of )en"er an" nu#(er a)ree#ent' Salience is calculate" (ase" on several intuitive factors that are inte)rate" into the al)orith# quite ele)antly' 1or e')' recency is enco"e" in the fact that salience is "e)ra"e" (y half for a ne* sentence' @quivalence classes are create" *hich contain all the referents that for# an anaphoric chain' @ach equivalence class has a *ei)ht associate" *ith it, *hich is the su# of all salience factors in *hose scope at least one #e#(er of the equivalence class lies -%aapin an" %eass 1774.' The equivalence classes, an" the saliency re"uction #easure for# a "yna#ic syste# for co#putin) the relative attentional salience of a referent' The factors that the al)orith# uses to calculate salience are )iven "ifferent *ei)hts accor"in) to ho* relevant the factor is' These factors are: Sentence Recency -1 . Su(:ect e#phasis -; . 2This factor enco"es the fact that su(:ects are #ore salient than other )ra##atical roles' @/istential e#phasis -7 . 2 $o#inal in an e/istential construction -There is F. is salient' Accusative e#phasis -5 . 2 5irect o(:ect is salient, (ut not as #uch as a su(:ect' !n"irect D(:ect an" o(lique co#ple#ent e#phasis 0 4 2 !n"irect o(:ect is less salient than "irect o(:ect an" is penali8e"' 6ea" noun e#phasis -; . 2 The hea" noun in a co#ple/ noun phrase is #ore salient than a non2hea", *hich is penali8e"' 6

$on2a"ver(ial e#phasis' -5 . 2 This factor penali8es $&3s in a"ver(ial constructions'

Thus all these salience #easures are #ostly structural an" "istance (ase"' 6o*ever, the salience calculations use" (y Centerin) approaches, *hich (asically #easure the center of an utterance (ase" on )ra##atical role -su(:ect G e/istential pre"icate no#inalG o(:ectG in"irect o(:ect or o(lique G "e#arcate" a"ver(ial &&. is capture" in this approach too' 6o*ever, the %appin an" %eass approach )ives *ei)hts to these factors, unli+e centerin) approaches' 2.1.' (orpus based %pproaches Charnia+, 6ale, an" <e -177;. present a statistical #etho" for resolvin) pronoun anaphora' They use a very s#all trainin) corpus fro# the &enn ?all Street =ournal Tree2 (an+ #ar+e" *ith co2reference resolution' They (ase their #etho" on 6o((3s al)orith# (ut au)#ent it *ith a pro(a(ilistic #o"el' The +in"s of infor#ation that they (ase their pro(a(ilistic #o"el are 1' 5istance (et*een the pronoun an" its antece"ent -enco"e" in the 6o((s Al)orith# itself. 2' Syntactic Constraints -also enco"e" in the 6o((s Al)orith# itself. 4' The actual antece"ents, *hich )ives infor#ation re)ar"in) nu#(er )en"er, an" ani#aticity' 4' !nteraction (et*een the hea" constituent of the pronoun an" the antece"ent' 5' The antece"ent3s >ention Count 2 the #ore nu#(er of ti#es a referent has occurre" in the "iscourse (efore, the #ore li+ely it is to (e the antece"ent' They assu#e that all these factors are in"epen"ent' Their e/peri#ent first calculates the pro(a(ilities fro# the trainin) corpus an" then uses these to resolve pronouns in the test corpus' Their "ata consiste" of 2477 pronouns - he, she an" it.' They use" a 1 2fol" cross vali"ation an" o(taine" results of ;2'7 percent correct' They also investi)ate the relative i#portance of each of the a(ove factors in fin"in) the pro(a(ility (y runnin) the pro)ra# incre#entally' They o(tain an accuracy of 65'4E usin) :ust the "istance an" syntactic constraints as i#ple#ente" in the 6o((s al)orith#' After a""in) *or" infor#ation to the #o"el -)en"er an" ani#aticity. the perfor#ance rises to 75'7E' A""in) +no*le")e a(out )overnin) cate)ories -hea"e"ness. i#prove" perfor#ance (y only 2'2E' 1inally, a""in) infor#ation a(out the #ention count i#prove" accuracy to the final value of ;2'7E' This #ention count appro/i#ately enco"es infor#ation a(out the topic of a se)#ent of "iscourse' Thus it is seen that i"entifyin) the topic accurately can i#prove perfor#ance su(stantially' 6o*ever, it coul" have (een #ore effective to enco"e infor#ation a(out the topic in ter#s of )ra##atical roles - as is "one (y centerin) approaches.' Since the al)orith# assu#es a "eep syntactic parse in any case, this *oul" not have (een an e/tra e/pense'

Thus *ithin the "o#ain of +no*le")e2(ase" approaches, there are t*o types: 1' The first approach *or+s (y first eli#inatin) so#e antece"ent can"i"ates (ase" on constraints -li+e syntactic constraints. an" then choosin) the (est of the re#ainin) (ase" on so#e factors such as centerin)' 2' The other approach consi"ers all can"i"ates as equal (ut #a+es "ecisions on ho* plausi(le a can"i"ate is (ase" on "ifferent factors' >it+ov -1777. co#pares the perfor#ance of these t*o approaches (y constructin) al)orith#s (ase" on the t*o approaches an" usin) the sa#e factors -para#eters. in (oth' 6e calls the first approach, *hich use" constraints an" preferences the !nte)rate" Approach -!A.' The secon" approach *hich uses A! uncertainty reasonin) approaches, is calle" the HRA -Hncertainty Reasonin) Approach.' So#e of the factors that he uses are 1' <en"er an" nu#(er a)ree#ent 2' Syntactic parallelis# 0 &reference is )iven to antece"ents *ith the sa#e syntactic function as the anaphor' 4' Se#antic consistency (et*een the anaphor an" antece"ent' 4' Se#antic &arallelis#: Those antece"ents are favoure" *hich have the sa#e se#antic role as the antece"ent' 5' Su(:ects 0 The su(:ect of the previous utterance is preferre"' 6' D(:ect preference of so#e ver(s 7' Su(:ect preference of so#e ver(s ;' Repetition2 repeate" $&3s are preferre" as antece"ents' 6e uses 144 occurrences of the pronoun Iit3 an" tests (oth approaches' 6e fin"s that (oth the !A an" HRA have co#para(le perfor#ances of ;4E an" ;2E' 6e then co#(ines (oth approaches to achieve a (etter accuracy than (oth' 6e conclu"es that anaphor resolution syste#s shoul" pay attention not only to the factors use" in resolution (ut also the co#putational strate)y for their application'

2.2 )nowledge"poor %pproaches

!n recent years, there is a tren" to*ar"s +no*le")e2poor approaches that use #achine learnin) techniques' Soon, $), %i# -2 1. o(taine" results that *ere co#para(le to non2learnin) techniques for the first ti#e' They resolve" not :ust pronouns (ut all "efinite "escriptions' They use" a s#all annotate" corpus to o(tain trainin) "ata to create feature vectors' These trainin) e/a#ples *ere then )iven to a #achine learnin) al)orith# to (uil" a classifier' The learnin) #etho" they use" is a #o"ification of C4'5 -Juinlan 1774. calle" C5, a "ecision tree (ase" al)orith#' An i#portant point to note a(out their syste# is that it is an en"2to2en" syste# *hich

inclu"es sentence se)#entation, part2of2speech ta))in), #orpholo)ical processin), noun phrase i"entification an" se#antic class "eter#ination' The feature vector consists of 12 features' 1' There *ere 5 features *hich in"icate" the type of noun phrase2 "efinite $&, "e#onstrative, pronouns, or proper na#es' 2' They ha" a "istance feature, *hich capture" the "istance (et*een an anaphoric $& an" its coreferrent' 4' $u#(er a)ree#ent 4' <en"er A)ree#ent 5' Se#antic class a)ree#ent *hich inclu"e" (asic an" li#ite" se#antic classes such as #ale, fe#ale, person, or)ani8ation, location, "ate, ti#e, #oney,etc' 6' 1eatures for i"entification of &roper na#es such as alias feature, proper2na#e feature, etc' 7' Appositive 1eature ?ith these features, they achieve" a recall of 5;'6E, precision of 67'4E, an" an 12 #easure of 62'6E 'They evaluate" their syste# on the trainin) an" test corpora fro# >HC26 an" >HC27' They #easure the contri(ution of each of the features to the perfor#ance of their syste# an" fin" out that the features that contri(ute the #ost are alias, string$match, an" appositi%e' Alias an" strin) #atch are features use" to "eter#ine co2referrin) "efinite $&3s, *hile the appositive feature i"entifies an appositive construction' All these features are concerne" *ith full $& co2reference' There are al#ost no features *hich "eal *ith pronouns specifically, e/cept for the )eneric nu#(er/)en"er a)ree#ent features' The error analysis is also focuse" only on noun phrase resolution an" "oes not tal+ a(out pronouns' The lac+ of syntactic features, an" salience #easures for pronoun resolution in"icates that the errors in pronoun resolution #ust (e contri(utin) to a lot of the lo* perfor#ance' Car"ie an" $) -2 2. trie" to #a+e up for the lac+ of lin)uistically #otivate" features in Soon, et al'3s approach' They increase" the feature set fro# 12 to 54' They intro"uce" a""itional le/ical, se#antic an" +no*le")e (ase" features, *ith a lar)e nu#(er of a""itional )ra##atical features, that inclu"e" a variety of lin)uistic constraints an" preferences' The a""itional features fall into the follo*in) cate)ories 1' !ncrease" le/ical features to allo* #ore co#ple/ strin) #atchin) operations 2' 1our ne* se#antic features to allo* finer se#antic co#pati(ility tests 4' A fe* positional features that #easure "istance in ter#s of nu#(er of para)raphs' 4' >ost i#portantly, 26 ne* features to allo* the acquisition of #ore sophisticate" syntactic coreference resolution rules' a' $& type (' <ra##atical Role

c' A)ree#ent in nu#(er an" )en"er "' 9in"in) Constraints, etc' Surprisin)ly, usin) these features, precision "roppe" si)nificantly, especially on co##on nouns in co#parison *ith pronouns an" proper na#es' !n a #o"ifie" version of the syste#, they han" selecte" features out of the ori)inal set to increase precision on co##on nouns' A result of this *as that there *as an increase in the precision for co##on nouns (ut a lar)e "rop in precision for pronouns' 6o*ever, overall, the syste# *ith the han"2selecte" features "i" consi"era(ly (etter than Soon3s syste# *ith 12 features *ith 12#easures of 7 '4E for >HC6' Car"ie an" $) 2 2 also trie" out three e/tra2lin)uistic #o"ifications to the Soon Al)orith# an" )ot statistically si)nificant i#prove#ent in perfor#ance' This supports the >it+ov -1777. results, in the "o#ain of #achine learnin) approaches, sho*in) that co2reference syste#s can (e i#prove" (y Kthe proper interaction of classification, trainin) an" clusterin) techniquesL -Car"ie an" $) 2 2.'

4 Co#parison of Approaches
A (roa" tren" seen in the research surveye" is that the ol"er syste#s "ealt *ith the resolution of only prono#inal anaphors' The ne* +no*le")e2poor #achine learnin) approaches are #ore concerne" *ith the (i))er pro(le# of noun phrase coreference resolution' This coul" (e (ecause of the inclusion of this tas+ in the Si/th an" Seventh >essa)e Hn"erstan"in) Conferences ->HC26 an" >HC27., *hich )ave an i#petus to research on this pro(le#' The +no*le")e2rich approaches typically use #anually preprocesse" input "ata' This preprocessin) can ta+e various for#s, such as #anual re#oval of pleonastic pronouns, etc' The +no*le")e2poor syste#s usually are en"2to2en" syste# that auto#ate all the preprocessin) sta)es too such as noun2phrase i"entification, #orpholo)ical an" se#antic class i"entification, etc' >it+ov -2 1. hol"s that inaccuracies in the preprocessin) sta)e in anaphora resolution lea" to a si)nificant overall re"uction in the perfor#ance of the syste#, for syste#s that use auto#ate" preprocessin)' 5ue to this, it is not entirely fair to co#pare #achine learnin) approaches that use auto#ate" preprocessin) *ith +no*le")e2(ase" techniques that ha" the a"vanta)e of havin) #anually preprocesse" input availa(le' The sa#e hol"s for co#parison (et*een the +no*le")e2poor approaches an" #achine learnin) approaches "iscusse" a(ove'

4 @valuation of &erfor#ance
!n the "o#ain of pronoun anaphora resolution, #ost stu"ies cover "ifferent sets of pronouns an" anaphors *hile e/clu"in) others' So#e cover only anaphors -*hich point to prece"in) "iscourse for their reference., *hile others also inclu"e cataphors -*hich point to su(sequent "iscourse.' They also use "ifferent "ata -fro# "ifferent )enres. for evaluation' 5ue to this, it is "ifficult to evaluate their co#parative perfor#ance' To facilitate co#parison, 9yron -2 1. su))ests so#e )ui"elines for researches to follo* *hile reportin) results' !n or"er to #easure ho* *ell a technique perfor#s *ith respect to the ulti#ate )oal of resolvin) all referential pronouns, she su))ests usin) a #etric calle" the Referential Rate -RR., in a""ition to the stan"ar" ones of &recision an" Recall' RRM C / -T C@. *here CM nu#(er of pronouns resolve" correctly T M all pronouns in the evaluation set @M all e/clu"e" referential pronouns Hsin) RR *ill re*ar" techniques that cover a (roa" ran)e of pronouns' She also proposes a stan"ar" "isclosure for#at in *hich: 1' The pronoun types inclu"e"/e/clu"e" in the stu"y are rea"ily apparent 2' Cate)ories an" ite#i8e" counts of e/clu"e" to+ens are clearly sho*n 4' RR can (e calculate" (ecause the e/clusions are enu#erate"' 1ollo*in) these )ui"elines can facilitate a realistic evaluation an" co#parison of results'

5 Aspect as an in"icator of (ac+)roun"in) or fore)roun"in)

All of the approaches a(ove that have ta+en into consi"eration attentional salience as a #easure to in"icate the li+elihoo" of an $& (ein) referre" to (y a pronoun, have only use" syntactic factors to #easure salience' ?hile )ra##atical role is an i#portant factor in "eter#inin) salience, there is evi"ence that so#e other factors #ay also play a role' Dne such factor that is not co#putationally "ifficult to #easure is aspect' 6arris an" 9ates -2 2. sho* that aspect plays a #a:or role in the the (ac+)roun"in) or fore)roun"in) -salience. of a no#inal' They e/a#ine sentences in *hich a prono#inal appears in a #ain clause an" refers ahea" to a full noun phrase that occurs later' They fin" that there is a statistically si)nificant "ifferent in the accepta(ility of sentences that use so#e types of aspect such as pro)ressive or pluperfect aspect, than in sentences that "o not use this aspect'

11

Thus in the follo*in) sentences -1., *hich has a si#ple past tense in the #ain clause is less accepta(le than -2., *hich has a pro)ressive aspect' -1ro# 6arris an" 9ates -2 -1. -2. 2..

6ei threatene" to leave *hen 9illyi notice" that the co#puter ha" "ie"' 6ei *as threatenin) to leave *hen 9illyi notice" that the co#puter ha" "ie"'

!n another e/peri#ent ai#e" at stu"yin) the infor#ational pro#inence of su(:ects in #ain clauses *hich have pro)ressive aspect, they )ave su(:ect sentences such as -4. an" -4. an" as+e" the# to orally pro"uce a continuation sentence' The i"ea (ehin" this e/peri#ent *as that the #ore attentionally pro#inent $& -=ac+ or 1ran+. *ill (e #ore li+ely use" in a follo*in) sentence' -1ro# 6arris an" 9ates -2 -2. 2. .

>ain Clause, Si#ple &ast Tense Case: &rank ru ed his tired eyes in fatigue when Jack spotted the car ill oard. ehind a

-4.

>ain Clause, &ro)ressive Aspect Case: &rank was ru ill oard. ing his tired eyes in fatigue when Jack spotted the car ehind a

-5.

Su(or"inate Clause Case: 'hile &rank ru ill oard. ed his tired eyes in fatigue, Jack spotted the car ehind a

Their results in"icate" that su(:ects *ere #ore li+ely to pro"uce a sentence *ith the secon" character3s na#e -=ac+., in the case of the su(or"inate clause sentence -5.' This is not surprisin) since it is *ell +no*n that the su(:ect of the #ain clause is #ore pro#inent than that su(:ect of a su(or"inate clause' This infor#ation is in"ee" enco"e" in so#e for# or the other #ost $%& syste#s that *e have seen a(ove' !n the #ain clause case, the su(:ects *ere #ore li+ely to pro"uce sentences *ith the first characters na#e than the secon" characters na#e' A)ain this is not surprisin)' 6o*ever, the pro)ressive aspect con"ition -4. *as si)nificantly "ifferent than the si#ple past tense con"ition -4., *ith su(:ects (ein) #ore li+ely to pro"uce a sentence *ith the secon" character3s na#e in the pro)ressive aspect con"ition than in the si#ple past tense clause con"ition' This sho*s that the aspect of #ain clause has a si)nificant effect on the pro#inence of the su(:ect of the #ain clause2 in particular, havin) a pro)ressive aspect in

12

the #ain clause lo*ers the pro#inence of the #ain clause su(:ect to lo*er than that of the su(or"inate clause su(:ect' This +no*le")e source has not (een e/ploite" at all in anaphora resolution research' The 9ates an" 6arris -2 2. paper only contrasts pro)ressive an" si#ple past tense' 6o*ever, it is li+ely that si)nificant "ifferences in infor#ation status are seen for other tenses an" aspects'

5 Conclu"in) Re#ar+s an" 1uture ?or+

?hile ol"er anaphora resolution syste#s "i" not follo* a co##on reportin) an" evaluation strate)y, #a+in) it "ifficult to co#pare perfor#ance in a(solute, quantitative ter#s, ne*er #achine learnin) techniques have )roupe" the anaphora resolution pro(le# alon) *ith the (i))er $& co2reference pro(le#, shiftin) the focus onto perfor#ance of the syste# as a *hole' !t appears that en"2to2en" fully auto#ate" syste#s that focus an" evaluate only anaphora resolution techniques are still rare' Car"ie an" $) 2 3s results i#plie" that #achine learnin) of anaphoric prono#inal reference an" $&2coreference resolution #ay (e (etter treate" as separate pro(le#s, *ith a "evelop#ent of "ifferent factor sets that are specifically tune" to the t*o "ifferent pro(le#s' !n a""ition, #ost #etho"s use only structural or syntactic consi"erations *hile "eter#inin) the salience of an $& -(ase" on the salience hierarchy of )ra##atical roles.' 9ates an" 6arris -2 2.3s fin"in)s re)ar"in) the role of aspect in (ac+)roun"in) #ay provi"e a further +no*le")e source that #i)ht (e *orth investi)ation' !t *oul" (e interestin) to see ho* #uch of a perfor#ance "ifference this *ill cause' 1ro# a cross2lin)uistic perspective, anaphora in "ifferent lan)ua)es present very "ifferent sorts of challen)es' >ost of the co##only researche" lan)ua)es such as @n)lish, 1rench, =apanese, Spanish have relatively si#ple anaphoric syste#s' !t *ill (e a challen)e to "evelop $%& syste# for lan)ua)es li+e !celan"ic, >arathi, etc' *ith #ore co#ple/ anaphoric pheno#enon'

14

References
-1. -2. 9ranco, Antonio, 2 1, 9in"in) >achines, in Co#putational %in)uistics, Nolu#e 2;, $u#(er 1' 9yron, 5onna A', 2 1, The Hnco##on 5eno#inator: A &roposal for Consistent Reportin) of &ronoun Resolution Results, in Co#putational %in)uistics, Nolu#e 27, $u#(er 4, 2 1' Cho#s+y, $oa#, 17;1, %ectures on <overn#ent an" 9in"in), 5or"recht: 1oris' <ros8, 9ar(ara, =oshi, Arvin", ?einstein, Scott, 1775, Centerin): A fra#e*or+ for #o"ellin) the local coherence of "iscourse' Co#putational %in)uistics, Nolu#e 21, $u#(er 2: 2 42226 6arris C' %' an" 9ates @' A', 2 2, Clausal (ac+)roun"in) an" prono#inal reference: A functionalist approach to c2co##an", in %an)ua)e an" Co)nitive &rocesses Nolu#e 17:247227 , 2 2' 6o((s, =erry, 1777, Resolvin) pronoun references, in Rea"in)s in $atural %an)ua)e &rocessin), <ras8, =ones an" ?e((er, e"s', >or)an Aauf#an &u(lishers, !nc' %os Altos, California, HSA' %appin, Shalo# an" %eass, 6er(ert, 1774, An al)orith# for prono#inal anaphora resolution, Co#putational %in)uistics, Nolu#e 2 , $u#(er 4: 5452 562 >it+ov, RuslanO 9o)uraev, 9rani#ir an" %appin, Shalo#, 2 1, !ntro"uction to the Special !ssue on Co#putational Anaphora Resolution in Co#putational %in)uistics, Nolu#e 27, $u#(er 4, 2 1' >it+ov, Ruslan, 1777, 1actors in Anaphora Resolution: They are not the only thin)s that #atter2 A case stu"y (ase" on t*o "ifferent approaches, in &rocee"in)s of the AC% I77/@AC% 377 ?or+shop on Dperational 1actors in &ractical, Ro(ust Anaphora Resolution' $iyu <e, =ohn 6ale, an" @u)ene Charnia+, 177;, A statistical approach to anaphora resolution, in &rocee"in)s of the Si/th ?or+shop on Nery %ar)e Corpora -177;.' $), Nincent an" Car"ie, Claire' !"entifyin) Anaphoric an" $on2Anaphoric $oun &hrases to !#prove Coreference Resolution in (roceedings of the !"th )nternational *onference on *omputational +inguistics (*,+)-./2002), 2 2'

-4. -4.

-5.

-6.

-7.

-;.

-7.

-1 .

-11.

14

-12.

$), Nincent an" Car"ie, Claire' !#provin) >achine %earnin) Approaches to Coreference Resolution in &rocee"in)s of the 4 th Annual >eetin) of the Association for Co#putational %in)uistics, Association for Co#putational %in)uistics, 2 2' $), Nincent, 2 2, >achine %earnin) for Coreference Resolution: Recent Successes an" 1uture Challen)es, >anuscript, Cornell Hniversity' Stru(e, >ichael an" H"o 6ahn, 1776, 1unctional Centerin), in &rocee"in)s of the 44th Annual >eetin) of the Association of Co#putational %in)uistics -AC% 376. Stuc+ar"t, Rolan", 2 1, 5esi)n an" @nhance" @valuation of a Ro(ust Anaphor Resolution Al)orith#, in Co#putational %in)uistics, Nolu#e 27, $u#(er 4, 2 1' Tetreault, =oel R', 2 1, A Corpus 9ase" @valuation of Centerin) an" &ronoun Resolution, in Co#putational %in)uistics, Nolu#e 27, $u#(er 4, 2 1' Soon, $), %i#, 2 1, A >achine %earnin) Approach to Coreference Resolution of $oun &hrases, in Co#putational %in)uistics, Nolu#e 27, $u#(er 4, 2 1' Reinhart, Tanya an" Reulan", @ric, 1774, Refle/ivity, in %in)uistic !nquiry, Nolu#e 24, $u#(er 4, 1all 1774: 657272 Juinlan, =' Ross, 1774, C4'5: &ro)ra#s for >achine %earnin)' >or)an Aauf#an, San >ateo, CA'

-14. -14.

-15.

-16.

-17.

-1;. -17.

15