TrialsinPrimaryCare:design, conductandevaluationof complexinterventions

DrGillianLancaster
PostgraduateStatisticsCentre LancasterUniversity
g.lancaster@lancs.ac.uk
CentreforExcellenceinTeachingandLearning

RSSPrimaryHealthCareStudyGroup
CoauthorsofSMMRpaper: MikeCampbell,Sheffield SandraEldridge,QueenMaryLondon AmandaFarrin,Leeds MauriceMarchant,EastSussexPCT SaraMuller,Keele RafaelPerera,Oxford TimPeters,Bristol TobyPrevost,KingsCollege GretaRait,UCL

1.Introduction
ResearchinPrimaryCareistime consumingandoftenchallenging Itrequiresextensiveplanning&prep Interventionsareoftencomplexand presentarangeofproblemseg.
Workinginhealthcaresetting Sensitivitytolocalcontext Logisticsofapplyingexperimentalmethods

Whatmakesanintervention complex?
Interactionsbetweencomponentsin experimentalandcontrolarms Difficultyofbehavioursrequiredbythose deliveringorreceivingtheintervention Organisationallevelstargetedbythe intervention Variabilityofoutcomes Degreeofflexibility/tailoringofintervention permitted Willitworkineverydaypractice?
NB.takenfromMRCguidelines

Guidance
MRCdocument
DevelopingandEvaluatingComplexInterventions www.mrc.ac.uk/complexinterventionsguidance CraigP.etal.BMJ2008,337:a1655

BMJpaper(CampbellNCetal.2007,334:4559)
DesigningandEvaluatingComplexInterventionsto improvehealthcare

Casestudies

KeystatisticaldesignissuesI
Phases given in MRC guidance framework Development Key elements in designing and evaluating complex interventions Background and context (For more information and examples see MRC and Campbell et al.) Defining and understanding the problem (See above docs) General points to consider Key statistical design issues addressed in our paper Socio-economic background; Underlying cultural assumptions; Health service system; Government initiatives; Preventative policies Prevalence of condition; Population most affected; How condition is caused/sustained; Potential for intervention and improvement Levels of complexity of health problem and co-morbidity; Risk factors and factors influencing changes over time; Patient beliefs, symptoms and adherence to treatment Systematic reviews; Epidemiological research; Qualitative research; Expert opinion Identify key processes and mechanisms for delivery; Potential beneficial effect; Define target group; Optimise best treatment combinations

Conceptualising the problem (See above docs)

Gathering evidence

Developing the intervention

Using evidence from primary studies, systematic reviews and qualitative studies to inform study design Conducting primary care research in the UK: complying with research governance and assessing quality of care using the Quality and Outcomes Framework

KeystatisticaldesignissuesII
Phases given in Key elements in MRC guidance designing and framework evaluating complex interventions Evaluation Developing and optimising trial parameters General points to consider Key statistical design issues addressed in our paper Pilot studies and pre-trial modelling; Selection of outcome measures for effectiveness and quality; Recruitment of practices and participants; Choosing the method of randomisation; Sample size and between trial variation Testing the feasibility and integrity of the trial protocol; Consideration of appropriate primary/secondary endpoints; Recruitment and retention strategies; Method of randomisation to minimise imbalance; Sample size considerations Data collection forms; Design of database; Monitoring procedures; Awareness of issues of data analysis for different study designs Publication and dissemination strategy; Stakeholder involvement; Benefits, harms, costs for decision making; Recommendations

Data collection and analysis

Choosing the method of analysis: cluster specific versus marginal models

Implementation

Getting evidence into practice (See new MRC guidance document)

2.Usingevidencefromprimary studies,systematicreviewsand qualitativestudiesinthedesign

Muchhighqualityresearchlacks generalisability (externalvalidity) Interventionmaynotbeeasilyimplemented inpractice(Who?How?Duration?) Strongargumentforcarryingoutresearchin themostappropriatecontextandsetting Eg.Canwetrustestimateofeffectsizewhen interventionstudiestolowerBPafterstroke aremostlycarriedoutinsecondarycare?
(Mant etalBMJ2006)

SystematicreviewsofRCTs
Usefulbecausebasedonclearlyformulated researchquestionsandmethodology Qualityofincludedpapershasbeenappraised Summary(pooled)estimateofeffectsize Feasibility,acceptabilityanduptakeof interventioncanbemeasuredbylevelofattrition ofparticipants Eg.Relativeattritionhasbeenusedtocompare levelsofattritionacrossoralanticoagulationand DiabetestypeIItrials(Hennekens etal.BMCRes.Methods2007) Systematicreviewsofdiagnostictestandmethod comparisonstudiesalsousefulforselectingan appropriatemeasurementmethod ortechnique

Qualitativestudies
Especiallyusefulwhenplanningorevaluatinga complexintervention Canbeused: Before thetrialtoexploreissuesofdesign eg.barrierstorecruitment;acceptabilityofthe randomisationfromapatientsperspective During thetrialtounderstandandunpackthe processesofimplementation andchange After thetrialtoexplorereasonsforthefindings eg.arefindingsinlinewithunderlyingtheory; acceptabilitytodeliverersandreceivers; comparisonswithpatientreportedoutcomes; thevalueoftheintervention asanevaluative assessmentandtoaidinterpretation

3.Conductingprimarycare researchintheUK
Publicgenerallytrustsacademicresearch Researchgovernance ensuresresearchintegrity toupholdthepublicsconfidence,toprotect participantsfromabuse,andtoprotect researchersfromaccusationsofmisconduct Widerangeoflegalrequirementseg.
EuropeanClinicalTrialsDirective DataProtectionAct

Ethicalapproval NB.muchdebateaboutwhetherRECsshould examinestatisticalissues&methodologicalrigour

Researchgovernance
GPsareusuallyselfemployedorworkwithina limitedcompany;contracttoNHS NHSPrimaryCareTrusts(PCTs)commissionGPs serviceswithintheirgivenarea PCTsfacilitateresearchlocallytoensure researchintegrity;researchreviewcommittees PrimarycareresearchofteninvolvesseveralGP centresacrossmultiplePCTs
verytimeconsumingtoobtainapproval; honorarycontracts;CRBchecksetc.(eg.6months)

NIHRhaverecentlyintroducedguidanceanda ResearchPassportSystemtohelptheprocess

ResearchpotentialofQOF
Tomonitorqualityofcareofpatients,financial incentives(upto30%ofGPincome)havebeen introducedthroughtheQualityandOutcomes Framework(QOF) QOFhas5domainsofincentivisation
o Clinicalcare o Organisation o Patientexperience o Educationandtraining o Otheradditionalservices

Pointsareawardedaccordingtotheworkload neededtoachievetargetsandprevalenceof disease(age,sex,deprivation)inthearea

ResearchpotentialofQOF
TouseQOFindicatorsinresearcheg.toassess differencesinqualityofcare,therearecertain problemstoovercome:
o Exclusionseg.failuretoattendforassessment, frailtyofcondition,refusetreatment o DifferencesbetweenGPPracticeseg.how conditionsarerecorded,howinterventionsare assessed,compositionandskillsofpracticestaff

QOFisprimarilypaymentdrivenandnotcreated forresearchpurposes Researchdatabaseshavebeencreatedeg.GPRD, Qresearch,usingsamplesofGPpractices

4.Useofpilotstudies
Importantprerequisiteforfunding Oftenadhocsmallstandalonestudies Subjecttopublicationbias Isthereadifferencebetweenafeasibility andapilotstudy? Pilotstudiesaddresstheintegrityofthe studyprotocol Needclearlistofkeyobjectives

Keyobjectivesofapilotstudy
Testintegrityofstudyprotocol Samplesizecalculation Recruitmentandconsentrates Developandtestimplementationanddelivery oftheintervention Acceptabilityoftheintervention Trainstaffindeliveryandassessment Selectionofmostappropriateoutcome measures(endpoints) Randomisationprocedure Pilotdatacollectionforms/questionnaires Prepareandplandatacollectionand monitoring

Example UKBEAMtrial
UKBackPain,Exercise,Activemanagement andManipulationtrial(Farrin etal.2005) Totesttheintegrityofthestudyprotocolusing aseriesofsubstudies Plannedasclusterrandomisedtrial 3treatments activemanagement(practice level);spinalmanipulationandexercise (patientlevel) Findings: Majorityofmethodsweresuccessful Problemwithdifferentialrecruitmentbetween practices changedtononclustereddesign

Pretrialmodelling
Example Fallspreventiontrial(Eldridgeetal.2005) Toinformdesignandtestlikelihoodofthe interventionbeingviableandeffective Costeffectivenessmodelusingpilotdata
o UsedprobabilitytreeandMarkovsimulation

Findings: Interventionwouldreduceproportionfalling byonly2.8%over12months Ifpolicymakerswerewillingtospend30,000 perQALYgained,therewasstillonlya40% chancetheinterventionwouldbecost effective

5.Selectionofappropriate outcomemeasure(s)
Distinguishbetweenprimaryandsecondary outcomemeasures Validandreliable(repeatable&reproducible) Directlymeasuredvs patientreported
o Includeadditionalobjectivemeasureswhenself reportingmaybeunreliableeg.selfassessedsmoking cessationandbiochemicalmeasure o HRQL usegenericanddiseasespecificmeasure

Selectmostappropriateoutcomeforevaluating theeffectivenessoftheintervention
eg.levelofkneepain, kneefunction,abilitytowork, satisfactionwithtreatment

Individuallevelvs group(cluster)level

6.Recruitment
Successfulrecruitmentrequiresacoordinated approachandgoodpilotwork Importanttoengagepracticesearlyon
o o o o IsresearchquestionimportantforPrimaryCare? Whatisitsprioritycomparedtootherissues? Howdoesitimpactonpatientdoctorrelationship? IsGPconfidenttoraiseresearchissuewithinasensitive consultation?

Timeconstraintsareamajorissue Needtofindefficientwaystoidentifythesample andgainconsent Complexinterventionscanhavedifferentlevelsof recruitment(practices&patients)

Principlesofgoodrecruitment
Engagewithallstakeholders(GPs,practicestaff andparticipants)
Brandfortrial(eg.BEAM,PANDA,SCAMPS) Welldevelopedmarketingstrategy,goodPR eg.BellsPalsytrialusedlocalcelebrityinmedia Wellwrittenpatientinformationdocuments

InvitationtotakepartcomingfromownGP UsetrainedstaffotherthanGPstoidentifyand consentparticipantseg.practicenurses Providestafftrainingindiseasetopicandresearch GetsupportfromlocalPCRNinfrastructure

ResearchReadyaccreditationscheme ePCRN (www.ePCRN.org)

Reimbursepracticesfortakingpart NB.Participantsareallowedtooptout

7.Methodofrandomisation
Byindividualorbyclustereg.GPpractices, households,nursinghomes
o relativecostsandjustification

Relativelyfewerclustersthanindividualsare usuallyavailable higherprob.ofimbalance

o inthesizeofeachtreatmentarm o inbaselinecovariatedistributionsatindividuallevel

Complexinterventionsinprimarycaremay havemultiplecomponents eg.simpleparalleldesignvs factorialdesign

Imbalanceinsizeoftrt groups
Tooptimisepowerneedtoensure
o anequalnumberofclustersineachtreatmentarm o anequalnumberofpeopleineachtreatmentarm

Toensurebalanceinnumbersofpeopleineach armcanuseblocking
o interventionsareassignedrandomlywithineachblock o varyingblocksizesreducespredictabilityofnext assignment

Allocationconcealmentisharderinclustertrials
o eachclustergetssameallocation o useofplacebosisnotusuallyfeasible

Imbalanceinbaselinecovariates
Imbalancemayaffectface validityofcomparisons andoverallconclusions Waystominimiseimbalance: Adjustmentbyanalysis mayresultindifferent unadjustedandadjustedestimatesoftreatment effects
o byeffectsizeandsignificance o difficultiesininterpretation

Atthedesignstage byidentifyingselected covariateswhichmaybeimportantpredictorsof outcome

o Randomiseusingstratification prepareaseparate randomisationscheduleforeachstrata o Useminimisation handleslargernumberofselected variables

8.Betweenpracticevariation andsamplesize
Variationbetweenpracticesintreatmentand referralpatternsiswellestablished
o hasimplicationsforgeneralisability ofastudy

Itisanimportantconsiderationinplanning clusterrandomisedtrialseg.samplesize Howdoweaccuratelymeasurethisvariation?

o Needgoodqualitydatafromlargedatasets eg.GPRD,Mediplus,MIQUEST,?QOF o Primaryresearch

Tradeoffbetweenbiasandprecision

Samplesize
Identifyprimaryoutcomemeasureand calculatesamplesizeforindividualtrial FindestimateofIntraclusterCorrelation Coefficient(ICC)
o Fortrialsrandomisinggeneralpracticeswith patientleveloutcomes,ICCsusuallyaround0.05. o PapershavebeenpublishedprovidinglistsofICCs

Multiply(inflate)samplesizebydesigneffect
o 1+(m1)xICC wheremisclustersizeassumingall clustersizesareequal

Pre2000manyCRTswereunderpowered

9.Methodofanalysis
IndividuallyrandomisedRCTs Welldocumentedstandardmethods Clusterrandomisedtrials Individualswithinclusters eg.GPpractices,nursinghomes Summarymeasuresapproach
o Simple usesmeansorproportionsforeachcluster o Givesequalweighttoeachcluster(wts canbeused) o Cannotadjustforindividuallevelcovariates

Populationaveraged(marginal)models
o UsesGEEs,minimumof20clusterspergroup

Clusterspecificmodels
UsesML,betterforsmallernumbersofclusters

Example DESMONDtrial
Comparisonof4methodsofanalysis:outcomeistheproportionof patientswithanHbA1cbelow7%,interventionisstructurededucn
Model Odds Ratio Cluster specific 1.085539 Standard Error 0.166037 0.54 0.592 (0.804362, 1.465007) z P > | z | (95% Confidence Interval)

Population averaged: Robust Independent errors Exchangable errors 1.079769 0.160086 0.52 0.605 (0.807480, 1.443876) 1.161681 1.161681 0.271156 0.162818 0.64 1.07 0.521 0.285 (0.735194, 1.835573) (0.882643, 1.528933)

10.Conclusion
Presentedmainstatisticalissuesforconducting complexinterventions Providesaflavouroftheissuescoveredinour PHCSGmeetingsoverpast8years Balancebetweenmethodologicalissuesand morepracticalissuesofrealliferesearch
o manyissuesnotuniquetoprimarycaresetting

Challengeremainsofmaintainingand expandingthecapacityofbothmethodological andappliedexpertiseinprimarycare

Reference
LancasterG.A.,CampbellM.C.,EldridgeS.E.,Farrin A.,Marchant M.,MullerS.,Perera R.,PetersT.J., PrevostA.T.,Rait G.(2010). TrialsinPrimaryCare:statisticalissuesinthe design,conductandevaluationofcomplex interventions. StatisticalMethodsinMedicalResearch19:34977. Facultyof1000publication. Primstat dataarchive
www.jiscmail.ac.uk/primstat presentationsandsummariesofdiscussionsfrom meetingsofPHCSG