Transcript: Twitter Journal Club Fourth Meeting 8pm, Sunday 26th June 2011

McManus et al. (2008) Graduates of different UK medical schools show substantial differences in performance on MRCP(UK) Part 1, Part 2 and PACES examinations, BMC Medicine - Available at http://bit.ly/jYA34O Hosted by Fi Douglas and Anne Marie Cunningham
twitjournalclub Good evening, everyone, and welcome to this week's TwitJC meeting. This evening's paper will be presented by @amcunningham. #twitjc -8:01 PM twitjournalclub An introduction to the paper can be found here: http://t.co/bRdT8wa #twitjc 8:01 PM twitjournalclub Chairing the discussion, as it were, is me, @fidouglas. If we can begin with introductions... #twitjc -8:02 PM amcunningham Hello, it's Anne Marie In Cardiff (and Belfast graduate!)- I hope you enjoyed reading http://t.co/bRdT8wa for #twitjc -8:03 PM themattmak Evening all, look forward to hearing what people have to say, I'm Matt, medical SHO in London #Twitjc -8:03 PM RichmondDoc @amcunningham @Neil_Mehta @gruntdoc @hrana Wish I could. Am out and about in DC. #twitjc #meded -8:03 PM parkerflash Hello, Ash here, soon-to-be F1 on the NETFS academic foundation programme. #twitjc -8:03 PM DrDLittle David, Clinical Teaching Fellow. Manchester Graduate 2008 #twitjc -8:03 PM PaedsSHO Will only be flirting with #twitjc tonight, looking forward to reading later. David. #Twitjc -8:03 PM fidouglas I'm Fi Douglas, a medical student between second and third years at Cambridge (competing interest declared!). #twitjc -8:04 PM PaedsSHO And yes, Birmingham graduate. #Twitjc -8:04 PM LizzyFerret Elizabeth, current Dundee student. #TwitJC -8:04 PM TWSY Tom Young, emergency physician, Leeds graduate #twitjc -8:04 PM thefourthcraw Mike, Paeds SpR in the East Midlands (Edinburgh graduate) #twitjc -8:04 PM EverythingVHost Callum, a student up in glasgow #twitjc -8:04 PM amreeeves Hey, Andy, u'grad 3rd year at Leicester here. #twitjc -8:04 PM FayetheGEP Evening all, I'm Faye, 1st year med student, looking forward to my first #twitjc may just read though, not sure how much I could contribute -8:05 PM alicexelizabeth My name is Alice; second year Peninsula student. Interested in learning more about papers and passionate about meded! #twitjc -8:05 PM iamdoctord Richard, CT2 Med, Belfast graduate signing in for #twitjc -8:06 PM mcumcu I'm a biochemistry student applying for medical school this year #twitjc -8:06 PM themattmak #Twitjc Oh, and Imperial College graduate (after we separated from University of London) -8:06 PM DrDLittle This is a great #twitjc for students to get involved with, even 1st years. #meded #twitjc 8:07 PM

fidouglas @DrDLittle Yup, it will be interesting to get lots of different perspectives on the matter. #twitjc -8:07 PM LizzyFerret RT @DrDLittle: This is a great #twitjc for students to get involved with, even 1st years. #meded #twitjc -8:07 PM DocLC #twitjc Hi all, I'm a soon to be F1. Liverpool graduate -8:08 PM sneekyfish #topgear or #twitjc? -8:09 PM twitjournalclub We've got 5 main discussion points this evening, which are on the blog. Thanks to @amcunningham for providing the introductory post. #twitjc -8:09 PM DrDLittle @sneekyfish top gear will be available on iplayer ;) #twitjc -8:09 PM LizzyFerret @sneekyfish: #topgear or #twitjc? both! :) -8:09 PM twitjournalclub Link here: http://twitjc.wordpress.com/2011/06/26/introduction-and-discussionmedical-schools-and-performance-in-mrcp/ #twitjc -8:09 PM behrouzn #twitjc Hi! Birmingham offer holder, grades permitting! Last exam tomorrow, so I won't be contributing as much as I would like to! -8:10 PM twitjournalclub The first point: What factors do you think might explain variation in performance in MRCP between medical schools? #twitjc -8:10 PM fidouglas @behrouzn Good luck tomorrow! #twitjc -8:11 PM teamilkwithone @twitjournalclub how early in career MRCP is attempted #twitjc -8:11 PM amcunningham @twitjournalclub I guess the authors here are trying to suggest that it some aspect of teaching quality may impact MRCP performance #twitjc -8:12 PM DrDLittle It is clear there is correlation between medical school entry requirements and postgrad performance. Unsurprising really #twitjc -8:12 PM amcunningham @teamilkwithone about a few years after qualification usually .... #twitjc -8:12 PM themattmak I would say the teaching offered and exams taken at the medical school, especially for Part 1 and PACES #twitjc -8:12 PM thefourthcraw @twitjournalclub Or that selection criteria for schools differ; that correlates to MRCP performance . Which isn't a surprise is it? #twitjc -8:13 PM amcunningham @DrDLittle #twitjc unsurprising? MRCP may assess different things to a-levels? or do people who do well in tests always do well in tests? -8:13 PM DrDLittle in my opinion the exams taken at medical school has a big effect here, would have been interesting to see this analysed. #twitjc -8:13 PM alicexelizabeth #twitjc I think the paper was clear that entry requirements correlated with how well students did with MRCP... -8:13 PM fidouglas I can only speak for pre-clinical exams, but there is e.g. quite a bit of variation in difficulty of anatomy exams. #twitjc -8:14 PM themattmak If you have good grounding in basic sciences at medschool, it helps for Pt 1, if your exams are similar setup to PACES, that helps #twitjc -8:14 PM DrDLittle @amcunningham I think the people that do well in test, always tend to do so. Possibly slightly different for PACES. #twitjc -8:14 PM teamilkwithone @amcunningham @DrDLittle some people are bad at taking exams for many reasons #twitjc -8:14 PM amcunningham @alicexelizabeth yes almost 2/3 of variation between medical schools is explained by offers to medical students... surprising? #twitjc -8:14 PM thefourthcraw Depends on test, but MRCP 1 (assuming sim to MRCPCH1 ) is a test to assess how good at taking tests as much as anything else. #twitjc -8:15 PM

themattmak @fidouglas Exactly! And only 4 or 5 medical schools still do Pathology finals now - it's sure to benefit them #twitjc -8:15 PM alicexelizabeth @fidouglas #twitjc anatomy exam difficulty certainly varies through different medical schools; same for other exams? -8:15 PM amcunningham they didn't really have a measure of teaching quality- so nothing to suggest that med school does much to influence MRCP results #twitjc -8:15 PM parkerflash @DrDLittle is there any way for research to quantify that some students simply get through exams better than others? #twitjc #meded -8:15 PM themattmak @amcunningham @alicexelizabeth Can the same be said now when most medical schools offer AAA/AAB? #twitjc -8:15 PM teamilkwithone is there is a big variation in how OSCE's/practical exams are done at medical schools? are some more like MRCP practicals? #twitjc -8:15 PM DrDLittle @thefourthcraw absolutely, those who are good at tests are obviously more likely to do better in A-levels and postgrad exams #twitjc -8:16 PM fidouglas @themattmak Oh, I know from others' experiences that the final MB pathology exams at Cambridge are utterly horrific. #twitjc -8:16 PM amcunningham @themattmak this was a limitation even then... commenters suggested correlation with ability of students entering may be higher #twitjc -8:16 PM DocLC #twitjc I think teaching whilst at med school has a huge impact, especially clinical science teaching -8:16 PM DrDLittle @amcunningham odd that they suggest the need for an exit exam but don't look at the difference in teaching. #twitjc -8:16 PM fidouglas With regard to A level marks - if there was a way they could have looked at actual marks, rather than grades. What about new A*? #twitjc -8:17 PM teamilkwithone @amcunningham @alicexelizabeth arn't all offers AAA now for medicine? isn't that why extra exams have been brought in?? #twitjc -8:17 PM thefourthcraw @fidouglas They say they had no access to that data, but I think it is main confounding factor #twitjc -8:17 PM sneekyfish the year that you sit your "finals" in uni. its 4th year here in aberdeen. no pressure to study in 5th year #twitjc -8:17 PM DrDLittle @DocLC I agree, but this is demonstrated in the paper. Perhaps because it is almost impossible to quantify? #twitjc -8:18 PM amcunningham @fidouglas they weren't looking at what students got... just the average offer that they were made... a little different! #twitjc -8:18 PM thefourthcraw @fidouglas Did people who got similar grades at A Level/Higher do similarly well in MRCP, regardless of medical school #twitjc -8:18 PM iamdoctord I think though that confounders such as pre-admission ability and/or postgraduate education exist. (from comments attached to paper) #twitjc -8:18 PM alicexelizabeth @teamilkwithone @amcunningham #twitjc yes, I believe so; but I think with this paper there were some differences (AAB/AAA) #twitjc -8:18 PM teamilkwithone @sneekyfish do u mean just theory exams are sat in 4th year? the end of it presume? do u still have OSCE's in 5th year? #twitjc -8:18 PM fidouglas @sneekyfish Is it normal that the hardest year is penultimate year? Final year still hard work, but apply for F1 based on 4th/5th yr #twitjc -8:18 PM amcunningham @iamdoctord pre=admission ability is not treated as a confounder here- it explains almost 2/3 of variance #twitjc -8:18 PM

thefourthcraw @sneekyfish We did staggered finals in Edinburgh, across 4th and 5th year #twitjc 8:18 PM themattmak @DrDLittle @thefourthcraw Exams will favour learners w/different learning styles. PACES will favour kinaesthetic learners naturally #twitjc -8:18 PM EverythingVHost @teamilkwithone @amcunningham not all, for example glasgow still asks for AAB #twitjc -8:19 PM iamdoctord In NI, the entry requirement has been AAA for years, yet MRCP marks aren't as good as for Oxbridge... #twitjc -8:19 PM teamilkwithone @alicexelizabeth @teamilkwithone @amcunningham this would be true as they would have entered med school some time ago #twitjc -8:19 PM fidouglas @EverythingVHost @teamilkwithone @amcunningham When I was looking around in 2006, still some places offering ABB. #twitjc -8:19 PM LizzyFerret @sneekyfish: the yr that u sit your "finals" in uni. its 4th yr here in aberdeen. no pressure to study in 5th yr #twitjc 4th in Dundee too -8:19 PM amcunningham I'm wondering how easy people found the actual paper to understand... it's quite complicated to know what they did actually! #twitjc -8:19 PM teamilkwithone @iamdoctord is belfast the only med school in NI? #twitjc -8:19 PM iamdoctord @amcunningham oh right, thanks #cantunderstandstats #twitjc -8:19 PM behrouzn What about GCSEs? Do you think it has an impact? For example Oxford, Birmingham, Cambridge are big on GCSE and they perform well. #twitJC -8:19 PM DrDLittle @themattmak @DrDLittle @thefourthcraw perhaps why the PACES results don't quite conform to the pattern in Pt1 & 2 #twitjc -8:20 PM fidouglas @amcunningham Yeah, I struggled to get my head around quite a few bits of it! #twitjc 8:20 PM amcunningham @DrDLittle @iamdoctord Belfast did a lot better in PACES than part 1/2 :) #twitjc -8:20 PM DrDLittle @iamdoctord but there is AAA and there is AAA. Need to look at marks rather than grades. #twitjc -8:20 PM thefourthcraw @DrDLittle We see that in the sort of person who pass Part I but don't pass the Clinical easily though; definitely different skills #twitjc -8:20 PM iamdoctord @teamilkwithone yep, and over 80% in the med school are NI locals #twitjc -8:20 PM McDawg Sorry, missed the intro bit (PC crashed) - interested very much in the philosophy of #twitJC - I'm Graham Steel. -8:20 PM DrDLittle I hoped that this paper would demonstrate that PBL is terrible but it doesn't really. #twitjc -8:21 PM teamilkwithone @behrouzn but then how far do we go back? do collect data on how old first word was spoken?? how far is too far back?? #twitjc -8:21 PM DrDLittle @behrouzn again, perhaps those people are just good at tests? #twitjc -8:21 PM LizzyFerret @amcunningham I'll be honest I started reading thru it earlier in the week and switched off until your blog post explanation #twitjc -8:21 PM amcunningham Remember the authors weren't looking at the aggregate A level results of those entering medicine, just the aggregate offer #twitjc -8:21 PM fidouglas @behrouzn I'm a bit old now, but when I applied in 2006, you could withold your AS grades completely. Emphasis on GCSE results. #twitjc -8:21 PM ChrisDickie BSc(Hons) Med Sci St Andrews entering year 3 MBChB Glasgow after summer. Likely reading more than writing #twitjc -8:21 PM

iamdoctord @DrDLittle but do you need to look at other skills other than marks/grades? NI didn't have pre-entry interviews till recently... #twitjc -8:21 PM fidouglas @amcunningham And many will exceed those offers. Especially with a subject as competitive as medicine. #twitjc -8:22 PM DrDLittle @amcunningham very complicated stats, particularly in the additional documents. I didn't understand all of it. #twitjc -8:22 PM amcunningham @McDawg I formulated discussion points this week for #twitjc and tried to focus more on meaning rather than technique of analysis. welcome! -8:22 PM teamilkwithone @fidouglas @amcunningham how can you exceed AAA? #twitjc -8:22 PM iamdoctord The multi-variate analyses, though commendable, were hard to make head/tail of #twitjc -8:23 PM DrDLittle @iamdoctord ideally yes, but I guess that is much more difficult to look at quantitively. #twitjc -8:23 PM amcunningham LOL! RT @DrDLittle: I hoped that this paper would demonstrate that PBL is terrible but it doesn't really. #twitjc -8:23 PM thefourthcraw @amcunningham Do the 'better' universities just cream off the better candidates? Self-fulfilling statistics? #twitjc -8:23 PM iamdoctord RT @thefourthcraw: @amcunningham Do the better universities just cream off the better candidates? Self-fulfilling statistics? #twitjc -8:24 PM twitjournalclub Discussion point 2: Is it surprising that the average offers given by a medical school may predict performance of graduates in MRCP? #twitjc -8:24 PM ChrisDickie looked over paper briefly a few days ago. Sad St Andrews isn't included but we are only pre-clinical #twitjc -8:24 PM behrouzn @DrDLittle @behrouzn that is important factor as those who can deal with exam stress+ have better studying techniques will be better #twitJC -8:24 PM amcunningham @iamdoctord as Belfast didn't have interviews then does that make performance in clinical part even more surprising? #twitjc -8:24 PM DrDLittle @amcunningham although still interesting that the two pioneers of PBL in the UK at this time, don't perform well... #twitjc -8:24 PM doctorblogs "Medical schools &MRCP performance" by @amcunningham http://t.co/bRdT8wa for week4 #twitjc -8:24 PM behrouzn @teamilkwithone @fidouglas @amcunningham new A* grade has been introduced and people take more than 3 A Levels #twitJC -8:24 PM LizzyFerret @iamdoctord: @teamilkwithone 80% are NI locals" As no1 wants to leave home; perception across the water" is somehow poorer standard #twitjc -8:24 PM DrDLittle RT @thefourthcraw: @amcunningham Do the better universities just cream off the better candidates? Self-fulfilling statistics? #twitjc -8:24 PM amcunningham @iamdoctord yes, that happens- the question is, does it matter? #twitjc -8:25 PM themattmak I think it's important to remember this is just about MRCP, and not the other college exams. Other medschools may have been better #twitjc -8:25 PM fidouglas @behrouzn But med school is very different to high school in many respects. Success at one doesn't guarantee success at another. #twitjc -8:25 PM teamilkwithone @behrouzn @teamilkwithone @fidouglas @amcunningham the more a levels will have been relevant but not the A* grades when these did it #twitjc -8:25 PM alicexelizabeth @iamdoctord @thefourthcraw @amcunningham If you were to fail a year at Oxbridge can you repeat? Or is that it for doing medicine? #twitjc -8:25 PM

amcunningham Interestingy strong correlation between MRCP and MRCP performance by graduates too- but I can't find ref's study @themattmak #twitjc -8:26 PM cebmblog RT @amcunningham: give me a minute to get up to speed #twitjc http://is.gd/AdhJVX 8:26 PM fidouglas @alicexelizabeth For pre-clinical, if you fail 2nd MB exams in June, you re-sit in Sept. Not sure about clinical school exams. #twitjc -8:26 PM DrDLittle @twitjournalclub do offers correlate at all with actual results? #twitjc -8:26 PM doctorblogs Not particularly MT @twitjournalclub Qu2:Is it surprising that average med school offers may predict graduate MRCP performance? #twitjc -8:26 PM FayetheGEP @twitjournalclub I don't think it's that surprising, higher offers suggests higher performing candidates... #twitjc -8:26 PM Shazmo Sorry I'm late! Here now #twitjc -8:26 PM themattmak I think uni offers are more complicated than just the grades - they look at GCSEs and interview, and now there's BMAT etc #twitjc -8:26 PM behrouzn @fidouglas of course, but surely sucess at high school must help at med school no? #twitJC -8:26 PM EverythingVHost @fidouglas Success isn't guaranteed, but it's not shocking to hear that those who work hard at school will continue to afterwards #twitjc -8:27 PM fidouglas @alicexelizabeth You can't repeat a year. You *can* drop out mid-year for e.g. health reasons and restart the next year. #twitjc -8:27 PM DrDLittle @FayetheGEP @twitjournalclub does it? #twitjc -8:27 PM mcumcu It'd be interesting to see whether there's a correlation between UKCAT score and MRCP performance. Perhaps UKCAT is still too young? #twitjc -8:27 PM doctorblogs ! RT @DrDLittle I hoped that this paper would demonstrate that PBL's terrible but it doesn't really #twitjc -8:27 PM fidouglas @behrouzn There's a correlation, for sure, but there are people who were exceptional at school, not so much now, and vice versa. #twitjc -8:28 PM Shazmo I didn't do A-Levels....... #twitjc -8:28 PM DrDLittle @mcumcu Yes, that data, if available, is still a couple of years away. #twitjc -8:28 PM EverythingVHost @mcumcu I think my year were the first UKCAT's and we're going into final year now #twitjc -8:28 PM amcunningham @themattmak but 2/3s of the variance was explained by just offer... how could we get data on those other bits? #twitjc -8:28 PM LizzyFerret @Shazmo: I didn't do A-Levels....... #twitjc Same here. -8:28 PM medicscott @mcumcu Those who sat UKCAT in it's first year have still to graduate I believe. #twitjc -8:28 PM behrouzn @fidouglas but is it all about grades? or is it also about the other aspects that are taken into account in admissions? #twitJC -8:29 PM thefourthcraw @amcunningham Do we know if there is any difference between university and likelihood to have problems completing PRHO/F1-2 years? #twitjc -8:29 PM cebmblog RT @alicexelizabeth: @iamdoctord @thefourthcraw @amcunningham If you were to fail a year at Oxbridge can you repeat? You can repeat #twitjc -8:29 PM fidouglas Very few med schools required BMAT 5 years ago. I was first cohort of UKCAT back in 2006. #twitjc -8:29 PM iamdoctord If a medschool is described as performing 'poorly', it would be fallacious to conclude that each student performs 'poorly'.... #twitjc -8:29 PM

themattmak @amcunningham I'm not sure! I think there's a lot to unpack there, as well as variance in medschool exams. Frustrating really! #twitjc -8:30 PM teamilkwithone as much as I want 2b part of #twitjc I just cant get on with this weeks subject matter. annoying, this is 1st week i've been able to join in -8:30 PM alicexelizabeth @doctorblogs @twitjournalclub #twitjc I am not surprised that higher entry grades correlates with better performance #twitjc -8:30 PM fidouglas @cebmblog Really? Can you? Didn't think you could at Cambridge... #twitjc -8:30 PM DrDLittle @thefourthcraw @amcunningham not that I'm aware of, again, would be interesting. #twitjc -8:30 PM alicexelizabeth #twitjc but there are always exceptions; exceptions that wouldn't come to light in many studies! -8:30 PM amcunningham You could suggest from this paper that rather than a natnl exit exam, #meded should have a common entrance exam. #twitjc UKCAT fit the bill? -8:30 PM themattmak @thefourthcraw @amcunningham A lot of that information is now coming to fruition with Transfer of Information forms and WBPAs #twitjc -8:31 PM fidouglas @teamilkwithone Sorry to hear that. Better luck next week? #twitjc -8:31 PM DrDLittle @BeccaSarahS yes, might not actually improve teaching. #twitjc -8:31 PM twitjournalclub Discussion point 3: Are the authors correct to conclude that this analysis suggests that a national exit exam should be introduced? #twitjc -8:31 PM GabrielScally #twitjc I have some sympathy with the referee for the paper who thought it was too weak to be published. -8:31 PM amcunningham @thefourthcraw I don't know up-to-date stats on not completing f1/2 but I think it is a pretty rare event... anyone else? #twitjc -8:32 PM LizzyFerret @thefourthcraw @amcunningham I can't find the link but @DundeeUniv consistently comes top for Foundation Year Preparedness. #twitjc -8:32 PM sarahcdixon @amcunningham would make for an interesting piece of extra analysis though #twitjc -8:32 PM Shazmo @amcunningham I think UKCAT *might* fit the bill. Would be good to see any evidence after 1st cohort graduate (1/2) #twitjc -8:32 PM dr_dyb @DrDLittle are postgrad exams written by ppl who taught/work at the high rank Uni? Undergrad exams influenced by mrcp style eg mcq #twitjc -8:32 PM mcumcu @amcunningham I think the fact that UKCAT is ignored by Liverpool and Bristol questions its value #twitjc -8:32 PM themattmak @thefourthcraw @amcunningham A lot of that information is now coming to fruition with Transfer of Information forms and WPBAs #twitjc -8:32 PM DrDLittle @twitjournalclub No, they don't demonstrate that these differences have anything to do with teaching. #twitjc -8:32 PM iamdoctord '@GabrielScally: I have some sympathy with the referee for the paper...' And I commend his honesty, because I felt the same! #twitjc -8:33 PM thefourthcraw @dr_dyb MRCPCH written by committee; volunteers from anyone who has already passed. I wrote some as an SHO3. Dunno about mRCP #twitjc -8:33 PM fidouglas Part of application for F1 jobs is based on which quartile you are in your medical school. Is this biased? #twitjc -8:33 PM cebmblog Be interesting to do same analysis for MRCGP #twitjc -8:33 PM DrDLittle @amcunningham @thefourthcraw I know of 2 out of 36 of my Foundation colleagues. #twitjc -8:33 PM

behrouzn @amcunningham is UKCAT really that important in terms of what is shows to medical schools? #twitJC -8:33 PM FayetheGEP @DrDLittle I think so, i'd think harder workers at A levels getting higher grades are more likely to work hard at med school? #twitjc -8:33 PM parkerflash ?outcome of a national exam would be directing performance to a new box to tick, risking diverting it away from clinical practice #twitjc -8:33 PM DocLC #twitjc Surely offers between med schools dont vary too much? My liverpool offer was the same as my friends Newcastle offer 5 years ago -8:33 PM amcunningham @GabrielScally I think we'll be getting to whether or not it should have been published in a bit! #twitjc -8:33 PM Shazmo @amcunningham I think care needs to be taken that an exam doesn't exclude those from certain backgrounds, eg lots of prep needed #twitjc -8:33 PM themattmak @DrDLittle I completely agree. There is no indication for a National Exam explained by these findings #twitjc -8:33 PM DrDLittle @dr_dyb @DrDLittle perhaps, an interesting point. #twitjc -8:33 PM alicexelizabeth @amcunningham #twitjc an entrance exam would seem a good idea. But I do not think UKCAT is the right kind of test for this. -8:34 PM iamdoctord Is the authors conclusion they used the term prima facie, for more info http://bit.ly/ig5Naq #twitjc -8:34 PM DrDLittle @FayetheGEP I agree with that but do better students get higher offers for med school? #twitjc -8:34 PM EverythingVHost I wouldn't say they were, different courses put emphasis on different subjects, a national exit exam ignores that entirely #twitjc -8:34 PM amcunningham @Shazmo already shown that those from ethic minorities already less likely to obtain an offer for given alevel results :( #twitjc -8:34 PM themattmak @fidouglas It's wholly ineffective as each medical school determines for itself what data to use to rank therefore not equitable #twitjc -8:35 PM fidouglas @Shazmo @amcunningham After 5/6 years of med school, Id question whether we should still be +vely discriminating based on background #twitjc -8:35 PM DrDLittle An exit exam would potentially demonstrate a significant difference due to difference in Alevel grades, would that be a concern? #twitjc -8:35 PM thefourthcraw @twitjournalclub How do you pick potential good doctor at 17? Do those who do well at clinical exams share any other features? #twitjc -8:35 PM fidouglas @themattmak I assumed rank was done based on cumulative performance in exams...? #twitjc -8:35 PM PasTest @amcunningham @thefourthcraw % completing F1/F2 is high, then with a massive drop off going into ST1 #twitjc -8:36 PM LizzyFerret @DrDLittle @fayethegep well like @Shazmo said some of us didnt even do ALevels thus making the point moot! #twitjc -8:36 PM thefourthcraw @PasTest Is that because we let people pass the PRHO /Foundation years that really we shouldn't? #twitjc -8:36 PM Shazmo @fidouglas Widening access to medicine? Would an entrance exam with lots of prep required favour private school apps? #twitjc -8:36 PM DrDLittle @PasTest as in getting a job at ST1 or completing ST1? Is there published data on that? #twitjc -8:37 PM EverythingVHost @Shazmo Pretty definitely, I remember folk going off to paid courses for BMAT, and buying books for their UKCAT #twitjc -8:37 PM

thefourthcraw @LizzyFerret It doesn't make it moot; the principle's the same whether 3As at A Level or 5 or 6 As at Higher, isn't it? #twitjc -8:37 PM fidouglas @Shazmo We're talking about exit exams, I thought? I agree with you at school-leaving stage... #twitjc -8:37 PM Shazmo @LizzyFerret I did the IB, and think that many of the IB offers were too high! Really limited the med schools I applied to. #twitjc -8:37 PM amcunningham @PasTest thanks! so graduation more closely correlated to completing f1/2, that completng f1/2 is correlated with entering st1? #twitjc -8:37 PM iamdoctord Do the authors risk suffering from an appeal to novelty (argumentum ad novitam)? Just because its a 'new idea' is it appropriate? #twitjc -8:37 PM DrDLittle @Shazmo @fidouglas IMO would potentially favour those that are best placed to prepare for it. i.e. private schools #twitjc -8:37 PM teamilkwithone @fidouglas maybe it should become a national quartile, facts are better people will get the better jobs #twitjc -8:37 PM behrouzn @Shazmo I thing that kind of exam will just favour some. Maybe interview is the key as you are by yourself showing what you can do #twitJC -8:38 PM Shazmo @fidouglas Haha oh right, at one point we were talking about entrance exams ;) #twitjc 8:38 PM themattmak @PasTest @amcunningham @thefourthcraw That's a different situation involving choice to progress immediately to ST1 though #twitjc -8:38 PM FayetheGEP @LizzyFerret @DrDLittle @Shazmo Surely it works for the equivalent of the qualifications you did? #twitjc -8:38 PM Shazmo @behrouzn I think interviews are really important, I don't understand why some med schools don't interview! #twitjc -8:38 PM amcunningham There was a lot of talk about a national exit exam at the time the paper was written- hence why this got published I guess #twitjc -8:39 PM PasTest this paper pre dates equating for MRCP as well; introduced to make comparison between candidates from different cohorts easier #twitjc -8:39 PM LizzyFerret @thefourthcraw I didn't do Highers either. I did GCSEs then went into Nursing and graduated with a 1st Class Hons. #twitjc -8:39 PM fidouglas @teamilkwithone the paper implies differing standards at graduation/shortly thereafter. quartiles of a national exit exam fairer? #twitjc -8:39 PM cebmblog RT @Shazmo: @fidouglas Would an entrance exam with lots of prep required favour private schools? #twitjc have to agree it would -8:39 PM amcunningham But the Tooke report ref's in paper was internediate one- there are no refs to a nnational exit exam in the final report #twitjc -8:40 PM DrDLittle @amcunningham do you think there should be a national exit exam? #twitjc -8:40 PM thefourthcraw @LizzyFerret Would you advocate going the US route and making medicine a secondary degree for all? #twitjc -8:40 PM twitjournalclub Discussion point 4a: How should we judge the performance of medical schools? #twitjc -8:40 PM behrouzn I don't think there should be a national exit exam to determine quartiles, I don't think it will be fair. #twitJC -8:40 PM amreeeves RT @cebmblog: RT @Shazmo: @fidouglas Would an entrance exam with lots of prep required favour private schools? #twitjc have to agree it would -8:40 PM LizzyFerret @PasTest: paper pre dates equating for MRCP; introduced to make comparison between candidates from dif cohorts easier #twitjc interesting! -8:41 PM

themattmak In the same way, a national exam would favour schools that prepared students solely for the exam and not to be good competent docs #twitjc -8:41 PM fidouglas @thefourthcraw There seems to be a lot of support for going down the graduate medicine course at the moment. #twitjc -8:41 PM thefourthcraw @twitjournalclub 4a) Short, medium and long term measures. All important, all different. #twitjc -8:41 PM amcunningham @DrDLittle I think that needs a whole other discussion- they are cetainly not justified by this paper #twitjc -8:41 PM adriamarilla #twitjc seems like better student have higher chances but is education equally in the uk? Apparently best post codes best scores -8:41 PM meddymaxwell I've just skimmed the article. It would be interesting to see how those who didn't access medicine via traditional methods performed.#twitjc -8:41 PM mcumcu I'm not sure a student survey has much value when judging medical school performance #twitjc -8:41 PM fidouglas @behrouzn Why is it unfair? Shouldn't all potential doctors be measured against the same scale? #twitjc -8:41 PM EverythingVHost Re point 3: Student satisfaction, satisfaction of teaching staff in local hospitals, etc, #twitjc -8:41 PM DrDLittle @twitjournalclub I think there needs to be more strict QA of medical education rather than an assessment of graduates #twitjc -8:41 PM LizzyFerret @thefourthcraw well at the risk of straying from tonight's topic I have to say I'm greatly in favour of that idea yes. #twitjc -8:41 PM thefourthcraw @adriamarilla 'best postcodes, best scores' - and only going to get worse in E+W with fees situation? #twitjc -8:42 PM iamdoctord Medical schools should be judged on the competency of the doctors they produce... #twitjc -8:42 PM FayetheGEP @behrouzn @Shazmo But there'll always be schools (usually private schools) who prep their students how to perform well in interviews #twitjc -8:42 PM alicexelizabeth #twitjc does this paper indicate that the differences in performance are so significant that an exit exam is needed? I think I would be... -8:42 PM TWSY Comparing medical schools, if that was the aim, would be facilitated by exit AND entrance exam, look at improvement of students #twitjc -8:42 PM david_colquhoun Is it possible, except in extreme cases? RT @twitjournalclub: Disc point 4a: How should we judge the performance of med schools? #twitjc -8:42 PM cebmblog One simple reason for result is finals at Oxf & Cam are similar to MRCP #twitjc -8:42 PM LizzyFerret @thefourthcraw very hard to decide who will make a good doctor at 17 and have their teenage years/background dictate their career. #twitjc -8:42 PM medicscott @fidouglas @thefourthcraw Though not from the government. Making medicine a grad only degree would be too expensive for the gov. #twitjc -8:42 PM DrDLittle @amcunningham I agree, a future #meded topic perhaps? #twitjc -8:42 PM thefourthcraw @LizzyFerret Not sure it is straying from topic - if issue is picking the best people to be future doctors and how we assess that? #twitjc -8:42 PM fidouglas @iamdoctord How do you measure competency though? #twitjc -8:42 PM amcunningham @fidouglas but why would we want to measure doctors against one scale? what is justification for that? #twitjc why have ranking? -8:42 PM iamdoctord And competency is not equivalent to 'passing an exam'... #twitjc -8:42 PM

EverythingVHost @fidouglas I guess the problem is who makes the scale, where the balance is between basic sciences/practical understanding/etc #twitjc -8:42 PM MhairiHepburn #twitjc nice paper, but who cares about MRCP? my hypothesis is that Aberdeen (& Dundee) are too busy producing the best psychiatrists... :)) -8:43 PM EverythingVHost RT @iamdoctord: And competency is not equivalent to 'passing an exam'... #twitjc -8:43 PM fidouglas @TWSY Isn't one of the Guardian table columns about value added? #twitjc -8:43 PM alicexelizabeth #twitjc ... concerned if such large discrepancies in teaching and achievement existed between medical schools -8:43 PM amcunningham RT @cebmblog: One simple reason for result is finals at Oxf & Cam are similar to MRCP #twitjc -8:43 PM DrDLittle @alicexelizabeth no, it doesn't demonstrate that the differences have anything to do with undergrad education. #twitjc -8:43 PM Shazmo @twitjournalclub I dont think one exam would suffice. Med schools have different strengths and so do doctors. 1 exam couldn't cover. #twitjc -8:43 PM fidouglas @amcunningham Well, ranking currently used in part to determine which F1 job you get. #twitjc -8:43 PM LizzyFerret @themattmak but are ALevels not by definition "national exams"? All the same exam taken by all the students nationwide. #twitjc -8:43 PM behrouzn @fidouglas but teaching methods will play a big factor. those who go to a small medical school will benefit from the small lectures #twitJC -8:43 PM themattmak @twitjournalclub The performance doesn't need to judged, it's medical education needs to be judged - which it is with QA #twitjc -8:43 PM david_colquhoun Or just yet more box-ticking like useless QAA? RT @twitjournalclub: How should we judge the performance of med schools? #twitjc -8:44 PM amcunningham @fidouglas because something is done is not a justification in itself/// #twitjc 8:44 PM iamdoctord RT @Shazmo: I dont think one exam would suffice. Med schools have different strengths and so do doctors. 1 exam couldnt cover. #twitjc -8:44 PM cebmblog best measure of doc: what your patients say about you & whether your practice reflects EBM RT @amcunningham: @fidouglas #twitjc -8:44 PM amcunningham @MhairiHepburn why do you think it is a 'nice paper'? #twitjc -8:44 PM behrouzn @FayetheGEP yes, but an interview is very unpredictable and some of them ie St George's interview are very difficult to prepare for. #twitJC -8:44 PM twitjournalclub Discussion point 4b: Is performance of graduates in post-graduate examinations important? #twitjc -8:44 PM DrDLittle @themattmak absolutely agree. QA needs to be much more thorough #twitjc -8:45 PM amcunningham @fidouglas none of the Guardian measures added anything to model beyond entrance requirements #twitjc -8:45 PM amreeeves #twitjc Was there any consideration of which trusts the students worked in for FY etc? Can variance in quality ... -8:45 PM amreeeves #twitjc ... of trusts affect the result as many stick around where they attended university? -8:45 PM behrouzn RT @cebmblog: best measure of doc: what your patients say about you & whether your practice reflects < I think this too #twitJC -8:45 PM medicscott @LizzyFerret @themattmak Not completely national, ALevels vary depending on the examboard and course the school chooses. #twitjc -8:45 PM

themattmak @LizzyFerret but we don't all do the same subjects (I would have died if I had to do maths or physics!) #twitjc -8:45 PM DrDLittle @twitjournalclub Performance in post-grad exams is important for the students! #twitjc 8:45 PM thefourthcraw @twitjournalclub 4b) Clinical exams, yes. Pre-clinical MCQs tend to act just as gatekeepers to see who gets to sit the clinical #twitjc -8:45 PM LizzyFerret RT @meddymaxwell: I've just skimmed the article. It would be interesting to see how those who didn't access medicine via traditional methods performed.#twitjc -8:45 PM fidouglas @amcunningham Not in the paper here, but in this year's Guardian tables, I'm sure one of the measures was value added. #twitjc -8:45 PM alicexelizabeth RT @meddymaxwell: I've just skimmed the article. It would be interesting to see how those who didn't access medicine via traditional methods performed.#twitjc -8:46 PM ChrisDickie needing to head out. Look forward to reading bout #twitjc later. Have fun! -8:46 PM DrDLittle The question is does performance in MRCP correlate to how good a doctor you are? #twitjc -8:46 PM FayetheGEP @behrouzn difficult but not impossible! Still agree interview is much better for selection than UKCAT though #twitjc -8:46 PM fidouglas @thefourthcraw Is it important that doctors still have a solid academic grounding (as arguably measured by MCQs)? #twitjc -8:46 PM Puffles2010 RT @david_colquhoun: Is it possible, except in extreme cases? RT @twitjournalclub: Disc point 4a: How should we judge the performance of med schools? #twitjc -8:46 PM amcunningham @amreeeves no, that wasn't looked at? you thing Oxbridge graduates have cushier posts that allow them more time to study? #twitjc -8:46 PM behrouzn 4.b. well it should be used to check that all medical school provide a similar standard of teaching #twitJC -8:46 PM thefourthcraw @fidouglas Yes, but again, I think part 1 tests ability to pass tests more than anything else. As much about strategy as knowldge #twitjc -8:47 PM showjumper42 RT @cebmblog: best measure of doc: what your patients say about you & whether your practice reflects EBM RT @amcunningham: @fidouglas #twitjc -8:47 PM fidouglas @behrouzn But isn't it better to do that at the point of graduation, rather than waiting for the doc to practice for a couple of yrs #twitjc -8:47 PM DrDLittle @behrouzn but you can't assess that using post-grad exams, this paper doesn't. #twitjc 8:47 PM themattmak @twitjournalclub I would argue this is more important - it now determines entry into higher training #twitjc -8:47 PM iamdoctord '@DrDLittle: The question is does performance in MRCP correlate to how good a doctor you are?' 360 appraisal anyone? LoL #twitjc -8:47 PM amreeeves @amcunningham haha not too sure, was just wondering as comparing where attended uni and exams that come at least a while post-grad #twitjc -8:47 PM LizzyFerret @iamdoctord: And competency is not equivalent to 'passing an exam'... #twitjc well it sort of is though - OSCEs? -8:48 PM thefourthcraw @DrDLittle Not absolutely. Good clinicians still fail clinical. Bad ones much less likely to pass though. #twitjc -8:48 PM showjumper42 RT @twitjournalclub: Discussion point 4a: How should we judge the performance of medical schools? #twitjc -8:48 PM behrouzn @FayetheGEP UKCAT is awful, I think. Interviews are the best method available at the moment to select students. #twitJC -8:48 PM

fidouglas @thefourthcraw @DrDLittle No test is perfect. I guess main thing is making it as a good as possible. #twitjc -8:48 PM tobyhillman MT @thefourthcraw: I think part 1 tests ability to pass tests more than anything else. As much about strategy as knowldge #twitjc -8:49 PM thefourthcraw @fidouglas The flipside of that is how we handle 'failing doctors'. Do we do right by them OR by the public at the moment? #twitjc -8:49 PM iamdoctord @LizzyFerret OSCEs I think at med school assess confidence in the skill, competency comes with experience and repetition... #twitjc -8:49 PM fidouglas @behrouzn @FayetheGEP I wonder what proportion of applicants get interviews at each med school? #twitjc -8:49 PM FayetheGEP @behrouzn Agreed, UKCAT is pointless. I'm struggling to think of a better way than interviews to select even tho they're flawed #twitjc -8:49 PM amcunningham How many of the medical schools in the study didn't use interviews to give offers? Belfast didn't. Others? #twitjc -8:49 PM TWSY Value added by medical schools hard to measure. MRCP only one distant outcome and dependent on input #twitjc -8:49 PM medicscott @behrouzn I think UKCAT has it's place. It tests ability to gather and use info. Interviews are good but don't really test ability. #twitjc -8:50 PM amreeeves @amcunningham Southampton doesn't, or at least didn't for my brother's application to start this September #twitjc -8:50 PM medicscott @amcunningham Southampton doesn't #twitjc -8:50 PM fidouglas @amcunningham I was interviewed at Imperial, Cambridge, and Bristol, though not Edinburgh (offered from all). #twitjc -8:50 PM tobyhillman @amcunningham Not sure that Southampton did at the time studied (didn't go there though) #twitjc -8:50 PM DrDLittle @amcunningham yet another variable, there are SO many potential variables that potentially influence MRCP performance. #twitjc -8:51 PM alicexelizabeth @DrDLittle would assessment of emotional intelligence add to finding more about how well doctors are performing? #twitjc -8:51 PM thefourthcraw @twitjournalclub Best medical school is the one that shows best IMPROVEMENT in its intake, not the one with best results? #twitjc -8:51 PM FayetheGEP @amcunningham Southampton that I know of #twitjc -8:51 PM Shazmo @amreeeves @amcunningham Southampton didn't interview when I applied. Only place I got rejected before interview! #twitjc -8:51 PM fidouglas Also, the nature of interviews at Imperial, Cambridge, and Bristol were all completely different. #twitjc -8:51 PM behrouzn @amcunningham Edinburgh does not interview, today. Not sure at the time of the study. #twitJC -8:51 PM DrDLittle @fidouglas @amcunningham but more places interview now than they did in 1994-6 #twitjc -8:51 PM tobyhillman @medicscott I thought that interviews were generally a poor way to select people for most things. #twitjc -8:51 PM thefourthcraw @alicexelizabeth How would you do it? (Emotional intelligence assessment) #twitjc -8:52 PM fidouglas @DrDLittle Oi! I'm not that old! I applied in 2006/7 academic year! #twitjc -8:52 PM amcunningham So it doesn't look as if interviewing or not seems to have a big impact on whether graduates will perform well in MRCP... #twitjc -8:52 PM

amreeeves @MedicScott @behrouzn plus, coaching for an interview is going to be a lot easier to come by for public schooled. #twitjc -8:52 PM LizzyFerret @DrDLittle: The Q is does performance in MRCP correlate to how good a doctor you are? #twitjc really don't think so! Head knowledge maybe -8:52 PM thefourthcraw @behrouzn Didn't interview in 1994 either. #twitjc -8:52 PM Shazmo @thefourthcraw Peninsula sort of do it that way.... #twitjc -8:52 PM DrDLittle @alicexelizabeth @DrDLittle maybe. #twitjc -8:52 PM themattmak @tobyhillman @medicscott including specialty training? consultant posts? #twitjc 8:52 PM fidouglas Emotional intelligence testing at 17 years old?! Really? Most are pretty immature at that age. #twitjc -8:53 PM cebmblog RT @amcunningham: Shouldn't the best med school be one that accepts variety in its candidates and reflects the make up of society #twitjc -8:53 PM DrDLittle @fidouglas haha! I know that, I meant that perhaps those places didn't interview back then? #twitjc -8:53 PM tobyhillman @thefourthcraw hmm, surely achieving some sort of standard important tho? wouldnt want a 'best improved' doc - i'd prefer a good one #twitjc -8:53 PM alicexelizabeth @thefourthcraw #twitjc with difficulty! Hard to measure and if people knew what it was for they could adjust their answers accordingly. -8:53 PM thefourthcraw @tobyhillman Oh, you'd still have to reach a reference standard as now, but would be interesting to analyse on those lines #twitjc -8:53 PM amcunningham @cebmblog I didn't say that... so don't know how you managed to rt me saying that:) #twitjc but it's an interesting suggestion -8:53 PM thefourthcraw @alicexelizabeth Yup; I think "EQ" measures too difficult to do with any reliability, especially at 17 #twitjc -8:54 PM cebmblog meant @amcunningham: Shouldn't best med school be one that accepts variety in its candidates and reflects the make up of society #twitjc -8:54 PM twitjournalclub Final discussion point: How helpful is it to read reviewers' comments on a paper? Is it something that all journals should aim for? #twitjc -8:54 PM medicscott @tobyhillman I think interviews are useful to assess personality and comm skills but don't suggest ability to be a good doctor IMHO. #twitjc -8:54 PM iamdoctord What about aptitude testing? Both pre-entry and post-grad? http://bit.ly/myC0xj #twitjc -8:54 PM themattmak @fidouglas the UKFPO are trialling situational judgement tests as ways of allocating to Foundation jobs though...#twitjc -8:54 PM EverythingVHost @cebmblog I'd vote yes to that, with the obvious caveat of not taking positive discrimination to a point that makes poor doctors #twitjc -8:54 PM tobyhillman @themattmak I think almost anything - but Med Dir certainly feels not great for cons posts. #twitjc -8:54 PM LizzyFerret @thefourthcraw @twitjournalclub but sure how could it "measure improvement" as all are starting out with no relevant knowledge? #twitjc -8:54 PM amcunningham Having met graduates from different medical schools... do you think they make comparable doctors? Are we doing OK? #twitjc -8:54 PM FayetheGEP @fidouglas Agreed! There's a reason why I scored 100 points higher on UKCAT 2nd time round #twitjc -8:55 PM LizzyFerret @fidouglas exactly - I had interviews for Barts GEP and Dundee 5 yr - both SO very different. #twitjc -8:55 PM

DrDLittle @LizzyFerret yes, head knowledge is important but doesn't mean you are a great doctor. #twitjc -8:55 PM amcunningham @cebmblog yes, diversity in medical schools in UK is reason most often given for not having a national exit exam #twitjc -8:55 PM LizzyFerret RT @cebmblog: RT @amcunningham: Shouldn't the best med school be one that accepts variety in its candidates and reflects the make up of society #twitjc -8:56 PM cebmblog Positive discrimination for some as role models worth a risk RT @EverythingVHost: I'd vote yes to that, #twitjc -8:56 PM DrDLittle @twitjournalclub really interesting to read the reviewers comments, I've never done that before but I agree with comments made #twitjc -8:56 PM tobyhillman @themattmak the variety of experiences at interview in this thread suggest not a great test... #twitjc -8:56 PM fidouglas @DrDLittle @LizzyFerret I wonder how med schools differ in numbers of students going into bench research. #twitjc -8:56 PM adriamarilla RT @cebmblog: RT @amcunningham: Shouldn't the best med school be one that accepts variety in its candidates and reflects the make up of society #twitjc -8:56 PM iamdoctord RT @DrDLittle: @LizzyFerret yes, head knowledge is important but doesnt mean you are a great doctor. #twitjc -8:57 PM themattmak @amcunningham I'm going to be controversial, and say yes. But there are always exceptions. #twitjc -8:57 PM Tom_Zorg20 Yes! RT @twitjournalclub: How helpful is it to read reviewers' comments on a paper? Is it something that all journals should aimfor? #twitjc -8:57 PM tobyhillman Final discussion point - YES - important to see how a paper has been 'spun' or developed following comment. #twitjc -8:57 PM DrDLittle @amcunningham I think grads from different universities differ significantly in my experience #twitjc -8:57 PM alicexelizabeth @twitjournalclub I really like the idea of reading reviewers' comments; I enjoy seeing any comments from papers and their thoughts #twitjc -8:57 PM fidouglas I think being able to see reviewers' (and indeed others') comments helps highlight possible problems with the paper. #twitjc -8:58 PM DrDLittle I agree with the reviewer who questioned how useful this paper is. It is v.interesting but not at all useful #twitjc -8:58 PM themattmak Sorry guys, have to leave the discussion - dinner's ready, really enjoyed it so far though, thanks @fidouglas and @amcunningham #twitjc -8:58 PM MhairiHepburn #twitjc joking aside, most drs dont end up doing MRCP anyway! most of the highest achievers from my yr (abdn 2005) arent physicians -8:58 PM tobyhillman @drdlittle for how long post qualification? is it something that gets replaced by specialty characteristics or rotation traits? #twitjc -8:58 PM DrDLittle @fidouglas @DrDLittle @LizzyFerret suspect that ox/cam has a high rate of grads entering research? #twitjc -8:58 PM iamdoctord Final discussion point - YES - and I agree with the reviewer with his 'highly political study' comment... #twitjc -8:58 PM EverythingVHost @twitjournalclub If tweets help our understanding of papers, of course contributions from experts in their field will #twitjc re:finalpoint -8:58 PM LizzyFerret RT @fidouglas: I think being able to see reviewers' (and indeed others') comments helps highlight possible problems with the paper. #twitjc -8:58 PM

cebmblog Papers much easier 2 understand when U simple present proportions (numbers who passed & failed) #twitjc lk 2 also poor use of odds ratio -8:59 PM thefourthcraw @MhairiHepburn MRCP acting as surrogate for any membership, in principle anyway? (as someone with MRCPCH) #twitjc -8:59 PM LizzyFerret @DrDLittle: I agree with the reviewer who questioned how useful this paper is. It is v.interesting but not at all useful #twitjc Hear Hear -8:59 PM mcumcu Definitely helpful to see comments from the reviewers, interesting that one of them had reservations about letting it be published #twitjc -8:59 PM tobyhillman @mhairihepburn ouch - how are you measuring achievment (COI - MRCP(UK) holder :) ) #twitjc -8:59 PM amcunningham @themattmak you're very welcome:) #twitjc my husband has brought me my dinner! #spoiled -9:00 PM fidouglas @DrDLittle Agreed. Does it suggest how to change teaching? Didn't we already know that diff med skls had diff grad success rates? #twitjc -9:00 PM alicexelizabeth @DrDLittle agree. Important topic but the paper isn't groundbreaking #twitjc 9:00 PM DrDLittle @tobyhillman I don't know, I think that it tends to even out quite quickly, or that is what I have observed. #twitjc -9:01 PM amcunningham @fidouglas but we have no evidence that teaching has anything to do with future mrcp performance... that's the surprising bit! #twitjc -9:01 PM fidouglas Are there studies looking into career paths from different med schools? Do some places produce more surgeons / GPs / etc.? #twitjc -9:01 PM loriblack @fidouglas @behrouzn #twitjc bit late to the party but thought I'd mention I could also withhold my AS grades (2008 entry). -9:02 PM DrDLittle @fidouglas not at all, it doesn't provide any evidence that performance is related to differences in undergrad teaching. #twitjc -9:02 PM MhairiHepburn #twitjc @tobyhillman @thefourthcraw how about MRCGP then- that's the most prevalent one? or look at all membership exams (difficult tho) -9:02 PM adriamarilla #twitjc Read reviewers' comments will help to understand possible problems with the paper. -9:02 PM tobyhillman @drdlittle I feel that uni effect peters out after about 3 years, but depends on distance moved. Higher trnees - rotation/specty #twitjc -9:02 PM cebmblog odds ratio only appears in discussion 67% passed in Newc 32% Liv 1st time round OR reported 4.3 - see why odds are no good RR 2 #twitjc -9:02 PM DrDLittle @amcunningham @fidouglas particularly surprising when they seem to suggest that this suggests a need for a national exit exam! #twitjc -9:03 PM PasTest @DrDLittle http://t.co/cGjvDkz - and not 100% posts are filled (Tooke report had graduating students at 7,900 nationally) #twitjc -9:03 PM behrouzn the pape does not show any relation with teaching. Bham is a big school while Oxbrdige has small tutorial but performance is similar #twitJC -9:03 PM MhairiHepburn #twitjc I wonder if students at some unis are more inspired to become physicians than others? possible confounder -9:03 PM amcunningham @tobyhillman they mention a paper on MRCGP but I can't find it published... can you? #twitjc -9:03 PM DrDLittle @fidouglas well this study begins to suggest that because different proportions of grads take MRCP. Not sure if any other evidence? #twitjc -9:03 PM

iamdoctord RT @MhairiHepburn: #twitjc I wonder if students at some unis are more inspired to become physicians than others? possible confounder #twitjc -9:03 PM thefourthcraw @MhairiHepburn I love the idea of a rogue medical school somewhere churning out nothing but surgeons #twitjc -9:04 PM mcumcu @fidouglas @amcunningham it'd be nice to see if course type (lecture based? PBL? A mix?) influences performance, esp in PACES #twitjc -9:04 PM tobyhillman @mhairihepburn I think RCGP provide a breakdown on their site - will take a look and try 2 get back. Definitely give sex/race/uk/img #twitjc -9:04 PM fidouglas @mcumcu @amcunningham How old is PBL? I was under the impression it was a fairly new invention... #twitjc -9:05 PM amcunningham @iamdoctord @MhairiHepburn yes, those med schools more inspired to be physicians in final year do better-but not sig in final model #twitjc -9:05 PM LizzyFerret @fidouglas that would be very interesting - with the amount of focus on CommSkills etc at @DundeeUniv id imagine it'd say ++ GPs! #twitjc -9:05 PM DrDLittle @alicexelizabeth absolutely, would be interesting to see more focused research into different teaching methods. #twitjc -9:05 PM EverythingVHost @fidouglas UK schools started using it in early '90s #twitjc -9:05 PM fidouglas @EverythingVHost Oh ok, so long enough to get some decent data. That would be worth investigating. #twitjc -9:06 PM iamdoctord Did any lower scoring school graduates get jibes after this paper was published? #twitjc -9:06 PM amcunningham RT @iamdoctord: Did any lower scoring school graduates get jibes after this paper was published? #twitjc -9:06 PM mcumcu Or, did the performance of a particular school change when they started to lean more on PBL? #twitjc -9:06 PM DrDLittle @fidouglas Manchester started it in UK in 1994 but has been around elsewhere since late 70s. (DOI: very anti PBL) #twitjc -9:06 PM MhairiHepburn #twitjc could i damage dundee's chances in any future repeat of this experiment by persuading everyone (esp the best) they want to do psych? -9:06 PM PasTest @fidouglas @mcumcu @amcunningham It was definitely around in 2000 at Manchester #twitjc -9:07 PM tobyhillman @mhairihepburn http://ht.ly/5qECL should go to pdf of MRCGP by med school etc #twitjc -9:07 PM DrDLittle @mcumcu no, I was hoping this would be demonstrated but it isn't. #twitjc -9:07 PM DrDLittle @mcumcu In-fact, they suggest Glasglow students did slightly better after introduction of PBL #twitjc -9:07 PM amcunningham @iamdoctord #twitjc the disliking reviewer suggested should have had ethical approval for this reason- bu (cont) http://deck.ly/~cP51L -9:07 PM MhairiHepburn @iamdoctord #twitjc no I didn't but then i have been described by Carstairs nurses as "f***ing scary" and don't generally get jibes ever :) -9:08 PM thefourthcraw @DrDLittle What was the natural year-to-year variation at Gladgow though? #twitjc -9:08 PM LizzyFerret @MhairiHepburn I don't think you'll manage to get this Dundee medic into psych... Sorry! :) Interesting but not for me! #twitjc -9:08 PM behrouzn @mcumcu that will be very interesting as strong PBL school seemed to perform worst at the time of the study. #twitJC -9:08 PM

thefourthcraw @DrDLittle What was the natural year-to-year variation at Glasgow though? #twitjc -9:08 PM amcunningham @tobyhillman can you find the more uptodate study cited on MRCGP- I couldn't? #twitjc -9:08 PM thefourthcraw @MhairiHepburn Carstairs nurses usually capable of taking on armoured tanks single-handedly. You MUST be scary. #twitjc -9:09 PM DrDLittle @amcunningham @iamdoctord problem that this data is automatically 8 yrs out of date therefore useless for choosing a med school #twitjc -9:09 PM twitjournalclub Right, I think we'll formally bring the discussion to a close, but feel free to keep chatting away. #twitjc -9:10 PM thefourthcraw @twitjournalclub Thank you for organising! #twitjc -9:10 PM amcunningham @tobyhillman it doesn't seem to have been published in format cites at least= no journal stated #twitjc -9:10 PM RichardJMcManus RT @cebmblog: best measure of doc: what your patients say about you & whether your practice reflects EBM #twitjc but how to measure? -9:10 PM DrDLittle @thefourthcraw there is a graph in the additional documents which seems to suggest that the improvement persisted. #twitjc -9:10 PM amcunningham #twitjc @DrDLittle the study isn't about helping to choose med schools it's about quality control in #meded -9:11 PM LizzyFerret Thanks to @twitjournalclub @fidouglas and @amcunningham for tonight's #twitjc -9:11 PM thefourthcraw @DrDLittle Damn. #twitjc #notagreatPBLfaneither -9:11 PM twitjournalclub Thanks to @amcunningham for helping! We'll have a transcript and summary up in the next few days. #twitjc -9:11 PM DrDLittle @twitjournalclub thanks :) Really interesting discussion on an interesting, albeit not that useful paper ;) #twitjc -9:11 PM alicexelizabeth @twitjournalclub #twitjc thank you; not sure if I added anything useful but enjoyed it. Thanks @fidouglas and @silv24 for setting it up! -9:11 PM twitjournalclub We'll tweet when the next paper has been decided. #twitjc -9:12 PM