Document OKXzREMJOR20QKjve9BZzNqK

Corporate Health-Physics CorporateOccupationalMedicine CorporateProduct Responsibility Corporate Toxicology 3M Medical Department 3M Center,220-2E-02 PO Box 33220 St.Paul,MN 55133-3220 651733 1110 Working Memorandum on Data QualityAssessment(oftheMean and Range of PFOS (ppb)LevelsinCurrent and HistoricaHluman Populations) ThisreportdocumentsBattellee'vsaluatioonfthestatistiqcuaallitaynd defensibiliotfy 12 datasourcesof serum perfluomoctanesulfona(tPeFOS) levelsfound innonoccupationahluman populations.These datawere summarized inthe3M whitepaper, "PerfluorooctanSeulfonate:CurrentSummary ofHuman Sera,Healthand Toxicology Data" providedtotheEPA inJanuary,1999.The purposeof thisinitiaclollectioonf human non-occupationaslerum samplesby the3M Medical Department was to determinethepresenceor absenceofPFOS inhuman blood from currentand historical samples. 3M requesteda reviewofthesedata by Battellteo determinewhat additional inferencescouldbe obtainedby analysesofthesedata.Battellsetatisticiafneslthatonly thepooled serum from the 18 U.S.blood banks was judged defensiblefora statistical analysis.Of thesedata,thegeometricmean PFOS blood levelwas 28.2ppb (95% confidenceinterva2l3.9-33.3).Regionaldifferencecsouldnotbe explainedwithout additionadlata and analyses. WOR93NG MEMORANDUM on DATA QUALITY ASSESSMENT by Bruce Buxton,Warren StraussO,wen BATTELLE September22,1998 Chang ThisreportdocumentsBattellce'vsaluatiotnoiateofthestatistiqcuaallityand defensibiliotfy 12 datasourcespresentedby 3M staffwho suggestedthatthesedatasourcesmight be used to estimatebloodlevelsofperfluorooctanesulfon(aPtFeOS) innon-occupationahluman populationsA.ccordingto3M, thesedatawere collectetdohelpaddressthreediscovery questionpsosedtothe3M MedicalDepartment:(1)IsPFOS detectablienindividuahluman bloodfrom a corpomte-bas'e3aM employeepopulatioknnown nottohaveworked in3M fluorochemicamlanufacwfingplants(butknowledgeableaboutfluorochemicals()2?)IsPFOS detectabliengeneralpopulatiopnooledbloodsamplesfrom differengteographicalocations acrosstheUr@ted States?and (3)IsPFOS detectablienhuman bloodwhich would havebeen collectepdriortotheintroductioonfPFOS-based chemicalsintothemarketplaceofthegeneral population?SubsequentlyB,attellweas askedwhetherany furthesrtatisticadlelfyensible inferencecsouldbe derivedfrom thesedata;thisreportaddressesthatquestion.Inthisreportwe (1)lisitmportantcharacteristwihcisch datasetsmust haveinordertosupportstatistically defensiblienferences(,2)judgeeachof3M's 12 datasourcesagainsthesecriteriaan,d (3) statisticaalnlaylyzetheavailabldeata,where appropriate. Our findingsfrom thisinitialssessmentofstatistidcatlaqualitcyanbe summarizedinthe followinpgoints: o We suggest12 criteriwahich defmc statisticadlelfyensibldeatasources.Of the12 data sourcespresentedby 3M staf&no sourcewas judgedsufficienotn all12 criteriaan,d only one source,the1998 pooledserum fromU.S.bloodbanks(datasourceA3),was judged sufficienftorsubsequentstatistiacnaallysis. o The 1998pooled serum bloodbank dataindicateadnationalgeometricmean PFOS blood levelof28.2ppb witha 95% confidenceintervaolf23.9to33.3ppb. Significanrtegional differenceasreapparentbutcannotbe explainedwithoutadditionadlataand analysis. Data CharacteristicfsorStatisticD2eifensibilill Inordertomake statisticadlelfyensiblceonclusionfsroma setofdata,seveml requirements must be met regardingdesignofthestudyandimplementatioonfthedatacouectionefforlEach oftheserequirementscanbe associatewdithimportantquantitatiavseswnptionsunderlyingthe numericalmodels which uc usedtomake statistiicnaflerencesI.nourassessmenthem, we consideredthreedifferenttypesofestimatiopnroblemsregardingPFOS blood levelwshich one might want toconsider:(1)temporalchangesinPFOS bloodlevelosverthepastseveral decades,(2)spatiatlrendsinPFOS bloodlevelfsorvariousregionsacrosstheU.S.,and (3) demographictrendsinPFOS bloodlevelsamong varioussiabpopulafloinnstheU.S.,or worldwide.Table I summarizes12 statistirceaqluirementfsordatadefensibuiiayn,d associates eachwiththethreetypesofestimatiopnroblemslisteadbove.Note inthistablethatthefirsftive Table1.StatistiRceaqluirementfsorDefensibQlueantitatiCvoenclusions StatisticRamllrtmentx 1 Detaileddocumentation of the study objectivesand approach, Includingsampling design (number and types of samplesi,and sampling and chemical analysisprotocols 2. Specificatioonr-awell defined targetpopulationabout which Inferenceisto be made (e.g., the entireU.S. population,or some specifiedsubpopulation) 3. Unbiased date collectionprotocolswhich ensure thatallsamples are demographically representativoef the targetpopulation;generallythisimpliesrandom sampling from the tarnatDopWation 4. Along withPFOS blood levels,accuratecollectioonf importantpredictoror explan@t-oryInformations,uch as demographics likeggplonder, race,ate. 5. Compositing of samples, when done, Isperformed only for samples withinthe same 3ubpopulationso thatknportant potentialdifferencesamong subpopulationsare preserved in the resultindgata S. Sampling Isperforfnedat several(i.e.a,t leastthrealpointsin time which arespaced at regulartimeinterval(se.g..everv fivefears) 7. At each pointIntime, sampffig Isperformed cross-sectionallfyrom the entiretarget populationorsubpopulations S. Sampling 1,p9erformed to provide uniform coverage of the entirespatialdoma7in of Interest le.g.t,he entireU.S.) 9. Date Iscollectedboth formultiplelocations(e.g.,regionsor citiesa)cross the spatialdomain - and tarmuldple Individualswithineach location 10. Alldata arecollect*dat approximatelythe sanro pointIn time to eliminatethe potentialfor confoundingdue to temporal trends 11. Specific;t-ioofntargetsubpopulationswhich arewell definedinterms of important demographic parameters (e.g.,age, gander,race.etc.1and from which sampling israndomly performed 12. Eitherfixsampling atone tifneand spatialocation(e.g..city)or ensure thatsampptlliiminegI.nfgor=all denwgfaphic subpopulationsIsperformed ina balanced design across the same timesfaaonnrdd'a" locations(i.e.t,o eliminatepotentiaclonfoundlni3due to time or location) Tv@as Temporal: @.Che'RUOS Estitnatior Spud& -Trehdt me be 2 requirementsaremore generalinnatureand applyequallyto allthreetypesof estimation problems,whilethelastsevenrequirementaspplymore specificaltloya differenttypeof estima@on problem. Assessment of 12 Data Sources On May 12,1998,3M staffaxedtoBattellaetablelistin1g2 datasourcestheybelievemight be used toestimatePFOS blood levelsinnon-occupationahluman populations.Thistablehasbeen retypedas shown inFigureI. Subsequently3,M staffsenttoBattellelectronifcilewsiththe PFOS blood datatheyhad availablea,swellas associatewdrittendocuinentatioonn thedata sourceswhich theyhad. Based on thisinformationB,attellaettempted,and was partially successfult,oobtainthroughliteratusrearchesmore extensivedocumentationon these12 studies.However, inmany_ casesno additionawlrittendocumentationcouldbe found. Given theinfomation senttoBattellbey 3M and otherdocumentationBattelloebtainedfrom the literaturseearch,Table 2 summarizes our assessmentofhow weU each of the12 datasources meets each of the 12 defensibilirteyquirementlsisteidnTable1. InTable2,a ",/"indicates thatthe.given datasourceseems tomeet theassociatedefensibilictryiteriownh,ilea 'X' indicatesthatthedatasourcedoes notmeet thecriterionT.his assessmentby Battelliesbased solelyon thewrittendocumentationwe were abletoobtain,eitherfrom 3M ortheliterature review. In a largemajorityof cases,no writtendocumentationwas found and so we can not judge whetherthedefensibilictryiteriaaremet by a datasource,ornot. In thesecases,a "T' is listedinTable 2. In some cases,notedas documentationmay have been found aboutthe initiasltudywhich collectetdhebloodsamples,butno documentationwas availabldeescribing how particulasrubsamples were selectedtobe sentto3AL Also,insome caseswejudged thata givendatasourcewould be appropriatfeoraddressinjgustone " ofestimatiopnroblem (i.e., temporalchanges,spatiatlrends,ordemographictrends)a,nd so defensibilictryiterifaorother problems would not be applicableT.hesecasesaredenotedby a "-" inTable2. Table3 lists more detailednotesabout eachofthedatasourcesforthosecriteriwahere sufficienitnformation was found inthewrittendocwnentationtowarrantcomment. FigureI Summary ofMean and Range ofPFOS fppb)LevelsinCurrentand HistoricHaulman Popula AllDataAnalyzedInFebruary- April1998 Populations A. CurrentPopulations(bloodcollectedIn 19981 1. Non-occupationallyexposed 3M employees 2. Commercial pooled serum samples lntergenLaboratory Sigman Laboratory 3. Pooledserum from 21 separateU.S. blood banks (seeFigureI formore daticill B. HistoricaSlamples (chronologicaolrder) 1. Korean War era U.S. militarryecruits1,948 to 1951 2. Swedish samples. 1957 3. Michigan BreastCancer Study, 1969-1971 4. Swedish samples, 1971 5. MRFIT pooled calibrationsamples, 1976 6. MRFIT pooled calibrationSaMP103, 1980 7. China samples (Linxian,ruralprovince),1984 S. MRHT Individual3aMP103, 1985 S. China samples lshandong, ruralprovincel,1994 Rounded to nearest ppb Not deto"od Dascriationof SamDlo 31 individuals 3 samples each with @--100donors 3 samples each with >-100 donors 3 to 6 pooled samples perlocationwith 5 to 10 donors per sample 10 pooled samples with 10 donors per sample 10 individualsamples 5 individuaslamples 10 individualsamples 6 pooled samples with unknown donors per sample 3 poolad samples with unknown donors per sample 6 Individualsamples 3 individuaslamples 6 Individualsamples number of number of Mean* (p 47 44 33 29 N.D. 2 33 1 31 23 N.D. 31 N.D. Note: This figurewas retyped from a fax sent to Battelleon 5112/98 withoutchanging possiblemistakes. 4 Table 2. OverallAssessment ofDefensibilitoyf12 Data Sources statmic@wniquireniom 1. Detailedocumentation Al A2 A3 :@8@ 'Of *1? 01? /? 2. Definedtargetpopulation ? 3. Unbiasedsamplingprotocols ? x7 4. Accurateexplanatory information 5. Appropriatceompositing ? ?7 6. Regularsamplingintime ? Z Ctiommepletesamplingateach ? S. Uniformsamplingacross space 9- Multiple35MPIOS locally 10. Samplingatsame iimo(r,) 11 . Defined demographic 77 7 subpopulations I' 12. Saanmdplloicnagattisoanm(est)ime(3) ? Adequatelydesignedand implemented X - Inadequateldy"ignod orimplemented ? = Unknown due toinsufficiednotcumentation Not applicablteo thisdatasource S3@ 1 M. 1(7 /? /? ? ??? ??? 7 ? ??? ? ? ? 777 ? :B5 Be -.67.1.B8 .89 if? WI? *I? xx x? ?? ?? ?7 7 ? ? ? ? 77 7 7 7 ? ? Table 3. NotesRegarding Defensibilitoyf12 Data Sources Al.:.No"66Upatlbi!My4.3iposod-.3Msi@pi6yeis. 1 Detaileddocumentation- The onlydocumentationisinGearyOlson's8/15/98letter 2. Definedtargetpopulatio-nThe targetpopulatiownas 3M employeeswithminimaloccupationaelxposure 4. Accurateexplanatoriynformatio-n Some demographicinformatiownas collecteadnd includeds,uch as age and gender 12. Samplingatsame tima(3)and location3(-,sT)he dataappearsto be onlyfrom employeesatthe corporate center A2.! ComiMr@Jig4coWd%ibrtim:sai@pi" 1 Detailedocumentation- The onlydocuimentatioinsinGearyOlson's8/15/98letter A3 bloodba6k& 1. Detaileddocumentation- The onlydocumentationisinGearyOlson's8/15/98letter 3. Unbiasedsamplingprotocol-s Thereispotentiabliasas a resulotfthe50% refusarlate 8. Uniformsamplingacrossspace- The samplesaregeographicaldliyspermw S. Multiplseampleslocally-Theraeremultiplreegionsand multipliendividuawlisthineachr!glon Bl!-K.o;r.ian-.WiroivU*.S.:WrFim@ 1. Detaileddocumentation- The onlydocumentationisinGoarvOlson's8/15/98letter 2. Definedtargeteopulation The targetpopulatioWnas U.S.militarryecruitfsromthe KoreanWar era 82.'iSwOdiih:i"O";..1957. 1. Detailedocumentation- The onlydocumemationisinGearyOlson's8/15/98letter B3.:M!S!ftahR"gt.cincer:BuWy;.1969-1971 1. Detailedocumentation- The onlydocurnantaftoinsinGearyOlson's8/15198letter B4. "$*"sh @umpi". .1971.1 - Detailed documentation - The only documentation is In Geary Olson's 13115198 letter 5 Table 3. Notes Regarding Defensibilitoyf12 Data Sources(Continued) 05. Mffff pboled@cigbr6tion 1976:' 1. Detailedocumentation- MRFIT isdocumented inGeary Olson's8115/98lonerand inmultipljeoumals includinJgAMA, Val.235, No. 6,Feb.23, 1976, but subsamplingfor3M isnot documented 2. Definedtargetpopulatio-n-The targetpopulatiownas men aged 35-57 with multiplreisksforheart disease 3. Unbiasedsamplingprotocol-s The samplingprotocolselectedcertaincharacteristiwcist,h multiple scroortings 4. Accurateexplanatoryinformatio-n Informatioanboutthe originaslample populatioWn83 recorded,butno informatioanbout thepooledcalibratisoanmple is,known 06: MRFIT ii66W..cabiit[6aiiiitiow,1980 1. Detailedocumentation- MRFIT isdocumented inGeary Olsen's8/15/98lettearnd inmukiplejournal$ includinJgAMA, Vol.235, No. 6, Feb.23, 1976, butsubsampfingfor3M isnotdocumented 2. Definedtargetpopulatio-n The targetpopulatiownas men aged 35-57 with multiplreisksforheart disease 3. Unbiasedsamplingprotocol-s The samplingprotocolselectecdertaincharacteristicwsi,thmultiple screenings 4. AccurateexplanatorIynformatio-n Informatioanbouttheoeig(insaalmple populatiownas recorded,but no informatioanboutthe pooledcaribratisoanmp a isknown 07 CNna. i4rOgg6*.'(tM)dtiumr,a.lo.tavin6i)1.984. 1 . Detailedocumentation- Documented inGeary Olson's9/16/98lettearnd inJournalof theNational Cancerlnst.V,ol.85, No. 18,Sept 15f 1993,butsubsamplingfor3M isnot documented 2. Definedtargetpopulation--The targetpopulafiownas alltheresidentosfL:mxianaged 40-69 without debilitatidnigseasesorpriorasophagealorstomach cancer 3. Unbiasedsamplingprotocol-s The sampleappearsto be representatiovfethetargatpopula6on 4. Accurateexplanatoryinformatio-n Informatioanbouttheoriginaslamplepopulatiownas recorded,butno informatioanboutthe individuaslampleisknown 'Ba.:'@MRRiTndMdUWsirii@l";:SlOS 1. Detailedocumentation--MRFIT isdocumented inGeary Olson's8/15198lettearnd inmultipljeournals includinJgAMA, Vol.235, No. 6, Feb.23, 1976, butsuboampringfor3M isnot documented 2. Definedtargetpopulatio-n-The targetpopulatiownas men aged 35-57 withmultiplreisksforheart disease 3. Unbiasedsamplingprotocols- The samplingprotocoslelectedcertaincharacteristiwcist,h multiple screenings 4. Accurateexplanatoriynformatio-n Informatioanbout-theoriginarlamplepopulationwas recorded,butno informatioanbout the pooledcalibratisoanmple isknown B9. Chkii"mplii'(Shohda;nd',.:itiiid!pr1"9c9i4e.).-,- I Detailedocumentation- The onlydocumentationisinGearyOften's8/16/98letter The main conclusionfmm our overallassessmentofthe 12 datasourcesisthatinmost cases thereissimply not enough writtendocumentationofthe studydesignand sampling protocolsto judge whether theresultindgataarestatisticaldleyfensibleo,r not. In some cases(e.g.N,MFIT) documentation isavailableon theoriginalstudyunder which an entiresetofblood samples was collectedb,ut no documentation isthenavailableon how thesubsampling was performed to send samples to3M forchemical analysis.IfeithertheoriginalSampling orthe subsampling was biased,thenthe dataobtainedby 3M willalsobe biased(i.e.s,ystematicalltyoo high or too low). Given thislackof documentation,thereiscurrentllyittlei,fany,justificatifoonrstatistically analyzing3M's dataand presentingStatisticcaolnclusions.Pooling the dataacrossstudies(e.g., tolook forblood temporal,spatialo,rdemographic trends)isnot reconunendod because of the widely differentstudy objectivesand targetpopulations2anongthe studies. Of allthe datasourcesprovidedby 3M, probably themost defmsible forstatisticaanlalysis purposes isthe datasetof 1998 pooled scruxnfrom U.S. blood banks (datasource A3). Although 6 detailedwrittendocumentationislackingi,tdoesappearthatthesamplesmay havebeen selecteidnan unbiasedmamer, atleaswtithina particulbalroodbank. We areconcomed with possiblenon-responsebiaspotentialliyntroducebdy thelargenumber ofbloodbanks (approximatelhyalfof thosecontactedw)hich declinetdosend3M thesamplestheyrequested. There arealsopotentiabliasesresulfinfgromtheprobablelackofcontrobly eachbloodbank concerningthedemographicsofthedonorsofthesampleswhich were sentto3M. However, recognizintghesepotentiasleriousdatadeficienciwehsich limitourabilittyomake valid inferencesi,nthenextsectionwe have subjectetdheresultinPgFOS bloodleveldatatoa limited statisticaanlalysis. St,atisticAanlalysisof Bloo-dB-ankData The recentlycollecteadnd analyzedpooledbloodsamplesfroman independenNtfinnesotand 17 geographicalldyispersedRod Crossbloodbanks havebeencombinedtoestimateanationally representativmean PFOS (ppb)levelH.owever, a varietoyfassumptionsarenecessary, including: 1) Each pooledsampleisrepresentatiovfetheregionaplopulatio(ni.ee.a,chmember ofthe populationhad an equalchanceofbeingrepresenteidnthesample). 2) Each pooled sample has a known number ofdonors,and eachdonorisrepresenteedquallyin thesample. 3) The sampling and analyticamlethods forobtainingserumPFOS levelissthesame acrossall samples. Itisunlikelythatthesamplesobtainedby 3M completelysatisfaylloftheaboveassumptions. Also,thedatahave thefollowinagdditionallimitations: 1) The bloodbanks represenotnlythosedw donatedblood- i.e.c,itiewshere thebloodbank(s) refusedtoparticipathead no chanceofbeingrepresenteidn3M's sample.Furthermorei,tis unlikelythatthesamplesobtainedfromparticipabnltoodbanks arerepresentatiovfethe generalpopulationineach geographicregion.Rather,thesesamplesarerepresentatiovfe peoplewho donateblood totheRed Cross. 2) Itisunlmown how many donorsam ineachpooledsample,norisitknown how eachdonor was selected. 3) Most ofthesesamples came fromRed Crossbloodbanks,soitislikeltyhosesampling methods arethesame. The Minnesotasamplesprobablydo nothavesignificantdliyfferent methods. As alloftheanalysiwsas performedby 3M, theanalyticmaelthods shouldbe the same. Although thesedatamay notsatisfaylloftheassumptionsdetaileadbove,thefollowingrandom effectmsodel was used toestimatea nationagleometricmean PFOS level: ln(PFOSij-) @L+ Bi -@cjjw,here 7 ln(PFOS#)representtshenatuml-lotgmwfonned PFOS levelassociatewdiththejthpooled sample from theithBlood Bank, expcu)isthenationalgeometricmen PFOS levelBi isthe differenc(eon thenatarallogscale)between themean PFOS levelattheithblood-bankand the nationalme=, and eijistheresiduaeltrorleftunexplainedby themodel (thedifferencbeetween theindividualsample PFOS measurements and themean PFOS levelatitscorrespondingblood bank). Itisassumed thatBi followsa normal distributiownithmean zeroand varianceebetween BloodBanks, and thateiifollowsa normal distributiwointh mean zeroand varianceo,2WitWn BloodBanks. Table 4 providestheestimatesobtainedfrom fittintghismodel tothedate-Table 5 listtshe individuarlegiongeometricmeans which were found.tofalloutsidethe95% and 99% confidenceintervalfsortheoverallgeometricmean. Table 4. Parameter EstimatesforBlood Bank Data ii.% confidends Not;onalGoometHc Mean PFOS Level(aOl(pobi 02 Between BloodBanks [in(pob)ll e WithinBlood Banks lim(pob)) 28.2 .095 I .052 (23.9,33.3) - -- - I 122.5,35.4) Table 5. Regions With Geometric Means FallingOutsideCoufldenceIntervabforOverall Geometric Mean Region. Santa Barbara,CA Omaha, NE BillingsM,T Grand Rapids,Mi Cheyenne, WY East Orange, NJ Newark, DE Anchorage, AK Las Vegas, NV Santa Rosa, CA Corpus C risfiT,X Kansas City,MO Davenport. IA Scottsdale,AZ Meridian.MS Minneapolis,MN Lafayette,LA Greenville,SC n' Tn c 0 b :-0i.litild90% C.i" -0t'i*tdd'e:99'C%l 3 15.9 5 16.2 3 22.4 3 23.1 3 23.3 3 23.5 5 23.9 4F 3 25.3 3 28.6 3 27.3 3 28.9 3 29.6 3 31.9 3 36.5 3 37.5 6 4S.6 3 45.7 3 51.5 The above resultsdemonstratethat12 of 18 bloodbanks had geometricme= PFOS levelsthat felloutsidethe95% confidenceintervaclonstractefdortheestimatednationalgeometricmean PFOS concentrationT.his high'nwnberofregionalmeans failinsgignificantalbyove and below theoverallmean isa resultof thefactthatthereissignificanvtariatiobnetween thegeometric mean PFOS concentrationosbservedateachbloodbank. Itispossiblethattherearemeaning&l factortshatcan explaina portionofthisvariabilitFyo.r example,thebloodbanksin MinneapolisM,N, LafayetteL,A and GreenvilleS,C may be locatedincloseproximityto industrietshatapply3M products(suchascarpetmanufactm-ersw)h,ileSantaBarbara,CA, Omaha6 NE and BillingsM,T may be isolatefdromsuchexposuresources.However, differenceisnthedonorpopulationateachbloodbank may be,anotherpossibleexplanatiofnor theobservedvariabilibteytweenbloodbanks. To investigatheereasonsfortheseregional differenceasd,ditionadlataanalysiwsithmore explanatoriynformatiownould be required. ResaTnWing the 19 Blood Bgnks 3M askedBattellteheiropinionofthestatistimcearlitofresamplingthese18 bloodbanksto helpevaluatteheuncertaintiynPFOS bloodlevelswithingeographicallocationsI.nthecurrent data,therearevariationosf20 ppb ormore betweentheminimum and maximum PFOS levels measured indifferenptooledsamplesfromthesame geographicalocation. Ifadditionailnformatiocnanbe collectefdromthebloodbanks,Battellreecommends thatthe most beneficiadlatawould be demogmpbic fiffonnation thepooledsampleswhich were previouslsyentto3M. We suggesthatitismost importanattthispointintimetobetter understandthefactorisnfluencintghePFOS bloodlevelvariabilitCyo.llectinngew blood sampleswithoutdemographicinformatiownould helpquantiftyhevariabiliotnye shouldexpect fromthe18 localpopulationsb,utitwould nothelpexplainthesource(so)fthatvariabilitFyo.r designingefficienftuturesamplingprograms,we needtoknow bow much variabilittoy.expect, butmore importantltyhemajorenviroranentbaelh,avioraln,utritionadlemographic, socioeconomic,cte.factortshataffecthatvariabilistoythatwe cantrytocontrolthesefactors (e.g.s,tratifby thesefactorst)oobtainmore accuratseiunplinrgesultsA. secondconcernwith resamplingthese18 locationissthattheymay constitutaebiasedsamplingofthegeneralU.S. populationB.y takingmore samplesfromthesecitiewse, mightbe simplygeneratinmgore precisebuthighlybiasedestimates. TOTAL P.iO