Document EQBNXLRL1DqE3G9ZqDd7Eqy0

How Well is NEPPS Working? A Summary Comparison of Several Recent Evaluations National Environmental Poerfftohremance Partnership System Prepared for 1999NoEvCeOmSb/eErP3A0 N- EDPePcSemWbeorrk2shop Baltimore, Maryland Prepared by Ross & As1s2o1c8iaTtehsirdEnAvvireonnume,enStuailteCo12n0su7lting, Ltd. Seattle, Washington 98101 Sierra Club v. EPA 18cv3472 NDCA Tier 10 ED 002061 00178452-00001 A Summary Comparison of Several Recent NEPPS Evaluations Introduction This paper summarizes and compares a number of recently conducted evaluations of the National EofnfviciiraolnsmanedntoalthPeerrifnotremreasntecde Ppaarrttnieesrswhhipo Sayrestceomns(NidEerPinPgSn).e*1xtItstiespinstteondimedprfoovreustheebNy ESPtaPteSapnrdocEePssA. In particular, attendees at the 1999 ECOS-EPA NEPPS Workshop, and senior EPA and EGOS roseyfpfniocoriptassl.siscThahiamergpsertdoocwehisitgshhojlofigidnhitstetkivlelaayltuifoaitnnidobinyngoitsfstonhfae,tNuasrEewPteePnlSldapssrtoothcseeismcsopmnlisafiyystfaeinnnddcietehlsiimsaianndhaetdelpidfffeuetlrapeirnlsicm.eRserea.amTdohenrissgb,wrtihitehef aannailnytseersesatnidnreucnodmermsteannddaintigonthsecoisnstuaeinserdaiinsetdheheorreigwinialll rbeepnoerftist.from reading the detailed problem bTyheloroepkionrgtsavtarayninumscboepreo. fSsotmateesexaanmd iEnePAtheReengtiiorenaNlEoPffPicSeps.rocOetshse, rdsrafwocinugs soynstaempiacrtcicounlcalrusEioPnAs Region's or State's experience, specific program areas, and/or one component ofthe NEPP Sprocess, e.g., the Performance Partnership Grant (PPG). EAnsvdiersocnrmibeendtainl tPheer1fo99rm5 EanCcOe SP/EarPtAneJroshinipt CSoymstemmit,mtehnet tpouRrpeofosermoOf NveErsPiPgSht aisndtoCr"esatrteenagNthaetnioonuarl protection of public health and the environment by directing scarce public resources toward iemnhparonvciinngg oeunrvaircocnomunetnatbaillirteysutoltsth, eaplluobwliicngansdtattaexspgayreeartse"r. fTlehxeibNilEitPyPtSo parcohgiervaem,thaocsceorrdeisnuglttso, athnids agreement, will "achieve more integrated environmental management, promote pollution prevention NanEdPePnSh:ance environmental results". The Joint Commitment defines seven principal components of ianncerewasaepdpuroseacohfteonvpirroognrmamenataslsegsosamlsenantsdbiyndSitcaatteosrs ednifvfeirroennmtiaelnotavleprseirgfhotrmance agreements performance leadership programs paujobliinct osuystrteemachevaanlduaitnivoonlvement, and Tinhfeivreemspaeicniifnicgcteaxtetgoorrgiaens,izroesugthhelyevcoalrureastipoonnsd'icnogntcoluthsieopnrsininctiopaalNseEt PoPf gSecnoemraplofinnednintsg:sj,oainntdsftirnadteinggics planning/priority setting, administrative flexibility/burden reduction, programmatic flexibility, performance measurement, and public participation. 1 The reports that were reviewed include: seven Office of Inspector General (OIG) Management Assistance Reviews (MARS) for the States of Alaska, Colorado, Connecticut, Delaware, Georgia, Indiana, and Texas; two OIG Regional Audits of Regions 4 and 8; a systemic review of the NEPPS system by the United States General Accounting Office (GAO); and meeting summaries from the Denver and the Providence NEPPS Training Sessions. The National Academy of Pubhe Administration (NAPA) is also reviewing the NEPPS process; no public review drafts were available to include in this summary document. 1 Sierra Club v. EPA 18cv3472 NDCA Tier 10 ED 002061 00178452-00002 A Summary Comparison of Several Recent NEPPS Evaluations Attached to this memo are two tables that provide a more detailed comparison and summary of each reeapchorrte'spfoirntd'sincgosn. cTluasbiloens1,otnhehNowEPwPeSllENvEaPluPaStiohnass SacuhmiemvaerdyitMs aotbrijxe,ctpivroevsiadneds apnoteeanstyiawl ainytthoesfciavne cexataemgpolreiessfrnoammeedacahbroevpeo.rt TinatbolePo2s,itSiveeleOctuedtcoFminedsinagnsdaRnedasEoxnasmfpolresS,ucocrgesasn,izEexsakmepylefsinodfinNgEsPaPnSd Successes, Barriers and Reasons for Lack of Success, and Recommendations. These more detailed findings and examples are also organized according to the five categories described above. General Findings The evaluations are generally quite consistent in the benefits and successes they attribute to NEPPS Rimepcloemmmenetnadtiaotnio, nass fwoerlilmasprionvtihnegptrhoebNleEmPsP, Sanpdruoncedsesr,lyhionwg ecvaeurs,evsaoryf twhoidseelyp.roTbhleemms,ajothrietyy iodfenthtiofsye. interviewed for these reports supported the philosophy and objectives of NEPPS. All the reports tfhoeunsdtatthea-tEpPaArticreiplaatniotsnsinhitph.e NInEPpPaSrtipcruolcaer,ssthbeelieevvaelduathtiaotnist hidaednbtiefieendbiemnepfriocviaeldancdomhamduinmicparotivoends bisestuwees,enasEfPreAquanendttlhyecsitteatdeNs,EaPsPwSelbl eansegfrietsa.teHrosweneivoerrm, manoasgteemvaelnutaatitotennptiaorntictoipparnotgsraalmsopbreiloireivtieedstahnadt progress to date falls substantially short of the overall promise and potential ofNEPPS to improve pthreocSetsaster/eEsuPlAtedpairntntehreshmiposatndsuecnchesasnfcuel tohuetcaocmhieesveinmethnot soef setnavteirsonamndenEtaPlAresRueltgsi.onTshtehaNtEwPePrSe csuopmpmorittteadndtoindvevoelvloempienngtpoefrftohremaagnecnec-yb'assteodpmleaandaegresmhiepn. t system, and that benefitted from the active Joint Strategic Planning and Priority Setting bTehteweeveanluEaPtiAonasnidndsitcaatetes tthharot uwghhiliemthperoNvEedPPcoSmpmroucneiscsahtiaosnfaancdilijtoaitnedt palamnonivneg,toawmarodrse aripgaorrtonuesrsahnipd robust joint strategic planning and priority setting process is needed for the success ofNEPPS. Joint ustnrdateergstiacndpilnangnbinetgweanend EpPrAioraintyd ssteattteinsganadreleacdritthicealreltaotiothneshNipEfProPmS opnreobceassesdionntahahtisttohreiycalfoEsPteAr ostvreornsgigehrtjorionlet ptloanonniengbapsreodceossn wpailrltneenrsahbilpe, fefodceurasledanodnsrteasteultasg.enTcihees etovatlaucaktiloenrselsautegdgeisstsuthesatoaf accountability, program flexibility, and differential oversight more effectively . The reports cited the following current barriers to joint strategic planning efforts: bDeitswagereenemSteantet sbaonthd wEiPthAi,naEbPouAt (thpearrtoicleulEarPlyAbsehtowuelednpRlaeygiinonesnfaonrdceHmeeandtqaunadrtceorms)p, laiasnwcee,llthaes level ofnecessary oversight, the types ofdata states should report to maintain accountability, and LthaeckamoofucnletaorfEwPoArkgushidaarinncgetahbaotusthhoouwld tooccimurp;lement joint strategic planning; and EPA's size and complexity inhibiting internal communication.2 Sierra Club v. EPA 18cv3472 NDCA 2 Tier 10 ED 002061 00178452-00003 A Summary Comparison of Several Recent NEPPS Evaluations Some successful examples ofjoint planning and priority-setting were identified. For example, the Region 8 OIG Audit highlighted the joint process used by the Utah Department of Environmental rQeugaiolinty',s eansvtiartoenmreegniotanlaplrhoebaleltmhsa.gency, and EPA Region 8 to develop solutions to some of the Selected recommendations for improving the planning process include: DSStteraveteenslgotaphneadnmEthoPerAesrtiaegtaoecrhsoeudlfse-jvaoseisnloetspsstmraagetrneegteipdcr-poulcpaeorsmns,itnowge/pplrlroiodvreiidftieynsmeedtotrpinergisotprairttoeiecasecsbcseotfhuornroteaubgeihnlittEeyrP;inAagngdtuhiedajnocinet; s(etr.agt.e,gwicithpolauntnfiunlgl apgrroeceemssenttoaimmopnrgovEePnAegmoatinaatgioenrss aabnoduttoperongsuraremfouvllebrsuiyg-hitnoafnNdEcPoPoSpesrtaattieosn, emphasis on accountability concerns constrains opportunities for differential oversight and burden reduction). Administrative Flexibility/Burden Reduction bTuhredeenvareludautciotinosniwndeirceamteixtheda.t NMEoPstPNSEpPaPrtSicpiparatnictsip'aenxtpseerxiepnecrieenwcietdhsaodmmeinaidsmtraintiivsteraftlievxeibflielixtyibialnitdy and savings; however, certain states actually noted increased burden. The evaluations identified two primary impediments to flexibility and burden reduction: EPA accountability concerns lead to identical or increased state reporting requirements, as EPA psyrostgermams pmroavneagtoersbearaeninefcfleincetidvetoarlteetraninatievxeis(ttihnigs mimepaesduirmesenotf isstactleosaecltyivtiiteydutnotijlorienstuslttsr-abtaesgeidc pSltaantenifnigna/pnrciioarlitmyasneattginegmeanntdsaygsrteeemmsebnutioltnfoinrteardnmalinpirsitoerriitnigesg);raanntds along media program lines constrain the resource flexibility that over-arching NEPPS grants can offer. Utilizing results-based performance measures is more difficult than tracking traditional activity measures and will in many cases tend to increase the cost of data collection and reporting. Therefore, NinEstPanPcSeps,arnteiceidpatontisnmcroevaisnegitnovwesatrmdsenatpseirnfodramtaanccoell-ebcatsioend manedasruerpeomrteinngt ,sywstheimchmmaayy, ibneatcloenatsrtasroymtoe many states' expectations of reduced reporting. Selected recommendations for enhancing flexibility and reducing burden include the following2:23 2 The 4/15/99 Addendum to 1997 Joint Statement on Measuring Progress under NEPPS: Clarifying the Use and Applicability o f Core Performance Measures, signed jointly by ECOS and EPA leadership, affirms the joint commitment to reducing unnecessary state reporting, and establishes a policy framework that uses the value of information, balanced with its cost, as criteria for determining what information is (or is not) necessary. The Addendum, and an April 7 memorandum from Peter Robertson entitled "State Partnership on Burden Reduction," set in motion a process whereby EPA and States are encouraged to discuss potential information collection/reporting changes that would result in higher value/lower cost information. 3 Sierra Club v. EPA 18cv3472 NDCA Tier 10 ED 002061 00178452-00004 A Summary Comparison of Several Recent NEPPS Evaluations aCnrdeating state financial management systems that can flexibly administer non-categorical grants; aSdumppinoirsttirnagtiv"edcioffsetrse,nthtiraolugohvestrrseignghtth,"entihnegrtehbeyjorinedt ustcrianteggisctpaltaennreinpgo/rptriinogritybuserdtteinng panrodceossthees.r Programmatic Flexibility NEPPS participants indicated that significant institutional barriers have impeded the utilization of programmatic flexibility potentially available through Performance Partnership Grants (PPGs). The evaluations identified the following barriers to programmatic flexibility: Individual media programs'program-specific focus results in resistance to funding multi-media pErPoAjecptrsotghraatmd-obyn-optrodgirreacmtlyobregnaenfiiztaatiospnecainfdic opproergartaimon;s management constrain funds transfer across media program lines (e.g., national program offices set rules on media-specific program mStoanteeyanudsfee,dwerhailclheginistluartinveli/mreigtsulsattaotery-lreevqeulimreemdeiantpsroofgtreanmeafrumndartkramnosfneerys)(;ananddrequire tracking) according to specific programs. The evaluations did identify instances where NEPPS participants successfully utilized programmatic falbelxeibtoilittayk. eFeofrfeecxtaivmeplaed, vManintangeesootaf,thpearftlleyxiabsilaitryesoufflteroefdawcirtohstsh-emirePdiPaGa.geNncoyrtrhe-Coargroalninizaaatinodn,Uwtaahs also implemented cross-media program fund transfers. Some evaluators suggested very specific ways to improve programmatic flexibility within the current institutional framework. Elowever, the evaluations suggest that, to realize the promise ofNEPPS mfleexdiibai-lsitpye,cEifPicAparongdrastmatemsawnailglenmeeedntt.o find ways to address systemic, institutional barriers such as Selected recommendations to improve flexibility include: EPA should improve guidance on the development and implementation of the PPA^PGs that EclParAifiaensdthsetacteirscushmosutaldnccerseautnedienrcewnhtiivcehscfroorssin-pdrivoigdruaaml mfuenddiainpgrtorgarnasmfesrstompaayrtoicccipuart;eainndmulti- media initiatives Performance Measurement NoatfEtdePenPvtiSeolnopptloaincpgeesarfmaosrotmrreoarnnegcseuelmmtspe-bahsaaussriesedmomneanentn.avTgiehrmoeneemnvtaelsnuytaaslttieopmne.rsfAroecrfmcleoacrndtcianecgmolyne,sateshnuesreuevmsatelhunaatttaisosunbassktpaeanyitdicasolumpbrspotoagnnreetinsastl hPaesrfboeremnamncaedeMineamsueraessuraignrgeperdougpraomn bpyerEfoPrAmaanncdeEaCndOeSnvarireornemgaerndtaeldoaustcaosmubess.tanTthiaelFimYp2r0o0v0emCoernet over prior versions, with fewer total measures and a greater percentage of outcomes. Despite progress, measures that truly link program efforts to environmental outcomes remain far too few, 4 Sierra Club v. EPA 18cv3472 NDCA Tier 10 ED 002061 00178452-00005 A Summary Comparison of Several Recent NEPPS Evaluations and most NEPPS participants believe that performance measurement still has a long way to go before ictotnrucleypstuopfppoerrtfsoormutacnocmeem-beaasseudremmaennatg,ethmeernet.arWe mhialenystvaateriaenddidfeedaesraablomuatnhaogwertsogiemnperleamllyensut pitp. oSrtotmhee states, such as Florida and Minnesota, have gone beyond the national CPMs and developed their own state-specific measures. The reports identify a number of reasons why performance measures have not yet fulfilled their potential. Confusion exists about whether reporting of CPMs by states is required, and about the uses and acpopmlimcaotniobnesnocfhCmPaMrkss,(oarre`pthaeinyttoa mnaetaiosnuarel pinicdtiuvrideu?a')l.3s*5tates'performance, compare states using Mcheaallseunrgeimngentthaonf tprraocgkrianmg mpraotigcraemffeacctitviveintieesss, aanndd erenqvuiriorensmaenltoalngre-tseurlmts cisominmheitrmenetnlty tmo obree successful. Challenges to developing effective performance measures include the following. -- aQrueallaitcykidnagt.a Cneecretasisnareyvfaoluractoionnnsecstuingggepsrtoegdrathmatpeitrfmoramyatnackee tyoeeanrsviorofndmaetantcaol lcloenctdiiotniontos support environmental trends measures. -- Idsroivlaetrisngsuecnhviarsonemnvenirtoanl magenentaclyaicmtivpiatcyt,s loifne esntyvlierocnhmanegnetasl, qeunavliirtyonfmroemntaolthaetrtitsuidgnesif,icaanndt climate variability has proven difficult. -- aSgtaetnecycaapcaticoitnys ttoo esnuvpiproonrtmtehnetarleosouutcrocem-iensteisnsciovnestprraoinceeds.s of developing measures tying Selected recommendations for improving performance measurement include: cCoonndtiitniuones,dwevheilleorpeivnigsinpgerefxoprmecatantcieonms feoarstuhreeisrdtehvaetlotpiemepnrot ganradmutilaicztaivtiiotinetso taockennovwirloendmgeetnhtaatl it is a large, expensive, and time consuming endeavor; RdaetvaisneeeexdpseocftapteiorfnosrmanadnicnevmesetmaseunrtepmIaennst;toanredco gnize the inherent complexity and the long term Clarify state CPM reporting requirements to minimize future confusion. Public Participation Most of the evaluations spent little time addressing the issue of public participation in the NEPPS ppurobcliecsws.ouSlodmgreeraetlpyoimrtsprnoovteedNtEhPePaSbsoeuntcceomofesp,utbhlriocupgahrtbicoitphaatdiodnitiaonndalsiungpguetsatnedd gthreaatteinrvkonlovwinlgedthgee NanEdPbPuSy-ainh.igSheevreprarliorreiptyo.rts suggested that EPA and the states make future public participation in 3 The 4/15/99 Addendum (cited above) attempts to address these questions. The Addendum may have been too recent to be well known to those interviewed in the course of many of the studies reviewed here. 5 Sierra Club v. EPA 18cv3472 NDCA Tier 10 ED 002061 00178452-00006 NEPPS Evaluations Summary Matrix Report Administrative Flexibility/ Burden Reduction Programmatic Flexibility Performance Measures Planning and Partnership Public Participation MAR Connecticut (NEPPS approach 97-99) / Experienced Ifdmlnistrative savings due to flexibility; data reporting increased / Difficulty in shifting funds across programs; marginal flexibility achieved Most CPMs incorporated, but not measuring environmental results; progress made on environmental indicators Communicatiorti& coordination improved PPA increased pace of outreach activities MAR Alaska P(9P8APwPGork&pl9a8n/)99 Increased flexibility to move resources within water programs AW Increased flexibility for State over what is to be accomplished, but program performance suffered, according to EPA CPMs not incdrporated into PPA/PPG; no data available for some CPMs / Communication between State & Region enhanced due to priority setting; priority setting took place too late & EPA program officials were left out AW public participation did not increase MAR Texas (97 PPA & PPG; air, water and RCRA programs) / State experienced reporting reductions & savings preparing the PPG, but EPA did not experience any savings Burden increased in order to learn how to manage programs differently Did not incorpirate all CPMs in PPA; not measuring environmental results Improved negotiations MAR Indiana a(9n7d P9P8APP&AP&PGPPG negotiation process) Financial management system increases administrative burden and decreases flexibility / Difficulty in shifting funds across programs, but some multi-media initiatives were funded Performance treasures incorporated, but not measuring environmental results / Communicatitin between Region and State improved; internal communication needs improvement MAR Delaware (97 PPG) Burden increased by placing administrative and technical requirements on top of existing ones Did not achievti flexibility No mention obmeasurement of environmental results Relationship strfeng, communication positive Note: The meeting summaries from the Providence and Denver NEPPS meetings are not included in this comparison chart because those meetings reflected a range of opinions and ideas, and did not result in summary findings in the key topic areas described here. 6 Sierra Club v. EPA 18cv3472 NDCA Tier 10 ED 002061 00178452-00007 Report MAR Georgia (97 PPG & PPA; air and groundwater programs) MAR Colorado (97 PPG & 97/98 PPA) Region 4 IG Audit (PPG implementation & oversight) Region 8 IG Audit (PPG implementation & oversight) GAO (national NEPPS implementation) NEPPS Evaluations Summary Matrix Administrative Flexibility/ Burden Reduction Programmatic Flexibility Performance Measures Planning and Partnership Realized administrative efficiencies, but no reduction in reporting to Region; new consolidated grant administration procedures instituted No savings / Some states "fed deduced administrative buiden while others had increased burden Aw Did not need to move funds, but feels that the flexibility exists / Difficulty shiffthg funds within & across programs, but had some success funding cross cutting initiatives AW States did not fully utilize flexibility for multi-media planning or prioritization of work CPMs startingio be incorporated, but not measuring results. Groundwater program developed performance measures CPMs incorporated, but not measuring results Delays in negotiating CPMs with states; not all states are including CPMs Process improved relationship by emphasizing partnership Partnership strengthened Lack of commifnication between Region 4 and states led to confusion Public Participation Minimal pasfpublic involvement in PPA/PPG process, but taking steps to involve public more Public participation increased / States realized varying degrees of administrative savings; Tribes realized large savings; in general, reporting increased under PPGs AW Flexibility not fully achieved due to barriers to shift funds AW Confusion about CPMs; CPMs not clearly identified in workplans and still relying on measures of output / Relationship Ofrerlgthened due to more joint partnership; in some instances the relationship between EPA & states was strained by NEPPS / Majority of strifes 'achieved modest reporting reductions; improved workload sharing & oversight reduction, and some states had increased reporting &/or oversight Limited ability ('shift resources across programs, but some multi-media projects funded Progress with tv 2000 CPMs-fewer measures, more outcome-based. Several challenges identified in developing results-based measures Improved EPA/fetate communication / relations; states more involved in joint enforcement planning & priority-setting; some instances of decreased oversight Limited public participation so far, with few exceptions. 7 Sierra Club v. EPA 18cv3472 NDCA Tier 10 ED 002061 00178452-00008 Legend (ratings inferred from evaluators' comments): modest improvement; some success ~ little or no progress / some success, some failure + 1 tried and failed More Specific Summary Results Administrative Flexibility/ Burden Reduction Programmatic Flexibility Performance Measures Positive Outcomes and Reasons for Success Planning and Partnership Public Participation --reduced reporting burden (MAR-TX, GAO) -reduced paperwork to apply for & receive federal support (MARs CT, GA) -2 year grant cycle (MAR-CT) -flexibility in staffing (MAR-CT, GAO) -reduced on-site reviews/ programmatic oversight (GAO) -money could be transferred across programs through PPG (GAO) -states/ programs that made progress already had performance based management system (GAO) -high priority cross-cutting initiatives were able to be funded (MAR-CO) -high management support & involvement (MARs CT, GA, CO) -central point of contact to coordinate, negotiate and manage PPA (MAR-CT) - provided opportunity to jointly discuss priorities, thereby increasing communication (MARs CT, AL, TX, IN, DE, GA, CO, Reg 8, GAO) -reduced number of grants (MAR-CT, Reg 8) -condensed individual work plans (MAR-DE, GAO) -workload sharing (GAO) Examples of NEPPS Successes Connecticut MAR Texas MAR Georgia (MAR, Reg 4) Maine, Florida, Georgia, Minnesota (GAO) Florida Quality Assessment Management Plan (GAO) Minnesota (reorganized agency to eliminate mediaspecific structure) (GAO) North Carolina (implemented multi-media inspection project) -Groundwater program in Georgia has developed new performance measures that may serve as a model (MAR-GA) -Minnesota has reorganized their agency to a multi-media structure and has made great progress in measuring performance (GAO) -Utah has a strong history of strategic planning and used NEPPS to bring Reg 8 and the regional health agency together to develop solutions to the region's environmental problems (Reg 8) - the PPA process increased public participation in Colorado and Connecticut (MARs CO & CT) 8 Sierra Club v. EPA 18cv3472 NDCA Tier 10 ED 002061 00178452-00009 More Specific Summary Results Administrative Flexibility/ Burden Reduction Programmatic Flexibility Performance Measures Planning and Partnership Public Participation Tribes (Reg 8) (GAO) Colorado was able to fund pollution prevention and community-based environmental protection initiatives (MAR-CO) Delaware (funded the Whole Basin Initiative, a multi-media project) (MAR-DE) Barriers/Reasons for Lack of Success -state financial management systems are separated by media; built for administering categorical grants (MARs IN, DE.&CO) -administrative & technical requirements superimposed on existing processes (MARs DE & GA, Reg 8) -State legislative requirements for tracking funding by program (MAR-CO, GAO, Reg 4, Reg 8) -difficulty shifting funds across programs, strong institutional barriers i.e. legal reporting requirements, strong constituencies, media-specific accounting/ information systems (MARs CT & CO, Reg 8, Reg 4, Denver, GAO) -lack of process to identify priorities & invest ordisinvest (Reg 8) -lack of or no data (MARs AL, IN, & CO, GAO, Reg 8, Denver) -hard to develop measures when there are different effects on the envt. and results aren't immediate (MARs IN & CO, GAO) -lack of resources to build infrastructure for obtaining quality data (MAR-CT, GAO) -states do not want new requirements (MAR-IN, MAR-DE, Providence) -internal communication within both States & EPA needs improvement (MAR-IN, Providence, MAR-AL) -Regional staff disagree on how the program should be run-this results in the same or increased oversight (Reg 8, Providence) -not enough HQ support for NEPPS (Denver) -EPA statutory/regulatory reporting requirements (GAO) -E P A 's reluctance to reduce oversight w/o measurable assurances that goals are met (GAO) -Challenge of EPA communicating through a complex organization (GAO) -conflict between measuring performance under NEPPS and outputs under GPRA; EPA still asking for outputs (GAO, Reg 8, Denver) -conflict over how much states can deviate from CPMs and if they have to include in PPA (GAO, MAR-TX) -Regional staff disagreed over issue of CPMs being enough to ensure accountability (Reg 8) -lack of NEPPS leadership & guidance from EPA (Denver, Providence) -overall confusion about NEPPS (Reg 8, Denver) -lack of resources-time, people, $ (Denver) -conflicting priorities and 9 Sierra Club v. EPA 18cv3472 NDCA Tier 10 ED 002061 00178452-00010 More Specific Summary Results Administrative Flexibility/ Burden Reduction Programmatic Flexibility Performance Measures Planning and Partnership Public Participation Recommendations -no agreement on who analyzes data for CPMs (Denver) agendas within EPA, within states and between EPA & states (Denver, Providence) -States should select & implement a comprehensive financial management system in order to be equipped to handle cross-program funding (MARs IN & DE) -Region & State should discuss oversight approaches and should include enforcement staff (MAR-CO, GAO, Denver) -EPA should evaluate current reporting w/ GPRA requirements (Denver) -EPA should develop written guidance on development and implementation of PPA/PPGs (MARs DE, IN & CO, Reg 8, Denver) -EPA & State should develop an organizational structure that supports multi-media and/ or geographic-based approach to environmental protection (MAR-DE) -decide what is required in annual year reports and joint process evaluation (Reg 8, MAR-GA) - implement 2 year grant cycle (MARs CT & GA) -EPA & State should facilitate priority setting, allow disinvesting in lesser ones and shifting of resources (MARs CT& CO, GAO, Reg 4) -State & Region should work together and establish a time frame to identify & eliminate duplicate reporting (MAR-CT, Reg 4) -work on developing performance measures that are better indicators of environmental results (MARs AL, TX, IN, DE, CO, GA & CT, GAO, Reg 8, Denver) -identify short term goals that would allow results to be seen early (MAR-IN) -clarify reporting requirements: define what needs to be reported on for CPMs (MAR-GA) -states & EPA should agree on whether CPMs are sufficient alone for accountability (Reg 8) -assign resp. to a specific office or person for monitoring progress in getting CPMs in PPGs & maintains communication (Reg 4) -need to better understand connection between GPRA & CPMs (Providence & Denver) -Region & State should jointly set priorities before State proposes budget (MAR-AL, Denver) -Region should keep State senior management & staff informed of NEPPS concept & its benefits (MARs IN & CO, GAO, Reg 4) -determine how effective public participation in NEPPS can be ensured (GAO, MARs GA & AL) -Region & State should ensure necessary people are trained in the process (to support cultural change) (MARs IN & CO, Providence) -roles and responsibilities and partnership need to be defined (MAR-GA, Reg 8) -Region & State should continue to work together on different issues such as CPMs, enforcement, annual evaluations, & work sharing (MARs CT & CO) -increase emphasis on adopting federalstate workload sharing (Providence) -establish a self-assessment process that will provide accountability and more informed differential oversight (Providence) -establish roles- determine whose responsibility it is to "roll up" data/analysis for CPM/GPRA (Denver) -Region 5 should consider reforming Indicators Steering Committee (MAR-IN) -senior managers should be more involved in cross-media discussions (MAR-DE, Denver) -Region & State should consider cross program teams to negotiate PPA/PPG (MAR-GA, Denver) 10 Sierra Club v. EPA 18cv3472 NDCA Tier 10 ED 002061 00178452-00011 More Specific Summary Results Administrative Flexibility/ Burden Reduction Programmatic Flexibility Performance Measures Planning and Partnership Public Participation -consider including performance measures as part of employee evaluation & reward system (Reg 8) -require regional staff to attend and participate in all PPG related meetings (Reg 8) -encourage state reps to include state legislative staff in a joint priority setting session (Denver) -bring enforcement & compliance into NEPPS agreements (Providence) Sierra Club v. EPA 18cv3472 NDCA 11 Tier 10 ED 002061 00178452-00012 A Summary Comparison of Several Recent NEPPS Evaluations List of References United States General Accounting Office, "Environmental Protection: Collaborative EPA-State Effort Needed to Improve New Performance Partnership System" GAO/RCED-99-171. June 1999. United States Environmental Protection Agency Office of Inspector General, Central Audit Division. Draft Report of Audit, "Region 8 Needs to Improve Its Performance Partnership Grant Program to Ensure Accountability and Improved Environmental Results". August 5, 1999. United States Environmental Protection Agency Office of Inspector General, Southern Audit Division. Draft Report of Audit, "Region 4's Implementation and Oversight of Performance Partnership Grants". August 3, 1999. United States Environmental Protection Agency Office ofInspector General, Northern D ivision, "Management Assistance Review oflndiana's 1997 Performance Partnership Grant andPerformance Partnership Agreement" IG Report 9400014. March 22, 1999. United States Environmental Protection Agency Region 10 and Office of Inspector General for Audits, "Joint Management Assistance Review Alaska Department ofEnvironmental Conservation Performance Partnership Agreement and Grant". July 29, 1999. United States Environmental Protection Agency Region 1 and Office of Inspector General "Connecticut Department of Environmental Protection (CT DEP) Performance Partnership Agreements and Performance Partnership Grants for Fiscal Years 1997 and 1998/99". April 21, 1999. United States Environmental Protection Agency Region 8 and Office of Inspector General "Management Assistance Review of Colorado's 1997 Performance Partnership Grant and 1997 and 1998 Performance Partnership Agreements" July 2, 1998. United States Environmental Protection Agency Office oflnspector General "Management Assistance Review of DNREC FY1997 Performance Partnership Grant". United States Environmental Protection Agency Region 6 and Office of Inspector General "Management Assistance Review ofthe PerformancePartnership Agreement and PerformancePartnership Grant for the State of Texas". September 29, 1999. United States Environmental Protection Agency Region 4 and Office of Inspector General "Management Assistance Review of the Performance Partnership Grant for the State of Georgia". April 23, 1999. Spring/Earth Day 1999 EPA NEPPS Training Session. Adams Mark Hotel. Denver, CO. April 27-29, 1999. National Environmental Performance Partnership System A Joint USEPA-States Workshop Final Report. Providence, Rhode Island. December 8-10, 1997. Sierra Club v. EPA 18cv3472 NDCA 12 Tier 10 ED 002061 00178452-00013