Fill and Sign the Confidential Rental Application Form

Peer Review of NIH Research Grant Applications Anthony M. Coelho, Jr., Ph.D. Review Policy Officer Office of the Director NIH Office of Extramural Research 1. The handout material = reference resource for you when you are working on your application 2. The handout contains more information than I will cover in my presentations. 3. Information that is important is repeated to remind you that it is important 4. You are responsible for making the handout material part of you Page 1 Anthony M. Coelho, Jr., Ph.D. Review Policy Officer responsible for: • developing and implement regulations, policies, procedures, methods, and guidance documents, governing NIH extramural review functions • ensuring standard approaches to the peer review of grants, cooperative agreements and Research and Development contracts. Anthony M. Coelho, Jr., Ph.D. Review Policy Officer Previous Experience: Scientific Review Administrator and Chief - Clinical Studies and Training Scientific Review Group - NHLBI 7 years Peer Reviewer Funded Investigator 12 years 18 years Page 2 • NIH Peer Review Process based on Laws • NIH Peer Review Practices based on Study Section Culture • My objective is to help you understand both NIH GRANT MAKING CHECKS, BALANCE, ACCOUNTABILITY Page 3 National Institutes of Health Most biomedical research in the United States is supported by the Federal Government, and primarily by the National Institutes of Health (NIH) National Institutes of Health Office of the Director * National Institute on Aging National Institute on Alcohol Abuse and Alcoholism National Institute of Allergy and Infectious Diseases National Institute of Arthritis and Musculoskeletal and Skin Diseases National Cancer Institute National Institute of Child Health and Human Development National Institute on Deafness and Other Communication Disorders National Institute of Dental and Craniofacial Research National Institute of Diabetes and Digestive and Kidney Diseases National Institute on Drug Abuse National Institute of Environmental Health Sciences National Eye Institute National Institute of General Medical Sciences National Heart, Lung, and Blood Institute National Human Genome Research Institute National Institute of Mental Health National Institute of Neurological Disorders and Stroke National Institute of Nursing Research National Center for Research Resources National Library of Medicine National Institute of Biomedical Imaging and Bioengineering National Center on Minority Health and Health Disparities National Center for Complementary and Alternative Medicine Fogarty International Center Clinical Center Center for Information Technology Page 4 Center for Scientific Review NIH 2004 Budget ~$28 Billion ~$26 Billion for Extramural Research i.e. money for your research FY 2004 President's Budget Request $27.9 Billion NIH Operation Training 3% $715 million Research Mgmt & Support 3% All Other Research Project Slice 1 Grants 55% $15.2 billion 5% Other Research 6% Research Centers 9% R&D Contracts 10% Intramural Research 9% Page 5 The Research Partnership Applicant Institution NIH Review Administrator Program Administrator Principal Investigator Authorized Institutional Official Sponsored Research Administrator Grants Management Administrator Applying for Funding Page 6 NIH Offices at NIH NIH The wrong way to request funds NIH Send $$ Page 7 Response to the wrong form of request Correct Way to request Funds Page 8 PHS Research Grant Application Kit (form PHS 398) Electronic Forms and Instructions Great Expectations NOBEL Prize + = Page 9 Dr. Me Peer Review NOBEL Prize Dr.Me + + Response to Unsuccessful Peer Review NOBEL Prize Dr. Me Page 10 NIH GRANT$ Formula for Grant Success Elements of Grant Success Good Ideas Good Reviewers Good Timing Good Luck Good Presentations Good Grantsmanship Page 11 Good Grantsmanship *Knowing + Understanding •What to do •How to do it •When to do it •What to do when things don’t go as planned *Being willing to do what is needed •Passion and Commitment *Doing it- doing what is needed •Commitment * Understanding Peer Review The “other” method of applying for grant funds Page 12 Understanding NIH Peer Review Page 13 Simple Model of the NIH Review Process for a Research Grant Application Not Funded N I H Referral Grant Review Program Award $ Principal Investigator REVIEW PROCESS FOR NIH RESEARCH GRANTS Research Grant Application (PI) Principal Investigator Initiates Research Idea School or Other Research Center (Applicant) Submits application National Institutes of Health Center for Scientific Review Assign to IC and IRG Scientific Review Group Review for Scientific Merit Institute Evaluate for Relevance Advisory Council or Board Recommends Conducts Research Allocates Funds $$ Action Institute Director Takes final action for NIH Director Page 14 Dual Review System for Grant Applications First Level of Review Scientific Review Group (SRG) Provides Initial Scientific Merit Review of Grant Applications Rates Applications and Recommends for Level of Support and Duration of Award Second Level of Review Advisory Council Assesses Quality of SRG Review of Grant Applications Makes Recommendation to Institute Staff on Funding Evaluates Program Priorities and Relevance Advises on Policy Rule #1 STUDY SECTIONS DO NOT FUND ! INSTITUTES FUND! Page 15 Rule #2 You must satisfy the needs of reviewers and the needs of the funding agency STUDY SECTIONS JUDGE Scientific and Technical Merit Institute staff use the evaluations as part of the process of considering the relevance of applications to the Institute’s mission, research priorities and portfolio of existing research STUDY SECTIONS DO NOT FUND ! INSTITUTES FUND! Page 16 Rule #3 Reviewers are never wrong, Reviewers are never right; they simply provide an assessment of the material that you provided to them in your application Rule #4 The comments in the summary statements are never about you as a person. The comments are about the material that you provided in your application and the way in which you provided the information Page 17 Rule #5 The comments in the summary statements only list some of the weaknesses not all of the weaknesses. When you revise your application use the time as an opportunity to improve the entire application. Rule #6 Always contact NIH staff before you submit an application and preferably when you are in the planning stages. Make sure that you give yourself and the NIH staffer enough time to work with together. Page 18 Rule #7 DO NOT write the application for the “Specialist” You MUST convince the entire review committee Research application PI can request IRG and IC Application to CSR CSR assigns to IRG, IC NIH Application Process Overview Review by CSR IRG RFAs & others CSR sends to IC Summary Statement to PI Review by IC Second Level Council Review Not Fundable Fundable PI Notified, Given Feedback IC Negotiates Award PI Evaluates Feedback Grant Ends, Renewal Application Prepared Revised Application Prepared Page 19 Grant Application Receipt and Assignment Applications Submitted to NIH • Approximately 60,000+ grant applications are submitted to NIH each year, • 25-30% are funded • Competing grant applications are received for three review cycles per year Page 20 Timeline Submission Oct 1/Nov 1*‘03 Review Feb Mar ‘04 Mar- Jun‘04 Feb 1/Mar 1* ‘04 Jun Jul ‘04 Jun 1/Jul 1* ‘04 Standard Receipt Date (new/ *revised and continuation) Post-Review Phase Sep 30‘04 May/Jun ‘04 Jul 1 ‘04 Sep/Oct ‘04 Dec 1 ‘04 Oct Nov ‘04 Nov- Feb‘05 Jan/Feb ‘05 Apr 1 ‘05 Council Meeting; Anticipated Funding Approved Award for Nonexpedited and Special Action Awards Funds Released for Payline Grants Chosen for Expedited SecondLevel Review Initial Peer Review Typical Timeline for a New Individual Research Project Grant Application (R01) There are three overlapping cycles per year: Cycle 1 Cycle 2 Cycle 3 –Submit in February June, October –Review in June October, February –Council in September January, May –Earliest award December April, July Page 21 Receipt Dates * ** Depend on the Type of Application • Jan, May, Sept 10: Institutional Training Grant • Jan, May, Sept 25: Academic Research Enhancement Award • Mar, Jul, Nov 1: Revised, Competing Continuations, and Supplements • April, Aug, Dec 1: Small Business Technology Transfer • April, Aug, Dec 5: Individual NRSA • April, Aug, Dec 1: Small Business Innovation Research • May, Sept, Jan 1: AIDS * RFA and RFP dates defined in the solicitations ** ALWAYS check with Institutes to verify dates ???? What Happens To Your Application When It Arrives at NIH ???? Page 22 Mail room 1 Center for Scientific Review (CSR) Focal Point for Initial Review at NIH •Central receipt point for PHS applications •Referral to Institutes (Funding Components) and to Study Sections (Review Components) • CSR study sections reviews of most investigator initiated research and research training applications for scientific merit Page 23 Sample Application Number Individual Research Grant 1 R01 New Application Serial Number CA 123456 National Cancer Institute Amended 01 A1 Grant Support Year Assignment Notification Letter Dear Dr. Sample: Your grant application entitled “CEREBRAL VESSEL INNERVATION IN HYPERTENSION” has been received by the National Institutes of Health and assigned to a Scientific Review Group (SRG) for scientific merit evaluation and to an Institute/Center for funding consideration. Specific information about your assignment is given below. The initial peer review should be completed by March, 2001, and a funding decision made shortly after the appropriate National Advisory Group meets in May, 2001. Questions about the assignment should be directed to the Scientific Review Administrator (SRA) or the Division of Receipt and Referral, Center for Scientific Review at (301) 435-0715. Other questions prior to review should be directed to the Scientific Review Administrator and questions after the review to the program staff in the Institute/Center. Page 24 Assignment Notification Letter (continued) Principal Investigator: Sample Pamela Assignment Number: 2 R01 HL12345 - 12A1 Dual Assignment: NS Scientific Review Group: Epidemiology and Disease Control Subcommittee 2 SS (EDC2) A roster of the membership of this Scientific Review Group located on the following website: http://era.nih.gov/roster/index.cfm Assignment Notification Letter (continued) Scientific Review Administrator: DR. DAVID MONSEES, SRA CTR FOR SCIENTIFIC REV 6701 ROCKLEDGE DR RM 3199 MSC7802 BETHESDA MD 20892 (301) 435-0684 Assigned Institute/Center: NATL HEART, LUNG, & BLOOD INST DIV/EXTRAMURAL AFFAIRS RK2 7100 NATIONAL INSTITUTES OF HEALTH BETHESDA, MD 20892 (301) 480-5295 Page 25 Assignment Notification Letter (continued) IMPORTANT NOTICE: Please review the information on human and animal subjects research located at: http://grants.nih.gov/grants/peer/hum_anim_notice.pdf as these requirements will affect the priority score on your application. Assignment to Institutes Applications are referred to an Institute or Center as the potential funding component: • Assignment is based on a match between the research proposed and the overall mission of the Institute or Center • Where applications are appropriate for more than one Institute or Center, multiple assignments are made Page 26 Assignment to CSR Study Sections Applications assigned to study sections known as Scientific Review Groups (SRG) based on: 1. specific referral guidelines for each SRG and 2. information contained in your application (Go to the Website http://era.nih.gov/roster/index.cfm to learn about study sections – their scientific mission and their scientific membership) Assignment to Study Sections (cont) TYPES OF REVIEW COMMITTEES: Chartered Study Sections • when the subject matter of the application matches the referral guidelines for the standing study section Special Emphasis Panels (SEPs) • when the subject matter does not fit into any study section, or • when assignment of an application to the most appropriate study section would create a conflict of interest, or • Special Mechanisms (RFA, Fellowships, SBIRs, AREAS, etc.) Page 27 Peer Review of NIH Support Mechanisms Who Reviews What ? CSR Institutes Research Project Grant (R01) Postdoctoral Fellowship (F32) Senior Fellowship (F32) Fogarty International Center Fellowship (F05, F06) Short-Term Training (T35) Small Business Grants (R41, R42 R43, R44) Academic Research Enhancement Award (R15) Biomedical Research Support Shared Instrumentation Grant (S10) Program Project Grant (P01) Center Grant (P30, P50, P60) Institutional Fellowship (T32) Academic Career Award (K07) Mentored Clinical Scientist Development Award (K08) Conference Grant (R13)* Marc Fellowships (F34, F36, T34) Minority Biomedical Support Grant (S06) Resource Grant (P40, P41, R24, R26, R28) RFA - Request for Applications R&D - Contracts WHO/WHAT DETERMINES WHICH GROUP REVIEWS THE APPLICATION? • Mechanism Type of application CSR or Institute Review • Referral and Review Staff • Past Review History (if any) of application • Principal Investigator Letter attached to application; self-referral Page 28 WHO/WHAT DETERMINES WHICH GROUP REVIEWS THE APPLICATION? YOU DO! • The words that are in your application • Your title • Your abstract • Your specific aims • Your methods Study Section Meeting Page 29 Study Sections at NIH • Study Sections are managed by a Scientific Review Administrator (SRA) who is a professional (at Ph.D. or MD level) whose scientific background is close to the expertise of the study section • Each standing study section has 12 - 24 members who are primarily from academia • 60 - 100 applications are reviewed at each study section meeting • Several hundred study section meetings SCIENTIFIC REVIEW GROUP Scientific Review Administrator •Recruits and selects reviewers •Insures that the review that is competent, thorough and fair (unbiased) •Proper review criteria used to evaluate application Reviewers •Some charter members; some temporary members •Scientists with appropriate expertise •High professional profiles •Dependable, reasonable, open minded Grants Technical Assistant •Mails material to reviewers •Handles paperwork •Organizes meeting room •Enters scores and codes •Assists with summary statements Page 30 WHO ASSIGNS REVIEWERS TO MY APPLICATION? • Scientific Review Administrator Assignment to Specific Reviewers • Based on application content • Based upon expertise of reviewers • Based upon knowledge of the field • May consult with Institute staff • May consult with chairperson • Suggestions from PI on type of expertise needed to evaluate (NEVER names) • Considers review history Center for Scientific Review Example of Varied Expertise on a Sample Study Section Surgery, Anesthesiology and Trauma Study Section Selected Areas of Competence of Members Biochemistry Burn Physiology and Electrolyte Metabolism Cardiovascular and Pulmonary Physiology Clinical Anesthesiology Drug Metabolism (Anesthetics) General Surgery Immunology and Transplantation Nutrition Pharmacology (Analgesics, Narcotics and Antagonists) Pulmonary Embolism Shock and Trauma Toxicology of Anesthetic Drugs Vascular Surgery Page 31 Criteria For Selection of Peer Reviewers • • • • • • • • Demonstrated Scientific Expertise Doctoral Degree or Equivalent Mature Judgment Work Effectively in a Group Context Breadth of Perspective Impartiality Interest in Serving Adequate Representation of Women and Minority Scientists Certification of No Conflict of Interest This will certify that in the review of applications and proposals by (study section) on (date), I did not participate in the evaluation of any grant or fellowship applications from (1) any organization, institution or university system in which a financial interest exists to myself, spouse, parent,child, or collaborating investigators; (2) any organization in which I serve as officer, director, trustee, employee or collaborating investigator; or (3) any organization which I am negotiating or have any arrangements concerning prospective employment or other such associations. ____________________ ____________________ ____________________ ____________________ ____________________ ____________________ ____________________ ____________________ SIGNATURES Page 32 Confidentiality • Review materials and proceedings of review meetings represent privileged information to be used only by consultants and NIH staff. • At the conclusion of each meeting, consultants will be asked to destroy or return all review-related material. • Consultants should not discuss review proceedings with anyone except the SRA. • Questions concerning review proceedings should be referred to the SRA. WHAT HAPPENS IN A STUDY SECTION MEETING? • Closed to the public (FACA rules apply) • Orientation Conflict of interest Developments of interest to the study section Changes in policy or procedure Introduction of persons present Role of persons present • Streamlining or list provisionally approved • Application by application discussion •Persons with conflicts of interest excused •Assigned reviewers give preliminary scores •Discussion of application’s scientific and technical merit •Assigned reviewers first, then other members •Range of scores set •Every member scores every application * •Assignment of gender, minority, and children codes, human subjects codes; recommended changes to budget Page 33 WHAT IS STREAMLINING? Process by which reviewers judge which applications are in the lower half of those assigned for review. Applications in the lower half are evaluated by the reviewers prior to attending the meeting but they are not discussed at the Scientific Review Group meeting. • Any member can object to the streamlining of an application • Requires that all reviewers agree to streamline an application • Streamlined applications receive written reviewer critiques Why? • Shortens meetings • Reviewers more willing to serve on committee • Allows more time for discussion of applications “Review” of Applications • Applications are not reviewed at the meeting. They are evaluated prior to the meeting. • The meeting is a time for discussion and negotiation of a priority score and for making a recommendation that best reflects the scientific and technical merit of the application. • Strong applications get brief discussion • Weak application get brief discussion • Marginal application get longer discussion to ensure fairness to the applicant Page 34 Review of Research Grants REVIEW CRITERIA: • • • • • Significance Approach Innovation Investigator Environment Described in detail in the PHS 398 application instructions Review Criteria • Significance: Does the study address an important problem? How will scientific knowledge be advanced? • Approach: Are design and methods well-developed and appropriate? Are problem areas addressed? • Innovation: Are there novel concepts or approaches? Are the aims original and innovative? • Investigator: Is the investigator appropriately trained? • Environment: Does the scientific environment contribute to the probability of success? Are there unique features of the scientific environment? Page 35 Research Involving Human Subjects Important Considerations that must be addressed in the application because they impact on priority score considered to be part of the Approach • Are there any risks* to the human subjects? • Are the protections adequate? • Are there potential benefits to the subjects and to others? • What is the importance of the knowledge to be gained? • Are the plans for inclusion of minorities, both genders and children adequately addressed? • Is the proposed study exempt from human subject review? • No page limits * “Risks” include the possibility of physical, psychological, or social injury resulting from research. Research Involving Human Subjects Areas of exemption • Education Research – normal educational practices • Educational Tests, Survey or Interview Procedures, or Observation of Public Behavior – subjects not identified – subjects’ privacy rights protected • Educational Tests, Survey or Interview Procedures, or Observation of Public Behavior Not Exempt in Previous Category if: subjects are public officials or public office candidates federal statute requires confidentiality without exception Page 36 Research Involving Human Subjects Areas of exemption • Collection or Study of Existing Data, Documents, Records, Pathological Specimens – information publicly available – subjects not identified • Research and Demonstration Projects Regarding Certain Public Benefit or Service Programs • Taste and Food Quality Evaluation and Consumer Acceptance Studies Using – foods without additives – U.S. Government approved food ingredient Inclusion of Women and Minorities in Clinical Research • Women and Minorities must be considered for inclusion in all clinical research supported by NIH or •Appropriate justification must be provided to explain why they are not included in the proposed research Page 37 Research Involving Children Children must be considered for inclusion in all human subject research supported by NIH or Appropriate justification must be provided to explain why they are not included in the proposed research Research Involving Children Children must be considered for inclusion in all human subject research supported by NIH Effective for all new applications received after October 1, 1998 • Child is defined as an individual under age 21 • If children are included, Investigator must address • age range • expertise of investigative team • facilities • sufficient numbers Page 38 Research Involving Children • If children are not included, must justify exclusion: • Topic irrelevant to children • Laws/regulations bar inclusion of children • Knowledge already available or being obtained • Separate study warranted • Unable to judge potential risk to children • Collecting data on pre-enrolled adults • Other special cases Animal Welfare Important Considerations • Will the anticipated results be for the good of society? • Will the work be planned and performed by qualified scientists? • Will the animals be treated so as to avoid any unnecessary discomfort, pain, anxiety, or poor health? • Species chosen? • Animals in short supply? Page 39 Scientific Review Group or Study Section Actions • Scored, Scientific Merit Rating • Priority scores: 1 (best) to 5 (poorest) and percentiles • Unscored (lower half) • Deferral Summary Statement After the review meeting is finished, the results are documented by the SRA in a summary statement and forwarded to the PI and to the assigned NIH Institute. The assigned NIH Institute is responsible for making a funding decision. The summary statement contains: •Overall Resume and Summary of Review Discussion •Essentially Unedited Critiques of Assigned Reviewer •Priority Score and Percentile Ranking •Budget Recommendations •Administrative Notes Page 40 National Advisory Council or Board Review Council Actions • Assesses Quality of SRG Review • Concurs with study section action or • Modifies SRG (study section) action Can not change priority score • Deferral for re-review of the same application – no changes allowed • Makes Recommendation to Institute Staff on Funding, Evaluates Program Priorities and Relevance and Advises on Policy Page 41 NIH Policy does NOT allow Rebuttal of Peer Review outcome There is an Appeal process however Differences of Scientific Opinion Can NOT be Appealed! NIH policy permits appeal of review outcome if 1. Procedural error in review process 2. Factual errors (not interpretations) REVISE & RESUBMIT Do Not Appeal Review Outcome NIH Appeal Outcomes: 1. Council Denies Appeal (bad outcome) 2. Council Accepts Appeal: Original Application and Letter of Appeal is sent to the Same Study Section for a second examination and evaluation (bad outcome) 3. Council Accepts Appeal: Original Application be sent to a new Study Section but without the Letter of Appeal (bad outcome) Page 42 Timeline Consequences Submit Review Council Earliest award Review 2 Council 2 Earliest Resubmission Earliest Award Review 2 Earliest Resubmission Council 2 Review 3 Earliest Award Council Earliest Award BestWay Feb 04 June 04 Sept 04 Dec 04 Revision Feb 04 June 04 Sept 04 Appeal Feb 04 June 04 Sept 04 Oct 04 Jan 05 March05 Apr 05 June 05 July 05 Sept 05 Oct 05 Dec 05 Feb 06 June 06 What Determines Which Awards Are Made? • Scientific merit + • Program Considerations + • Availability of funds Page 43 You do not want a reviewer to make this comment about your application: “This application is characterized by ideas that are both original and scientifically important. Unfortunately the ideas that are scientifically important are not original and the ideas that are original are not scientifically important.” You do not want a reviewer to make this comment about your application: “In addition to proposing a research design that is a fishing expedition, the applicant also proposes to use every type of bait and piece of tackle ever known to mankind.” Page 44 The research that you propose in your application must be innovative and focused NIH Information Sources Page 45 NIH GUIDE for Grants and Contracts U.S. Department of Health and Human Services • Announces NIH Scientific Initiatives • Provides NIH Policy and Administrative Information • Available on the NIH Web Site : http://www.nih.gov http://grants.nih.gov/grants/guide/index.html http://crisp.oit.nih.gov Page 46 Learn the mission of the study section ! Page 47 Learn the membership of the study section! Learn about special funding opportunities ! Page 48 Learn about special funding opportunities ! Program Announcements are very important for you • Invites grant applications in a given research area • May describe new or expanded interest in a particular extramural program • May be a reminder of a continuing interest in a particular extramural program • Generally has no funds set aside • Applications reviewed in CSR along with unsolicited grant applications Page 49 Requests for Applications (RFA) are very important for you • Announcement describing an institute initiative in a well-defined scientific area • Invitation to submit research grant applications for a one-time competition on a specific topic • Set-aside of funds for a certain number of awards • Applications generally reviewed within the issuing institute Selected Sites of Interest • National Institutes of Health http://www.nih.gov • Office of Extramural Research http://www.nih.gov/grants/oer.htm • Grants Policy http://www.nih.gov/grants/policy/policy.htm • NIH Study Section Rosters http://era.nih.gov/roster/index.cfm Page 50 • Office of Extramural Research: Grants Page http://grants1.nih.gov/grants/index.cfm •Center for Scientific Review http://www.csr.nih.gov •Referral and Review http://www.csr.nih.gov/refrev.htm •Overview of Peer Review Process in CSR http://www. csr.nih.gov/review/peerrev.htm •NIH Peer Review Notes http://www.csr.nih.gov/prnotes/prnotes.htm Office of Extramural Research • Handles requests for grant applications, program guidelines, general information on grant applications and review policy Office of Extramural Research National Institutes of Health 6701 Rockledge Drive, Suite 6095 Bethesda, Maryland 20892-7910 PHONE: 301-435-0714 FAX: 301-480-0525 e-mail: grantsinfo@nih.gov Page 51 NIH GRANT$ Formula for Grant Success Good Grantsmanship *Knowing + Understanding • • • • What to do How to do it When to do it What to do when things don’t go as planned *Being willing to do what is needed *Doing it- doing what is needed Understanding Peer Review Page 52 Thank You Page 53