About
The Research Experiences for Undergraduates (REU) Survey, managed by the CRA Center for Evaluating the Research Pipeline (CERP), is an evaluation resource that provides critical information to the community on how participation in undergraduate research opportunities impacts student pathways to computing graduate education and computing careers.
The project began as a contract between CERP and the National Science Foundation (NSF) Directorate for Computer and Information Science and Engineers (CISE) in 2021, wherein CERP provided an evaluation of the NSF CISE REU programs (including REU Sites and REU Supplements). In 2024, CERP entered into a cooperative agreement with NSF CISE (Award# 2335072). During that time, REU Survey activities transitioned within the umbrella of the cooperative agreement along with CERP’s extensive portfolio of projects.
CERP currently provides evaluation for NSF CISE REU programs at no cost to the PIs. Sign up is free and easy! Simply complete our online interest form and you will receive a confirmation email from a CERP team member (cerpreu@cra.org) within 1-2 business days. You should complete the interest form regardless of whether you are only in the proposal phase or have a currently active REU project.
Meet the team
Burçin Campbell, Project oversight
Heather Wright, Project manager
Ama Nyame-Mensah, Lead evaluator
Brendan Kane, Administrator
Evelyn Yarzebinski, Data Manager
Audrey Rorrer (UNC-Charlotte), Project consultant
Acknowledgements
The CISE REU Evaluation Project is currently supported through U.S. National Science Foundation cooperative agreement #2335072. Previous support came from U.S. National Science Foundation contract (49100421C0010) and was approved by the Office of Management and Budget (OMB control number 3145-0265).
Evaluation Process
Depending on whether you are running an REU Site or you have an REU Supplement, we will use an evaluation method most appropriate for your awarded project and student population.
Data Collection
- Four weeks prior to your REU start date, we will send PI instructions and a PI form. This form must be completed within 1 week and is a required form.
- Two weeks prior to your REU start date, we will send a suggested participant recruitment email and the pre-survey link for you to send to your REU participants. Participants will have until 1 week after the REU starts to complete this pre-survey (a total of 3 weeks). We will monitor that data collection and request reminders as appropriate.
- Six days before the end of your REU, we will send you the second PI form for you to complete (completion required). Two days later, we will send you the post-survey link and suggested participant recruitment email to be sent to all REU participants. All forms and surveys must be completed within 3 weeks after your REU ends. We will also monitor that data collection and request reminders as appropriate.
Reporting
Approximately 2-3 months after data collection is complete, you will receive (depending on sample sizes):
- A key findings page – a one-page report summarizing the evaluation survey data collected from your REU participants.
- Two separate reports – one summarizing your REU participants’ responses alongside a comparison group of REU participants at similar institutions and a second comparing your REU participants’ responses to those of participants participating in projects in similar research areas.
If you are in an REU Supplement, your report may include aggregated findings with other REU Supplements if you only have a few students in your project. The reporting timeline for Supplements will vary depending on the type of Supplement and when the project concludes.
Measures
Our REU Evaluation surveys include various measures tailored towards different experiences of students that participate in NSF CISE REUs. Below is a non-exhaustive list of measures included.
Click here to request a copy of our participant surveys.
Previous REU experiences
- Year, setting, and details of previous REUs that students have participated in
General attitudes and beliefs regarding computing
- Perceptions of abilities, skill sets, and accomplishments
- Levels of identification with computing and sense of belonging
- Perceived support from professional network and mentors
Future plans and career aspirations
- Highest degree intentions
- Employment and graduate school plans
- Future career interests
- Career values and perceptions of computing careers
Previous and current program details and experience
- Academic background (e.g. pre-college CS and Math courses)
- Enrollment information and reasons for matriculation and persistence
- Perceptions of degree program and institution
- Degree of involvement in curricular, co-curricular and extracurricular activities
Demographics
- Gender
- Race/ethnicity
- Sexual orientation
- Disability
- Marital status
- Socio-economic measures
- Parents’ education and career
Evaluation for an Active NSF CISE REU Site or Supplement Award (Active Awards)
- If you are interested in having CERP evaluate your currently active NSF CISE REU program, please complete our interest form below and select “active award”. Please complete the form as completely as possible so that we do not need to request additional information from you later.
- After you complete the interest form, you will receive an email from the CERP team (cerpreu@cra.org) within a few days confirming receipt of your form response.
Evaluation for an NSF CISE REU Site or Supplement (Proposal Stage)
- If you are interested in having CERP evaluate your NSF CISE REU that is in a proposal stage (as a new project or renewal), please request a proposal paragraph by completing our interest form below and selecting “proposal”. This general proposal paragraph will state your collaboration with CERP as your evaluator. If you have more than one proposal, please complete a form for each proposal.
- After you complete the interest form below, you will receive an email from the CERP team (cerpreu@cra.org) confirming receipt of your form response that includes the proposal paragraph to include in your proposal.
IRB
Thank you for your interest in CERP’s evaluation of your NSF CISE REU project. While CERP has received IRB approval to conduct this evaluation, your institution’s IRB may require you to get separate approval to conduct the evaluation. CERP urges you to consult your institution’s IRB to determine whether they will require their own separate review.
Click here to request our IRB approved materials.
Guidelines for talking to your local IRB about the REU Evaluation The information below is intended to support PIs with guidelines and talking points for approaching your own institution’s IRB about the REU evaluation. The Center for Evaluating the Research Pipeline (CERP) has received IRB approval from Solutions IRB for this REU Evaluation project as an Exempt, Category 2 project. All currently-funded NSF REU projects in the U.S. are eligible to use evaluation services being offered by CERP and are covered by this IRB approval. Based on current OHRP definitions of when an institution is considered to be “engaged” in research, most institutions we work with do not require separate review and approval from their own institution’s IRB. This is because the local REU PI distributes anonymous pre-program and post-program survey links to the REU participants on behalf of CERP but has no other involvement in the research process (e.g., creation of the survey instruments, consent process, data collection, data analysis, or any other aspect of the study), and the PI will not have access to any data files or individually-identifying information resulting from this study. If you are a PI seeking to clarify your local IRB’s requirements, here are some key points to cover: If your institution’s IRB does require separate review and approval, you or another colleague will need to serve as PI for the portion of the research project that will occur with students at your institution. Please complete the request form linked above to receive CERP-created materials that will help you with your local submission. You can choose to receive a version of the materials that only pertains to the local implementation (recommended), or you may receive a version of materials that includes the comprehensive set of materials for the full national evaluation being conducted by CERP. Both sets will include a document “IRB resources for individual institutions,” which includes common IRB application questions and suggested answers.
If you have any other questions or need any additional information to satisfy the requirements for a particular question on your local IRB forms, please contact us at cerpreu@cra.org.
Frequently Asked Questions about CERP REU Evaluation
What is the cost of working with CERP evaluation?
With funding from the U.S. National Science Foundation, CERP’s evaluation services are free for NSF REU Sites and REU Supplements. PIs do not need to include a budget for CERP’s evaluation in their proposals.
If I work with CERP, do I need to hire another evaluator to fulfill NSF’s evaluation requirement?
CERP’s evaluation fulfills NSF’s evaluation requirement, so another evaluator is not necessary for this purpose.
Will CERP collaborate with other evaluators?
While CERP is happy to provide other evaluators with the methodology and copies of the survey, CERP is not directly collaborating with other REU Site and Supplement evaluators for this project.
Will CERP share data with other evaluators?
Due to privacy considerations, CERP does not share raw data collected through this project. PIs are encouraged to share the summary reports that CERP will provide each year (assuming that there are enough participant responses) with other evaluators as needed.
When I signed up to work with CERP for evaluation, I received a proposal paragraph. How do I include it in my REU Site/Supplement proposal?
The proposal paragraph that we provide should be inserted directly into the required section of your proposal on evaluation. As it is a general paragraph, please feel free to add minor modifications to the language so that the paragraph includes any project-specific language or institutional information.
As an REU PI working with CERP for evaluation, do I need to have my own IRB approval?
CERP will have IRB approval to conduct evaluation through Solutions IRB. As an REU PI, you are encouraged to contact your IRB office to determine whether additional oversight is required by your institution. Once we have our IRB-approved materials, we will supply a copy of the approval letter and templates on our website that you can use to initiate that conversation.
Can I add specific questions to the CERP survey?
We have some limitations in our ability to change our surveys, as our project requires approval of all study instruments by our IRB. You may contact us if you have suggestions for new measures. We will keep a record of any requested additions and will consider suggestions for future surveys.
Will surveys also be sent to mentors/staff?
CERP will not survey REU mentors and staff for evaluation of their individual experience. However, CERP will collect information about the REU from PIs and other project leadership to better understand the structure of each REU. Data collected about the project from the PIs will also be used for providing appropriate comparison groups.
Who is responsible for distributing the evaluation surveys to student participants?
Each REU project will be responsible for distributing the evaluation surveys to participants. Prior to data collection, CERP will provide the PIs (or the determined contact person) with all materials necessary for distribution, such as recruitment templates and the survey link. All communications from CERP will come from cerpreu@cra.org.
Where can I share feedback about my experience working with CERP?
If you have feedback or suggestions for improvements, let us know by sending an email to cerpreu@cra.org or completing our Evaluation Feedback Form.
