Expanding the Research Pipeline – An update on Scaling the Early Research Scholars Program

The Early Research Scholars Program (ERSP), founded by Christine Alvarado in the Computer Science Department at UC San Diego (UCSD), provides authentic research experiences to early undergraduates over one academic year. Since its inception in 2014, the program has transformed the landscape of undergraduate research at UC San Diego by significantly broadening access to research among early students, with over 350 student participants, 60% of whom identify as women or non-binary, and 22% as Black, Latinx, or Native American students. Further, ERSP has retained most participants (97% since the program matured). Figure 1 summarizes the main outcomes of the program from the first four years of its implementation at UCSD.

 

Expanding ERSP beyond UC San Diego: Three new partners
ERSP has been a great success at UCSD, but how well does it work at other institutions? A team of faculty from University of Illinois Chicago (UIC), Stanford, and UC Santa Barbara (UCSB) worked with Dr. Alvarado to expand ERSP to their institutions. With support from the National Science Foundation, ERSP launched at UCSB in 2018, and other institutions followed within the next year. Outcomes, experiences, and insights from this five-year effort have helped understand how to create positive mentoring relationships, how well ERSP works in different contexts, its impact on participants, and, more broadly, the institutions themselves. We share the main highlights of the expansion effort with the hope that it will be valuable to others looking to take a community-driven approach to scale undergraduate research.

The partner universities share certain similarities; most importantly, they are research-active doctoral-granting institutions in the US. But they also differ in size, selectivity, and timeline (quarter vs. semester) (Table 1). UCSD, UCSB, and UIC are all public institutions; Stanford is private. UCSD, UCSB, and Stanford are highly selective institutions, while UIC is somewhat less selective. UCSB and UIC are classified as Hispanic Serving Institutions (HSI), though the participation of Latinx students in computing is relatively low (25% or less). Finally, UCSD, UCSB, and Stanford operate on a quarter-system (three 10-week quarters per academic year); UIC uses a semester system (two 15-week semesters per academic year).

School	Timeline	HSI	# CS majors	% women	% BLN
UIC (CS) UIC (ECE)	Semester	Yes	1700 (CS) 647 (ECE)	21% (CS) 16.7% (ECE)	34.7% (ECE)
Stanford	Quarter	No	950	34.4%	15.2%
UCSD	Quarter	No	1959	22%	10%
UCSB	Quarter	Yes	591	22%	9%

Table 1: Partner institution context and demographics

The ERSP implementation at all partner sites preserved the essential components of the program, described in the next section. However, contextual differences required modifying the original ERSP model in several ways, discussed in detail in earlier work [2, 5]. Our main findings are that implementations at partner sites have seen similar diversity and initial success (as summarized in Table 2). Further, they have taught us how to implement the program in different contexts (e.g., quarters vs. semesters, different credit structures).

	# students (% retention)	Women/non-binary (%)	BLNPI (%)
UC San Diego (2014 – 22)	339 (97%*) (*since program matured)	202 (60%)	75 (22%)
UCSB (2018 – 22)	80 (95%)	40 (50%)	11 (13%)
UIC (2019 – 22)	83 (94%)	51 (61%)	21 (25%)
Stanford (2019 – 22)	135 (97%)	54 (40%)	29 (21%)

Table 2: Retention rates of ERSP participants at partner sites by gender, race/ethnicity (BLNPI=Black, Latino/a, Native, Pacific Islander)

Essential components of ERSP
ERSP has two primary goals: (1) to connect second-year undergraduates (particularly those from marginalized groups) with cutting-edge computing research projects and (2) to create an inclusive and supportive community around research in computing. Towards these goals, the program has four essential components that are preserved across partner sites – a research methods course that gives students a formal introduction to research, group-based research to help students learn valuable teamwork skills and to support each other through the ups and downs of research, a dual mentoring structure consisting of a central mentor and technical mentors who play complementary roles, and a focus on fostering equity and inclusion by engaging groups marginalized in computing and centering ERSP instruction and mentoring in culturally inclusive approaches to research and group-work [6, 7].

 

Mentoring early undergraduates using a dual mentor approach

The dual-mentoring structure is one of the unique aspects of ERSP. We conducted a qualitative study involving 64 undergraduate researchers from two partner universities to understand students’ perceptions of the dual-mentoring model [3]. Our goal was to understand the role of the central mentor vs. the technical mentor, whether these roles are complementary, and how students’ relationships with their mentors change (or stay the same) during the program.

We found the central mentor primarily provides high-level support and project guidance, helping students to reflect on their research, encouraging them to think more deeply, and helping them articulate questions for their technical mentors. Seemingly counterintuitive, this freed faculty/technical mentors to provide more critical low-level support to students, which the central mentor cannot offer.

Further, we found that students tended to see research as a series of assignments to complete rather than an independent discovery process. However, we observed some growth and comfort with less guided research as the program progressed, confirming prior findings that projects for early undergraduates should be appropriately scoped [4].

We found that one of the essential characteristics of a positive mentoring relationship was the perceived friendliness of a mentor. This characteristic captured the mentor’s ability to keep interactions casual, provide emotional support, and, most importantly, make students feel comfortable asking simple questions.

“She [central mentor] also keeps the mentor/student relationship more casual and friendly, so it makes it easy to tell her about any difficulties we’re having.”

While friendliness seemed a critical quality that students looked for in a mentor, it required proactive effort. Even when mentors say students should feel comfortable, this result is not always achieved with a hands-off approach.

“He [technical mentor] takes a hands-off approach, but if we ask, he will always give us advice and point us in a better direction… I’m not really sure what to ask him sometimes… It’s still intimidating for me to meet with him, I feel I barely know anything compared to him, but he’s expressed that we can be comfortable with him.”

Our findings suggest potential benefits to involving an auxiliary mentor, even when fully adopting the ERSP model is not an option.

Expansion beyond the initial partners
Based on success with the initial expansion, we continued recruiting new and more diverse partners interested in implementing ERSP. With financial support from Google and NSF, we provided $25K-$45K in seed funding for new partners. We held an open application process, and in 2020 and 2021, four new schools joined our team: University of Massachusetts-Amherst (UMass Amherst), North Carolina State University (NC State), Southern Connecticut State University (SCSU) and Texas Southern University (TSU). UMass Amherst and NC State are similar to the original four partner schools in that they are large R1 institutions, while TSU and SCSU bring more diversity to our team. Both TCU and SCSU offer only Master’s degrees at the graduate level in computer science and have faculty with greater teaching responsibilities and less time for research. Additionally, TSU is a historically Black university. All four schools run on a semester timeline.

UMass Amherst and NC State are in the second year of their ERSP implementations, while TSU and SCSU are in their first year. At UMass, the program has been run very similarly to prior implementations, with similar successes in terms of retention in the program. The other three institutions have adjusted the program to accommodate the context at their universities. First, these three institutions have fairly constrained computing major requirements in students’ first two years. Thus, students have difficulty accommodating the research class and the subsequent independent study research in their second year. At NC State, there have also been challenges integrating ERSP with the large existing research program, which involves mainly upper-class students doing individual projects. So while the ERSP research class has launched successfully at NC State, it reaches relatively few early undergraduate students, and the program has not (yet) been able to implement the group-based research structure.

In contrast, undergraduate research structures at TSU and SCSU were originally insufficient, so these universities chose to preserve all of the elements of ERSP, but to focus on later-career students (juniors and seniors). This shift succeeded, allowing the course to easily be taught as an upper-division course.

Finally, TSU faced an additional challenge: Due to resource limitations, they had to combine the ERSP course with their existing capstone course. Thus, TSU adopted the ERSP model used by Stanford (see [2]), where students not only propose but also complete a research project within a single term (a 16-week semester at TSU and an 11-week quarter at Stanford). While this format was challenging for TSU students, we still saw the positive effects of ERSP, including increased understanding, confidence, and interest in CS research. TSU is currently exploring ways to implement the model over a longer timeframe.

While ERSP was designed to operate at R1 universities with strong research infrastructures, partnerships with TSU and SCSU allowed us to test how well the model would work at institutions with a more even balance between research and teaching. It has launched successfully at both universities, partly because ERSP streamlines and supports undergraduate research by integrating research into coursework, creating appropriate student credit structures, and mentoring students through the research process. However, one of our partners still observes:

“My institution being an R2, faculty have heavy teaching loads, and few engage in research. It would be good to have a network of ERSP technical mentors outside of the institution who could integrate a team from an R2 institution within their research team.”

                             – Dr. Lila Ghemri, ERSP director at TSU

Indeed, ERSP partnerships between less research-focused and more research-focused universities is a promising idea for giving even more students access to undergraduate research opportunities. More broadly, successful undergraduate research opportunities create better alignment between the educational system, research enterprise, and faculty and student goals, among other factors [1].

Impact of ERSP on participants and institutions
At most partner institutions, ERSP has created a pathway into research for early undergraduates where few or none existed. In addition, it has successfully engaged and retained diverse students across all mature implementations. However, we expect the true benefits of participation in ERSP to be long-term. To study the long-term impact of ERSP on its participants, we recently conducted our most extensive retrospective analysis in collaboration with Burcin Tamer and Heather Wright from the CRA Center for Evaluating the Research Pipeline (CERP). We studied longitudinal data collected from the CRA Data Buddies Survey (DBS) from 2020-2022 for the 2018-2019, 2019-2020, and 2020-2021 cohorts of ERSP at UIC, UCSB, and UCSD. We examined the impact of ERSP on students’ desire to persist to graduate school, their self-efficacy, their sense of belonging, and their desire to remain in their computing major while controlling for gender, race/ethnicity, first-generation college student status, socio-economic background, and academic background related to computing.

Examination of students’ sense of belonging showed that ERSP participants felt more welcomed in computing than students who did not participate in any REU [8]. On average, ERSP participants were three times more likely than the comparison group to want to pursue a graduate degree. However, participating in ERSP did not change the odds of considering leaving computing [8].

Finally, the end-of-program student surveys at multiple sites have been overwhelmingly positive. Students report that ERSP has helped them develop their identity as computer scientists, feel more confident in pursuing independent work, and are more comfortable working in a research environment.

“My group was unique in that we were all women. Our mentor was also a woman. Seeing representation so early in my academic career helped me see a future for myself in computer science.”

“I just learned so much in ERSP. In addition to learning about research in CS and how to participate in it, I learned how to work in a team, improved my resourcefulness, and gained so much confidence as a CS student.”

Join our growing community in implementing ERSP!
We invite you to use our resources to implement ERSP at your institution: https://ersp.eng.ucsd.edu/resources/startup-course-materials and join our growing community of partners (email cjalvarado@ucsd.edu)

Acknowledgments
This material is based upon work supported by the National Science Foundation under Grant No DUE – 821521, 1821415, 1821449, 1821501.

About the authors:
Diba Mirza is an Associate Teaching Professor at the University of California, Santa Barbara (UCSB). She has been involved in expanding ERSP (beyond UC San Diego) to the Computer Science Department at UCSB, where she has directed the program for the past four years. During this time, Diba has helped grow and institutionalize ERSP, co-mentoring 80 (mostly early) undergraduates in research projects, including 40 women and 11 students from underrepresented racial and ethnic groups.

Christine Alvarado is a Teaching Professor at the University of California, San Diego. She founded ERSP at UC San Diego in 2014 and directed the program until 2022. To date, ERSP has engaged 339 undergraduates—59% of whom identify as women or non-binary, and 22% of whom identify as Black, Latinx or Native American—in academic year-long research projects. She has led the expansion of ERSP to 7 additional universities, including UC Santa Barbara (launched in 2018), University of Illinois-Chicago (2019), Stanford (2019), UMass Amherst (2021), NC State (2021), Texas Southern University (2022) and Southern Connecticut State University (2022). She also has extensive experience directly supervising undergraduates in research, most of whom are women and/or members of racial groups underrepresented in computing.

Diana Diaz is a Clinical Assistant Professor in the Computer Science department at UIC. She has more than ten years of teaching experience and believes in education as a pathway toward social equality and out of poverty. She teaches computer design, a team-based course that challenges students to build hardware projects using Arduino boards. Dr. Diaz is in charge of implementing ERSP in the Computer Science Department at UIC in collaboration with Dr. Renata Revelo.

Renata Revelo is a first-generation college student who migrated from Ecuador to the United States as a teenager with her parents and sister. She is the first in her family to obtain a Ph.D. She is currently a Clinical Associate Professor at the University of Illinois, Chicago, in the department of Electrical and Computer Engineering. Her research focuses on shifting the culture of engineering via the study of engineering identity, which centers on students of color and examines systemic change.

Cynthia Lee is a Senior Lecturer in the Computer Science Department at Stanford. She specializes in team-based and question-based course design and founded peerinstruction4cs.org to support educators in flipping their computer science classrooms using peer instruction. Her teaching awards include the Lloyd W. Dinkelspiel Award for exceptional contributions to undergraduate education at Stanford, a “Top 10 Papers of All Time” award at the 50th anniversary of the ACM SIGCSE technical symposium, and the Stanford Society of Women Engineers’ Professor of the Year. She has a Ph.D. in high-performance computing from UC San Diego. Her industry work experience includes NASA Ames and startups.

References
[1] Rhea Sharma, Atira Nair, Ana Guo, Dustin Palea, and David T. Lee. 2022. “It’s usually not worth the effort unless you get really lucky”. https://doi.org/10.1145/3501385.3543976

[2] Christine Alvarado, Joseph Hummel, Diba Mirza, Renata Revelo, and Lisa Yan. 2022. Scaling and Adapting a Program for Early Undergraduate Research in Computing. https://doi.org/10.1145/3478431.3499336

[3] Christine Alvarado, Alistair Gray, Diba Mirza, and Madeline Tjoa. 2021. The Role of Mentoring in a Dual-Mentored Scalable CS Research Program. https://doi.org/10.1145/3408877.3432364

[4] Jane G. Stout, N. Burçin Tamer, and Christine J. Alvarado. 2018. Formal Research Experiences for First Year Students: A Key to Greater Diversity in Computing? https: //doi.org/10.1145/3159450.3159472

[5] Renata Revelo, R. A., and Hummel, J. (2022, August). Early Research Scholars Program at UIC Adjustments. In 2022 ASEE Annual Conference & Exposition.

[6] Christine Alvarado, Expanding the Pipeline – Engaging Undergraduates in Research: UC San Diego Early Research Scholars Program, CRA Newsletter, August 2017, Vol. 29/No.7.

[7] Michael Barrow, Shelby Thomas, and Christine Alvarado. 2016. ERSP: A Structured CS Research Program for Early-College Students. https://doi.org/10.1145/2899415.2899436

[8] Burcin Tamer, Early Research Scholars Program (ERSP) 2018 – 2021 Cohorts Aggregated Evaluation Report, Center For Evaluating the Research Pipeline, pp 9 -13, September 2022.