This article is published in the June 2018 issue.

Expanding the Pipeline: The Computer Science Outreach Program Evaluation Network –  Increasing Quality and Capacity

By: Karen Peterson, Founder and CEO, The National Girls Collaborative ProjectIn: /

The National Girls Collaborative Project (NGCP) brings together organizations throughout the United States that are committed to informing and encouraging girls to pursue careers in science, technology, engineering, computer science, and mathematics. NGCP serves more than 35,000 programs in 41 states and uses a collective impact model that builds the capacity of educational programs.

With a desire to support the development of effective and equitable computer science opportunities for girls and the acknowledgment that many nonprofits lack the evaluative capacity needed to ensure high-quality programming, the Google K-12 Outreach team and NGCP developed the CS Outreach Program Evaluation Network (CS OPEN). Launched in 2015 at an event co-hosted by the Center for Gender Equity in Science and Technology and the White House Council on Women and Girls, CS OPEN provides support to develop the evaluative capacity of network grantees. The CS OPEN is designed to improve opportunities and empower underserved girls through CS education by boosting program knowledge on exemplary evaluation practices in studying CS education.

Google, in partnership with Haynie Research and Evaluation, provides expertise and professional development to promote evaluation of CS education initiatives within the NGCP network. The overall goal of the network is not only to enhance the participating programs, but also to inform the field of CS education. The CS OPEN is designed to meet a wide range of needs through professional development opportunities and individualized support. NGCP identifies potential applicants, leads the selection process, and manages the grantee community. Grantees receive the following benefits:

  • The CS OPEN network community of practice, including monthly web-based meetings with topic-based trainings and sharing of successes and challenges
  • Grantee involvement in their own evaluation process, including writing an evaluation plan, developing instruments, collecting data, analyzing data, and reporting
  • A web repository of online evaluation resources
  • Individualized support and mentoring

From 2015-2018, 16 CS OPEN projects have served a total of 6,823 participants, of which 5,028 were girls. Project evaluations were conducted by each of the grantees. Each of the project evaluations included an evaluation plan, evaluation questions, methodology, a data collection plan, analysis work, and a final report to the CS OPEN team. Because of CS OPEN funding, eight new project evaluations were implemented and eight existing evaluation efforts were expanded. Grantees were asked through the surveys, interviews, and a final report to describe their growth in evaluative capacity, as well as the ways they attribute this growth to participating in the CS OPEN initiative.

 “It was helpful to learn I wasn’t alone – there are a lot of people all over the United States doing this stuff. It was encouraging to me. The ideas and plans that everyone was doing. Different ideas, but at the end we all have the same goal.”—Regarding Monthly Community Meetings

Grantees were asked through surveys, interviews, and a final report to describe their growth in evaluation capacity. These are some of the skills and advancements grantees self-reported as correlated with participation in the CS OPEN initiative:

  • Actively streamline evaluation tools across all programs
  • Created and developed better evaluation tools and instruments with more relatable context
  • Empowering all staff
  • Qualitative evaluation approaches
  • Creating milestones and measures, developing survey schedules
  • Transcribing data

When analyzed collectively, the broader evaluation findings on engaging girls in CS education were as follows:

  • Tap girls’ natural enthusiasm at young ages. Girls are interested and want to be engaged in computer science.
  • Consider best environments. Girls do well in all-girl settings. Physical activity can be integrated.
  • Offer hands-on, exploratory activities. Use hands-on activities, not lectures. Make activities engaging and relevant.
  • Provide support and encouragement. Provide help that is always available.
  • Use longer delivery formats. More contact time can support stronger outcomes.
  • Provide role models, a vision for future career. Linking CS to the real world is key.

The CS OPEN community meets quarterly to focus on evaluation capacity building, responding to the needs of the community. Applications for the next cohort of CS OPEN grantees will be announced in early 2019.  To learn more, visit the NGCP website, and contact Karen Peterson at kpeterson@ngcproject.org to get involved in the community.

Grantees and Members of the CS OPEN Community
Digital Girl, Inc.
DIY Girls
LA Makerspace
TechGirlz
Boys & Girls Clubs of Austin Area
CodeNow
CU Science Discovery
GEMS: Girls Excelling in Math and Science
Girl Scouts of Connecticut
Mobile Apps for Hartford
OSU Open Campus
STEM Academy @ OSU
UMBC
University of New Hampshire
Women in Science and Engineering Community Outreach
Young Women in Computing

About the Author
Karen Peterson is the founder and CEO of the National Girls Collaborative Project. She has more than 25 years of experience in education as a classroom teacher, university instructor, teacher educator, program administrator, and researcher. Peterson is also Co-Principal Investigator for Leap into Science:  Cultivating a National Network for Informal Science and Literacy, STEM Integration into Digital Forensics Science Learning, SciGirls Code: A National Connected Learning Model to Integrate STEM Learning with Middle School Girls, and Code: SciGirls! Media to Engage Girls in Computing Pathways. All of these projects are funded by the National Science Foundation and address gender, racial, and socioeconomic underrepresentation in STEM fields. Many of them access the NGCP’s national network and dissemination tools to distribute, scale-up, and/or replicate project outcomes. These projects have leveraged Karen’s expertise in STEM equity project development, effective national scale-up strategies, dissemination, and capacity building.

The National Girls Collaborative Project™ has been partially funded by a grant from the National Science Foundation, “GSE/EXT: National Girls Collaborative Project: Building the Capacity of STEM Practitioners to Develop a Diverse Workforce,” grant HRD-1532643.