Great Innovative Idea- Wide-Field Ethnography: Studying Software Engineering in 2025 and Beyond
The following Great Innovative Idea is from David Socha from the University of Washington Bothell. Socha and his colleagues, Robin Adams (Purdue University), Kelly Franznick (Blink UX), Wolff-Michael Roth (University of Victoria), Kevin Sullivan (University of Virginia), Josh Tenenberg (University of Washington Tacoma), and Skip Walter (Factor, Inc), published a paper called Wide-Field Ethnography: Studying Software Engineering in 2025 and Beyond, which was the first place winner at the Computing Community Consortium (CCC) sponsored Blue Sky Ideas Track Competition at the 38th International Conference on Software Engineering (ICSE), May 14-22, 2016 in Austin, TX.
The Innovative Idea
Wide-field ethnography (WFE) refers to an approach of gathering and collaboratively analyzing large, multi-modal, multi-stream datasets of physical-social-economic-cyber systems (PSECs) in action. While our paper framed the WFE vision around physical-cyber-social systems (PCSSs), our research now focuses on PSECs in order to also include the important economic aspects of such systems as exhibited through a wide variety of value exchanges across multiple units of analysis (e.g., person, team, organization).
WFE is a result of the combination of increasingly rich sensing technologies, highly scalable computing, and advanced analytics that presents radical new possibilities for observing, recording, analyzing, understanding, and improving complex PSECs, such as software engineering teams and organizations and their products and services. We can capture high-resolution, multi-perspective, large-scale data of both physical phenomena (e.g., people’s speech and physical actions), and digital phenomena produced by human-machine interactions (e.g., software commits to version-control repositories) enabling rich inquiry into individual and team work in PSECs.
Impact
Wide-field ethnography makes work visible across modalities, systems, space, and time by gathering widely with multiple data recorders running concurrently. Collecting so widely provides datasets that help researchers to individually and collectively:
- Follow work over space and time.
- Pursue a wide range of research questions using a variety of units of analysis, such as an individual, task, type of work (e.g., triage), location in the room, time of day, topic (e.g., power structures), or practice (e.g., standup meetings).
- Do inter-disciplinary analysis by communities of researchers using a diversity of theoretical and analytical frameworks.
- Widen the observational field beyond what in situ human observation affords.
- Mitigate against researchers’ preconceptions of what type of data to collect.
- Reduce risk when you don’t know what you might want to later go back and study.
- Argue more strongly about patterns by evidencing them across larger datasets.
- Embrace emergence since the datasets often will contain data to support even more interesting questions uncovered during analysis.
- Develop and evaluate machine-learning algorithms on naturalistic datasets (e.g., laughter detection).
- Access sufficient context to understand particulars (e.g., speech, gestures, documents).
Wide-field ethnography is flexible, scalable, and widely applicable. WFE can be used for preliminary observations, a single session in one room, or for a multi-site study using dozens of data collectors over months. Since WFE makes work visible in such a rich manner, WFE is valuable for any discipline interesting in studying social aspects of systems. This includes software engineering, computer supported cooperative work, human computer interactions, user research, human-centered design, anthropology, sociology, education, and a variety of other human-centered and engineering disciplines.
Other Research
My general area of research is collaboration in the wild, where my colleagues and I study how humans collaborate in naturalistic settings (i.e., not laboratories) to co-design and co-evolve complex systems. This research area includes studying agile software development practices, how software developers design, and how students design. As part of WFE’s blending of Thick Data with Big Data, my colleagues and I are working on a variety of machine learning algorithms to automatically detect events of interest for further exploration and analysis (e.g., we created a laughter detection system). See my Collaboration in the Wild website for more details.
An emerging area of research is using WFE to study educational practices in situ, such as active learning. University of Washington Bothell is instrumenting a set of classrooms in order to create a laboratory in which we can do contextually rich inquiry into what teaching practices work well in which contexts. Our interdisciplinary team plans to gather a set of WFE datasets of teaching and learning in action and connect these with other metrics of learning and performance within and across classes.
Researcher’s Background
I am an Assistant Professor in the Computing and Software Systems Division in the School of Science Technology Engineering and Math at University of Washington Bothell. Prior to joining UW Bothell, I spent 19 years in industry working as a software developer, engineering manager, and agile coach in a variety of industries from startups to large Telecoms and working on a diverse set of products such as voice mail, mapping tools for emergency responders, Nintendo 64 games, and large-scale microsimulation systems for urban planning. At UW Bothell I teach and research about how people collaboratively understand, design, build, and evolve complex physical-social-economic-cyber systems. I have a BS in Zoology from the University of Wisconsin, Madison, a Masters and Ph.D. in Computer Science and Engineering from the University of Washington, Seattle, and a black-belt in aikido.
Links
For those of us who like to build software tools, WFE presents an opportunity, since WFE datasets are too large and complex for a single researcher to easily understand, navigate, browse, filter, annotate, and analyze without tool support. For our WFE vision to come to fruition, researchers need a substantial ecosystem of software tools to leverage the opportunities latent in such large and complex datasets. Creating WFE tools will require attending to a range of technical, methodological, and privacy issues, but the resultant tools and approach promise to help us:
- better understand and automate important aspects of software engineering organizations and the PSECs they create, and
- benefit a wide spectrum of researchers in other disciplines who study PSECs.
My research group has begun building these WFE tools in order to enable research and teaching. In the near future when the WFE tools are sufficiently advanced, we expect to open the BeamCoffer dataset to analysis by other research groups around the world.
More on wide-field ethnography is available on my research website: http://depts.washington.edu/citw/wordpress/. This includes a video of a June 2, 2016 talk I gave at Microsoft Research that expanded on the themes in our award-winning ICSE 2016 paper.
