Making data science systems work

Abstract

How are data science systems made to work? It may seem that whether a system works is a function of its technical design, but it is also accomplished through ongoing forms of discretionary work by many actors. Based on six months of ethnographic fieldwork with a corporate data science team, we describe how actors involved in a corporate project negotiated what work the system should do, how it should work, and how to assess whether it works. These negotiations laid the foundation for how, why, and to what extent the system ultimately worked. We describe three main findings. First, how already-existing technologies are essential reference points to determine how and whether systems work. Second, how the situated resolution of development challenges continually reshapes the understanding of how and whether systems work. Third, how business goals, and especially their negotiated balance with data science imperatives, affect a system’s working. We conclude with takeaways for critical data studies, orienting researchers to focus on the organizational and cultural aspects of data science, the third-party platforms underlying data science systems, and ways to engage with practitioners’ imagination of how systems can and should work.