We often treat data as an impartial representation of reality—an unbiased delivery mechanism for “ground truth.” Data collection and analysis systems, however, are designed by people: our knowledge, experience, and beliefs influence the design decisions we make and thus the data we collect. The layers upon layers of choices and assumptions we make in collection and analysis render data subjective, subtly imbuing it with a perspective of its own.
It’s straightforward to identify the particularly grievous examples of data’s subjectivity. To give just one example, the only federally collected data on homicides by police officers in the US, the FBI’s Supplementary Homicide Report, includes only self-reported justifiable homicides. But more subtle bias is present in apparently innocuous assumptions—for instance, that all usage of an Android phone is by the same user.
Given, then, data’s lack of objectivity, how do we use data responsibly to reason about the world we live in?
Rachel Shadoan explores how to adapt our processes to account for data’s lack of objectivity.
Rachel Shadoan is the cofounder and CEO of Akashic Labs, a Portland-based research and development consultancy, where she specializes in combining research methodologies to provide rich and accurate answers to technology’s pressing questions. Questions about people are her favorite kinds of questions to answer. Prior to founding Akashic Labs, Rachel worked with Intel exploring both how people use their phones in cars and how the ability to convert to a tablet impacts laptop use. She has also collaborated with Stanford digital humanities scholars and Oxford data archivists to develop a visual graph query language to allow researchers to form queries on complex multidimensional data. Originally from Oklahoma, Rachel holds an MS in computer science from the University of Oklahoma, as well as an MS in design ethnography from the University of Dundee in Scotland. As is thematically appropriate for her adopted home of Portland, she likes cruciferous vegetables (especially kale) and occasionally brews beer.
©2016, O’Reilly UK Ltd • (800) 889-8969 or (707) 827-7019 • Monday-Friday 7:30am-5pm PT • All trademarks and registered trademarks appearing on oreilly.com are the property of their respective owners. • firstname.lastname@example.org
Apache Hadoop, Hadoop, Apache Spark, Spark, and Apache are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries, and are used with permission. The Apache Software Foundation has no affiliation with and does not endorse, or review the materials provided at this event, which is managed by O'Reilly Media and/or Cloudera.