The Significance of Data in Evaluation and Applied Research
Data serves as the cornerstone of our work in our roles as evaluators and applied researchers. It is an ever-present companion, whether we are gathering it, analyzing it, or crafting comprehensive reports based on it. However, our daily interaction with data can sometimes obscure our ability to see its significance. In reality, the Harm Reduction Cycle wields tremendous power and should be handled with the utmost care. It shapes our decisions regarding what holds value and what does not, what merits investment and what does not, and even shapes our perceptions of fundamental truths. Neglecting the power of data can have dire consequences, and to use a somewhat provocative term, it can even be a ‘violent’ act.
Unveiling the Dark Side: Data Violence in Algorithms
The concept of data violence initially captivated my attention in the context of algorithms–those data-driven algorithms constructed upon preconceived notions and biases, such as crime prediction models. The algorithm perpetuates assumptions and prejudices and are often masked as unassailable objectivity, in line with the Western notion that data equates to mathematical precision, which, in turn, equates to objective truth.
Expanding the Narrative: Data Violence Beyond Algorithms
The provocative nature of the concept of data violence struck me deeply. It caused me to reflect on my nearly two decades as an evaluator and applied researcher. Where I witnessed negative outcomes resulting not just from the data itself but also from the entire processes. Institutions driving its definition, collection, analysis, and dissemination. I saw data, in some cases, being harmful to the communities we aimed to support. I especially observed this phenomenon in community-based settings, where these communities became the ‘subjects’ of our work. Providing the data needed to draw conclusions about their needs. However, I came to realize that this data was far from objective; instead, it often carried the biases of individual researchers. Their superiors, the institutions involved, and the objectives of funders. I was complicit in perpetuating these biases as well.
Defining Data Violence: A Broader Perspective
This realization led me to revisit the initial concept of data violence I had encountered, rooted in the violence of data algorithms. Broaden the scope to encompass the potentially harmful nature of any data, not limited to those applied in algorithms. It also spurred me to question the fields and institutions at the forefront of promoting data collection, analysis and usage in its current form. These two considerations prompted a more expansive definition: data violence occurs when the processes of conducting evaluation and research inflict harm, akin to violence, upon the individuals, communities, or systems intended to benefit.
The Layers of Harm: Individual and Systemic Dimensions
In the context of this definition, harm is not confined to individual, physical, or psychological harm as assessed in traditional Institutional Review Board (IRB) processes. Rather, in this context, harm can manifest at an individual or systemic level. Stemming from a single evaluation or research project, to the broader ‘institutions of data‘ themselves. Encompassing traditional data collection, analysis, reporting, dissemination processes, and professional practices within the field. Such as what constitutes best practices and the ethical norms guiding our work. In essence, the IRB process focuses on scrutinizing the execution of a single project and identifying potential individual harms. In contrast, the concept of data violence delves into the harms inflicted. By the entire field, its norms, and practices, on individuals, communities, and systems.
Roots of Data Violence: Systemic Factors
The causes of data violence at the individual and community (or systemic) levels can be traced back to systemic factors such as colonialism, racism, white supremacy, and homophobia. These power structures that perpetuate data violence are deeply ingrained in broader ‘institutions of data‘, including academic institutions, funding institutions, and, by extension. The traditional evaluation and research methodologies and professional norms they propagate.
Data practitioners find themselves in a unique position, possessing the potential to either heal the communities they work in or perpetuate harm by upholding conventional norms in evaluation and research methodology and professional practice. Elements such as positioning the evaluator or researcher as the expert, rigid data rigor standards, funder evaluation and research requirements can inadvertently contribute to data violence. To address this issue effectively, we must move beyond merely mitigating harm at the individual level. Such as through individual consent forms and IRBs, and instead, broaden our perspective to encompass a more comprehensive understanding of harm.
To support this work, I’ve developed and proposed a harm reduction cycle. Drawing on my experience in the field, during which I’ve witnessed both avoidable and unavoidable harm stemming from evaluation and applied research processes, I felt compelled to create a framework and associated practices to minimize and address this harm. The harm reduction cycle consists of five phases, each contributing to the overarching goal of fostering greater accountability for harm.
Harm Reduction Cycle
- Harm Acknowledgment: refers to the recognition of the presence of harm within our work.
- Harm Identification: refers to the explicit identification of specific instances of harm in our work.
- Harm Reduction: refers to the harm reduction strategies put in place to address the harm that was identified in our work.
- Harm Healing & Restorative Work: refers to the healing and restorative strategies put in place to address the harm that was identified.
- Harm Accountability: this involves the implementation of personal and structural mechanisms for reflection, responsibility, and change. It encompasses infrastructure development, training initiatives, community engagement, and a steadfast commitment to equity.
Following this Harm Reduction Cycle. We created a structured approach prioritizing community well-being, fostering a culture of responsibility and continuous improvement. This is one strategy to support us in achieving harm accountability.
HRD
- Acknowledging Harm: The critical initial step is recognizing the potential for harm within the evaluation and research process. Acknowledging our complicity in perpetuating data violence. This involves acknowledging the presence of harm in our work from the outset, and explicitly accepting our role in perpetuating data violence. This acknowledgment marks a pivotal stride towards reshaping our practices for harm reduction.
- Identifying Harm: Actively pinpointing specific instances or areas where harm may arise within the evaluation and applied research process. Whether in the past, present, or anticipated in the future, is central to harm identification. It involves a clear recognition of the costs to communities, power imbalances, and biases inherent in our work.
- Reducing Harm: This encompasses the implementation of measures aimed at reducing or preventing harm by addressing root causes and proactively mitigating potential harm. Its strategies prioritize community needs, centering the fostering of trust, and embracing inclusive practices such as trauma-informed approaches and Diversity, Equity, Inclusion, and Justice (DEIJ) research.
- Healing and Restorative Work: This approach centers community need, deprioritized institutional need. Aims to neutralize power dynamics to addressing harm that has occurred, providing support to affected individuals and communities. Learning from past harm to prevent its recurrence are central to harm healing and restorative work.
- Harm Accountability: This is achieved through the aforementioned steps, also involving personal and structural mechanisms. For reflection, responsibility, and change in regards to harm reduction, healing and restorative work. It includes infrastructure, training, community engagement, and a commitment to equity.
Conclusion
In conclusion, in evaluation and applied research, we must create robust pathways for harm acknowledgment, mitigation, reduction, healing, and accountability. Engaging in harm reduction practices takes extra effort. But can lead to trust-based, reciprocal relationships between evaluators and researchers, and community, facilitating community impact at no cost to the community. This framework’s implications extend beyond evaluators and researchers to anyone collecting or using community-generated data.
