Implications and Limitations of the Use of AI in Criminal Justice in Germany
In the next two white papers from the AI Team at CIHR, Visiting Scholar Dr. Carsten Momsen dives into the impact of AI, and more specifically big data analytics, and their impact on German criminal procedure, from data collection and analysis to predictive policing and counterterrorism. The bias inherent in datasets and training data risk not only replicating biases (especially racial and gender) in the original data, bur risk introducing new biases. Tracing the “black box” effects of AI in policing from defendants’ rights to due process, the presumption of innocence and predictive recidivism, Dr. Momsen argues for case-based ethical approaches to decision-making that go beyond binary logic inherent in the “authored texts” of algorithms.
The next white paper in this series will be on digital evidence and criminal defense in Germany, accompanied by a series of blogs from our team on the recent Encrochat and Anom cases illustrating these problems in the context of encrypted mobile application data seizure in both European and US-Australia led investigations of transnational criminal networks.
— Dr. Marie-Michelle Strah
Implications and Limitations of the Use of AI in Criminal Justice in Germany by Dr. Carsten Momsen
In law enforcement practice globally, Big Data analytics (still) play a greater role than the use of specific AI, insofar as this goes beyond the collection and analysis process of the various data streams. A somewhat different picture emerges when looking at threat prevention. Here, AI is increasingly being used, especially in the area of predictive policing, where it goes beyond retrograde analysis to make real predictions and thus (at least) predetermine human decisions. In recent years, a new phenomenon has emerged: the same tools are being used in both security and law enforcement. This mixed situation arises primarily from the surveillance of “threats” in the area of preventive counterterrorism. What is particularly novel is the fact that the legal requirements for use are standardized almost identically in Germany. Another example of this from the area of general crime is the use of so-called “extended DNA analysis” (Forensic DNA Phenotyping — FDP).
1. Changes in Central Areas of German Criminal Procedure
In addition to these structural aspects, the new technologies harbor specific risks. Some central potential dangers of the use of AI can be described with the key words “lack of understanding of processes,” “lack of transparency,” “lack of individual fairness,” “promotion and reinforcement of existing inequalities,” “lack of evaluation level,” and “problem of trust-based decisions.” In addition, there are documented instances of discrimination and racism in algorithmic bias and datasets, behind which lie real social problems, including in law enforcement. The use of data analytics seems to make these problems at least more salient, as tendentious analyses can often still be tracked.
Bias can arise outside the analysis process performed by the AI, particularly by those involved in the design, process, and evaluation of data processing. In this context, the origin of such an error is de facto in the human domain — such as incorrectly or carelessly selected learning data or a lack of ability to interpret it properly. However, within the processes automated with the help of an AI, certain factors, such as the design of the algorithm based on certain assumptions and classifications, can also amplify, modify or distort the error. AI can increase the impact of errors or make them more difficult to detect. Multiple errors can occur, for example, because both the selection of the input data and the interpretation of the result of an analysis as evidence are not carried out appropriately. In the area of security and also law enforcement, they can lead to serious misinterpretations and misjudgments, such as the surveillance and prosecution of innocent people or the violation of elementary principles, e.g., the presumption of innocence.
Access the full white paper here: https://www.jjay.cuny.edu/sites/default/files/contentgroups/center_international_human_rights/implications_and_limitations_of_the_use_of_ai_in_criminal_justice_in_germany.pdf
