AI in Justice: Balancing Innovation and Human Rights

Moroccan justice officials have raised concerns about the potential risks associated with the use of artificial intelligence (AI) in the legal system. During a recent training session in Fez, the country's top prosecutor, Hisham Blaiwi, emphasized the need to balance the benefits of AI with the protection of human rights and fundamental freedoms.

Blaiwi acknowledged the promising opportunities that AI presents for enhancing investigative methods, data analysis, and the detection of cybercrimes and transnational criminal activities. However, he cautioned that the use of AI in criminal justice raises significant challenges, including the accuracy of results, interpretability, and the protection of personal data and privacy.

He also highlighted the potential risks of bias and the impact on fair trial guarantees. Blaiwi stressed the importance of subjecting AI outputs to rigorous scrutiny and ensuring that they are not treated as definitive facts. He underscored the need to distinguish between technical indicators and legally admissible evidence, and to verify the origin, integrity, and lawfulness of digital evidence, as well as the reliability of the tools used in its analysis.

Blaiwi further emphasized the importance of human oversight, clear responsibilities, and the provision of safeguards for accountability and error correction. He noted that the use of AI should not compromise the presumption of innocence, the right to defense, or judicial independence.

Blaiwi also referenced international frameworks for AI governance, including UNESCO's 2021 Recommendation on the Ethics of AI, the OECD's 2019 Principles on AI, and the forthcoming Council of Europe's Convention on AI, Human Rights, Democracy, and the Rule of Law. These frameworks emphasize the importance of respecting human rights, promoting transparency and accountability, combating discrimination, and ensuring human oversight.

Meanwhile, researchers from the United States have warned about the emotional and psychological impacts of AI, particularly in the context of conversational AI applications. These applications, which simulate parental emotions, can lead children to believe that a real person is interacting with them through a device. Experts caution about the blurring of lines between AI applications and humans or other devices.

Sherry Turkle, a psychologist at the Massachusetts Institute of Technology, has highlighted the risks of forming emotional attachments to AI, noting that such relationships can lead to existential dangers. She argues that these relationships can form despite the AI's mechanical nature, as users may begin to perceive the AI as a living being that cares for and loves them.

Researchers from the Universities of Zurich and Amsterdam have also noted that even the most advanced chatbot software struggles to accurately mimic the full range of human linguistic expression. A recent study published in the Proceedings of the National Academy of Sciences found that AI-generated texts can be systematically distinguished from human communication. The study, based on an analysis of social media messages and posts, showed that AI consistently faces challenges in replicating the emotional depth and intended meaning of user messages.

Previous research has also suggested a link between AI use and potential declines in mental health. A team from the University of Oxford and London College warned last year about a potential feedback loop where conversational AI applications could exacerbate certain mental health conditions, such as psychosis.

**Context/Background:** The integration of AI in various sectors, including justice and daily communication, presents both opportunities and challenges. While AI can enhance efficiency and provide new insights, it also raises concerns about privacy, bias, and the potential for misuse. Ensuring that AI is used responsibly and ethically is crucial for protecting human rights and maintaining public trust.