Argument

We were asked to address the question:

To what extent can cyber evidence repositories, and digital and open—source evidence, facilitate the work of the OTP, and the ICC more generally?

We should point out that we have both advised the Court and the OTP on technology for more than five years and currently serve on the Court’s Technology Advisory Board. One of us (Enrique Piracés) is a human rights practitioner with a long history of developing and adapting ICTs for human rights documentation and advocacy in both resource-limited and well-resourced contexts. He is the creator of Digital Evidence Vault, a widely-used system for collecting and preserving digital evidence from social media and the internet. The other (Jay Aronson) is a historian and social scientist who has devoted his career to understanding and improving the role that science and technology can play in achieving just outcomes in the domains of law and human rights. He has written extensively on the value of large media collections and first-person video in the investigation of human rights violations, as well as the ways that computer vision, machine learning, and artificial intelligence can (and cannot) be used to analyze this data. Along with a statistician colleague, the authors run the Center for Human Rights Science at Carnegie Mellon University, which was founded in 2011 to facilitate partnerships between scientists and researchers in academia and human rights practitioners with data gathering and analysis challenges that they cannot address on their own. As such, we understand the general issues associated with the collection, preservation, and analysis of cyber evidence and open source intelligence and the unique challenges and opportunities that the Court and the OTP faces in this domain.

As we noted in the introductory section, we believe that the Court and the OTP have a tremendous opportunity to make use of the content and evidence repositories being built in situations of interest to the Court. We know that tools and methods currently exist for the internal staff of the Court to gather and collect evidence that will positively impact the ability of the Court to investigate crimes against humanity (as we have already seen in the Werfalli case), but that these tools and methods are not currently being used by the Court in the most effective ways. We also believe that computer vision, machine learning, and AI can be used to increase the effectiveness and efficiency of the OTP and the Court. However, the successful implementation of this vision will require changes in the structure and function of the Court and the OTP beyond just the adoption of new technologies.

In what follows, we offer some thoughts on issues the Court and the OTP will need to think through if they are going to make the most of open source human rights media, digital evidence repositories, and artificial intelligence in the future. We do not provide our views on the value of open source evidence here since we have done that elsewhere.1 Nor do we make the case for open source investigations in human rights accountability efforts more generally or in the context of the ICC (the former has already been done in an extraordinary new volume co-edited by Sam Dubberley, Alexa Koenig, and Darragh Murray2 and the latter has been done by Lindsay Freeman in the aforementioned edited volume and by Nikita Mehandru and Alexa Koenig in a recent Harvard Human Rights Journal article3 and in several reports by them and their UC Berkeley Human Rights Center colleagues). Nor do we address the legal dimensions of admissibility because it is not our area of expertise.

The ICC is the main international institution with the mandate to investigate grave crimes against humanity and prosecute wrong-doers in countries that are unwilling or unable to address them. It is constrained at many levels, not least of which is that it must wait for a request for investigation from the UN Security Council, or the Prosecutor herself must decide to launch an investigation into a situation in which dire human rights violations are taking place and perpetrators are not being held accountable. This means that the time between crimes against humanity and the start of an investigation is often very long (sometimes years), that the country in which the crimes are being investigated is hostile to the investigation, or both. These challenges make the job of the investigator incredibly difficult and ensures that ICC officials are never the first to arrive at a crime scene.

In the past, this meant that in order to build a case, the Court was required to find eyewitnesses to the crimes, victims of crimes, or relatives of victims and then make arrangements to get statements from them about events that happened in the past. Such witness statements can sometimes be weak and easy for defense attorneys to discredit. Both critics of the Court and its supporters have long advocated for changes in the way that the OTP made its case in Court.

At least since the Syria Civil War (and certainly earlier on a limited basis), conflicts and human rights violations have been well-documented on video and in pictures by people on the ground, whether they be bystanders, victims, or the violators themselves. Some of these first responders have been ordinary people, while others are local humanitarian responders, journalists, or human rights advocates. This has made it possible for the Court and its investigators to gain access to evidence produced directly by first responders.

While the Court has been looking for ways of communicating directly with first responders for several years, the most likely means for investigators to gain access to this material is through public social media accounts. In terms of access, the challenge presented by citizen evidence, and overall online content, is not that it per se is volatile in nature, but rather that it is held by private third parties that are globally distributed and with different terms of service. This determines what content can be published, removed, or preserved (as well as what the mechanisms are to request removal or preservation). Many organizations, like the UC Berkeley Human Rights Center and WITNESS, have engaged with the private sector to find avenues for materials that may have probative or evidentiary value to be made available to human rights fact-finding efforts, and however valuable they are, they can only satisfy a number of situations.

In the absence of agreements with the main platforms, the only way to ensure that material that is published on social media is available for eventual ICC action is to preserve it as quickly as possible after it is uploaded. However, this opportunistic preservation is often done without a clear focus on what type of information will eventually be needed, leading to too much or too little material being preserved. Further, while there might be a mechanism for working directly with the platform company to do the collection and preservation, the process might be too onerous or inefficient, on the one hand, or lead to a deluge of data on the other. In general, though, platforms have not developed clear guidelines for the human rights community to obtain data from them.

In the absence of the ability to carefully collect just the right media, investigators are often left to ingest as much data as possible in the short term, with the cleaning of the collection and tagging of media for future reference having to wait for later processing. Volume can create a challenge for the ICC OTP as it may burden analysts and lawyers with the need to review content and disclose its findings to the defense.4 This is where computer vision, machine learning, and AI can have the biggest impact. AI is very good at recognizing patterns in data and looking for these patterns repeatedly in large data sets. It is not very good at developing nuanced interpretations of the meaning of these patterns or placing them in historical, social, and political context. Nor is it particularly good at evaluating the authenticity of the source of the data. Once a dataset has been filtered and tagged automatically, authentication and determination of probative value of relevant material are much better handled by human investigators.

The Court and the OTP cannot be expected to become a scientific institution or employ experts in all relevant emerging technologies. The Court could learn from the experience of other communities of practice where practitioners often find solutions by relying on trusted networks with specialized organizations. The Court should, as much as possible, distribute these connections and knowledge widely across the institution so that no one unit becomes the gatekeeper to accessing technology. Strategically, the Court should also favor systems that are open, both as in source and open for use by all units in the Court, including the defense bar.

The big challenge is how to make this happen in an institution that is not designed for innovation and that has distributed management, as well as ownership, of key responsibilities and resources around technological issues. A sustainable and effective way to approach this is by providing analysts, lawyers, and judges with access to knowledge about the technologies that are behind the growth in digital content as well as the technologies and methods that may have the most potential to address these new challenges. Empowering analysts, in particular, by creating opportunities for learning, experimentation, and creativity, may be the best way to adapt to the new challenges.

The OTP could find a good fabric to support such efforts in the myriad of organizations and projects that have been working at the intersection of human rights, science, and innovation over the past decade. For that to be effective, there would need to be a new form of engagement that facilitates rapid response to relevant issues and that also reduces the institutional barriers that traditionally hinder change.

In our invitation to contribute to this forum, we were given a question to answer. In our conclusion, we wanted to pose a related question for consideration: How can an unique institution, and its particular bureaucracy, create agile mechanisms to stay up to date with the latest forms of evidence and the technology needed to process it; increase the capacity of analysts, lawyers, and judges to understand, adapt, and forecast the use of technology; remove administrative roadblocks to technology adoption when useful; and create spaces for exposure to emerging technologies? We are not suggesting that the Court must adopt new technologies simply because they are available, but it must develop a more robust and less institutionally constrained system for evaluating, adopting, and adapting relevant technologies that make its difficult job easier to accomplish.

Endnotes — (click the footnote reference number, or ↩ symbol, to return to location in text).

1. 1.

   Jay D. Aronson, Computer Vision and Machine Learning for Human Rights Video Analysis: Case Studies, Possibilities, Concerns, and Limitations, 43 Law & Soc. Inquiry 1188 (Mar. 1, 2018), available online, doi; Jay D. Aronson, McKenna Cole, Alex Hauptmann, Dan Miller & Bradley Samuels, Reconstructing Human Rights Violations Using Large Eyewitness Video Collections: The Case of Euromaidan Protester Deaths, 10 JHRP 159 (Apr. 18, 2018), paywall, doi; Jay D. Aronson, The Utility of User-Generated Content in Human Rights Investigations, in New Technologies for Human Rights Law and Practice 129 (Molly K. Land & Jay D. Aronson, eds., Apr. 2018), available online, doi; Jay D. Aronson, Preserving Human Rights Media for Justice, Accountability, and Historical Clarification, 11 Genocide Stud. & Prevention 82 (May 2017), available online, doi. ↩

2. 2.

   Digital Witness: Using Open Source Information for Human Rights Investigation, Documentation, and Accountability (Sam Dubberley, Alexa Koenig & Darragh Murray, eds., Feb. 19, 2020). ↩

3. 3.

   Nikita Mehandru & Alexa Koenig, Open Source Evidence and the International Criminal Court, 32 Harv. Hum. Rts. J. (Apr. 15, 2019), available online. ↩

4. 4.

   See International Criminal Court, Rules of Procedure and Evidence, ICC-ASP/1/3 (2d ed. Sep. 9, 2013), available online, archived. ↩

5. Suggested Citation for this Comment:

   Jay D. Aronson & Enrique Piracés, The OTP and ICC Can Take Advantage of Open Source Evidence and Digital Evidence Repositories, Core Elements of Almost All Grave Crimes Investigations, if They Undertake Cultural, Procedural, and Bureaucratic Changes to Create a More Agile and Open Institutional Environment, ICC Forum (Jun. 1, 2020), available at https://iccforum.com/cyber-evidence#Aronson.

   Suggested Citation for this Issue Generally:

   To What Extent Can Cyber Evidence Repositories, and Digital and Open-Source Evidence, Facilitate the Work of the OTP, and the ICC More Generally?, ICC Forum (Jun. 1, 2020), available at https://iccforum.com/cyber-evidence.

Subsequently, in the case of Al Mahdi,4 the Court was presented with a significant quantity of open source evidence including satellite images taken from Google Earth as well as content from YouTube and audio content sourced online.5 The conflict in Al Mahdi occurred in 2012, and was among the first cases to be tried whose events had occurred during a period in which digital recording devices and satellite and drone technology were in common use. In this respect, the case offers an insight into the necessity and prevalence of open source evidence in future international criminal prosecutions. In 2017, the Prosecutor relied on open source evidence in the issuance of the Al-Werfalli arrest warrant.6 The warrant was notable for its reliance on open source evidence drawn from social media, specifically YouTube, as well as five video files drawn from social media generally.7

However, Bemba et al.8 is perhaps the most concrete example of reliance on social media evidence by the Court to date. In Bemba, the defendant was alleged, subsequent to other crimes of which he had been convicted, to have engaged in witness tampering.9 Specifically, the Prosecution argued that the defendant had bribed witnesses to change their testimony.10 In support of this allegation, the Prosecution submitted evidence of a wire transfer as well as pictures from Facebook showing the two witnesses alleged to have been bribed together.11

Authentication of Open Source Evidence

Once open source evidence has been obtained, the issue for the Court is whether such evidence can be used. While open source evidence is by its nature generally already in the public sphere, its use is contingent on authentication.12 Thus, while there appear to be few practical barriers to the acquisition of such evidence, its collection is troubled by concerns over integrity, compatibility, and standardized authentication procedures.13

There is currently no designation of what constitutes admissible evidence within the ICC Rules of Procedure and Evidence. Rather, the Rules establish a framework of evidentiary analysis in accordance with which evidence is admitted or rejected based on its relevance, probative value, and prejudicial impact.14 In addition, the Court has developed standard procedures for uploading and presenting evidence in the form of the e-Court Protocol.

The Protocol provides for direct disclosure, in-court provision of digital materials, and consistent information exchange.15 However, it is notable that while the Protocol offers some guidance on authentication, by specifying that metadata should be attached to digital files for example,16 it is largely limited to harmonizing the format, means of storage, and presentation of digital evidence within the Court system. The Protocol does not address the probative value of digital evidence which, under Rule 63(2), remains flexible, broadly drawn, and subject to the graduated grounds for action under the Rome Statute.

The result is that there seems to be little procedural guidance on the risks and challenges posed by open source evidence and its authentication. Given the minimal involvement of open source evidence in trials previously, this was not a significant issue. But as the events considered by the Court edge into the digital age, as in Al Mahdi, the necessity of employing a coherent and consistent approach to authentication will become increasingly necessary. This necessity is now heightened by the emergence of tools for the manipulation of the images, audio, and video which form a crucial part of open source evidence. To date the most potentially problematic of these are deepfake technologies.

Deepfake Technologies

Deepfake videos are produced by artificial intelligence and machine-learning tools. Machine learning is a branch of artificial intelligence that permits computer systems to learn from data and “experience” rather than through programmed rules. These tools can merge, combine, replace, and superimpose audio, images, and video onto existing files, creating “fake” content which nevertheless appears authentic. Deepfake videos, for example, allow one person’s face to be convincingly superimposed onto a video clip or image of another person. Thus, using deepfake technology, a video of an extra-judicial killing could be manipulated so the face of one individual was superimposed over the face of another, or the uniform of a non-state actor was superimposed over that of a militia group.

Concerningly, access to deepfake technology is not limited to the technologically sophisticated or state actors. As Wittes and Blum have noted, even elaborate and ostensibly tightly controlled “dangerous” technologies diffuse rapidly17 lagging only where the technology experiences a chokepoint as a result of its tangible and thus limited nature (for example enriched uranium) or institutional controls. In the context of deepfakes, such scarcity is not present, nor it is apparent that institutional awareness of the need to curtail the technologies underlying deepfakes is high. Indeed, at present, the most well-known deepfake technology, Google’s Tensorflow, and Adobe which is incorporating AI into its programs (including Scene Stitch) are all available on a direct to consumer basis.

The result, Citron and Chesney argue, is that the capacity to generate deepfakes while already present for those with above average technological capacity will diffuse rapidly as the volume of publicly available deepfake tools increases extending the reach of such technologies beyond specialized users.18 Given the fragmentary and incomplete control of the digital environment by States and the democratization of access to global audiences which social media and online platforms provide, this diffusion carries the potential for such content to reach large and international audiences, and to proliferate to such an extent that the real and the fake are difficult to differentiate.

While the implications of deepfake technologies for the integrity of elections and democratic processes are significant, such technologies pose particular problems for evidential integrity broadly, but specifically including international criminal prosecutions. How then to authenticate open source evidence and thus fight against deepfakes and their impact on due process and the administration of justice?

Of course, the manipulation and alteration of photographic and video evidence long predates digital technologies.19 However, prior to the emergence of digital technologies, a forensic investigator had only to compare an actual photograph with its negative to verify authenticity. In digital imaging, a RAW file format is the counterpart of this negative. The RAW file is produced by a digital device and constitutes the original or unaffected pixel information of an image. Post-processing alterations of an image leave traces which are thus detectable when the RAW file is examined. Such authentication is complicated by open source deepfakes, however, which are made available in compressed formats which may cancel or fatally compromise RAW files such that the processing history of the associated content is either partially or entirely beyond reach.20

Moreover, the machine-learning algorithms and AI on which deepfake technologies rely are designed to constantly improve their performance. As such, authentication tools must constantly evolve alongside deepfake technologies themselves. This was pointedly illustrated by a group of researchers in 2018 who, led by Andreas Rössler, created a dataset using some half million images and developed an algorithm that could detect manipulated or otherwise altered images based on their dataset.21 However, the same technology could also be used to improve the manipulation of the images such that detection was harder.22

In light of such challenges, the US Defense Advanced Research Projects Agency (DARPA) has launched a media forensics research initiative to facilitate the creation of deepfake detection technologies that automatically seek out and assess digital visual media to determine whether images are authentic. However, the resulting technologies remain in development and have not been tested.23

In a research context, Citron and Chesney have predicted that such problems with authentication may be ameliorated by the development of what they refer to as “immutable life logs” or authentication trials that permit a possible victim of a deepfake to produce a certified alibi credibly proving they did not do or say what is depicted.24 Citron argues such logs are enabled by the introduction of wearable tech, and the internet of things more broadly, but might also be specifically created for the purpose of combatting deepfakes.

In the context of international criminal law, however, the solution based on a life log is problematic. Conflict zones and those in them do not necessarily benefit from the uninterrupted high latency internet connection necessary to produce an unbroken record of an individual’s movements. Indeed, those participating in conflicts may actively seek to go “untracked” such that only intermittent and anonymous phone signals are available. This solution, moreover, presumes that individuals can be associated beyond reasonable doubt with a device, and that such devices are present—neither being a given in conflict areas and in less developed countries.

Perhaps the more encouraging avenue towards authentication lies with an industry partnership. Several of the market dominant platforms from which open source evidence is derived now recognize the challenges that deepfakes present. Facebook, for example, is taking measures to combat misleading or false material on its platforms including through verifying the sources of content uploaded to its platforms, while Google and Twitter have announced similar plans and policies against deepfakes.25 Given the resources necessary to successfully detect deepfakes, it is likely that a partnership with an academic centre of industry is the most practical means of ensuring the Court can consistently evolve its authentication methodologies in light of the evolving nature of the AI and machine learning on which deepfakes rely.

Endnotes — (click the footnote reference number, or ↩ symbol, to return to location in text).

1. 1.

   See Róisín Á Costello, International Criminal Law and the Role of Non-State Actors in Preserving Open Source Evidence, 7 Cambridge Int’l L.J. 268 (Dec. 1, 2018), paywall, doi. ↩

2. 2.

   Peggy O’Donnell, Alexa Koenig, Camille Crittenden & Eric Stover, UC Berkeley HRC, Beyond Reasonable Doubt: Using Scientific Evidence to Advance Prosecutions at the International Criminal Court, 7 (Jan. 7, 2013), available online. ↩

3. 3.

   The Prosecutor v. Abdallah Banda Abakaer Nourain and Saleh Mohammed Jerbo Jamus, ICC-02/05-03/09 OA 4, Judgment on the appeal of Mr Abdallah Banda Abakaer Nourain and Mr Saleh Mohammed Jerbo Jamus against the decision of Trial Chamber IV of 23 January 2013 entitled “Decision on the Defence’s Request for Disclosure of Documents in the Possession of the Office of the Prosecutor” (AC, Aug. 28, 2013), available online; The Prosecutor v. Bahar Idriss Abu Garda, ICC-02/05-02/09, Decision on the Confirmation of Charges (PTC I, Feb. 8, 2010), available online. ↩

4. 4.

   The Prosecutor v. Ahmad Al Faqi Al Mahdi, ICC-01/12-01/15, Decision on the confirmation of charges against Ahmad Al Faqi Al Mahdi (PTC I, Mar. 24, 2016), available online. ↩

5. 5.

   Id. ↩

6. 6.

   The Prosecutor v. Mahmoud Mustafa Busayf Al-Werfalli, ICC-01/11-01/17, Warrant of Arrest, 8 (PTC I, Aug. 15, 2017), available online. ↩

7. 7.

   Id. 8–9. ↩

8. 8.

   The Prosecutor v. Jean-Pierre Bemba Gombo, Aimé Kilolo Musamba, Jean-Jacques Mangenda Kabongo, Fidèle Babala Wandu and Narcisse Arido, ICC-01/05-01/13, Public Redacted Version of Judgment pursuant to Article 74 of the Statute (TC VII, Oct. 19, 2016), available online. ↩

9. 9.

   Id. ↩

10. 10.

    Id. ↩

11. 11.

    Id. at 83. ↩

12. 12.

    See The Prosecutor v. Vujadin Popović, Ljubiša Beara, Drago Nikolić, Ljubomir Borovčanin, Radivoje Miletić, Milan Gvero, and Vinko Pandurević, ICTY-05-88-T, Decision on Admissibility of Intercepted Communications, 4, 22, 26, 33–35 (ICTY TC II, Dec. 7, 2007), available online. ↩

13. 13.

    International Bar Association, Evidence Matters in ICC Trials, 18–20 (Aug. 9, 2016), available online; Aida Ashouri, Caleb Bowers & Cherrie Warden, An Overview of the Use of Digital Evidence in International Criminal Courts, 11 DEESLR 115 (2014), available online, doi. ↩

14. 14.

    See International Criminal Court, Rules of Procedure and Evidence, ICC-ASP/1/3, R. 64, 68, 72 (2d ed. Sep. 9, 2013), available online, archived. ↩

15. 15.

    See Janet Keddie & Alvaro Flores, ICC, The Use of eCourt Technology at the International Criminal Court (Jun. 27, 2008), available online. ↩

16. 16.

    The Prosecutor v. Germain Katanga and Mathieu Ngudjolo Chui, ICC-01/04-01/07, Prosecution’s Communication of eCourt Protocol Metadata to the Defence on 23 June 2008 and Communication of the Original to the Registry (PTC I, Jun. 23, 2008), available online. ↩

17. 17.

    Benjamin Wittes & Gabriella Blum, The Future of Violence: Robots and Germs, Hackers and Drones—Confronting a New Age of Threat (Mar. 10, 2015), paywall. ↩

18. 18.

    Bobby Chesney & Danielle Keats Citron, Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security, 107 Cal. L. Rev. 1753 (2019), available online, doi. ↩

19. 19.

    See generally, Mia Fineman, Faking It: Manipulated Photography Before Photoshop (2012), paywall. ↩

20. 20.

    Andreas Rössler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, & Matthias Nießner, FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces (Mar. 24, 2018), available online; Paolo Bestagini, Simone Milani, Marco Tagliasacchi & Stefano Tubaro, Local Tampering Detection in Video Sequences, IEEE Xplore (Nov. 11, 2013), available online, doi. ↩

21. 21.

    Rössler, supra note 20. ↩

22. 22.

    This Algorithm Automatically Spots “Face Swaps” in Videos, MIT Tech. Rev., Apr. 10, 2018, available online. ↩

23. 23.

    Steven Melendez, How DARPA’s Fighting Deepfakes, Fast Company, Apr. 4, 2018, available online. ↩

24. 24.

    Chesney & Citron, supra note 18. ↩

25. 25.

    See Nick Dufour & Andrew Gully, Contributing Data to Deepfake Detecting Research, Google AI Blog (Sep. 24, 2019), available online; David McCabe & Davey Alba, Facebook Says It Will Ban ‘Deepfakes,’ N.Y. Times, Jan. 7, 2020, available online; Mike Schroepfer, Creating a Data Set and a Challenge for Deepfakes, Facebook AI (Sep. 5, 2019), available online; Shirin Ghaffary, Twitter Is Finally Fighting Back Against Deepfakes and Other Deceptive Media, Vox Recode, Feb. 4, 2020, available online. ↩

26. Suggested Citation for this Comment:

    Róisín Á Costello, Facilitating the Use of Open Source Evidence at the International Criminal Court: Authentication and the Problem of Deepfakes, ICC Forum (Jun. 1, 2020), available at https://iccforum.com/cyber-evidence#Costello.

    Suggested Citation for this Issue Generally:

    To What Extent Can Cyber Evidence Repositories, and Digital and Open-Source Evidence, Facilitate the Work of the OTP, and the ICC More Generally?, ICC Forum (Jun. 1, 2020), available at https://iccforum.com/cyber-evidence.

What Is Digital and Open Source Evidence and Who Creates It?

Digital evidence is “information and data of value to an investigation that is stored on, received, or transmitted by an electronic device.” 1 It can be closed source, meaning accessible only to some people and stored on devices such as personal cell phones. It can also be based on open source information, which is “publicly available information that can be obtained through observation, request or purchase.”2

Digital evidence encompasses a wide range of content. In this comment, we primarily focus on video and associated data, including user-generated and machine-generated data. Eyewitness videos taken at the scene of an incident with mobile phones and cameras are a type of user-generated data. These videos can be created by ordinary people who happen to witness something, human rights defenders who purposefully document abuses, and perpetrators of abuses themselves. These videos are often stored and shared on mobile phones, computers, USB sticks, hard drives, and social media.

Perpetrator video is a term used to describe videos taken at the scene of an incident by a perpetrator of abuse or an accomplice, often with the express intent to do harm. The filmer may create and share the video in order to spark fear, promote hate, dehumanize an individual or community, glamorize violence, recruit new members to an organization, entertain abusers, share tactics, mislead the viewing public, or even simply to document their actions. Ultimately, the distinction between perpetrator video and other eyewitness video is who created the video and for what purpose.3

Unlike user-generated data such as a written description of a video, machine-generated data is generated by devices (like mobile phones) or platforms (like Facebook). Devices and platforms collect and generate data for purposes such as providing services and monetization. This machine-generated data can include personal information, usage and transaction data, and technical attributes. For example, one app on your phone could generate a username, a log of location data, and a list of other people you interact with using the app. Such data is typically retained by the service provider for a period of time, and may also be shared with third-parties.

How Is Digital Evidence Collected and Stored?

Digital evidence of interest to the OTP resides on various types of electronic devices that are owned or controlled by diverse stakeholders, who determine level and method of access. Closed source information typically resides on devices in the possession of content creators, such as portable hard drives, or on third-party services such as cloud storage providers. Content owners typically control access to their devices or online accounts, although devices and services vary in terms of their vulnerability to unauthorized access.

Open source information resides with the platforms that publish content or allow users to publish content. Content creators or owners typically upload content to these platforms for the purpose of sharing information publicly or to a closed network. For example, an eyewitness may post a video in a closed Facebook group, or a nontraditional media group may publish a video publicly on their YouTube channel. Users also commonly download and re-upload copies of open source content, creating many duplicates, which can complicate efforts to determine the original source of the content. Platforms display or takedown, and retain or delete uploaded content at their discretion.4 Most platforms disclose data to law enforcement in accordance with their terms of service and local laws.5 Platforms may authorize and enable users to download data, sometimes for a fee. In many cases, third-party tools will allow users to download data from platforms without explicit authorization.

Archives and civil society organizations that collect open source information employ both authorized and unauthorized methods of downloading from platforms. They also use different methods to identify, select, ingest, catalogue, preserve, and provide access to content. Some may use their collections for their own research and analysis, and publish the data or reports. These collections can potentially serve as valuable sources of evidence for the OTP, especially for open source content that is no longer available online due to deletions or takedowns.

The Role of Civil Society in Digital Evidence

In the case of eyewitness video, there is, by definition, a disconnect between the people who record and publish the content and the people who intend to use it to build cases for court. As with all international justice proceedings, analysts are often trying to understand video content without direct knowledge of the local context. This situation risks harming accuracy and excluding people and communities affected by rights violations. Local and international stakeholders have been successfully working together to address these gaps and facilitate collaboration with innovative strategies, platforms, and new tools. Currently, some of the richest sources of expertise on digital evidence can be found in civil society organizations. The OTP should collaborate with these stakeholders, as well as legal and academic institutions such as the UC Berkeley Human Rights Center, both for specific cases where appropriate, and for general knowledge and advice on using digital evidence.6

Civil Society Organizations Working in the Field to Collect Documentation and Evidence Collection in Collaboration With Investigators and Legal Experts

• Founded in 2011 in Aleppo by a group of lawyers, school teachers, and academics, the Syrian Institute for Justice (SIJ) focuses on human rights and criminal justice.7 They monitor and document violations and investigate crimes. They also support civil society through programs and training courses. The SIJ is recognized for their quality documentation and investigations, such as their investigation of the Queiq River massacre in Aleppo, Syria.8

• Mwatana for Human Rights is an independent organization that defends human rights through field research, creating reports and gathering documentation of violations by all sides to the current conflict in Yemen.9 Mwatana collaborates with regional and international partners, such as the European Center for Constitutional and Human Rights10 and the Global Legal Action Network (GLAN)11 to investigate the potential involvement of various European economic and political actors in alleged war crimes in Yemen.12

Civil Society Organizations Working on Aggregation, Authentication, (Emergency) Preservation and Analysis in Collaboration With Legal and Technology Experts and Other Relevant Stakeholders Working in the Justice and Accountability Space

• Mnemonic created the Syrian Archive in 2014 as their first country and issue-specific archive. The Syrian Archive was designed as a proof of concept for future archives, to experiment and establish a working methodology and stable technical workflow.13 Since 2014, Mnemonic has captured 1.4 million videos of the Syrian conflict, and made them available in a queryable and publicly available database. Mnemonic shares information about their tools and methods publicly. This includes their step-by-step process for identifying, collecting, preserving, and verifying data, as well as the technical tools they use, which are free and available online.14 The Syrian Archive has partnered and collaborated with myriad international organizations and initiatives in the human rights and citizen journalist sectors. These include the United Nations Office for High Commissioner of Human Rights: Inquiry on Syria, UC Berkeley’s Human Rights Center, Bellingcat, WITNESS, and, of course, a diverse network of Syrian journalists, lawyers, and human rights defenders. Findings by the Syrian Archive have been used by international organizations such as Amnesty International and Human Rights Watch, as well as by intergovernmental groups like the Organisations for the Prohibition of Chemical Weapons, and the Independent International Commission of Inquiry on the Syrian Arab Republic. Mnemonic is applying what it’s learned together with its infrastructure and networks from the Syrian Archive to its new Yemeni Archive, and other locations and contexts around the world.15

• GLAN is a non-profit organization that pursues innovative legal actions across borders, challenging states and other powerful actors involved with human rights violations.16 Since 2018, GLAN has been building and actively maintaining a database of Saudi/UAE-led Coalition airstrikes and cluster bomb attacks in Yemen.17 Their user-friendly system enables the cross-referencing of large quantities of evidence and will be available to partner organizations seeking to promote accountability in the future. It brings together key pre-existing technologies to allow online user generated content and private evidence to be stored and viewed together. GLAN is also collaborating with Mnemonic’s Yemeni Archive and Bellingcat on a new project that seeks to apply core legal principles to open source investigations, so that it can increase the reliability and usefulness of the evidence.18 The methodology aims to balance adherence to procedural standards on legal evidence with workability. The information in the database can be made available to litigators, national investigators, and prosecutors, who can use it as primary evidence or as “lead information” for their own investigations.

• Benetech applies advances in technology, including machine learning and computer vision, to help sort, identify, preserve, and analyze evidence of abuses and potential war crimes.19 The Benetech Initiative works with Syrian civil society organizations and supports the mandate of the United Nations’ International, Impartial, and Independent Mechanism on Syrian accountability.

Organizations Supporting Documentation and Evidence Collection on a Local Capacity Through Training and Resources

• WITNESS works with civil society organizations and international experts—including the OTP —to develop guidance so that first responders and non-professional human rights documenters can capture and preserve video that meets the highest admissibility standards. This guidance is published through the Video as Evidence Field Guide20 and Activists’ Guide to Archiving Video.21 The guidance, combined with trainings, results in better coordination between civil society organizations and international justice institutions and helps provide a more robust evidentiary record.

Case Study: Hasiam Omar Sakhanh

The Swedish Prosecutor’s Office and the District Court of Stockholm relied partially on video evidence to find Haisam Omar Sakhanh guilty of a war crime.22 Sakhnah was sentenced to life in prison for his participation in the killing of seven captured soldiers in Syria in 2012. The rebel group that Sakhanh belonged to documented the execution on video. The video showed nine men posing casually, standing over seven shirtless men with hands tied behind their backs and backs bearing marks of torture. The rebels’ commander recites a revolutionary monologue before ordering the killing of the seven men. Sakhanh, appearing on the far left, is shown toting a Kalashnikov with which he used to kill one of the bound men at his feet.

In 2013, the video became public when a former rebel provided the video to The New York Times.23 The Swedish prosecution was able to build a strong case through collaboration with local stakeholders from Syria as well as other countries and institutions. This put the criminal act in the right context, showing that it was not seven cases of murder but rather a war crime. The prosecutors found additional evidence online, including a video taken shortly before the executions in which the rebels appeared posing with their captors and explaining where they had been captured. They also found information online that specified the time and location the prisoners were captured. Some of this information was posted by the rebels themselves. This evidence, and additional information, helped the prosecution establish a timeline, showing only 41 hours passed between the time the soldiers captured and executed the prisoners. The prosecution argued that there couldn’t have been any form of quasi-judicial procedure or due process based on the circumstances and rapid timeline. This timeline, and the video evidence of the particularly cruel manner in which the rebels had treated and executed the defenseless prisoners, led to Sakhanh being found guilty.24

Challenges and Key Considerations for Working with Digital Evidence

Digital Video Evidence from Closed and Open Sources

Digital video evidence from closed sources usually resides on an electronic device or user-controlled cloud storage accounts. The biggest challenge for the OTP is identifying and reaching these sources, and obtaining content from them. Local civil society groups can play an important role in this respect, fostering connections between individual eyewitnesses or documenters and large international institutions. In the case of such collections, authenticity can often be established by more standard means—forensics can be applied to the storage device, people who came into contact with the device can be interviewed, etc. Digital open source videos present more challenges, including the staggering volume of content available on social media sites. However, it is that very availability that makes the challenges worth overcoming. Open source videos may be available in situations where corroborating or contextual evidence is essential to building a case but is otherwise impossible to obtain.

Ethics, Security, and Privacy

The OTP must closely consider the related challenges of ethics, security, and privacy. Videos have multiple stakeholders, including the people who created them, the people who may be depicted, the people creating repositories, and the communities the repository is ultimately aimed at supporting. The OTP must consider the potential impact on all of these stakeholders who could be endangered by the creation of collections and use of videos. Videos that depict targets of human rights abuses in a way that makes them identifiable, or that make the location of the filmer identifiable, could lead to retaliation against that person or their family and loved ones. Fortunately, the OTP has experience in risk assessments and shielding victims and is well equipped to integrate these types of concerns into its protocols.

The OTP must also consider the ethical implications of pulling videos out of their original context, processing them, storing them somewhere else, and making them available for uses far beyond what they might have been intended for. This is possible with closed source evidence, for example, if an individual provided a collection to a civil society organization that then provided it to the OTP without communicating the level of consent to the OTP. However, this is a far greater challenge with open source evidence. Of course, the OTP must comply with local laws such as the European Union’s General Data Protection Regulation (GDPR). The GDPR may apply to processing of videos and related content that include personal identifying information, and it is important that the OTP do a thorough analysis prior to creating any digital evidence database.

More broadly, it is easy to argue that once something has been posted on a social media site, it is public. It is even possible to look at the terms of use for such sites and argue that people posting there have given up their rights over their content. But this is both incorrect and ethically wrong. First, as noted, it is a far cry from posting a video on Facebook to having that video featured in an international legal proceeding. Second, few people read the terms of use on social media sites, and those terms are famously difficult to understand. Third, even if a video were originally posted online, putting it into a repository may allow it to persist after an owner has deleted it.

Finally, security poses a big challenge. Digital videos may be stored online or offline, directly by the OTP or by a third-party service. Different storage methods present unique security risks. Offline repositories are in many ways the safest method, as access to such collections is naturally limited, but they are still susceptible to physical destruction. Repositories can be self-run or stored on a third party service. Third-party storage relies on the security of whoever is hosting the repository, while self-hosted storage relies on good security knowledge and skills.

The OTP must assess potential risks based on the characteristics of the collection itself and those who might access it. They must consider issues such as how stored data is encrypted, whether to allow remote access at all, and if so under what requirements, i.e. using an OTP-issued laptop. Every person who has access to these collections poses another potential risk. Background checks for working with the Court are the norm, but someone must also vet adherence to minimum security practices, for example having a secure password on any device used to access the OTP’s collection.

Discovering Digital Evidence

The large volume of videos documenting human rights abuses in conflict zones can be a serious deterrence to finding the right video, on social media sites and in private collections. For example, with at least 500 hours of video uploaded to YouTube every minute, it can be difficult to discover relevant content if it lacks an informative and relevant title, description, or tags.25 Videos that lack sufficient metadata or description to enable users to discover them can be unfindable, both on social media sites and in private collections.

Fortunately, the civil society groups identified in this comment have all created methodologies for finding content. For example, Mnemonic has a large list of trusted sources for video, and Benetech is using new technology to help human rights groups “deduplicate” content.26 Without collaboration with civil society, most digital video evidence will remain undiscoverable. With collaboration, the OTP will be able to wade through the large volume of videos to find the right content.

Permanence and Availability of Digital Evidence

The permanence and availability of content posted on social media; and any content created by individuals, hosted by a third-party, and accessed on the internet; is precarious. Videos may be voluntarily removed or made private by the person who originally posted them. Most web hosts and social media platforms make no commitment to keep uploaded content available over time. In fact, through the process of content moderation, platforms actively remove content that violates their terms of service: the rules by which a user agrees to abide as a condition of using the platform.

Takedowns especially impact human rights videos, which sometimes contain graphic or violent imagery. Social media sites increasingly deploy machine-learning algorithms, i.e. artificial intelligence (AI), to detect so-called “terrorist and violent extremist content,” putting human rights content at risk.27 AI is not good at telling the difference between content posted to expose or document a human rights abuse, and content posted to incite or glorify human rights abuses. In 2017, YouTube’s machine-learning-based algorithm removed hundreds of channels and thousands of videos documenting atrocities in Syria.28 Mnemonic has documented this removal as much as possible, but there are doubtless innumerable videos that will never be counted because they were deleted so fast.29

The increase in AI content moderation is moving forward at a rapid pace due to pressure from policymakers in a number of contexts, including a pending “Preventing the dissemination of terrorist content online” regulation in the European Union, the Christchurch Call to Eradicate Terrorist and Violent Extremist Content Online, and more.30 These policy conversations do not contemplate the evidentiary and documentary nature of content that could accidentally be captured by AI. They also haven’t addressed the biased nature of lists of designated terrorist and violent extremist groups such as the US State Department’s list of Foreign Terrorist Organizations and even the Consolidated United Nations Security Council Sanction List.31 These lists are disproportionately composed of Islamic extremist groups, to the exclusion of white supremacist and other groups that are responsible for increasing levels of violence around the world. In fact, based on the experience of civil society, it seems that content from conflicts in Yemen, Syria, and other Arabic-speaking and/or Muslim majority places is far more likely to be removed.

Content takedowns are undoubtedly damaging online repositories of evidence. By example, the 2017 ICC arrest warrant for Al-Saiqa commander Mahmoud Mustafa Busayf Al-Werfalli was based largely on video documentation of seven incidents posted on social media. With this evidence, the Court was able to charge Werfalli with committing and ordering murders as a war crime in Libya. However, less than three months after the videos were posted, the first video had already been deleted from Facebook. It would not exist today if it had not been downloaded and saved elsewhere.32

Platforms can also go out of business and shut down their services. Justin.tv, Yahoo! Video, and Orkut are just some examples of social media platforms that have shut down in recent years. When services go out of business, they often give a timeframe during which users may download their media content before it is lost.33 Unfortunately, not all platforms give users adequate warning before they shut down.

While they are operational, many third-party platforms and web hosts create barriers to downloading social media content and its accompanying metadata. Twitter, for example, places limits on its free Search application programming interface (API) for providing access to Twitter data; a full Search API is available but at a monthly cost.

Even when open source information persists online, it remains susceptible to dislocation or so-called “link rot”—a term that applies when external links no longer point to their intended content. This makes information difficult to find over time, as links from web pages or documents are broken.34

Authentication—Reliability

Authenticating open source information requires significant expertise, time, and labor. Unlike digital content obtained directly from its creator in its original form, the original source of open source content is difficult to clearly establish. Online video platforms and messaging services typically do not make the video available in its original form and with its original metadata. Video posted online can be easily decontextualized or mis-contextualized, making it very susceptible to problems of identification and verification, such as establishing the correct time, date, and location of the incident depicted. There is seldom a traceable chain of custody, making it difficult to assert that the information has been handled properly and has not been manipulated or altered.

Those intending to use open source content in criminal investigations and proceedings should approach this content with flexibility and adaptability if traditional means of ascertaining authenticity cannot be applied. They should take advantage of the wide variety of tools and methods created by civil society organizations to determine authenticity, and look to additional forms of media provided by eyewitnesses and other relevant and more closely informed stakeholders (digitally and physically) to help address information gaps.35 Civil society groups like Mnemonic, for example, combine both new approaches and traditional ones, to assess the authenticity of open-source information. Mnemonic uses a variety of information and tools to verify videos, such as geolocation and “cross referencing the publishing date on social media platforms (e.g. YouTube and Facebook) with dates from reports concerning the same incident.”36 Importantly, Mnemonic’s knowledge of the context and landscape means they can rely on trusted sources, and, when needed, can assess sources’ credibility by looking at factors like pre-existing knowledge of the source, past behavior, and location.37

Manipulated video in the form of “deepfakes” and “shallowfakes” poses a specific kind of authentication challenge, in addition to generally eroding trust in video evidence.38 This false or misleading content may be shared on social media for nefarious purposes, or accidentally by well meaning individuals. It is intentionally created to mislead, and must be weeded out for collections to remain reliable. Deepfakes, and other forms of AI-enabled techniques for synthetic media generation, enable the creation of realistic representations of people doing things they never did or saying things they never said. It can make a realistic creation of people or objects that never existed, or of events that never happened.39 So-called “shallowfakes” are less sophisticated but still highly effective techniques such as mislabeling a real video or lightly editing a real video.

Shallowfakes can usually be detected using well-established authentication techniques. Deepfakes are a bit more complicated. Creators of deepfakes have, until this point, mainly used them to target women for harassment, but their potential for muddying the waters of digital evidence is clear.40 For-profit organizations are developing sophisticated and effective tools to detect deepfakes. Unfortunately, as outlined in WITNESS’Ticks or it Didn’t Happen report, there is a real risk that these tools will not be easily available to civil society organizations that create repositories.41 As this technology becomes more widely available, WITNESS will continue to advocate for solutions that are available to all stakeholders—in particular, groups creating databases of digital video evidence.

Managing and Preserving Digital Evidence

As discussed above, digital evidence lives precariously online, necessitating timely capture as a first step if the evidence is to be preserved. If the OTP does not have the capacity to actively monitor and rapidly capture evidence when it appears, it can collaborate with civil society organizations that have dedicated themselves to this effort. The OTP can develop protocols to request and obtain collected content from these groups, and provide guidance to ensure that chain of custody and evidence of authenticity is sufficiently documented.

Once obtained, maintaining collections can be expensive, time consuming, and require constant attention to ensure security and accuracy and keep up with changing technology. This can be challenging for institutions that have less experience using digital evidence. Fortunately, other relevant stakeholders in the justice and accountability space have been working to develop new systems, technologies, and workflows to help navigate the digital deluge of data.

Once the OTP obtains content, it must take on the responsibility for that content’s storage and management. The OTP’s preservation strategies should be customized to its circumstances, the nature of its collections, and the needs of its intended use. Preservation activities can take place on a spectrum depending on available resources. The key to any approach is that it be planned, consistently applied, and documented to ensure effective long-term preservation. Protocols must ensure data is not improperly altered or mishandled during its access, collection, transmission, and storage.

It would be a mistake to jump directly to thinking about tools and technologies for collecting evidence before first making decisions about what the OTP intends to collect and how it will be collected and processed, and then codifying these decisions into policies so that they are enacted consistently. The case of The Prosecutor v. Jean-Pierre Bemba Gombo et al. helps to illustrate the importance of an intentional approach to collecting. During the trial, the prosecution sought to introduce screenshots of a Facebook page as evidence linking Bemba’s sister and another witness. The prosecution argued that:

[The] documents are open source materials from Facebook and, thus, prima facie authentic and reliable. The authenticity and reliability of these documents is further corroborated by their general appearance, which bears indicia that they originate from Facebook […].42

The defense challenged this assertion, arguing that the screenshots could not be taken as prima facie evidence, since screenshots of a Facebook page are neither the Facebook page nor do they “originate from” Facebook. The defense argued that the screenshots neither show whether the images are actually from Facebook, nor who actually created the profile page, nor when and how the screenshots were taken, nor who appears in the screenshots.43

The OTP must therefore determine in advance how objects should be captured and preserved in terms of their technical, intellectual, structural, or aesthetic characteristics, to ensure the object’s accessibility, usability, interpretability, and authenticity for the Court. By analyzing and identifying the significant properties44 of the information it is responsible for preserving, the OTP can assess the relative importance of each property to its use in the Court, and to shape its policies with this evaluation in mind.

Considering that the Internet and related tools are in constant evolution, the OTP should also assess how it intends to maintain authenticity of its stored objects in the face of technological change. Transformations like reformatting or media migration can be necessary for preservation or for rendering/playback, and can sometimes involve changing the digital object. The OTP must determine what properties are significant and what evidence of authenticity is sufficient, and choose transformation techniques and workflows that preserve an object’s significant properties and authenticity, and document those changes appropriately. Considerations when making transformations might include, for example, the audiovisual quality of the transformed object compared with the original object, the structure of the transformed object compared to the original object, and the transparency or reversibility of the transformation.

Keeping in mind constraints in finances and manpower, the OTP must realistically assess what is involved in collecting and maintaining digital evidence over time. Preserving the availability, identity, persistence, renderability, understandability, and authenticity of a digital object requires much more than just saving its content. Additional information must also be maintained for each object, including reference, provenance, context, access rights, and fixity information. Data management is a key and labor-intensive function. It includes creating and administering databases, performing queries for access purposes, generating reports, keeping the data accurate and up-to-date, and ensuring the associations between stored objects and their metadata are maintained over time.

Unlike with non-digital objects, storage is also not a passive function nor a one-time cost. It involves storage hierarchy management, storage media replacement, error checking, fixity checking, disaster recovery, and locating and returning stored objects. It requires ensuring the persistence of intact and authentic objects, and the ability to find and retrieve them over time, regardless of their physical location.

Conclusion

Digital video evidence is not a technological silver bullet, but it is an extremely valuable resource, and worth the trouble it takes to “do it right.” In close partnership with affected communities, video evidence has provided insight into atrocities in places like Syria.

The baseline in using digital video evidence has to be “do no harm.” That means first assessing the need for digital video evidence in conjunction with people who understand local contexts. In many cases, the OTP will be able to find NGOs and groups of human rights defenders in a country or in a diaspora that can help with this. Next, it means creating a thorough collection and authentication plan and working through the steps to find, process, and store digital evidence, keeping in mind ethics, security, privacy, changing technology, and maintenance of such collections.

The OTP has many impressive examples to work from. Groups like Mnemonic can help the OTP properly handle digital video evidence, opening up new avenues to achieving justice across borders. Finally, as noted, the OTP should make the value of this evidence clear, in order to assist technology companies and policymakers in finding solutions to the problem of “terrorist and violent extremist content,” that do not put this evidence in jeopardy.

Endnotes — (click the footnote reference number, or ↩ symbol, to return to location in text).

1. 1.

   National Institute of Justice, Electronic Crime Scene Investigation: A Guide for First Responders (2d ed. Apr. 4, 2008), available online. ↩

2. 2.

   Berkeley Protocol on Open Source Investigations, OHCHR & UC Berkeley HRC (forthcoming 2020) [hereinafter Berkeley Protocol]. ↩

3. 3.

   Raja Althaibani, New Series Launch: Perpetrator Video in the Middle East and North Africa, WITNESS Media Lab (Sep. 2018), available online. ↩

4. 4.

   See, e.g. Terms of Service, TikTok, ¶ 7, available online (last visited May 23, 2020)

   (“We have the right to remove, disallow, block or delete any posting you make on our Services if, in our opinion, your post does not comply with the content standards set out at ‘Your Access to and Use of Our Services’ above. In addition, we have the right—but not the obligation—in our sole discretion to remove, disallow, block or delete any User Content (i) that we consider to violate these Terms, or (ii) in response to complaints from other users or third parties, with or without notice and without any liability to you. As a result, we recommend that you save copies of any User Content that you post to the Services on your personal device(s) in the event that you want to ensure that you have permanent access to copies of such User Content.”). ↩

5. 5.

   See, e.g., Information for Law Enforcement Authorities, Facebook, available online (last visited May 8, 2020). ↩

6. 6.

   Shakiba Mashayekhi, The Berkeley Protocol on Open Source Investigations, Soc. Sci. Matrix (Jan. 28, 2020), available online

   (In particular, the UC Berkeley Human Rights Center will soon be releasing a Protocol on Open Source Investigations that will provide important guidance on “the effective use of open-source information in international criminal and human rights investigations.”). ↩

7. 7.

   Syrian Institute for Justice, Facebook, available online (last visited May 8, 2020). ↩

8. 8.

   Luke Mogelson, The River Martyrs, The New Yorker (Apr. 22, 2013), available online. ↩

9. 9.

   Home Page, Mwatana Hum. Rts., available online (last visited May 8, 2020). ↩

10. 10.

    Home Page, ECCHR, available online (last visited May 8, 2020). ↩

11. 11.

    Home Page, GLAN, [hereinafter GLAN], available online (last visited May 8, 2020). ↩

12. 12.

    European Center for Constitutional and Human Rights, Questions and Answers: Accountability for Alleged War Crimes in Yemen—European Arms Exporters’ Responsibility Is a Case for the International Criminal Court (Jan. 2020), available online. ↩

13. 13.

    Home Page, Syrian Archive, available online (last visited May 8, 2020). ↩

14. 14.

    Tools and Methods, Syrian Archive, [hereinafter Tools and Methods], available online (last visited May 8, 2020). ↩

15. 15.

    Home Page, Yemen Archive, available online (last visited May 8, 2020). ↩

16. 16.

    GLAN, supra note 11. ↩

17. 17.

    Yemen Airstrike Evidence Database, GLAN, available online (last visited May 8, 2020). ↩

18. 18.

    Yemen Project, Bellingcat, available online (last visited May 8, 2020). ↩

19. 19.

    Justice AI: Turning Conflict Data into Actionable Evidence, Benetech, available online (last visited May 8, 2020). ↩

20. 20.

    Kelly Matheson, Video as Evidence Field Guide, WITNESS (Mar. 29, 2016), available online. ↩

21. 21.

    WITNESS, Activists’ Guide to Archiving Video (Aug. 30, 2017), available online. ↩

22. 22.

    Stockholms Tingsrätt [Stockholm District Court], B 3787-16, Case of Haisam Omar Sakhnah, Judgment (Feb. 16, 2017) (Swed.), available online. ↩

23. 23.

    C.J. Chivers, Syrian Asylum Seeker Linked to Mass Killing Is Arrested in Sweden, N.Y. Times, Mar. 14, 2016, available online. ↩

24. 24.

    Christina Anderson, Syrian Rebel Gets Life Sentence for Mass Killing Caught on Video, N.Y. Times, Feb. 16, 2017, available online. ↩

25. 25.

    Anmar Frangoul, With Over 1 Billion Users, Here’s How YouTube is Keeping Pace with Change, CNBC, Mar. 14, 2018, available online. ↩

26. 26.

    Lale Tekisalp, How Should Human Rights Advocacy Balance the Opportunities & Risks of Artificial Intelligence?, Partnership on AI (Dec. 10, 2019), available online, archived. ↩

27. 27.

    Hadi al Khatib & Dia Kayyali, YouTube is Erasing History, N.Y. Times, Oct. 23, 2019, available online. ↩

28. 28.

    Dia Kayyali & Raja Althabani, Vital Human Rights Evidence in Syria is Disappearing from YouTube, WITNESS (Aug. 30, 2017), available online. ↩

29. 29.

    Tech Advocacy, Syrian Archive, available online (last visited May 8, 2020). ↩

30. 30.

    Dia Kayyali, Human Rights Defenders Are Not Terrorists, and Their Content Is Not Propaganda, WITNESS (Jan. 21, 2020), available online. ↩

31. 31.

    See, e.g., Cat Zakrzewski, Instagram Faces Backlash for Removing Posts Supporting Soleimani, Wash. Post, Jan. 13, 2020, available online; Abdul Rahman Al Jaloud, Hadi Al Khatib, Jeff Deutch, Dia Kayyali & Jillian C. York, EFF, Syrian Archive, and WITNESS, Caught in the Net: The Impact of “Extremist” Speech Regulations on Human Rights Content (May 30, 2019), available online. ↩

32. 32.

    How a Werfalli Execution Site Was Geolocated, Bellingcat (Oct. 3, 2017), available online. ↩

33. 33.

    E.g., A Fond Farewell!, Bambuser (Nov. 30, 2017), archived

    (Bambuser was a mobile live streaming service that was used widely by activists during conflicts in Syria, Ukraine, and Egypt. When it shut down in 2018, it instructed users to download their videos within two months or lose them). ↩

34. 34.

    See, e.g., Jonathan L. Zittrain, Kendra Albert & Lawrence Lessig, Perma: Scoping and Addressing the Problem of Link and Reference Rot in Legal Citations, 127 Harv. L. Rev. F. 176, 178 (Mar. 17, 2014), available online

    (50% of URLs cited in US Supreme Court decisions since 1996 no longer linked to the originally cited information).

    Hany M. SalahEldeen & Michael L. Nelson, Losing My Revolution: How Many Resources Shared on Social Media Have Been Lost?, arXiv (Sep. 13, 2012), available online

    (Within the first year of publishing, nearly 11% of resources shared on social media are lost, and subsequently continue to be lost at a rate of 0.02% per day). ↩

35. 35.

    See, e.g., Bellingcat, Online Investigation Toolkit (v5.1 Mar. 9, 2020), available online (last visited May 23, 2020)

    (Bellingcat’s freely available online open source investigation toolkit covers “satellite and mapping services, tools for verifying photos and videos, websites to archive web pages, and much more”). ↩

36. 36.

    Tools and Methods, supra note 14. ↩

37. 37.

    Id. ↩

38. 38.

    Sam Gregory, Deepfakes Will Challenge Public Trust in What’s Real. Here’s How to Defuse Them., Defusing Disinfo (Feb. 19, 2019), available online. ↩

39. 39.

    Backgrounder: Deepfakes in 2020, WITNESS (Apr. 1, 2020), available online. ↩

40. 40.

    Henry Ajder, Giorgio Patrini, Francesco Cavalli & Laurence Cullen, Deeptrace, The State of Deepfakes: Landscape, Threats, and Impact (Sep. 2019), available online

    (96% of deepfakes online are pornographic content). ↩

41. 41.

    Gabi Ivens & Sam Gregory, WITNESS, Ticks or it Didn’t Happen: Confronting Key Dilemmas in Authenticity Infrastructure for Multimedia (Dec. 2019), available online. ↩

42. 42.

    The Prosecutor v. Jean-Pierre Bemba Gombo, Aimé Kilolo Musamba, Jean-Jacques Mangenda Kabongo, Fidèle Babala Wandu and Narcisse Arido, ICC-01/05-01/13, Prosecution’s Fifth Request for the Admission of Evidence from the Bar Table (TC VII, Nov. 30, 2015), available online. ↩

43. 43.

    The Prosecutor v. Jean-Pierre Bemba Gombo, Aimé Kilolo Musamba, Jean-Jacques Mangenda Kabongo, Fidèle Babala Wandu and Narcisse Arido, ICC-01/05-01/13, Defence Response to Prosecution’s Third Request for the admission of Evidence from the Bar Table (TC VII, Sep. 10, 2015), available online. ↩

44. 44.

    Gareth Knight, InSPECT, Framework for the Definition of Significant Properties (Feb. 14, 2008), available online

    (The InSPECT Project usefully classifies significant properties the Court can consider into five general categories: content [e.g. the intellectual material, like words on a page], context [e.g. creator, creation date, and other metadata], rendering [the way the visual or audio elements look and sound], structure [the way pieces of content are related to each other, like pages or attachments], and behavior [the way the object functions or interacts.]). ↩

45. Suggested Citation for this Comment:

    Dia Kayyali, Raja Althaibani & Yvonne Ng, Digital Video Evidence, When Collected, Verified, Stored, and Deployed Properly, Presents New Opportunities for Justice, ICC Forum (Jun. 1, 2020), available at https://iccforum.com/cyber-evidence#Kayyali.

    Suggested Citation for this Issue Generally:

    To What Extent Can Cyber Evidence Repositories, and Digital and Open-Source Evidence, Facilitate the Work of the OTP, and the ICC More Generally?, ICC Forum (Jun. 1, 2020), available at https://iccforum.com/cyber-evidence.

History

Since its inception, an ongoing hurdle for the OTP has been securing sufficient evidence to corroborate witness testimony, in part because of the difficulty of getting to physical and documentary evidence quickly given its ephemerality in remote regions and sites of conflict. There are several reasons for the delay, some of them outside of the Court’s control: the ICC may not have jurisdiction until months or years after an event, delaying investigators from getting on the ground to conduct fact-finding. Additional constraints may be political, such as when a government leader opposes an investigation, or operational, such as security concerns.

This means that digital information that can be accessed remotely from the scene of alleged crimes may have particular utility for the ICC, and especially during the preliminary examination phase of a situation when the OTP does not yet have full investigatory powers.2

The OTP’s need to find new ways of securing corroborating information took on a particular urgency in 2011, when the Chambers admonished the OTP that they needed to increase and diversify their acquisition and use of non-testimonial evidence.3 Soon after, UC Berkeley’s Human Rights Center began working with the OTP and a number of other NGOs, including the Open Society Justice Initiative, to host a series of workshops designed to explore potential uses of digital and other scientific content to strengthen the evidentiary foundations of cases. The workshops joined parallel efforts started by the OTP and others to bring together civil society organizations, technologists, and investigators experimenting with new models of fact-finding and verification, including remote sensing, and how to harness user-generated content,4 such as the photos and videos that people were increasingly capturing with their smartphones.

In 2013, the OTP released its strategic plan for 2012–2015,5 which underscored the Chambers’ request for more “diversified evidence” from the OTP. The OTP explained that it had simultaneously begun “facing an explosive growth in its access to digital data,” leading the OTP to “focus on expanding its capabilities to collect different forms of scientific evidence.”6 The document also detailed upcoming changes specific to the Investigations Division, which included “enhancing its capabilities to collect other forms of evidence in addition to witness statements, in particular scientific evidence,”7 including digital content. Soon after, the OTP began meeting with social media companies such as Facebook, YouTube, and others to figure out how to maximize access to evidence of grave international crimes being posted to those sites.8 Meanwhile, companies like SITU Research and Forensic Architecture were experimenting with creative ways to aggregate open and close source digital data to help investigators, lawyers, and judges better understand facts on the ground in atrocity cases.9

A former ICC investigator, Alison Cole, has played a key role in driving the evolution towards greater collection and use of digital evidence for ICC purposes. She has frequently served as a bridge between the OTP and NGOs, noting the “unique questions” raised by digitally-derived evidence, including the applicability of various rules of procedure and evidence and how they might need to be adjusted to accommodate new forms of digital evidence. As she explained in a 2015 article, figuring out how to effectively integrate digital evidence into investigations has greater importance for the ICC than for most courts. This is due to the barriers the OTP faces in collecting “traditional evidence, such as documents or subpoenaed testimony” given its lack of enforcement powers and its dependency on voluntary cooperation from friendly states.10 As she explains:

For the ICC, the ‘best evidence’ rule is far more likely to involve evidence processed through technological means [than for national courts], particularly evidence generated online and available through open-source investigations. The ICC is therefore most likely to be the first major global jurisdiction to focus predominantly on the newest emerging forms of evidence, including technology-based, cyber, online, and digital evidence.11

Cole provided the Libya case as an example of both the pressures the OTP was under to adapt to an increasingly digital world, and how information had changed in ways that were potentially relevant to the Court:

The legal challenges [to using digital evidence] became most apparent during the opening of the ICC investigations in Libya. The extent of citizen engagement in fact-finding was unprecedented, with an overwhelming deluge of potential evidence uploaded on YouTube, Twitter, and Facebook documenting the conflict in real time. The complexities of social media verification reached crisis mode as high-stake ICC allegations of mass rape, allegedly captured on video and then removed from YouTube, were questioned by the chairman of the UN Commission of Inquiry. It became obvious that it is essential to build ways of managing technology-generated digital evidence and to join the ICC in preparing for the tidal wave of new technology before it hit the ICC and flooded the courtroom with untested evidence.12

Her comments about the need for and potential ability of the ICC to pivot proved prescient. In 2017, the Court reached a milestone with its release of an arrest warrant for Mahmoud Al-Werfalli, commander of the elite Libyan special forces unit, Al Saiqa.13 As has been widely reported,14 the warrant was the Court’s first to rely heavily on social media content. This had recently been preceded by another open source milestone, the case of The Prosecutor v. Al Mahdi,15 in which a geolocation report that pulled together both open and close source digital information was submitted as evidence of the destruction of cultural heritage property by the Accused.

Ultimately, this need and resulting capacity building has situated the ICC as a somewhat unexpected and relatively innovative leader in the digital information as evidence space. Given the dearth of national guidance, the OTP has created standard operating procedures for handling digital content—a generic version of which can serve as a guide and support for national courts and others who have yet to similarly pivot. The Court has also played an important role in the development of the Berkeley Protocol on Digital Open Source Investigations,16 a set of international guidelines for using online open source information for evidentiary purposes that will be published this Fall by the United Nations Office of the High Commissioner for Human Rights and the Human Rights Center at Berkeley Law.17

Barriers to Securing Digital Content and Opportunities for Addressing Those Barriers

Despite this progress, the Court’s ability to efficiently utilize digital evidence has limits. Thus, external partnerships are crucial to the ICC’s functioning, just as they have been for other international, ad hoc, and hybrid tribunals.18 For example, at the Extraordinary Chambers in the Courts of Cambodia, the NGO community provided diverse support to help meet the needs of civil parties who participated in the trials or were impacted by historic events.19 Similarly, the ICC needs and has been receiving state support, as well as assistance from the circle of civil society organizations that hover around it. For example, groups like WITNESS and Videre est Credere have been training individuals in situation countries to document atrocities with an eye to meeting the needs of legal investigators, while organizations like eyeWitness to Atrocities, TruePic, and the Whistle have developed tools for secure data collection and storage, and implemented hashing and other digital technologies to help provide some form of chain of custody.

One especially acute challenge that the OTP continues to face is how to manage the vast scale of potentially relevant digital data. As has been famously reported, as of the time of writing, approximately 6000 tweets are created every second20 and more than 500 hours of video are uploaded to YouTube every minute.21 However, ironically, at the same time that the OTP risks drowning in data, much of the most powerful information that is relevant to international crimes is at risk of removal from online public spaces, due to its (sometimes) graphic nature.22

To help navigate these challenges, over the last few years NGOs have been working with the OTP and social media companies to explore whether there might be a way to create an evidence locker for social media and other user-generated content. To advance that conversation, the UC Berkeley Human Rights Center will soon release a study of the types of digital archives that have been established to collect user-generated content, with the goal of exploring the strengths and weaknesses of these different types of repositories for holding social media content that has relevance to international crimes and is at risk of removal. Student researchers have created a typology of archives, which vary across a number of key variables. These include how an archive receives information, who it receives information from, who can access data in the archive, what kinds of content the archive ingests, and whether the social media companies’ engagement with the archive is legally-mandated or voluntary.23

At one extreme are archives that receive content from the social media companies themselves as required by law, which the research team is calling the “Legal Compulsion Model.” Using an example from the United States, Facebook and other social media companies must remove and submit information about child exploitation to the National Center for Missing and Exploited Children, which works with law enforcement and others to “help find missing children, reduce child sexual exploitation, and prevent child victimization.”24

Another category of repository, which the teams calls the “Voluntary Partnership Model” receives content directly from the social media companies, but on a voluntary basis. A particularly illustrative example is the Global Internet Forum to Counter Terrorism (GIFCT), which was established by a number of tech companies to flag extremist content that their peers might want to know about in order to better detect and remove such content from their platforms. The GIFCT’s mandate has recently been expanded and its structure reformulated in the wake of the Christchurch Massacre in New Zealand to “improve the capacity of a broad range of technology companies, independently and collectively, to prevent and respond to abuse of their digital platforms by terrorists and violent extremists,” among other goals.25

Yet another—the “Aggregated Model”—compiles user-generated and other digital content from a variety of sources. An example is the International, Impartial and Independent Mechanism for Syria (IIIM), that collects information from various organizations and individuals. Sources include “the International Impartial Independent Commission of Inquiry on the Syrian Arab Republic, the UN-OPCW Joint Investigative Mechanism, States, international or regional organizations, entities of the United Nations system, non-governmental organizations, foundations, and individuals.”26

As for its processes and the broader services it provides, the IIIM:

[R]eviews the information and evidence—both inculpatory and exculpatory—already collected by others and identifies possible gaps […] [to build] a more complete and nuanced understanding of the crimes and alleged perpetrators, for use in future accountability proceedings.27

While the IIIM will likely never provide digital evidence for an ICC case given the Court’s lack of jurisdiction over crimes in Syria, it still provides an interesting example of how digital open source information may be aggregated, analyzed, and preserved to support legal accountability. And regardless of jurisdiction and subject matter, IIIM staff engage with the OTP in knowledge exchange that can be used to strengthen the quality of evidence for a diverse array of potential end users, such as national courts and war crimes units,28 in line with its complementarity function.

On the far opposite end of the spectrum is the “Disaggregated Model,” whereby a wide array of civil society groups run a decentralized set of repositories that share a focus on a particular conflict or crisis. One example is the Syrian Archive (now Mnemonic), which proactively identifies, collects, and preserves content from publicly accessible sites. Such disaggregated archives have a defined geographic, temporal, and/or substantive scope (for example, specific to Myanmar between the years of 2016 and 2018, or Syria between 2011 and the present). Under this model, diverse NGOs begin grabbing potentially overlapping data related to atrocities tied to their interests in real time—and in a race against either manual or automatic take down by third party sites (such as Facebook and YouTube) that host that content. At their best, these organizations bring much needed innovation and flexibility to the data collection space, for example, scraping digital content from social media sites and then preserving and indexing that data for future fact-finders, including legal investigators. This is especially needed as a gap filler between the occurrence of potential international crimes and when the OTP gets authorization to investigate atrocities (if ever), since the companies may not be alert to the need and/or incentivized to preserve that content. Just as precious physical evidence may be lost in the interim between crime and courtroom, digital content may prove similarly ephemeral—for example, if alleged perpetrators delete content they’ve posted once a warrant of arrest is issued, or a social media company removes content that violates its community guidelines. These independent archives may therefore become the sole repositories of precious evidence, filling a critical hole in the digital information-to-evidence lifecycle.

So, in summary, why are these cyber evidence repositories so needed?

First, as discussed above, they are important for addressing temporal lags in acquiring or asserting jurisdiction and the intervening risk of loss.

Second, they can be helpful for mitigating the scale of digital content and making fact-finding more efficient. NGOs can curate user-generated content through digital archives, helping to ensure that relevant information—including exculpatory information—is available to safeguard due process.29 Through these archives, civil society can play an important role in locating relevant digital data, finding signal in the noise, and creating a first rough cut to minimize the amount of information that the Court has to comb through and preserve. This is especially valuable when that curation honors the investigatory principles of objectivity and completeness. Those external parties ultimately assume the costs of sorting through and tagging that information, as well as providing the server space needed to hold the information, which can be helpful for the under-resourced ICC.

Third, such processes help ensure that legal requirements can be met, including (as mentioned above) obligations to disclose exculpatory information to the defense. If the OTP ingests large quantities of information without the personnel and equipment to adequately comb and index that information, they won’t know what they have, and yet could be held responsible for any nondisclosure.30 Instead, the OTP can make targeted requests of external repositories, limiting the risk of over-ingestion. Ultimately, so long as the digital information is coded and tagged for relatively straightforward retrieval, this “outsourcing” of some of the initial collection avoids the risk of court actors over collecting information and being burdened with siphoning through large quantities of data that it does not have the capacity to mine.

However, these repositories also come with major challenges. One is knowing that they exist—and thus who may be holding critical information. Another is standardizing what is held and how it is organized, so that the data can “speak” to each other, and so that legal investigators can efficiently locate what they need, when they need it. A third is ensuring that content has been captured and preserved in ways that maximize its value for court processes, for example, by including metadata or preserving some form of chain of custody, so that judges, investigators, and lawyers know how reliable the information is, and can weight it accordingly. This is where the Berkeley Protocol and any future public version of the OTP’s standard operating procedures may be especially helpful, namely by setting norms and helping standardize collection and organization of digital data in ways that maximize efficiencies for court purposes.

What’s Needed Next

First, when it comes to accessing information collected by others, the Court should continue establishing and formalizing relationships with external bodies that grab and preserve large quantities of content related to core international crimes as that content comes online. The benefit of this symbiotic practice is that the NGO can become an aggregator of content, while the OTP remains an important—yet nonexclusive—consumer. In order to make this work, however, the OTP needs to continue communicating with those external NGOs in order to:

1. provide critical feedback about how data can be structured in ways that increase its utility and overall value for court processes, and

2. strengthen relationships with organizations and networks that will lead them to relevant datasets in the first place.

Second, funders need to increase their support for these external actors, especially those who have a mission to support international justice. Otherwise these archives may prove nearly as ephemeral as the content they gather—risking the loss of not only evidence, but the historical record of major world events. Many archives that have sprung up to preserve such content regularly struggle to pay for everything from staff to servers and yet are increasingly recognized as a critical link in the chain of accountability.

Third, the Court should continue exploring how the benefits of digital information extend beyond the OTP. Evidence collection is just one potential objective of digital fact-finding. While the investigations division has struggled, at times, to get the information it needs, other units have struggled with effectively communicating with the Court’s diverse audiences. One way to think about the potential for utilizing digital open source content is in terms of how to maximize the flow of information in two directions: from users on the ground to the Court, but also from the Court back to users on the ground. Capacity building, which can include strengthening relationships with social media companies, may also be helpful for Outreach, or the Victims Participation and Reparations Section, for example. Such entities could benefit from:

1. working directly with technology companies to facilitate more effective digital communications with survivors in situation countries,

2. conducting online open source investigations to strengthen their analysis of country conditions,

3. providing ongoing training on open source investigations for ICC staff, etc.

Ultimately, all information collection and communication functions can benefit from appropriately tapping into digital resources. Given the rapidly-shifting technological environment, however, the OTP and other units need resources to regularly train personnel and update their capacities—especially with regards to how to verify the authenticity of digital content in light of the increasing risk of “deep fakes.”31 Each unit will also require a deep and broad understanding of:

1. what technologies are being used in locations of interest (for example, Facebook in Myanmar versus Weibo in China); and

2. how use of various platforms differs across everything from age to gender to class to the urban/rural divide.

For example, in the United States, children may use TikTok, while their parents hang out on Facebook. Ultimately, a careful and continual assessment of the technological landscape is key to effectively mining relevant information and deploying technologies to educate populations around the world about the Court’s accomplishments.

Conclusion

Given shelter-in-place orders in response to the COVID-19 pandemic, and an ever-increasing reliance on digital means of communication, the ability to use digital information for fact-finding is more urgent now than ever. While the OTP has come far since 2011, expanding their “expert capabilities in the field of voice communication, electronic data and cyber investigations as well as [their] reliance on partners,”32 they still have far to go.

Relying on external digital repositories may be a powerful model for overcoming some of the ICC’s temporal, legal and financial limitations. While the ICC generally, and the OTP specifically, have made significant and important strides in their use of digital content, the ICC and the states that support it must continue to invest in such processes for the long haul.

Endnotes — (click the footnote reference number, or ↩ symbol, to return to location in text).

1. 1.

   See Berkeley Protocol on Open Source Investigations, OHCHR & UC Berkeley HRC (forthcoming 2020) [hereinafter Berkeley Protocol]. ↩

2. 2.

   Office of the Prosecutor, ICC, Policy Paper on Preliminary Examinations ¶ 12 (Nov. 2013), available online; Rome Statute of the International Criminal Court, Adopted by the United Nations Diplomatic Conference of Plenipotentiaries on the Establishment of an International Criminal Court, Jul. 17, 1998, UN Doc. A/CONF.183/9, as amended [hereinafter Rome Statute], Art. 15, ¶ 2 available online

   (describing the powers of the prosecutor to initiate a preliminary examination).

   See also Office of the Prosecutor, ICC, Report on Preliminary Examination Activites 2019 ¶ 11 (Dec. 5, 2019), available online; Alexa Koenig, Felim McMahon, Nikita Mehandru & Shikha Silliman Bhattacharjee, Open Source Fact-Finding in Preliminary Examinations, in Quality Control in Preliminary Examination: Volume 2 (Morten Bergsmo & Carsten Stahn, eds., 2018), available online. ↩

3. 3.

   See, e.g., The Prosecutor v. Mathieu Ngudjolo Chui, ICC-01/04-02/12, Judgment pursuant to article 74 of the Statute (TC II, Dec. 18, 2012), available online; The Prosecutor v. Callixte Mbarushimana, ICC-01/04-01/10, Decision on the confirmation of charges (PTC I, Dec. 16, 2011), available online; The Prosecutor v. Bahr Idriss Abu Garda, ICC-02/05-02/09, Decision on the Confirmation of Charges (PTC I, Feb. 8, 2010), available online.

   See Lindsay Freeman, Digital Evidence and War Crimes Prosecutions: The Impact of Digital Technologies on International Criminal Investigations and Trials, 41 Fordham Int’l L.J. 283 (2018), available online

   (excellent discussion of this history and the Court’s response).

   See also Office of the Prosecutor, ICC, Strategic Plan June 2012–2015 (Oct. 11, 2013), [hereinafter Strategic Plan], available online

   (noting the chambers requirement that the OTP diversify forms of evidence). ↩

4. 4.

   See Rebecca J. Hamilton, User-Generated Evidence, 57 Colum. J. Transnat’l L. 1 (Nov. 7, 2018), available online

   (provides an overview of the utility and potential limitations of user-generated content as potential evidence). ↩

5. 5.

   Strategic Plan, supra note 3. ↩

6. 6.

   Id. at 22. ↩

7. 7.

   Id. at 7. ↩

8. 8.

   Alexa Koenig, The International Criminal Court at RightsCon: Upping its Cyber Game, Huff. Post, May 11, 2014, available online. ↩

9. 9.

   See, e.g., SITU Research, ICC Digital Platform: Timbuktu, Mali (2016), available online, overview (last visited May 24, 2020)

   (examples of outputs that SITU Research and Forensic Architecture have created to support trials).

   The Murder of Pavlos Fyssas, Forensic Arch., available online (last visited May 5, 2020). ↩

10. 10.

    Alison Cole, Technology for Truth: The Next Generation of Evidence, Int’l Just. Monitor (Mar. 18, 2015), available online. ↩

11. 11.

    Id. ↩

12. 12.

    Id. ↩

13. 13.

    The Prosecutor v. Mahmoud Mustafa Busayf Al-Werfalli, ICC-01/11-01/17, Warrant of Arrest (PTC I, Aug. 15, 2017), available online. ↩

14. 14.

    See, e.g., Emma Irving, And So It Begins… Social Media Evidence in an ICC Arrest Warrant, Opinio Juris (Aug. 17, 2017), available online; Alexa Koenig, “Half the Truth is Often a Great Lie”: Deep Fakes, Open Source Information, and International Criminal Law, 113 Am. J. Int’l L. Unbound 250 (Aug. 19, 2019), available online, doi. ↩

15. 15.

    See Office of the Prosecutor, ICC, Report on the Implementation of the OTP Strategic Plan 2016–2018, n.43 (Aug. 23, 2019), available online

    (noting with regards to the use and aggregation of digital data that “during the presentation of the evidence in the Al-Mahdi trial in August 2016, the OTP utilized an interactive digital platform to present its case. The demonstrative platform displayed 360-degree visuals and incorporated different sets of photographs, satellite imagery and video footage. After the presentation, the platform allowed the judges to further explore the platform and interact with the evidence themselves. Technology-based presentation of evidence was also used in other trial proceedings, such as in the cases of Ntaganda, Ongwen and Bemba et al. Also, the two pending warrants of arrest in the Al-Werfalli case relate to (execution) incidents that were filmed, with graphic videos posted on social media, with this visual record being authenticated and related to other evidence”). ↩

16. 16.

    Berkeley Protocol, supra note 1. ↩

17. 17.

    Id. ↩

18. 18.

    See, e.g., Peggy O’Donnell, Alexa Koenig, Camille Crittenden & Eric Stover, UC Berkeley HRC, Beyond Reasonable Doubt: Using Scientific Evidence to Advance Prosecutions at the International Criminal Court, 3 (Jan. 7, 2013), available online

    (noting that Court investigators “have frequently relied upon information and assistance from a wide variety of sources including governments, journalists, peacekeepers, human rights researchers, forensic scientists, and intelligence specialists”). ↩

19. 19.

    See, e.g., Eric Stover, Mychelle Balthazard, & Alexa Koenig, Confronting Duch: Civil Party Participation in Case 001 at the Extraordinary Chambers in the Courts of Cambodia, 93 Int’l Rev. Red Cross 503 (Jun. 30, 2011), available online, doi. ↩

20. 20.

    See, e.g., Kit Smith, 60 Incredible and Interesting Twitter Stats and Statistics, Brandwatch (Jan. 2, 2020), available online. ↩

21. 21.

    See, e.g., Kit Smith, 57 Fascinating and Incredible YouTube Statistics, Brandwatch (Feb. 21, 2020), available online; Anmar Frangoul, With Over 1 Billion Users, Here’s How YouTube is Keeping Pace with Change, CNBC, Mar. 14, 2018, available online. ↩

22. 22.

    See, e.g., Avi Asher-Schapiro, YouTube and Facebook are Removing Evidence of Atrocities, Jeopardizing Cases Against War Criminals, The Intercept, Nov. 2, 2017, available online. ↩

23. 23.

    See UC Berkeley Human Rights Center, Digital Lockers: Options for Archiving Social Media Content with Evidentiary Value (forthcoming 2020). ↩

24. 24.

    About Us, NCMEC, available online (last visited May 6, 2020); see also Reporting Requirements of Providers, 18 United States Code § 2258A (2008), available online. ↩

25. 25.

    Evolving an Institution, GIFCT, available online (last visited May 4, 2020). ↩

26. 26.

    Information and Evidence Collection, IIIM, available online (last visited May 4, 2020). ↩

27. 27.

    Id. ↩

28. 28.

    Elise Baker, et al., Joining Forces: National War Crimes Units and the Pursuit of International Justice, 42 Hum. Rts. Q. __ (forthcoming 2020). ↩

29. 29.

    See Berkeley Protocol, supra note 1

    (more on key principles related to digital open source investigations).

    See also Alexa Koenig & Lindsay Freeman, Open Source Investigations for Legal Accountability: Challenges and Best Practices, in Digital Witness: Using Open Source Information for Human Rights Investigation, Documentation, and Accountability (Sam Dubberley, Alexa Koenig & Daragh Murray, eds., Feb. 19, 2020), paywall, paywall. ↩

30. 30.

    See Lindsay Freeman & Raquel Vazquez Llorente, Justice 3.0: International Criminal Evidence and Procedure in the Digital Age (Int’l Nuremberg Principles Acad., forthcoming 2020). ↩

31. 31.

    See Koenig, supra note 14

    (more on strategies for addressing deep fakes). ↩

32. 32.

    Strategic Plan, supra note 3. ↩

33. Suggested Citation for this Comment:

    Alexa Koenig, Digital and Open Source Information Can Play a Critical Role in Improving the Overall Efficiency and Efficacy of the International Criminal Court, ICC Forum (Jun. 1, 2020), available at https://iccforum.com/cyber-evidence#Koenig.

    Suggested Citation for this Issue Generally:

    To What Extent Can Cyber Evidence Repositories, and Digital and Open-Source Evidence, Facilitate the Work of the OTP, and the ICC More Generally?, ICC Forum (Jun. 1, 2020), available at https://iccforum.com/cyber-evidence.

Argument Continued

“Finding a Needle in a Pile of Needles”

The conflict in Syria has become the most documented crime base in human history. Information about today’s events on and off the Syrian battlefield is instantaneously disseminated around the globe through formal and informal media and social networks. From the beginning of the uprising, and in real-time ever since, citizen journalists wielding smartphones from the grassroots began uploading videos and photographs of the revolution, the government’s crackdown, and the ensuing armed conflict at a rate never before seen in previous conflicts.6 Among all this information are millions of video clips purporting to show the targeting of civilians and civilian objects, the execution of captured combatants and perceived opponents, the use of chemical and other indiscriminate weapons, and other international offenses. These have been uploaded onto YouTube and other social media platforms. This degree of citizen activity is particularly remarkable given the heretofore autarkic nature of the Syrian state. The sheer volume of information is overwhelming. Google has estimated that there are “more hours of footage of the Syrian civil war on YouTube then there actually are hours of the war in real life.”7

Because the current information environment is increasingly internet-based and digital, human rights investigators have had to update their collection, storage, authentication, and analytical protocols.8 Activists and their allies are thus exfiltrating and digitizing regime documents, taking witness/victim testimonials remotely on new communications platforms, scrubbing social media sites for potential open-source evidence, digitizing gigabytes of data that are then subjected to big-data and statistical modeling techniques, crowdsourcing identification and authentication information, generating three-dimensional crime scene renditions and other forms of data visualization to convey the full horror of the conflict, and securing potential evidence in encrypted digital vaults. All of these efforts are prompting, and benefiting from, innovations in technology around data management, secure storage and communication, optical character recognition or OCR (no easy feat with Arabic script), machine learning and artificial intelligence, and digital forensics.9 This includes technologies to de-duplicate, code, and organize reams of data and to make sense of the contents at scale. Journalists, citizen documenters, and advocates are not necessarily well-versed in ICL doctrine, investigative strategies, or the state of the art security protocols, and so NGOs —such as the Institute of International Criminal Investigations, Videre est Credere, New Tactics for Human Rights, eyeWitness to Atrocities, and WITNESS—are creating training programs, field guides, and technology tools to help citizen documentarians and others ensure that their collections achieve maximum utility (with manageable risk) in any criminal prosecution or transitional justice process.10

Much of the information recorded depicts what purports to be violations of international criminal and humanitarian law. Although this crime base information is valuable, when it comes to legal accountability, it is equally important—if not more so—to collect potential linkage evidence that speaks to individual responsibility. Linkage evidence can help identify not only the direct perpetrator(s), but also his or her confederates, co-conspirators, superiors, subordinates, and enablers, all of whom may be equally liable through doctrines of complicity, aiding and abetting, conspiracy, joint criminal enterprise, common plan, instigation, and superior responsibility, depending on the operative legal framework. Evidence of “key linkage themes”—the order of battle and the objectives of military operations; the functioning of a regime’s political, military, police, or paramilitary structures; the procurement and movement of arms; communication patterns and logistical support; command, control, and communications (C3) structures; and the available disciplinary procedures—may be less susceptible to open-source research.11 Instead, prosecutors will still need to collect linkage clues in hard copy documents, electronic intercepts, and testimony (from witnesses, insiders, defectors, and experts). That said, investigators are undertaking social network analyses of social media platforms and other datasets to reveal connections between individuals (and their devices).12 These sources can be marshalled to prove incitement to violence as well as membership in, or fealty to, an armed group, particularly in the form of enlistment records or allegiance videos.13

From the perspective of advancing criminal accountability, these developments present a number of opportunities and challenges. For one, this collection work is often undertaken contemporaneously, before evidence of war crimes can be hidden, tampered with, deliberately or inadvertently destroyed, or removed from public view by corporate entities enforcing their terms of service and community standards against terrorist activity, hate speech, and bigotry.14 Because far-reaching justice may be years—or even decades—in the making, it is imperative that evidence of crimes is amassed in real time and preserved for when the time is ripe for justice and accountability. Civil society actors—including first responders—will inevitably start to operate well before the OTP can, or is invited to, turn its attention to an unfolding atrocity situation. This speaks to the need for the international community to capacitate digital documentation as soon as the threat of international law violations appears.

Additionally, there is the need to refine these raw data into more structured information and then, ultimately, into admissible evidence. The sheer quantity of documentation collected makes it harder to separate the wheat from the chaff, described as an exercise of “looking for a needle in a pile of needles.”15 Indeed, an abundance of information can paralyze action just as easily as a dearth of information can.16 That said, this information, even if not fully dispositive, may be important for lead and background purposes. And, even absent an accountability process, the material collected remains vitally important for other transitional justice processes of truth-telling, vetting/lustration, restitution, reparation, memorialization, and institutional reform.

Even with all these documentation efforts underway, we still only have what two statisticians call “snapshots of violence,” given the difficulty of gaining access to a complete record of the conflict and all its consequences.17 As a result, the “dark figure of crime” in ICL —data on those who were killed or mistreated who have not been counted or accounted for—remains disconcertingly high.18 Open source data is akin to a “convenience sample” that is inherently incomplete and only captures a fraction of that which is observable. In addition, some acts of violence are more likely to be recorded than others (e.g., public incidents with a large number of victims versus custodial abuses in clandestine cells) and conflict dynamics (such as siege warfare) make documentation of some events difficult.19 All told, different conflicts will have different documentation dynamics, which impact the availability of recorded data.20 Finally, there is the ever present risk of inadvertently relying upon mis- and dis-information.21 As the saying goes: “the first casualty of war is truth.” Content creators can easily fabricate information and even metadata can be manipulated. The internet is full of information pollution that can distract or undermine a criminal investigation. The Syrian government, and its the shadowy Syrian Electronic Army,22 has proven itself quite adept at falsifying and manufacturing information before and during the war.23

Actors & Action in the Field

When it comes to the Syrian conflict, much of this documentation, verification, forensic, and analytical work is being done in the non-governmental sphere by classic civil society documentation centers (such as the Syrian Archive and the Syria Justice & Accountability Centre); new private and for-profit entities (such as the Commission on International Justice & Accountability (CIJA) and Hala Systems, respectively); and investigative journalists pioneering open source investigations (like Bellingcat). By way of example, the Syrian Archive began as an offshoot of the Tactical Technology Collective with the simple goal of providing a safe information repository for people monitoring peaceful protests in Syria. It now works to support citizen documentarians through the use and dissemination of open source tools and replicable methodologies for collecting, authenticating, and preserving user-generated visual documentation (such as the Electronic Discovery Reference Model commonly used in domestic e-discovery). It maintains public databases of information on a range of international crimes being committed in Syria, including chemical and cluster munition attacks, disappearances, and airstrikes. Each video is given a unique hash value, or digital identification fingerprint, at the point of collection to help with de-duplication, geo-location, and authentication.24 Cryptographic hashing is like a digital pixel fingerprint that will reveal whether there have been any changes to, or corruption of, the information and metadata and also help establish the chain of custody. The Archive has partnered with Visual Forensics and Metadata Extraction (VFRAME) to use machine learning with 3D printing and a purpose-built computer vision/object detection program (a subfield of machine learning) to track the use of indiscriminate weapons or weapons remnants, such as cluster munitions.25

As another example, CIJA signifies a new model of “privatized” investigations (in the sense of not being sanctioned by any sovereign entity) focused on captured Assad regime and Islamic State in Iraq and the Levant (ISIL) documents with a focus on linking horrific crimes to specific perpetrators. Funded largely by Western governments, CIJA is staffed by veterans of international courts and military intelligence units who are focused less on amassing additional information about the Syrian crime base and more on collecting linkage evidence to the highest criminal law standard to ensure its maximum utility.26 It produces prosecution-ready files, proto-indictments, and evidentiary briefs on responsible individuals and units, particular crimes, and the structure and functioning of the Assad regime and ISIL.27 Both organizations have already informed domestic criminal proceedings involving events in Syria.28

In the multilateral sphere, numerous United Nations fact-finding efforts are also underway. Most important is an innovative new mechanism created by the UN General Assembly—the International, Impartial and Independent Mechanism to Assist in the Investigation and Prosecution of Persons Responsible for the Most Serious Crimes under International Law Committed in the Syrian Arab Republic since March 2011 (IIIM)—that has been tasked with consolidating and analyzing existing documentation with an eye towards supporting future criminal trials. The IIIM is acting as a central repository of evidence concerning Syria, although it is not necessarily trying to vacuum up all documentation from all sources, which might not only overwhelm the IIIM but also threaten smaller documentation efforts whose holdings are akin to their intellectual property. Rather, it is endeavoring to operate as a hub between multiple stakeholders to index or catalogue what potential evidence is out there and cross-reference it to the information it gathers directly in a gap-filling role. Un-siloing these datasets enables record linkages across the different collections and the construction of a more complete understanding of events.29 Every piece of evidence is given a unique alphanumeric identifier (the “hash”), which will allow anyone to locate it from the main catalog and identify the “cleanest” version of any particular image or video (e.g., the version devoid of logos or watermarks, and free from tampering, etc.). The result of all this collection work is a massive record of international crimes being committed in Syria; the next question is how to effectively utilize these repositories of data in the service of accountability?

New Technologies & Techniques

These actors are all exploiting new technologies and documentation techniques that have revolutionized human rights documentation, reporting, and analysis.30 For example, eyeWitness, which is affiliated with the International Bar Association, has created a smartphone application that helps users structure the gathering of evidence of atrocity crimes, including key metadata (i.e., data about data that is not necessarily immediately visible in a digital artifact).31 At the source of capture, the app creates a chain of cryptographic and digital fingerprints that render the data resistant to editing. The information assembled using the app is automatically uploaded into a secure evidence vault, which creates a certifiable chain of custody. The platform also allows for verification and analysis by international lawyers with an eye towards enhancing its utility in accountability processes. Similarly, Physicians for Human Rights has created MediCapt, a similar mobile solution, that helps convert a standardized medical intake form into potential forensic documentation to secure evidence of rape and other forms of sexual assault on an encrypted and high-fidelity digital platform. A secure mobile camera facilitates the preservation of evidence of physical injuries, and a mapping feature tracks geographic trends.32

Technologists at organizations like Benetech are working to develop software tools to expedite and mechanize processes of collection, verification, prioritization, and analysis. Benetech’s Connected Civil Society Initiative is using machine learning algorithms to automate the analysis of videos collected by Syrian documentation organizations, including through clustering, distance mapping, matching, and de-duplicating videos that are related or depict the same event(s). It has also designed a system to extract key metadata and has developed a set of archetypal classifiers to expedite object and audio recognition. These new tools will enable NGOs and the IIIM to compare their holdings, find the “best evidence” of a particular incident, and avoid duplication of efforts. Data visualization tools allow bulk information to be disaggregated along desired characteristics (such as the age and sex of victims or cause of death) and conveyed to multiple audiences—including the general public, policymakers, and judges—in compelling and accessible ways.33

Academic institutions are making key contributions to this field. The Syrian Archive has partnered with UC Berkeley’s Human Rights Investigations Lab and Amnesty International’s Digital Verification Corps to harness student energy around the conflict and train the next generation of human rights advocates and investigators.34 Researchers at Carnegie Mellon University have developed the Event Labeling Through Analytic Media Processing (E-LAMP) system. E-LAMP harnesses dynamic machine learning technologies to identify predefined events and key objects within multimedia images.35 The system can undertake color, shadow, illumination, and photogrammetry (taking measurements from images) analysis and be “taught” to identify significant semantic concepts (“classifiers”) through a user interface. These can be objects (such as weapons, smoke plumes, or vehicles), actions (e.g., tank movements or explosions), text (through OCR), speech acts and sounds (like helicopters or gunfire), human behaviors (such as running, screaming, or crying), and classes of people (including soldiers, corpses, or children).36 Researchers used over five hundred videos from Aleppo, Syria, to train the system to recognize, with a high degree of accuracy, key munitions being deployed in the Syrian battlespace.

Human rights investigations have become increasingly interdisciplinary, borrowing from tools and techniques developed in other contexts. Forensic Architecture (FA), a multidisciplinary collective of investigators based at the University of London, has employed architectural rendering software to synchronize multimedia information and create groundbreaking computer models of potential crime scenes.37 These can shed light on the circumstances of particular armed attacks on civilians and civilian objects. When it comes to Syria, for example, FA has recreated the sites of chemical attacks and built a three-dimensional rendition of Saydnaya prison where detainees have credibly alleged they were tortured, all by using spatial imagery, computer aided design/architectural software, and acoustic modeling.38 In addition, FA examined the destruction of the Sayidina Omar Ibn Al-Khattab Mosque in al-Jinah, Syria, which was hit by US airstrikes on March 16, 2017. The United States originally denied having caused multiple civilian casualties and insisted that the venue was a community hall where regional members of Al Qaida were meeting on the night in question. Based on a reconstruction of the building prior to the attack and other data, FA concluded that the building was clearly a mosque being used for religious purposes and that thirty-eight civilians were killed.39 The United States eventually admitted that the strike had hit part of a “mosque complex” and that “a more deliberative pre-strike analysis should have identified that the target was part of a religious compound,”40 but continued to argue that appropriate precautions were undertaken.41

Human rights advocates are attempting to rehabilitate and augment the utility of satellite imagery for human rights purposes in Syria and elsewhere.42 Such imagery can support the identification of observables and points of interests, the construction of a timeline or before-and-after comparisons, the creation of geographic configurations of a crime scene, and even the development of a narrative of intent where one area or group is singled out for violence.43 For example, Amnesty International has forged a partnership with the Geospatial Technologies and Human Rights Project of the American Association for the Advancement of Science; and the Standby Volunteer Task Force Satellite Team, an online volunteer community established in 2010 at the International Conference of Crisis Mapping; to provide dedicated live mapping support to organizations in the humanitarian and human rights space, including in Syria.44 Using high-resolution imagery from DigitalGlobe (now Maxar), this team has employed crowd-tasking to document the presence of regime forces and armored vehicles in civilian areas and identify the commission of potential war crimes, such as the destruction of civilian objects45 in Aleppo and elsewhere, through before-and-after damage assessment.46 The project depends largely on volunteers who are recruited through micro-tasking platforms (starting with Tomnod, also succeeded by Maxar), which invite volunteers to solve real-world problems using satellite imagery.

Crowdsourcing (a form of non-probability sampling) offers a useful way to aggregate, verify, and authenticate data of relevance to accountability. It is valuable because it can occur in real-time (or close to real-time) in constrained collection environments.47 The Syrian Tracker project of Humanitarian Tracker has been crowdsourcing and live mapping the Syrian conflict since April 2011.48 In an effort to consolidate eye witness accounts and leverage the work of citizen reporters, it accepts anonymous reports—via email, Twitter, phone, the website, and other encrypted means—which are tagged and catalogued by type of attack. Where possible, these contributions are then cross-referenced with media reports and other validation sources using data mining tools. Only a small percentage of submissions are published, given limited resources for de-duplication and verification. The site is built upon the crowdmap technology first developed by Ushahidi to track post-election violence in Kenya.49

Crowdsourcing visual images can be used to identify victims and perpetrators. In January 2014, a number of media outlets began circulating gruesome photographs that had been exfiltrated by a former military photographer who worked in Damascus hospitals—an extraordinary cache of government-generated proof of human rights abuses in Syria. The defector, code-named “Caesar” to protect his identity, had smuggled out of the country, on thumb drives and his phone, over fifty thousand images depicting more than eleven thousand victims. Half of the photos depicted cadavers showing signs of torture, starvation, and mistreatment (the other half are likely battlefield deaths).50 The Syrian Association for Missing and Conscience Detainees posted the entire collection online so that people whose loved ones had disappeared could search for and identify the victims. This approach generated some controversy given the privacy rights of victims and the risk of traumatization to family members.51 Other Syrian NGOs, such as the Syrian Network for Human Rights, notified the families if they were able to identify the victims.52 On the other side of the aisle, Adalmaz is crowdsourcing photographs of fighters to identify anonymous perpetrators and generate leads for law enforcement. The loved ones of one of the victims depicted in the Caesar photos and their lawyers at the Guernica Center for International Justice initiated the first criminal case against Syrian officials in domestic courts, although the case was dismissed on jurisdictional grounds.

Where these techniques and technologies are potentially game changing is in processing and triaging bulk collections and large datasets. Eventually, however, there can be no doubt that manual analysis by experts—of satellite imagery or the outputs of machine learning algorithms, which are only probabilistic53 —will be necessary before the OTP can rely on this information for its purposes.54 Expert testimony will be crucial not only to contextualize the results of these analytical processes but also to offer attestations of validity and to authenticate the technology and methodologies employed. Indeed, according to a recent report from OpenGlobalRights, the key determinant of the admissibility of satellite imagery in human rights litigation is the ability of a human witness to testify credibly about what the images depict.55

It is hoped that much of this evidence will be considered self-authenticating such that the original source will not always need to testify in Court (let alone be identified).56 This can lessen the dependence on witness testimony by offering corroborating evidence and eliminating the need to call multiple witnesses to a single event.57 There are efforts afoot to render international trials less dependent on viva voce testimony, including through the use of probabilistic methods and other social science research tools—the topic of an earlier issue of the ICC Forum.58 This reflects the worrisome reality that witnesses are the soft underbelly of any criminal prosecution.59

To be sure, no prosecutor worth her salt will mechanically adopt external work product, but this analytical and verification work by trusted partners will jump start and facilitate prosecutorial efforts when suspects are within reach. In particular, the IIIM —as an internationally-mandated organization—can undertake the sort of deep historical, cultural, and contextual research that is necessary to launch a successful war crimes prosecution. This would include gathering proof of the chapeau (or circumstantial) elements of war crimes and crimes against humanity, such as the existence of an armed conflict or a widespread and systematic attack against a civilian population. This singular focus on one crime base—coupled with the ability to pursue both structural and target-based investigations—might be difficult for multiple national prosecutors to undertake. The same goes for the OTP, which is already over-stretched with its existing docket of prosecutions, investigations, and preliminary examinations.60

Conclusion

The Nuremberg proceedings relied heavily on captured Nazi documents—amounting to upwards of three thousand tons of records and photographs—with witnesses playing lesser roles in establishing the commission of international crimes.61 With the revival of the Nuremberg promise in the 1990s, international prosecutors have depended more heavily on viva voce testimony, with all that such reliance entails in terms of witness protection, reliability deficits, and persistent problems with interpretation in vernacular languages.62 The advent of the digital revolution suggests that we may have come full circle when it comes to the ICL evidentiary landscape. Although activating international jurisdiction has proven to be a bridge too far when it comes to Syria,63 all this documentation on Syria is contributing to episodic cases that are beginning to materialize in domestic courts around the globe—the most promising avenue for justice to date.64 When it comes to the OTP, a ready judicial forum exists; what is needed is probative and reliable evidence that proves the commission of international crimes beyond a reasonable doubt. New digital repositories and techniques have the potential to make an important contribution in this regard and should be embraced.

Endnotes — (click the footnote reference number, or ↩ symbol, to return to location in text).

1. 1.

   Elements of this comment are drawn from a chapter in my forthcoming book, Beth Van Schaack, Innovations in International Criminal Law Documentation Methodologies and Institutions (Feb. 5, 2019) in Imagining Justice for Syria (OUP, forthcoming Sep. 2020), paywall, doi. ↩

2. 2.

   United States Office of the Director of National Intelligence, Intelligence Community Directive No. 301, National Open Source Enterprise (Effective Jul. 11, 2006), available online; Syria Files, Wikileaks (Jul. 5, 2012), available online (last visited May 24, 2020).

   (Wikileak’s Syria Files provide an example. It is distinct from other forms of intelligence, such as signals intercepts or human intelligence).

   See What is Intelligence?, US DNI, available online (last visited May 24, 2020)

   (outlining the six categories of intelligence). ↩

3. 3.

   Rome Statute of the International Criminal Court, Adopted by the United Nations Diplomatic Conference of Plenipotentiaries on the Establishment of an International Criminal Court, Jul. 17, 1998, UN Doc. A/CONF.183/9, as amended [hereinafter Rome Statute], Art. 69(7), available online

   (excluding evidence obtained by means that violate internationally recognized human rights if the violation undermines reliability or admission would seriously damage the integrity of the proceedings). ↩

4. 4.

   Office of the Prosecutor, ICC, Strategic Plan 2016–2018, ¶¶ 54, 96 (Jul. 6, 2015), available online; Office of the Prosecutor, ICC, Strategic Plan 2019–2021, ¶ 46 (Jul. 17, 2019), available online. ↩

5. 5.

   Andras Vamos-Goldman, The Importance of Professional Expertise in Gathering Evidence of Mass Atrocities, Just Security (Oct. 27, 2017), available online. ↩

6. 6.

   See Rebecca J. Hamilton, User-Generated Evidence, 57 Colum. J. Transnat’l L. 1, 5 (Nov. 7, 2018), available online

   (arguing that user-generated evidence is “the most visible sign yet of the fundamental disruption underway within the investigatory ecosystem” of international criminal law). ↩

7. 7.

   Armin Rosen, Erasing History: YouTube’s Deletion of Syria War Videos Concerns Human Rights Groups, Fast Company, Mar. 7, 2018, available online

   (quoting Google executive). ↩

8. 8.

   See Brianne McGonigle Leyh, Changing Landscapes in Documentation Efforts: Civil Society Documentation of Serious Human Rights Violations, 33 Utrecht J. Int’l and Eur. L. 44 (Apr. 12, 2017), available online, doi; Els De Busser, Open Source Data and Criminal Investigations: Anything You Publish Can and Will be Used Against You, 2 GROJIL 90, 91 (Dec. 5, 2014), available online, doi; Lindsay Freeman, Digital Evidence and War Crimes Prosecutions: The Impact of Digital Technologies on International Criminal Investigations and Trials, 41 Fordham Int’l L.J. 283 (2018), available online. ↩

9. 9.

   Enrique Piracés, Collecting, Preserving, and Verifying Online Evidence of Human Rights Violations, OpenGlobalRights (Jan. 30, 2018), available online. ↩

10. 10.

    See, e.g., Craig Silverman ed., Verification Handbook for Investigative Reporting, EJC (Apr. 27, 2015), available online; Craig Silverman ed., Verification Handbook for Disinformation and Media Manipulation, EJC, available online (last visited May 24, 2020); Berkeley Protocol on Open Source Investigations, OHCHR & UC Berkeley HRC (forthcoming 2020), announcement online; Digital Witness: Using Open Source Information for Human Rights Investigation, Documentation, and Accountability (Sam Dubberley, Alexa Koenig & Daragh Murray eds., Feb. 19, 2020), paywall. ↩

11. 11.

    Ewan Brown & William H. Wiley, What Can and Should the Next ICC Prosecutor do to Improve the ICC’s Investigative Techniques?, Just. in Conflict (Apr. 13, 2020), available online. ↩

12. 12.

    Valdis Krebs, Connecting the Dots: Tracking Two Identified Terrorists, Orgnet (2008), available online (last visited May 24, 2020); Francesco Calderoni, Social Network Analysis of Organized Criminal Groups, in Encyclopedia of Criminology and Criminal Justice (Gerben Bruinsma & David Weisburd, eds., Nov. 27, 2018), paywall, doi, earlier version (Jan. 2014) online. ↩

13. 13.

    Rao Kumar, How to Digitally Verify Combatant Affiliation in Middle East Conflicts, Bellingcat (Jul. 9, 2018), available online. ↩

14. 14.

    Ellery Roberts Biddle, “Envision a new war”: The Syrian Archive, Corporate Censorship and the Struggle to Preserve Public History Online, Advox (May 1, 2019), available online. ↩

15. 15.

    Saving Private Ryan ( DreamWorks Pictures Jun. 24, 1998); Keith Hiatt, Vice President of Human Rights, Benetech, Stanford University Panel Discussion on Syria (Feb. 13, 2018), remarks available online. ↩

16. 16.

    See Patrick Philippe Meier, Digital Humanitarians: How Big Data Is Changing the Face of Humanitarian Response (Jan. 22, 2015), paywall. ↩

17. 17.

    See Megan Price & Patrick Ball, Big Data, Selection Bias, and the Statistical Patterns of Mortality in Conflict, 36 SAIS Rev. of Int’l Aff. 9 (2014) [hereinafter Mortality in Conflict], available online, doi. ↩

18. 18.

    See Megan Price, Anita Gohdes & Patrick Ball, Documents of War: Understanding the Syrian Conflict, Significance 14, 15 (Apr. 2015) [hereinafter Documents of War], available online.

    (The “dark figure” is the number of events not counted in any documentation process). ↩

19. 19.

    Mortality in Conflict, supra note 17, at 10

    (discussing event size bias in human rights documentation). ↩

20. 20.

    Documents of War, supra note 18, at 19. ↩

21. 21.

    Claire Wardle & Hossein Derakhshan, COE, Information Disorder: Toward an Interdisciplinary Framework for Research & Policy Making, 5 (Sep. 27, 2017), available online.

    (Misinformation is information that is false but not necessarily intended to cause harm. Disinformation is deliberately created to mislead). ↩

22. 22.

    Stewart Kenton Bertram, “Close Enough”—The Link between the Syrian Electronic Army and the Bashar al-Assad Regime, and Implications for the Future Development of Nation-State Cyber Counter-Insurgency Strategies, 8 J. Terrorism Res. 1 (Feb. 8, 2017), available online, doi. ↩

23. 23.

    Maya Abyad, Regime Wages War of Documents on Syrians, SyriaUntold (Mar. 31, 2017), available online. ↩

24. 24.

    See Douglas Carner, Detecting and Preventing File Tampering, Forensic Protection, available online (last visited May 24, 2020). ↩

25. 25.

    See Jay D. Aronson, Computer Vision and Machine Learning for Human Rights Video Analysis: Case Studies, Possibilities, Concerns, and Limitations, 43 Law & Soc. Inquiry 1188 (Mar. 1, 2018), available online, doi.

    (Aronson heads the Center for Human Rights Science at Carnegie Mellon University). ↩

26. 26.

    See generally Melinda Rankin, Investigating Crimes against Humanity in Syria and Iraq: The Commission for International Justice and Accountability, 9 GR2P 395 (Nov. 26, 2017), paywall, doi

    (recounting history of CIJA). ↩

27. 27.

    Marlise Simons, Investigators in Syria Seek Paper Trails That Could Prove War Crimes, N.Y. Times, Oct. 7, 2014, available online. ↩

28. 28.

    Caroline Fehl, The Partial Return of Universal Jurisdiction: Syrian Torturers on Trial in Germany, Global Pol’y J. (May 12, 2020), available online. ↩

29. 29.

    See Megan Price, Anita Gohdes & Patrick Ball, HRDAG, Updated Statistical Analysis of Documentation of Killings in the Syrian Arab Republic (Aug. 2014), available online.

    (In work commissioned by the UN High Commissioner for Human Rights, the Human Rights Data Analysis Group (HRDAG) undertook this with casualty data collected early in the conflict. From these data, HRDAG was able to shed light on the dark figure of victims and estimate the total number of casualties through a process of multiple systems estimation).

    See Amelia Hoover Green, Multiple Systems Estimation: The Basics, HRDAG (Mar. 11, 2013), available online; See Documents of War, supra note 18, at 16. ↩

30. 30.

    Eric Reidy, Technology Exposed Syrian War Crimes Over and Over. Was it for Nothing?, MIT Tech. Rev. (Oct. 18, 2019), available online. ↩

31. 31.

    See Mark S. Ellis, Shifting the Paradigm—Bringing to Justice Those Who Commit Human Rights Atrocities, 47 Case W. Res. J. Int’l L. 265 (2015), available online

    (discussing the app’s origins). ↩

32. 32.

    PHR’s Mobile App MediCapt Puts Cutting Edge Technology in the Service of Preventing Sexual Violence, Physicians for Hum. Rts., available online (last visited May 24, 2020). ↩

33. 33.

    See Katharina Rall, Margaret L. Satterthwaite, Anshul Vikram Pandey, John Emerson, Jeremy Boy, Oded Nov & Enrico Bertini, Data Visualization for Human Rights Advocacy, 8 JHRP 171 (Jun. 29, 2016), available online, doi. ↩

34. 34.

    See, e.g., Anna Banchik, Leenah Bassouni, Michael Elsanadi, Hannah Ellis, Sonia Hamilton, Natalia Krapiva, Danny Lee & Andrea Trewinnard, Hum. Rts. Investigation Lab, Chemical Strikes on Al-Lataminah, March 25 & 30, 2017 (Jan. 18, 2018), available online

    (analyzing open source visual content regarding attacks in March 2017).

    Amnesty International, No Where Is Safe for Us: Unlawful Attacks and Mass Displacement in North-West Syria (May 6, 2020), available online. ↩

35. 35.

    Wei Tong, Yi Yang, Lu Jiang, Shoou-I Yu, ZhenZhong Lan, Zhigang Ma, Waito Sze, Ehsan Younessian & Alexander G. Hauptmann, E-LAMP: Integration of Innovative Ideas for Multimedia Event Detection, 25 Machine Vision and Applications 5 (Jul. 9, 2013), available online, doi. ↩

36. 36.

    Grigorios Kalliatakis, Shoaib Ehsan, Maria Fasli & Klaus D. McDonald-Maier, DisplaceNet: Recognising Displaced People from Images by Exploiting Dominance Level, arXiv (May 3, 2019), available online; Grigorios Kalliatakis, Shoaib Ehsan, Aleš Leonardis, Maria Fasli & Klaus D. McDonald-Maier, Exploring Object-Centric and Scene-Centric CNN Features and their Complementarity for Human Rights Violations Recognition in Images, 7 IEEE Access 10045 (Jan. 9, 2019), available online, doi; Grigorios Kalliatakis, Shoaib Ehsan, Maria Fasli & Klaus D. McDonald-Maier, GET-AID: Visual Recognition of Human Rights Abuses via Global Emotional Traits, arXiv (Feb. 11, 2019), available online. ↩

37. 37.

    SITU Research, ICC Digital Platform: Timbuktu, Mali (2016), available online, overview (last visited May 24, 2020).

    (The OTP has already explored the potentiality of 3D reconstruction of crime scenes in connection with the destruction of sacred tombs in Mali). ↩

38. 38.

    Michael Kimmelman, Forensics Helps Widen Architecture’s Mission, N.Y. Times, Apr. 6, 2018, available online. ↩

39. 39.

    See Christiaan Triebert, The Al-Jinah Mosque Complex Bombing—New Information and Timeline, Bellingcat (Apr. 18, 2017), available online.

    (Forensic Architecture worked in conjunction with Bellingcat and Human Rights Watch in this investigation).

    Human Rights Watch, Attack on the Omar Ibn al-Khattab Mosque: US Authorities’ Failure to Take Adequate Precautions (Apr. 18, 2017), available online. ↩

40. 40.

    Transcript of Pentagon’s Al Jinah Investigation Media Briefing, Airwars (Jun. 27, 2017), available online. ↩

41. 41.

    See Shane R. Reeves & Ward Narramore, The UNHRC Commission of Inquiry on Syria Misapplies the Law of Armed Conflict, Lawfare (Sep. 15, 2017), available online. ↩

42. 42.

    See, e.g., Human Rights Watch, Syria: New Satellite Images Show Homs Shelling (Mar. 2, 2012), available online

    (discussing evidence of the bombardment of the Baba Amr neighborhood of Homs). ↩

43. 43.

    See generally Ben Yunmo Wang, Nathaniel A. Raymond, Gabrielle Gould & Isaac Baker, Problems from Hell, Solutions in the Heavens?: Identifying Obstacles and Opportunities for Employing Geospatial Technologies to Document and Mitigate Mass Atrocities, 2 Stability 1, 6–12 (Oct. 2013), available online, doi; Katia Savchuk, All the Dead We Cannot See: How a Tech Geek Is Using Machine Learning to Hold Human Rights Abusers Accountable, Pacific Standard, Dec. 28, 2018, available online. ↩

44. 44.

    See Patrick Philippe Meier, Combining Crowdsourced Satellite Imagery Analysis with Crisis Reporting: An Update on Syria, iRevolutions (Sep. 19, 2011), available online. ↩

45. 45.

    Satellite Imagery Analysis for Urban Conflict Documentation: Aleppo, Syria, AAAS (Jun. 27, 2017), available online. ↩

46. 46.

    Meier, supra note 44. ↩

47. 47.

    See generally The Engine Room, Benetech & Amnesty International, DATNAV: How to Navigate Digital Data for Human Rights Research (Jun. 2016), available online. ↩

48. 48.

    See Lorenzo Franceschi-Bicchierai, How Data Scientists Are Uncovering War Crimes in Syria, Mashable, Aug. 8, 2014, available online. ↩

49. 49.

    Will Haydon, #IndexAwards2015: Digital Activism Nominee Syria Tracker, Xindex (Mar. 11, 2015), available online. ↩

50. 50.

    Priyanka Motaparthy & Nadim Houry, HRW, If the Dead Could Speak: Mass Death and Torture in Syria’s Detention Facilities (Dec. 16, 2015), available online. ↩

51. 51.

    Syria Justice and Accountability Centre, The Publication of Victims’ Photographs Online Jeopardizes Security and Accountability in Syria (Mar. 26, 2015), available online. ↩

52. 52.

    Syrian Network for Human Rights, Analytical Study About the Leaked Pictures of Torture Victims in Syrian Military Hospitals: “The Photographed Holocaust” 5 (Oct. 15, 2015), available online. ↩

53. 53.

    Aronson, supra note 25, at 1204. ↩

54. 54.

    Patrick Philippe Meier, Verifying Crowdsourced Social Media Reports for Live Crisis Mapping: An Introduction to Information Forensics, iRevolutions (Nov. 29, 2011), available online. ↩

55. 55.

    See Theresa L. Harris, Jonathan Drake, Jessica M. Wyndham, Susan R. Wolfinbarger, Stephen D. Lott & Michael Lerner, AAAS, Geospatial Evidence in International Human Rights Litigation: Technical and Legal Considerations (Jul. 25, 2018), available online. ↩

56. 56.

    Hamilton, supra note 6, at 55; id. at 45

    (“There is a valid legal argument to make that the app itself is the witness” quoting Wendy Betts, Program Director, eyeWitness to Atrocities). ↩

57. 57.

    See generally Keith Hiatt, Open Source Evidence on Trial, 125 Yale L.J. F. 323 (Mar. 3, 2016), available online

    (discussing the promise and perils of open source investigations). ↩

58. 58.

    See Anne-Marie de Brouwer, Cases of Mass Sexual Violence Can Be Proven Without Direct Victim Testimony, in Contemporary Issues Facing the International Criminal Court 282 (Richard H. Steinberg ed., Apr. 8, 2016), doi, earlier version (Jun. 26, 2012), available online.

    John Hagan, The Use of Sample Survey Interviews as Evidence of Mass Rape, in Contemporary Issues Facing the International Criminal Court 295 (Richard H. Steinberg ed., Apr. 8, 2016), doi, earlier version (Jun. 26, 2012), available online. ↩

59. 59.

    See Press Release, FIDH, The Kenya Cases at the International Criminal Court: Q&A on “Rule 68” and Witness Tampering (Feb. 11, 2016), available online.

    (The Kenyan cases before the ICC collapsed due to unprecedented witness tampering and intimidation). ↩

60. 60.

    Patryk I. Labuda, Less is More: Rediscovering the Prosecutor’s Core Mandate, Just. in Conflict (Apr. 14, 2020), available online. ↩

61. 61.

    Edna Friedberg, How the Nuremberg Trial Bore Witness to the Nazis’ Worst Crimes, Slate, Nov. 20, 2015, available online. ↩

62. 62.

    Chris Stephen, Book Review (reviewing Nancy A. Combs, Fact-Finding Without Facts: The Uncertain Evidentiary Foundations of International Criminal Convictions), 22 EJIL 602 (May 2011), available online, doi. ↩

63. 63.

    Caroline Fehl & Eliška Mocková, PRIF Spotlight, Chasing Justice for Syria (May 2017), available online. ↩

64. 64.

    See Beth Van Schaack, Accountability in the Time of COVID-19: Syria & Iraq, Just Security (May 6, 2020), available online; Beth Van Schaack, National Courts Step Up: Syrian Cases Proceeding in Domestic Courts, SSRN (Feb. 2, 2019), paywall, doi

    (discussing many of these recent cases on Syria). ↩

65. Suggested Citation for this Comment:

    Beth Van Schaack, The Fourth Industrial Revolution Comes To The Hague, ICC Forum (Jun. 1, 2020), available at https://iccforum.com/cyber-evidence#Van-Schaack.

    Suggested Citation for this Issue Generally:

    To What Extent Can Cyber Evidence Repositories, and Digital and Open-Source Evidence, Facilitate the Work of the OTP, and the ICC More Generally?, ICC Forum (Jun. 1, 2020), available at https://iccforum.com/cyber-evidence.