Defending in the age of open source digital user-generated evidence

By Michael G. Karnavas and Noah Al-Malt

Recently we had the opportunity to attend a panel discussion at the Leiden University Grotius Center for International Legal Studies on Filling an Accountability Gap? How a Standing UN Investigative Mechanism Would Further International Justice. Bluntly, it seemed more like a promotional event for the International Commission of Jurists’ (“ICJ”) concept of establishing a Standing Independent Mechanism (“SIIM”) as set out in its September 2022 report: “Options for the establishment of a Standing Independent Investigative Mechanism (SIIM).”

The panelists (but for one appearing remotely) were all in The Hague for the International Criminal Court (“ICC”) Assembly of State Parties, which, with all the side events, has turned into an annual convention for networking and promotion – a regular law-fest. Kudos to the Grotius Center for taking advantage of the presence of the panelists and organizing the discussion on the benefits, needs, and presumptions associated with establishing a SIIM. Noteworthy, the proponents for a SIIM did not argue that the either the International, Impartial and Independent Mechanism for Syria (“IIIM”) or Independent Investigative Mechanism for Myanmar (“IIMM”) – both of whom heavily rely on digital evidence of all sorts – were underperforming because of the lack of a SIIM.

Maybe there is merit in establishing a SIIM to support the work of other accountability mandates (we think not, for a host of reasons). But isn’t the real elephant in the room how to ensure quality control of open source digital user-generated evidence, from collection to preservation to admission to assessment to reliance. As practitioners, we think so.

Gathering evidence from the net is the easy part (thanks to Google, YouTube, Facebook, Instagram, Twitter, and other search engines and social media outlets). The conundrum is in figuring out what is real and what is fake, what is accurate and what is misleading, what is legitimate information and what is misinformation (or even disinformation).

A trial, stripped to its essence, can be reduced to two basic precepts: getting good evidence admitted and preventing bad evidence from being admitted. Of course, strategic thinking, case preparation, investigation, skillful examination of witnesses, astute motion practice, and proficiency in rhetoric and the art of persuasion, are also involved. Intrinsically, they tie in to the two fundamental precepts of what a trial is about – whether prosecuting or defending or judging. Recognizing that there are a host of issues to consider when it comes to the admissibility and use of open source digital user-generated evidence in court, we take this opportunity to share our thoughts. Spoiler – much of what we have to say applies to pretty much all evidence: it all comes down to foundations.

Open source digital user-generated evidence in a nutshell  

Open source investigation is the practice of collecting and analyzing information gathered from publicly available sources – including print media, internet sources such as social media websites and blogs, public government data, professional and academic publications, commercial data, and so on – to produce actionable information or evidence.((European Commission, Open-Source Intelligence, 2 May 2022.)) Digital open source information (i.e., public information in digital format) includes both “machine-generated” and “user-generated” data.((Berkley Protocol on Digital Open Source Investigations, para. 1.)) Curt Monash, a leading analyst and strategic advisor to the software industry, defines machine-generated data as “data that was produced entirely by machines or data that is more about observing humans than recording their choices,” such as network logs, satellite imagery, location data such as GPS, and call data records.((Curt Monash, Examples and Definition of Machine-Generated Data (italics in original) (last visited 8 February 2023). )) Digital user-generated evidence, by contrast, is “information recorded by an ordinary citizen,” such as photographs, video, and text (such as social media posts) uploaded to the internet for the purpose of helping achieve legal accountability for crimes.((Trust in User-Generated Evidence, Analysing the Impact of Deepfakes on Accountability Processes for Human Rights Violations (last visited 8 February 2023); Rebecca Hamilton, User Generated Evidence, 57 Columbia J. Transitional L. 1, 3 (2018). ))

With the proliferation of smartphones with audio-visual recording capabilities, the current debate on open source investigations has primarily focused on digital user-generated evidence (for example, see here, here, and here). Now, millions of people around the globe carry these devices, uploading photographs and video footage of crimes being committed onto the internet. Simply type “Syria,” “Ukraine,” “Myanmar,” “Iran,” or “Syria” into Google’s image search bar and you will find a seemingly immeasurable catalogue of images showing acts worthy of criminal prosecution.

Civil society organizations, such as eyeWitness to Atrocities, have created apps where ordinary citizens can upload information such as photographs and images.((eyeWitness, FAQs (last visited 8 February 2023) )) eyeWitness’s app will automatically collect GPS coordinates, date and time indicators, and encrypt the information so that it cannot be edited or altered in any way. Users of the eyeWitness app also have the option to remain anonymous or provide their contact details. The hope or expectation, perhaps, is that user-generated evidence collected through the app will be admissible in international criminal trials without having to identify the user or have him or her testify in court.((Rebecca Hamilton, User Generated Evidence, 57 Columbia J. Transitional L. 1, 4 (2018). )) This would significantly impact on the fair trial rights of suspects and accused, potentially turning trials into a parody, a charade, a pretense.

Open source digital user-generated evidence is a reality. Indeed, it is virtually inescapable in mass atrocity trials – despite its inherent, though not necessarily fatally flawed, characteristics. Chambers will be reluctant to outright dismiss this type of evidence, so accept the fact that it will be admitted. But so what? The real issue is not the admission of this evidence, but the weight accorded to it. And here is where evidentiary foundations come in.

Evidentiary foundations

Evidentiary rules and principles assist a Trial Chamber in admitting and assessing the evidence, and, invariably, in making findings of facts to which the applicable law is applied in reaching legal conclusions. Simply put, evidentiary rules and principles inform the Trial Chamber what evidence should be admitted, what evidence should be excluded, and how much weight if any should be given to the evidence that is admitted.

International(ized) criminal courts and tribunals generally apply the “principle of free evaluation of evidence,” which favors the admissibility of evidence.((International Criminal Tribunal for the former Yugoslavia Rules of Procedure and Evidence (“ICTY RPE”), Rule 89; International Criminal Tribunal for Rwanda (“ICTR”) RPE, Rule 89; ICC RPE, Rule 63(3); Extraordinary Chambers in the Courts of Cambodia (“ECCC”) Internal Rules, Rule 87; Special Tribunal for Lebanon (“STL”) RPE, Rule 149(C); Prosecutor v. Stakić, IT-97-24-PT, Provisional Order on the Standards Governing the Admission of Evidence and Identification, 25 February 2002, p. 4. )) The Rules of Procedure and Evidence of these courts and tribunals do not regulate the admissibility of evidence in detail, but rather contain general principles of evidence. Normally, the Trial Chambers will adopt guidelines that specify and govern the admissibility of evidence.((See e.g., Prosecutor v. Gicheru, ICC-01/09-01/20-189, Directions on the Conduct of Proceedings, 7 October 2021.)) Under the ICC’s statutory regime, the Trial Chambers are free to adopt whatever procedure they feel is most suited for the expeditiousness of the proceedings.((Rome Statute, Art. 64; ICC RPE, Rule 64.)) Some Trial Chambers opt for a more common law approach where evidence is screened up front as it is being introduced (albeit loosely and seemingly with a presumption for admission).((Prosecutor v. Katanga and Ngudjolo, ICC-01/04-01/07-2635, Decision on the Prosecution’s Bar Table Motions, ICC-01/04-01/07-2635, 17 December 2010, para. 13.))  Other Trial Chambers prefer the civil law approach where the screening of the evidence and its assessment for relevance, authenticity, reliability, and probative value are made after hearing all of the evidence.((Prosecutor v. Bemba, ICC-01/05-01/08-1022, Decision on the admission into evidence of materials contained in the Prosecution’s list of evidence, 19 November 2010, para. 35.))

Given that the triers of facts at the international(ized) criminal courts and tribunals are “professional judges” and not lay persons, the expectation is that the judges will scrupulously analyze the facts and unaffectedly apply the evidentiary rules and principles. Effectively, it matters not which approach a Trial Chamber takes to admitting the evidence. What does matter, however, is that before any evidence is introduced with a view to its eventual reliance in determining facts or matters relevant to the outcome of the case, foundational evidence is introduced. This is especially important when dealing with digital user-generated evidence. This preliminary (predicate) evidence is necessary to show that the material evidence is authentic, reliable, accurate, trustworthy, and relevant. The type of preliminary evidence necessary to lay the proper foundation depends on the form and type of material evidence offered. An objection is likely to be made if the proper foundation is lacking.

Generally, a foundation must be established to show that the evidence is:

• • Relevant – it has a tendency to make the existence of a fact or consequence more probable or less probable than without the evidence;((Prosecutor v. Katanga and Ngudjolo, ICC-01/04-01/07-2635, Decision on the Prosecution’s Bar Table Motions, ICC-01/04-01/07-2635, 17 December 2010, para. 16; ICC Statute, Art. 69(4), ICC RPE, Rule 64(3). See also ICTY RPE, Rule 89(C); ICTR RPE, Rule 89(C); STL RPE, Rule 149(C). ))
  • Authentic – the matter or thing in question is exactly what it is claimed to be (for example, a knife, a gun, a video, etc.);((Prosecutor v. Katanga and Ngudjolo, ICC-01/04-01/07-2635, Decision on the Prosecution’s Bar Table Motions, ICC-01/04-01/07-2635, 17 December 2010, para. 22. See also ICTY RPE, Rule 89(E); ICTR RPE, Rule 89(D); STL RPE, Rule 149(E). )) and
  • Reliable – the evidence is in the same condition as when it was used or located (e., it was not tampered with or altered), can be independently verified or tested (i.e., it is not based on untestable and uncorroborated hearsay), and/or has other indicia of reliability considering the source of the evidence, its nature and characteristics, the purpose for which it was generated, and so on.((For a list of factors normally considered by ICC Chambers in assessing the reliability of evidence, see Prosecutor v. Katanga and Ngudjolo, ICC-01/04-01/07-2635, Decision on the Prosecution’s Bar Table Motions, ICC-01/04-01/07-2635, 17 December 2010, para. 27.))

All three criteria should be satisfied before a tangible piece of evidence (as opposed to viva-voce testimonial evidence) can be received in evidence for the Trial Chamber’s consideration. For each tangible piece of evidence (such as a printout of a social media post or video), the proponent should satisfy, through the testimony of one or more witnesses, each of the three criteria by laying the foundation necessary for its acceptance as evidence.

Examples of laying a foundation

    Photograph

• • • Representation of the scene at the time of the incident
    • True
    • Accurate
    • Fair

Voice

• • • Prior knowledge of the voice
    • Ability to hear
    • Recognition of the voice

While the ICC Trial Chambers are not required to make a separate ruling on the admissibility of evidence up front and are permitted to defer their admissibility assessment after hearing all the evidence, Rule 64(1) of the ICC Rules of Procedure and Evidence requires that “[a]n issue relating to the relevance or admissibility must be raised at the time when the evidence is submitted to a Chamber.” Thus, while objections related to the foundations of the evidence may not be sufficient to render the evidence inadmissible, they can affect the Trial Chamber’s determination of the weight accorded to the evidence.

Proffering & objecting to the admission of open source digital user-generated evidence

One primary difficulty of proffering (and objecting to the admission of) digital user-generated evidence is that the source of the information (such as the original poster on social media) may be anonymous. According to eyeWitness to Atrocities Program Director Wendy Betts, “[t]here’s a valid legal argument to make that the app itself is the witness.”((Interview with Wendy Betts, cited in Rebecca Hamilton, User Generated Evidence, 57 Columbia J. Transitional L. 1, 46-47 (2018) (italics in original). )) However, while the eyeWitness app may include vast amounts of metadata (GPS coordinates, cell tower data, etc.) and establish a secure chain of custody so that the data is not compromised, the app cannot establish the primary criterion of foundations: reliability. As the saying goes, garbage in, garbage out. If unreliable and untrustworthy data is uploaded into the app, neither the vast amounts of metadata nor secure chain of custody can render the unreliable evidence reliable.

Also, without having the original poster come to testify (and be subjected to robust cross-exanimation), the content of the information provided by the anonymous witness is unverifiable anonymous hearsay, which is generally accorded low probative value.((Prosecutor v. Bemba, ICC-01/05-01/08-424, Decision Pursuant to Article 61(7)(a) and (b) of the Rome Statue on the Charges of the Prosecutor Against Jean-Pierre Bemba Gombo, 15 June 2009, para. 50. )) For instance, in expressing their dissatisfaction with the prosecution’s use of anonymous evidence recorded in NGO reports and press articles, the Gbagbo Pre-Trial Chamber considered:

Proving allegations solely through anonymous hearsay puts the Defence in a difficult position because it is not able to investigate and challenge the trustworthiness of the source(s) of the information, thereby unduly limiting the right of the Defence under article 61(6)(b) of the Statute to challenge the Prosecutor’s evidence, a right to which the Appeals Chamber attached “considerable significance”. Further, it is highly problematic when the Chamber itself does not know the source of the information and is deprived of vital information about the source of the evidence. In such cases, the Chamber is unable to assess the trustworthiness of the source, making it all but impossible to determine what probative value to attribute to the information.((Prosecutor v. Gbagbo, ICC-02/11-01/11-432, Decision Adjourning the Hearing on the Confirmation of Charges Pursuant to Article 61 (7) (c) (i) of the Rome Statute, 3 June 2013, para. 29. ))

Corroboration, according to the Gbagbo Pre-Trial Chamber, would also “be difficult, if not impossible, to determine whether and to what extent anonymous hearsay in documentary evidence corroborates other evidence of the same kind.”((Id., para. 30.)) This is because it will be difficult to determine whether to or more anonymous sources are truly independent of each other (especially with numerous groups operating/investigating in situ) – and the Chamber should not be speculating in making its findings.((Id.)) Acknowledging that in exceptional circumstances anonymous evidence may be corroborated by independent sources, the Gbagbo Pre-Trial Chamber considered that it “may still not have enough information about the trustworthiness of these sources” and thus must be “extremely cautious in attributing the appropriate level of probative value.”((Id.))

Another difficulty in objecting to the use of such evidence concerns challenges to the method of collection: i.e., whether any data collected may have been altered, deleted, manipulated, or compromised in any way thus rendering it unreliable. The ICC Office of the Prosecutor (“OTP”) has a Cyber Unit staffed with experts and investigators trained in digital forensics to explain away the technicalities. Gratefully, the OTP has put together an impressively useful virtual training on Digital Forensic Capabilities and Presentation in Court and was generous enough to have Yvan Cuypers of the OTP Cyber Unit explain it all in a recent training organized by Counsel Support Services (“CSS”). But this is not enough. Not if the defence is to be fully due diligent as required by Article 5 of the ICC Code of Professional Conduct for Counsel (as well as the national codes of ethics and professional responsibility to which defence counsel are bound), the defence will need its own expert to challenge the prosecution’s foundations. Hiring an expert with the right qualifications comes at a high cost. With virtually all suspects and accused at the ICC being indigent and reliant on CSS for providing adequate resources for a defence – presumably amounting to equality of arms – query whether the defence will have access to qualified experts in digital forensics.

Providentially, there are two excellent guidelines available, which, in our opinion, should be the advocate’s first port-of-call when dealing with digital user-generated evidence. Though they have not been formally adopted by the ICC or other international(ized) criminal courts and tribunals, they are thorough and comprehensive. Whether you are in the process of collecting digital user-generated evidence for future judicial proceedings or seeking to have such evidence admitted or rejected (or afforded no weight), we advise you consult them.

Best practices guidelines

The Berkley Protocol on Digital Open Source Investigations and the Leiden Guidelines on the use of Digitally Derived Evidence were developed to assist practitioners and civil society understand the essential elements which should be considered when conducting digital open source investigations and in submitting the results for use in international criminal trials. The two documents have distinct aims and coverage.

Primarily aimed at those conducting open source investigations, the Berkley Protocol provides standardized methodologies for collecting and securely storing evidence, in particular, to ensure that open source investigators understand the importance of:

• • Tracing the provenance of online content and attributing it to its original source, where possible;
  • Evaluating the credibility and reliability of online sources;
  • Verifying online content and assessing its veracity and reliability;
  • Complying with legal requirements and ethical norms;
  • Minimizing any risk of harm to themselves, their organizations and third parties;
  • Enhancing protection of the human rights of sources, including the right to privacy.((Berkley Protocol on Digital Open Source Investigations, para. 10.))

The Berkley Protocol also sets out guiding principles to which those conducting open source investigations should adhere:

• • Accountability – ensuring that open source investigators are transparent and accountable for their actions, such as clear documentation, record-keeping, and oversight.
  • Competency – ensuring that open source investigators have the proper training and technical skills to carry out their investigations in a professional and ethical manner.
  • Objectivity – a principle that applies to all investigations, whether online or offline, under which investigators must understand the potential for personal, cultural, or structural biases that affect their work, for example, by deploying multiple working hypotheses and not favoring any particular theory to explain a case.
  • Legality – ensuring that open source investigations comply with the applicable laws such as data protection and the internationally recognized human right to privacy.
  • Security awareness – ensuring that individuals conducting open source investigators have basic operational security awareness to ensure that they minimize their digital trail and avoid potential risks to themselves, their sources, or others.((Berkley Protocol on Digital Open Source Investigations, paras. 25-29.))

By contrast, the Leiden Guidelines are intended for legal practitioners, “comprehensively outlining the essential elements which should be considered before submitting [digitally derived evidence] to an international court or tribunal.”((Leiden Guidelines on the use of Digitally Derived Evidence, Introduction.)) Acknowledging that each international(ized) criminal court and tribunal has their own procedural and evidentiary rules, the Leiden Guidelines draw primarily from the ICC’s approach to the admissibility of evidence as expressed by the Bemba Trial Chamber:

[F]or an item to be admitted into evidence it must satisfy the three-part test under which it must (i) be relevant to the case; (ii) have probative value; and (iii) be sufficiently relevant and probative as to outweigh any prejudicial effect its admission may cause. Further, [the] determination on the admissibility into evidence of an item has no bearing on the final weight to be afforded to it, which will only be determined by the Chamber at the end of the case when assessing the evidence as a whole.((Prosecutor v. Bemba, ICC-01/05-01/08-2721, Decision on the admission into evidence of items deferred in the Chamber’s “Decision on the Prosecution’s Application for Admission of Materials into Evidence Pursuant to Article 64(9) of the Rome Statute” (ICC-01/05-01/08-2299), para. 9. ))

After providing definitions of relevance, probative value, and prejudice in the introduction, the Leiden Guidelines go on to provide succinct summaries of jurisprudence from the various international(ized) criminal courts and tribunals concerning the admission and use of specific types of evidence: videos, photographs, aerial and satellite images, intercepts, call data records, and audio recordings. For example, with video evidence, the Leiden Guidelines provide the following practice points:

• • A1. Instead of excerpts, videos should be submitted in their entirety.
  • A.2 A video and its associated transcripts and translations must be seen as forming integral parts of the same evidence.
  • A3. Videos not in a working language of the Court should be translated into one of the working languages of the Court and made available to the Chamber and all parties within the time limit fixed by the Chamber.
  • A4. When a witness appears on a video that the party intends to tender into evidence, the video should be tendered through the witness during the examination-in-chief and not through the bar table.
  • A5. The Court can make an inference from the content of a video to the extent that it allows the Court to make a definite finding.
  • A6. Videos can be admitted into evidence if relevance and prima facie authenticity is demonstrated by providing information about the date, the location, the events depicted, the author, the source, and/or the chain of custody.
  • A7. Video evidence of interviews conducted during an armed conflict by a party to the conflict may not be objective and reliable and therefore low probative value may be attached to the video.
  • A8. The consent of witnesses and others affected by the work of the Court whose image is depicted in video evidence is required.((Leiden Guidelines on the use of Digitally Derived Evidence, A. Videos.))

When it comes to digital user-generated evidence, however, the Leiden Guidelines are scant, noting that while digital user-generated evidence such as social media posts have been admitted in ICC proceedings, “none of the Chambers discussed the admissibility of social media evidence or specific evidentiary requirements and as such, no authoritative guidelines could reasonably be deduced or formulated.”((Leiden Guidelines on the use of Digitally Derived Evidence, Introduction.)) Perhaps this is because the ICC Chambers (and other international(ized) criminal courts and tribunals) have treated digital user-generated evidence just like any other piece of evidence.

For example, the Bemba et al. Trial Chamber admitted screenshots of publicly available Facebook profiles, which were relevant to establishing the relationships between the accused and witnesses.((Prosecutor v. Bemba et al., ICC-01/05-01/13-1498, Public redacted version of the “Prosecution’s Fifth Request for the Admission of Evidence from the Bar Table”, 27 November 2015, ICC-01/05-01/13-1498-Conf, 30 November 2015, para. 17.)) It considered that the documents were open source material from Facebook and thus prima facie authentic and reliable: the authenticity and reliability of the documents was corroborated by their general appearance, bearing the indicia that they originated from Facebook, including the Facebook logo, the layout of the webpage in the screen shot, and its structure.((Id.)) Further, their veracity and reliability was independently corroborated by other evidence in the case, including the testimony and statements of three witnesses.((Id., para. 18.))

In another instance at the Special Court for Sierra Leone, the defence was permitted to confront a witness with a social media post, having laid a foundation:

Q. Do you recognise this as being a Facebook page, Ms White?

A. I do.

Q. Do you see just below the photos –

Q. “Jeanna Ridout, what an inconvenience. Friday at 11.05 a.m.” 11.05 a.m. The UK is one hour behind The Hague, so that’s 10 o’clock here in The Hague. So at that time would your employees be in the office?

A. 11.05 on Friday morning, yes.((Prosecutor v. Taylor, SCSL-2003-01-T, Transcript, 9 August 2010, pp. 45783, 45788.))

Approaching open source digital user-generated evidence as any other type of evidence

The long story short is that despite the technicalities involved in collecting, storing, and using digital-user-generated evidence, it is no different than any other piece of evidence in terms of admissibility and assessment in court.

In introducing any digital user-generated evidence, it is incumbent upon the advocate to lay a foundation to show that the evidence in question is relevant, authentic, and reliable in order for any weight to be accorded to it. This may require an expert to demonstrate and explain how the evidence was collected and stored, including what measures were taken to minimize the risk of the data being altered or compromised. It may also require having the source of the material in question testify in court. While this entails security concerns for the source, there are mechanisms to ensure witness security such as the possibility of testifying under pseudonym and relocation.

As for the advocate objecting to the use of digital user-generated evidence (or any evidence being submitted for that matter), objections must be made prior to the testimony of any witness who is going to introduce and use the evidence during his or her testimony. Although this advice is very adversarial in nature and limited to common law jurisdictions, if the advocate is unsatisfied with the foundation that has been laid, he/she should seek leave to question the witness in the form of a voir dire limited to the issues of foundation prior to the witness using the evidence in his or her testimony. While leave for voir dire is unlikely to be granted at the ICC (especially by civil law-trained judges), it merits at least an attempt to highlight the foundational concerns. In other words, do not wait until cross-examination. During cross-examination, the advocate should also deal with foundations as the preliminary issue. Even if the advocate were granted the ability to do a limited voir dire, this should not prevent him or her from supplementing at the beginning of cross-examination, provided that it is not repetitious.

Parting remarks

In this new brave world of digitalization and artificial intelligence – such as the text-generating ChatGPT, AI-synthesized “deepfakes”, and who knows what’s next in technological advancement – digital evidence will increasingly be challenged in arguing for or against the admission of evidence whose provenance, authenticity, and reliability is difficult to discern to a high level of certainty (beyond reasonable doubt). At the end of the day, however, the basic evidentiary rules and principles of foundations relevant to ordinary evidence are just as applicable – indeed, the nettles that must be grasped. The Berkley Protocol and Leiden Guidelines are useful, but without a deep and abiding understating of evidentiary foundations, the advocate is less likely to fully appreciate them.

For an excellent primer on foundations, we highly recommend Edward J. Imwinkelried, Evidentiary Foundations, Twelfth Edition. While it primarily relates to the United States Federal Rules of Evidence, the principles related to evidentiary foundations are relevant irrespective the legal tradition, be it civil law, common law, or hybrid.

About Author

Michael G. Karnavas

Michael G. Karnavas is an American trained lawyer. He is licensed in Alaska and Massachusetts and is qualified to appear before the various International tribunals, including the International Criminal Court (ICC). Residing and practicing primarily in The Hague, he is recognized as an expert in international criminal defence, including pre-trial, trial, and appellate advocacy.

See author's posts

Noah Al-Malt

Noah Al-Malt is an American attorney licensed in the State of New York. For the past nine years, he has been working as a member of defence teams before the International Criminal Tribunal for the former Yugoslavia, Extraordinary Chambers in the Courts of Cambodia, and International Criminal Court.

Author: Michael G. Karnavas

Michael G. Karnavas is an American trained lawyer. He is licensed in Alaska and Massachusetts and is qualified to appear before the various International tribunals, including the International Criminal Court (ICC). Residing and practicing primarily in The Hague, he is recognized as an expert in international criminal defence, including pre-trial, trial, and appellate advocacy. View all posts by Michael G. Karnavas