In-House Vs. Outsourced Trolls: How Digital Mercenaries Shape State Influence Strategies

1. Digital mercenaries refer to the industry of private digital marketing firms, citizen troll networks, and public relations firms that execute disinformation campaigns. We define a disinformation campaign as a coordinated effort to spread content with the intent to deceive.

2. For example, in April 2021 the U.S. issued sanctions against Russia, including increasing restrictions on the ability of U.S. financial institutions to deal in Russian sovereign debt, in part in response to Russian state-backed information operations (Treasury Sanctions Russia with Sweeping New Sanctions Authority, Citation2021).

3. Unpublished dataset from the Stanford Internet Observatory. Similarly, the Oxford Internet Institute found that there was evidence of private firms running political disinformation campaigns in 48 countries in 2020 and 25 countries in 2019 (Bradshaw et al., Citation2021). This does not imply governments are implementing fewer disinformation campaigns, just that social media companies are finding more operations that can be attributed to digital mercenaries than operations that can be attributed to governments.

4. Facebook attributed posts to the GRU, while Facebook and Twitter attricuted posts to the IRA.

5. Author analysis of post-level datasets of GRU Facebook data.

6. In this time period, Russia had a particularly vested interest in shaping global narratives on Syria as Russia’s intervention in Syria in September 2015 marked the first time since the end of the Cold War that Russia became militarily engaged in an armed conflict outside of the former USSR; and the conflict also presented an opportunity to distract from domestic unrest at home (Rogers & Reeve, Citation2015). Moreover, the Syria crisis also presented an opportunity to highlight Western shortcomings – both militarily in Syria and regarding handling of the refugee crisis.

7. The IRA Twitter operation began focusing on Syria in early 2014 and the IRA Facebook and Instagram operation began discussing Syria in mid-2015. The GRU Facebook operation began in early 2014, but the Syria part of the operation only began toward the end of 2015. For start and end dates of each dataset see Table A1.

8. One such example is the UK’s Information Research Department, a division of the British Foreign Office, which created untrue stories during the Cold War – in one instance creating a fake press release that they attributed to a Budapest-based Communist-supported organization. https://www.bbc.com/news/uk-politics-47571253.

9. For examples see Ponce de León and Suárez Pérez (Citation2020) on Ecuador, Le Roux and Rizzuto (Citation2020) on Egypt, DiResta et al. (Citation2019) on Saudi Arabia, Bandeira et al. (Citation2019) on a digital marketing firm in Israel, and Cryst et al. (Citation2020) on a US public relations firm that managed Latin American disinformation operations.

10. https://home.treasury.gov/news/press-releases/jy0126.

11. In some cases narratives can be laundered across publications intended for specific audiences, but the ability to inflect narratives by audience is blunter than on social media.

12. GRU is an abbreviation from a prior name.

13. DOJ indictment https://www.justice.gov/file/1080281/download.

14. The term “troll” in internet parlance is used to denote participants in online activity who post or comment with the deliberate intent to rile up their audience or disrupt an online community.

15. https://intelligence.house.gov/social-media-content/.

16. The SSCI requested that a collection of technical advisers to the committee assess the data and produce reports on the material therein. Those analyses, as well as extensive interviews, expert testimony, and assessments from the intelligence community and Department of Justice were then incorporated into SSCI’s own comprehensive reports. See e.g., https://www.intelligence.senate.gov/sites/default/files/documents/Report_Volume2.pdf. One of the authors of this paper, Renée DiResta, was a technical adviser with access to the the IRA Facebook and Instagram data, and Stanford Internet Observatory researchers subsequently comprised a team that assessed the GRU Facebook data for research. Due to privacy concerns, portions of the Facebook and Instagram data sets have been released with appropriate redactions (https://intelligence.house.gov/social-media-content/social-media-advertisements.htm). We are not currently permitted to release the full scope of the material used in this analysis publicly. A majority of the data provided by Twitter has since been released to the public via Twitter’s Transparency Center. (See https://transparency.twitter.com/en/reports/information-operations.html).

17. While it would have been ideal to focus on a wider array of topics, we were limited by the fact that the overwhelming majority of the social media takedown concretely attributed to the GRU focused only on Syria. Despite this limitation, the IRA’s Syria work was split across many of its pages, and those pages ran similar strategies for a range of other issues. Additionally, the GRU has run similar narrative laundering strategies on other topics as well. This increases our confidence in describing Russian tactics more broadly, even though our analysis is limited to the Syria case.

18. They had names and biographies: Sophia Mangal (ISMC’s “co-editor”, who sometimes went by Sophie), Anna Jaunger, Said Al-Khalaki, Mehmet Ersoy, and Mariam Al-Hijab.

19. Sometimes they plagiarized from Syrian and Iranian state media.

20. We used 25 topics (k = 25), used the Spectral initialization, and included a date variable as a covariate.

21. We first used the quanteda package to remove English stopwords, punctuation, and urls from our data. We then used quanteda’s textatfrequency() function to identify the top unigrams and bigrams in our data.

22. While this threshold of 100 words is somewhat arbitrary, going further down the list yielded almost entirely irrelevant words across topics.

23. We coded posts that purely contained news headlines as neutral, regardless of whether the headline could be interpreted as positive or negative.

24. We note that other platform features may shape their use, including character count restrictions on Twitter, and norms of long posts on Facebook.

25. These figures are based on the proportion of posts that have any clickbait attributes, not the proportion of clickbait attributes per total characters of text.

26. Per DiResta et al. (Citation2019) it is possible that some of this engagement came from clickfarms.

27. It is unclear to what extent the third-party news sites that accepted these contributions were aware that they were laundering content for fake personas. Some likely did not know, but simply had weak vetting procedures for what they published; some have owned up to being duped and removed the fabricated-persona posts, https://www.counterpunch.org/2017/12/25/go-ask-alice-the-curious-case-of-alice-donovan-2/ while others have left the content on their website despite the link between ISMC and the GRU being revealed.

28. For example http://en.insidesyriamc.com/2018/07/12/breaking-syrian-opposition-and-western-ngos-hire-actors-for-chemical-weapons-provocation/.

29. For example https://insidesyriamcen.wordpress.com/2018/07/12/breaking-syrian-opposition-and-western-ngos-hire-actors-for-chemical-weapons-provocation/. The wordpress URLs provided us with more text from the original articles than the Facebook posts, which only displayed a few sentences.

30. We developed this protocol after observing that sometimes the re-posts changed one or two words from the title or article. After finding what appeared to be a re-post of an ISMC article, we investigated whether the article was in fact taking content from a third domain, SANA, that ISMC occasionally took content from. This was straightforward to assess, as the articles would say something like ”source: SANA,” indicating that the original source was Syrian state media. Those instances are typically not counted as re-posts. In some instances, ISMC changed a few words from the original article, and the article then appeared on other sites with ISMC’s modified text. We do count those as re-posts.

31. There were 2,014 English ISMC article in total, the majority of which were never reposted.

32. CrowdTangle is a social media analytics platform owned by Facebook that tracks public posts on Facebook made by public accounts or groups as well as public interactions (likes, reactions, comments, shares, upvotes) to these posts.

33. To make that more concrete, here is an example ISMC re-post: https://www.globalresearch.ca/eastern-ghouta-do-western-countries-support-terrorists-in-Syria/5632417. We are able to collect the number of times that article was posted to Facebook – both on public and private accounts – and how many interactions it received on Facebook. We collect the following metrics from CrowdTangle: Shares to public Facebook pages and groups: The number of times the URL was posted to Facebook pages and public groups; Total followers of public pages and groups: The total number of followers and members for those pages and groups; Total public and private engagement on Facebook: The number of Reactions, Comments, and Shares on page and group posts that this URL was shared to, in addition to private posts that the URL was shared to.

34. Importantly, we are unable to measure the reach of readership beyond social engagement. It is possible that there is significant consumption of this content happening off of social media platforms that we do not capture here.

35. These include globalresearch.ca and veteranstoday.com. We also see a cluster of African sites – newsghana.com.gh, southafricatoday.net, and gnnliberia.com. An investigation into Southafricatoday.net found that it is a right-wing website run by South African expats in Thailand: https://www.news24.com/news24/southafrica/news/conservative-website-traced-to-sa-father-and-son-in-Thailand-20180615. Gnnliberia.com removed the ISMC content after being alerted to its links to Russia. Newsghana.com.gh is run by a Ghanaian man. Another site that reposted ISMC content, media.mehrnews.com, is Iranian state media.

36. We only conducted the reposting analysis on the GRU dataset, as the IRA did not engage in similar article re-posting tactics.

37. Attribution in these and other operations is challenging and imprecise, making it difficult, using takedowns to date, to test our hypotheses with a larger dataset.