Introduction
‘News outlets’ named ‘Media Site,’ ‘World News’, and ‘Breaking News’ [1] are the next generation of platform exploitation tactics by Islamic State supporters currently laundering official Islamic State media for larger audiences. They are branded news outlets that can hoodwink moderators and audiences into falling for an elaborate ruse while continuing to be a successful frontline of the propaganda war by the Islamic State media mujahideen.
Extremists exploiting platforms for their own ends and learning along the way is a tale as old as the internet and one that has become even more pronounced in the era of ubiquitous access to social media. Over the past three years, a set of exploitation and evasion tactics have become central for Islamic State supporters online, and they are only getting more elaborate. It has been documented that at one point, Islamic State supporters were using third-party applications to acquire digital phone numbers linked to real users on platforms such as Facebook to lock them out of their accounts and repurpose them for propaganda dissemination. Now, Islamic State supporters are developing stand-alone brands for unaffiliated news outlets to pump out whitewashed Islamic State content to thousands across multiple platforms. The outlets have their own editorial standards, branding, and regional expertise.
Understanding both old and new tactics and strategies by Islamic State supporters online is key to anticipating the next moves by these terror groups and their supporters and how they adapt to innovations in moderation by tech companies. This Insight will provide a historical overview of Islamic State supporters’ platform exploitation and evasion tactics and highlight gaps in platform moderation and current research into Islamic State ecosystems on platforms such as TikTok, Facebook, X [formerly known as Twitter], and Telegram. It will focus on what has been termed the Islamic State supporters’ ‘exploitation and evasion toolbox’ and five new tactics that continue to confound platform moderation. These strategies are becoming a staple of terrorist support ecosystems, underlining the lengths supporters of the Islamic State will go to evade platform moderation and how adept they have become at eluding takedowns and spreading propaganda unencumbered.
Understanding the Islamic State Supporters’ Exploitation and Evasion Toolbox
In 2020, research by the Institute for Strategic Dialogue mapped out six central content moderation evasion and platform exploitation tactics used by Islamic State supporters. The research was then corroborated in a brief article in the group’s weekly al-Naba newsletter published a week after the report’s release in July 2020. Those six tactics continue to be a mainstay of the Islamic State platform exploitation, ingrained into the modus operandi of the group’s supporters who continue to hack accounts, mask their content, link share, hijack hashtags, coordinate raids, and game text analysis efforts. Three years on, there are now five new key strategies and tactics used by Islamic State supporters central to their content surviving on platforms attempting to remove it. These tactics are part of an all-out strategy to evade moderation that evolves with time: content delivery and content creation.
Content delivery requires significantly more investment in developing mechanisms such as the branding and concepts behind ‘independent’ news outlets and media personalities suited for a specific region of influence. For Iraq, Islamic State supporters online developed a brand for news outlets called ‘Media Site’, named after on-the-ground Islamic State propaganda dissemination sites in Mosul and elsewhere, and adapted for distributing propaganda on Facebook. The same outlets developed a 42-emoji code system to get around text-based moderation. A key was distributed to followers to get around the moderation of keywords such as ‘killing’, ‘ambush’, and ‘rafidah’ [rejectionists].
Content creation, on the other hand, relies on low-cost innovative measures such as ‘scribbling’ and picture-in-picture content to evade both artificial intelligence and manual moderation efforts. The content delivery evasion toolbox contains tactics like overlaying audio on unrelated videos, the distortion of video content, and new branding not tied to official Islamic State outlets or the 60+ Islamic State support groups operating in more than 23 languages globally. Some of these unaffiliated brands include their own logos and symbols, such as ‘Glory’, which posts daily Islamic State news on Facebook to an audience of 1,100 followers, often using a ‘Breaking News’ digital poster stripped directly from an Islamic State bulletin (Fig. 1). Banned on X, Telegram and LinkTree, ‘Glory’ continues to hold onto its presence on Facebook and the website development site Wix.
Fig. 1: ‘Glory’ branding from their ‘news outlet’
Content Delivery: News as Terrorist Propaganda
The primary tactic and strategy of Islamic State supporters in this modern era is the use of ‘independent’ news outlets, all of which are unaffiliated with known Islamic State brands or support groups. These outlets exploit social media platforms’ inability to verify news organisations. The outlets mask their support for the Islamic State by using the word ‘Daesh’ to describe the group, eschewing the notion that referring to the Islamic State as Daesh neither frustrates nor insults its supporters. Considered a diminutive term, supporters now embrace it as a source of pride. Similarly, in closed spaces such as Telegram, the supporters behind the news outlets also welcome the use of ‘terrorist’ as a badge of honour, drawing on the content of an Abdallah Azzam speech in which the journeyman jihadist of the 1980s self-identified with the term.
In 2022, ISD researchers tracked 38 Islamic State news outlets across Telegram, Facebook, TikTok, and X [formerly Twitter]. All were flagged and removed by a number of these platforms, only to be reinstated again within days and flourishing once more within months. These news outlets have had a significant impact beyond their follower bases; videos shared on Facebook by Islamic State news outlets garnered more than 1 million views between 2021 and 2022. This figure is over 20 times their collective follower base, indicating that a significantly larger and broader audience views content shared by Islamic State news outlets. The strategic goal of these outlets is three-fold:
Resilience
The primary function of pro-Islamic State news outlets is to spread Islamic State’s version of current events on as many popular platforms as possible. The outlets represent a quasi-coordinated disinformation operation of key nodes branded as media outlets or personalities. These outlets deliberately provide false information and covertly spread rumours to influence public opinion or obscure the truth. These outlets act as disinformation mechanisms that can evade platform takedowns and moderation by being self-branded, stand-alone news operations claiming to provide independent news focused on Iraq, the Middle East and Africa. These outlets also target specific areas or populations with an Islamic State footprint, such as Somalia, Congo, Mali, Nigeria and Mozambique. By playing a fulcrum-style role in the wider disinformation landscape in Iraq, Islamic State news outlets can openly share polarising propaganda to wider audiences than their official news outlets, which are targeted by technology companies and relegated to niche platforms and messaging applications.
Creation
They also develop the narrative ammunition for networks of supporters attacking governments for their stances toward Sunni populations, civil society, and media in the Middle East and Africa. These disinformation operatives and outlets cast doubt on the legitimacy of official government information and mainstream media outlets in Iraq but similarly focus on regional and global news too. By undermining official government accounts of Islamic State skirmishes and overplaying the Islamic State’s perspective, the effect of this tactic is specifically to overinflate the threat of the Islamic State in these regions.
Contestation
Finally, these news outlets combat anti-Islamic State narratives from governments and the mainstream media by providing alternative and localised narratives that portray the global Sunni population as marginalised, discriminated against, and subjected to state violence at the hands of regional governments that are supported by Western states. The rationale behind this framing is to compel audiences to attack state institutions and security services. Often based on mainstream media accounts of violence or intimidation of Sunni populations in Iraq and beyond, these outlets are feeding on real-world injustice to spur violent action from their audiences.
Content Creation Tactics: Scribbling Away and Overlaying Affiliation
Scribbling
Altering, adjusting, and embedding terrorist content into non-terrorist content are content creation and evasion tactics that are central to the Islamic State supporter’s platform exploitation repertoire. At the heart of these evasion tactics is the altering of branded terrorist content to evade automated detection of official videos linked to the Islamic State. Scribbling is the process of drawing marks over the branding on a piece of terrorist video content (Figs. 2 & 3).
Fig. 2 & 3: Examples of scribbling over official Islamic State content
Video-in-Picture Content
‘Video-in-picture’ content is another simple, effective, low-cost way to avoid online content moderation efforts. By embedding video into an image of a television, for example, supporters can still disseminate video propaganda online. This tactic allows the offending video to be embedded in non-violative content, making it harder for moderation systems to detect. Al-Qaeda affiliates such as al-Shabaab have similarly learned how to use this technique to evade moderation (Figs. 4 & 5).
Fig. 4 & 5: Examples of the ‘video-in-picture’ technique to spread propaganda
Audio Overlaying
Islamic State supporters are acutely aware of the shortcomings of platforms’ content moderation policies regarding audio content. Audio, therefore, plays an equally important role to video in the Islamic State’s content creation process. By overlaying the official narration of an Islamic State video onto an unaffiliated piece of content, such as nature videos, Islamic State supporters can quickly and effectively game moderation efforts. This method means that content can remain on mainstream platforms for weeks or months. A prime example of this was the use of the al-Battar Media Foundation audio version of the al-Naba newsletter article ‘Practical Steps to Fight the Jews’, released by an Islamic State support group to incite attacks on Jewish communities and targets during the conflict between Hamas and Israel. The audio of the Islamic State article was overlaid on a video montage of a sunset rather than any readily identifiable Islamic State content.
On TikTok, for instance, users can create ‘original sounds’ using their own audio, which other users can repurpose. In the Islamic State supporter TikTok ecosystem, the original speeches by the now-dead spokesman Abu Muhammad al-Adnani are readily available and are often repurposed to accompany seemingly innocuous video content of nature scenes or fast cars. One 28-second video of an al-Adnani speech with more than 14,000 views on TikTok was used as an original sound for 433 other videos, illustrating the ease with which a single piece of terrorist content can taint a platform’s ecosystem.
The author found 168 public Facebook pages sharing the on-air programs from the now-defunct Islamic State al-Bayan FM radio station. These pages have a collective followership of 34,749 and brand their materials as ‘al-Bayan Radio’, making no attempt to hide their affiliation with the Islamic State. Over the past year, these 168 pages have had over 120,000 views of their 2,538 videos – all audio from former programs on the al-Bayan radio station.
Alternative Branding
Similar innovations to exploit platforms by Islamic State supporters include using alternate branding for established Islamic State media outlets. The al-Naba newsletter is a recognisable brand on the radar of almost all major platforms as violative of their ‘dangerous individuals and organisations’ policies. Islamic State supporter networks understand this dynamic and have developed alternative branding for al-Naba, referring to it as the “newspaper of record for Muslims” and creating alternative logos for the content. This tactic has been used on Facebook and X, illustrating its success in getting around moderation efforts. The author found the latest issue of al-Naba and 14 back issues with alternative branding shared through a ‘news outlet’ operating on Facebook, demonstrating the success of this tactic and its ability to evade moderation.
These methods for creating content that can evade moderation efforts often go hand in hand with the content delivery mechanisms described in this Insight. Islamic State supporters’ ability to deploy content delivery and creation mechanisms to evade takedowns and expand their influence while remaining on platforms is not just a single-platform problem but a perennial multiplatform challenge.
Platform Recommendations: Stemming Exploitation
The Islamic State supporter platform exploitation toolbox poses a serious challenge to the health of the overall information ecosystem online in the Middle East, Africa and beyond. Narratives espoused by Islamic State supporters undermine governments, technology companies and wider civil society by suggesting they are puppets of Western states and incompatible with the Islamic State’s ideology. They are often coupled with sectarian tropes focused on the real and perceived injustices faced by Muslims at the hands of various governments and their security forces. Their capacity to camouflage as news outlets requires a deeper understanding of Islamic State networks online and how they mobilise to survive in the face of increased moderation. Practical steps are needed to deal with these challenges:
First, there is a clear need for more expert-led moderation and investigative tracking of accounts and their spheres of influence. This should come with greater investments in Trust and Safety divisions in smaller or less profitable markets. The ability of these support networks to survive takedowns necessitates in-depth tracking on all platforms in question and the use of subject-matter, linguistic and contextual expertise in several regions, such as East, Central, and West Africa.
A model of cross-platform network coordination around content removal should be developed for these networks rather than relying on piecemeal takedowns that focus on single-actor behaviours or single platforms. Technology companies can demonstrate the same coordination as Islamic State supporters through entities such as the Global Internet Forum to Counter Terrorism (GIFCT). Equally, Europol demonstrated this possibility in 2019 when it targeted Islamic State channels in a wide-ranging, multiplatform takedown effort.
Issues persist regarding the role of artificial intelligence in moderation efforts. To enhance the capabilities of artificial intelligence to recognise content that has been ‘scribbled’, systems should be trained on ‘scribbled’ content that has been previously flagged and identified via a manual review. Further, creating mechanisms by which companies can detect variations in language that allow supporters to continue to publish inflammatory content is an important endeavour.
While platforms such as Facebook and X continue to play pivotal roles in assisting independent media outlets globally, Islamic State news outlets are disseminating terrorist content and disinformation under the guise of independent and ‘objective’ news sources. Unlike independent journalists, outlets, and civil society groups, however, these outlets are only focused on Islamic State actions and objectives.
Taken together, these platform evasion and exploitation tactics and strategies by the Islamic State seem to indicate that the current moderation strategies of platforms are ill-equipped to deal with a much more nimble-footed set of users adapting to platform moderation techniques. What is needed is a cross-platform acknowledgement of these strategies and an adjustment in how moderating these Islamic State support communities can inhibit further support in an era where they evade and exploit platform lapses to remain and expand.
[1] All outlets’ names have been altered throughout the piece to avoid driving traffic to the accounts.