Click here to read GNET's latest report GNET Survey on the Role of Technology in Violent Extremism and the State of Research Community-Tech Industry Engagement

Tracking Violent Extremism Online and the Challenge of Open-Source Intelligence

Tracking Violent Extremism Online and the Challenge of Open-Source Intelligence
2nd June 2021 Lyria Bennett Moses

Understanding online behaviours of extremist groups requires observation, analysis and often dissemination. Academic researchers, government agencies, and civil society groups collect and analyse social media feeds, possibly through application programming interfaces (APIs), to understand violent extremist networks. This collection and use is subject to a range of legal requirements across various jurisdictions. Legal issues include contract law requirements around the enforceability of terms and conditions agreed to when accessing social media feeds, privacy and data protection requirements, and copyright law restrictions. This post provides an overview of the restrictions these legal requirements create concerning open-source intelligence activities aimed at understanding violent extremist networks.

Terms and Conditions, and Ways Around Them

The first legal obstacle pertains to restrictions researchers or government agencies agree to when accessing social media feeds on particular platforms. The public availability of a feed is often limited so that at least some ‘public’ information can only be accessed via a login and password authentication. This authentication is often tied to acceptance of a contract, limiting the permitted use of the service. These contracts, which may or may not be enforceable, make the legal position complex.

A single social media post can often be viewed without agreeing to any contractual terms. A tweet, for example, can be seen using a direct link in a manner that avoids an enforceable contract between a viewer and Twitter. Twitter’s terms of service state that “access or search” by non-approved interfaces without agreement from Twitter are not permitted. However, if it is not shown to a user and the user has not agreed to terms with Twitter, it will be unenforceable. One can often search for, find and read social media posts without agreeing to contractual terms.

Another possibility is the use of a web scraping tool. The legality of web scraping remains contentious and varies by jurisdiction. In the United States, for example, legality may depend on whether it can be done without bypassing a login/password screen (hiQ Labs, Inc. v. LinkedIn Corp.; appeal pending, but see here). In Europe, the lawfulness of web scraping hinges on the purpose served and whether notification requirements are complied with as required. Beyond law, what can be accessed through scraping will depend on the technological measures adopted by platforms to block bulk access and user privacy settings.

Alternatively, a researcher or analyst may agree to the terms of service offered by the platform or a third-party API. Some of those terms of service prevent use for “surveillance” (Facebook; Twitter for public sector entities). Gab, an alt-right preferred social media platform, prohibits “any manual process to monitor or copy any of the material on the website” without “prior written consent.” In this way, private companies are dictating the terms on which their data feeds are made available. Such contracts may be difficult to enforce and may even conflict with a legislative function. In circumstances where enforcement of a contract term would be contrary to public policy (including those relating to national security and law enforcement agencies) or a pre-eminent legal obligation, that term may be inoperable, or the entire contract rendered void. Given that there are public policy justifications for government surveillance, it is arguable that terms that prohibit such use would be treated as contrary to public policy, thus unenforceable.

Nevertheless, researchers and government agencies should be aware of the terms and conditions they agree to when accessing social media feeds and any subsequent legal risks incurred should they be breached.

Privacy and Data Protection

As the GNET community acknowledges, observation of violent extremist groups requires “[b]alancing user privacy, state security and human rights.” At first blush, analysis of publicly available social media posts presents few concerns in this regard. Posts published on the Internet are not private, so it might seem open-source intelligence raises no concerns. However, privacy and data protection laws typically protect all personal information or personal data (terminology is jurisdiction-dependent), not just secret information. Privacy law definitions focus on the identifiability of individuals rather than the extent to which they have sought to keep information secret. Identifiability will depend on what information is collected, but it may include a person’s name, date of birth and/or location. While national security agencies may be able to rely on broadly framed exemptions from such laws, they clearly bind academic researchers and, in some situations, law enforcement intelligence. Where the collection of such information derived from social media platforms falls under privacy or data protection laws, this will generally come with requirements about transparency of collection and use, limited circumstances in which material can be collected without consent, and restrictions on secondary uses. For example, “legitimate interest” is one of six legal bases for processing personal data under the European General Data Protection Regulation (GDPR). While “legitimate interest” is broad, processing must be necessary (the purpose could not reasonably be achieved by less intrusive means) and proportionate to the subject’s interfered rights and freedoms. Before personal data is processed, researchers should carefully justify the applicable legal basis and ensure compliance with relevant conditions. The GDPR also extends to data processed in a third country (Schrems II) and thus has extra-territorial application. This is important as data protection requirements in other jurisdictions may be weaker. For example, in Australia, an individual’s data can often be collected with implied consent.

Copyright Infringement

Making copies of or altering some social media posts may also constitute copyright infringement. Copyright laws are generally broad enough to include at least some text-based quotes as ‘literary works’ and some images or memes as ‘artistic works’. The applicability of copyright law will thus depend heavily on the nature of the material collated. Further, use for non-commercial research purposes may in some circumstances fall under fair use or fair dealing doctrines. It is also possible to procure licences to copy content, and some jurisdictions have specific provisions about copying works where the author is unknown or uncontactable. The legality of making copies of social media posts in the context of scanning and analysing social media posts will depend on rules applying in the relevant jurisdiction, as well as the specific conduct concerned.


While some countries have specific laws that govern the collection and analysis or bulk retention of publicly available information, these only apply to particular government agencies. Whether or not access to and analysis of publicly available material is lawful is context and jurisdictionally specific, and the law in this area continues to evolve. The platform’s terms of service and the technologies employed remain relevant in this context. Hence, both researchers and government agencies must consider how the legal framework relevant to them applies to the collection, analysis and dissemination of open-source material used to track violent extremism online.