Open Source Intelligence Analysis — What Businesses Need to Know About OSINT

While social media platforms have developed and trained algorithms to provide a personalized feed of information tailored to our unique interests, corporate security and safety leaders are often left to their own devices trying to make sense of this ever-growing world of unstructured data. Monitoring and analyzing social conversations and content from countless other data sources is of critical concern as organizations attempt to quickly identify pertinent information that poses a material risk to their business or people. Sara Pratley, Alertmedia’s SVP of Global Intelligence, puts it succinctly:

Nearly everyone who has faced making a critical decision with incomplete information has longed for clearer or better data. At its foundation, open source intelligence is meant to provide decision-makers with more complete, timely, and actionable data—connecting dots and for a more accurate picture. But this breadth of intelligence is also rife with misinformation, demanding rigorous analysis by experts.

In this post, we’ll cover what OSINT is, why it has grown in popularity, and what businesses need to know about leveraging it as part of their risk monitoring and employee safety strategies.

What Is Open Source Intelligence?

Open source intelligence, or OSINT, refers to information derived from sources available to the general public. Here’s how the U.S. Department of State defines it:

The term “open source” is used broadly to refer to public information that can be accessed and used freely without permission or special accommodations. This includes both public sources— popular websites, social platforms, and forums—as well as less discoverable sources of information, such as publicly accessible files and documents, webpages hidden behind a login/paywall, and more.

With 5.25 billion people using the internet—roughly 66% of the world’s population—the volume of information published daily is almost beyond comprehension. On average, individuals generate approximately 1.7MB of new data every second, and researchers estimate that the vast majority of the world’s data was created in the past two years alone. Consider the following:

This wealth of information also represents an invaluable source of insight for governments, intelligence agencies, and private businesses looking for a competitive advantage or to keep tabs on what’s happening in the world around them. While sifting through billions of data sources to find a handful of valuable and relevant pieces of information may sound like looking for a needle in a haystack, OSINT researchers and analysts have developed various tools and methods to help discover and take advantage of the information available to them.

Open source intelligence tools and analysis

While it’s easy to think about OSINT as the words or pictures seen on a website or in a social media post, its scope is far greater. Created in 2016, the OSINT Framework provides a helpful overview of the breadth and depth available to researchers and analysts, ranging from usernames, phone numbers, and email addresses to transportation records, public registries, and the global terrorism database.

Some of the available sources OSINT researchers use for gathering information include

Consumer search engines (e.g., Google, Bing, Yahoo, etc.) are invaluable tools for OSINT researchers. However, specialty search engines like Shodan are also used to find obscured data from various “Internet of Things” (IoT) devices—like webcams, smart TVs, license plate readers, and more—to aid cybersecurity investigations and other research. TweetDeck is also a popular OSINT tool used to curate real-time social content based on keyword, content type, or location. Tools like Spiderfoot and Maltego help OSINT investigators and information security (infosec) teams by automating data collection, creating graphical links, and visualizing the relationship between individuals and topics of interest.

Passive vs. active OSINT

Given the sheer volume of information publicly available on the web, there’s no shortage of OSINT available to researchers and analysts. However, there are limits to what researchers can access without taking some proactive steps—whether it be creating an account to access specific social media platforms or being invited to specific moderated communities, such as invite-only groups, messaging services, subreddits, or message boards. Depending on the techniques used to collect the information, it is considered either passive or active OSINT.

OSINT for Cybersecurity and Physical Threat Detection

While governments and intelligence agencies continue to use open source information for various purposes, its origins in the private sector can be traced back to the 1980s. Today, security researchers and teams at companies of all sizes use this intel to reduce risk and inform decision-making in the physical world and cyberspace.

Physical OSINT

The protests initially started as small, isolated pockets of unrest that became widely publicized. As governments cracked down on traditional media and public gatherings, protestors turned to social media, which boosted their visibility even more. This had the paradoxical effect of invigorating political movements in other countries, resulting in several regime changes like in Egypt and Tunisia. 

At the same time, businesses in those areas that previously relied on government support to protect their employees were suddenly left in the lurch as the governments concerned themselves with the protests. Luckily, savvy companies used those same social media platforms to keep abreast of developments and get their employees out of dangerous areas before tensions erupted further.

Investigating security incidents

Corporate security teams commonly use OSINT when investigating or responding to reported security incidents, including active shooter situations, suspicious activity, bomb threats, and more. Once the situation is reported, security teams may use OSINT to verify critical details, such as the location or time an incident occurred, or to provide the most up-to-date information about rapidly developing events on the ground.

Cybersecurity OSINT

Hackers and other digital bad actors are closely connected to OSINT, for better and for worse. As cyberattacks have become more widespread and well-known, the impetus for hacking victims to publicly share details about the attack is much stronger. From companies like Sony and Colonial Pipeline to government agencies, no organization is immune to unauthorized digital intrusion.

By collaborating with other legitimate organizations and governments to share information on these attacks, organizations and individuals contribute to the overall OSINT ecosystem and have the potential to prevent the unnecessary spread of computer viruses or other malware.

OSINT can also flow directly from the threats themselves: Cybercriminals’ online activity can help companies quickly detect when sensitive company information is leaked. This “paydata” is often discussed or published on deep web or dark web message boards, enabling security teams to investigate cyber threats and learn about the vulnerabilities hackers may have exploited to access the information.

Unfortunately, it also works the opposite way. Most companies have a wealth of information publicly available to anyone with malicious intent. From employees’ social media profiles to media coverage, hackers are keen to exploit seemingly innocent information to inform their attacks, including social engineering attacks like phishing attempts or password hint analysis.

4 Challenges to OSINT Analysis

While open source intelligence is a well-established field with countless use cases and applications, the majority of organizations are ill-equipped to navigate the ever-changing field of OSINT on their own. There are many reasons why organizations might find themselves unsuited to this approach to risk monitoring—from navigating legal and ethical debates around OSINT data collection to the extensive human and financial resource requirements to monitor online conversations and world events effectively, 24/7/365. Here are some of the ways OSINT falls short.

1. OSINT data collection is highly time-consuming

While collecting OSINT data is far simpler now than ever before, making sense of it remains a highly specialized skill set requiring years of training. Unfortunately, due to the volume of data generated every day, OSINT analysts must sift through copious amounts of information from a wide range of sources to piece together a coherent story about what’s actually happening at any given time in a given location. Sara Pratley knows from deep experience: “It takes expertise—and sometimes numerous experts to manage the pure amount of information and perform careful vetting.” 

Meanwhile, if the situation is urgent—such as in an emergency or business-critical event—the time required to gather and analyze the information may be longer than the time in which the organization can take meaningful action to alter the outcome, thus calling into question the value of the collection process itself.

2. OSINT is noisy and difficult to filter

Not only does it take time, but the need to constantly monitor, search, and filter voluminous troves of available information to identify relevant information is also tedious for even trained analysts. While AI, machine learning, and other specialized tools help researchers parse information faster, it is challenging for any sized organization to identify all critical, time-sensitive events as they happen. This is particularly true for multi-location businesses that monitor hundreds of distinct facilities or thousands of remote employees.

3. Unverified OSINT can’t always be trusted

False positives are another primary concern within the OSINT community. For example, last year, a video of a woman in Singapore rejecting the country’s COVID-19 mask mandate went viral. Shortly after, the woman in question was taken into custody by local police; however, various internet figures took it a step further, identifying her as the CEO of a digital security firm based on publicly available sources—reports that turned out to be false. Unfortunately, the misinformation resulted in her company and its employees having to endure damage to the company’s brands and personal threats before the initial reports were corrected.

4. Artificial Intelligence can mislead (but also reveal)

The recent advancements in artificial intelligence (AI) have sent reverberations throughout our daily lives, meaning open source information is highly vulnerable to manipulation. OSINT analysts rely on up-to-the-minute reports for the situations they’re monitoring, but what if someone fabricated that information—and convincingly so? The AI technology has proven itself capable of working as a high-speed disinformation assembler, intentionally and unintentionally, using text, images, and deepfakes to disseminate falsehoods. Sara Pratley explains that researchers must focus even more on corroborating information and fact-checking sources to preclude bad insights. This extra attention can slow things down significantly.

On the other hand, AI has been used to sniff out faked media successfully. At best, it seems like we’ll be witness to an AI arms race between bad actors and legitimate organizations, with OSINT analysts caught in the murky spaces between.

What to Look for in a Threat Intelligence Tool

Fortunately, there are several ways organizations can leverage the benefits of OSINT to improve situational awareness and aid decision-making without hiring a dedicated team of analysts to comb through the data. Many are turning to purpose-built threat intelligence solutions that use OSINT data while also providing a layer of curation and verification to provide businesses with a cleaner, more actionable view of what’s happening.

If your organization is considering investing in an intelligence solution, here are five things to consider when evaluating your options.

Speed

Whether you have a small security team or a large GSOC, knowing as quickly as possible about critical events that pose a risk to your employees or business is crucial. When assessing tools, ask how intelligence is delivered and how easy it is to implement so you can alert those in harm’s way. Is threat intelligence offered 24/7/365? Can it be integrated with your organization’s mass notification system to streamline response and recovery workflows? Activating your emergency response isn’t only about how quickly you have verified information to deliver; it also requires a mechanism to notify those impacted.

Depth and breadth of coverage

Your provider should offer both around-the-clock monitoring and comprehensive coverage of a wide range of threat types. What types of incidents are captured? What data sources are used? Does the provider cover all major geographies? Use this information in conjunction with a business continuity checklist to understand if there are any gaps in your preparedness efforts so you can address them before moving forward.

Noise-free, accurate information

If the phrase “drinking from a fire hose” came to mind when reading about the volume of data created daily, you’re not alone. Trying to monitor and make sense of everything posted on the web—even if you were to monitor only a fraction of available data sources—is guaranteed to leave your organization overwhelmed and with little more insight than if you had done nothing at all. How does your provider analyze and filter information? What parameters do they use to determine relevance? How does the provider differentiate between unverified and verified sources? Consider requesting a demo to see real-world threat alerts around one of your locations so you can independently assess whether the information provided is adequate.

Location-aware risk assessments

Moving from raw data to actionable intelligence is all about context. For organizations to truly understand the impact of threats, they must also understand if, how, and to what extent each threat poses a risk to their people, facilities, and assets. Be sure to ask potential threat intelligence providers how threat data integrates with location intelligence. Do they support real-time data syncing with your HRIS? How does the solution account for threats near remote employees or business travelers?

Analyst verification

We’ve stressed the danger of information overload in OSINT operations, where analysts have more on their plate than what they have time for. Thankfully, high-quality threat intel tools provide direct analyst access, where trained intelligence professionals can jump in and assist your organization’s OSINT efforts with their specialized knowledge, allowing you to scale your capabilities without expanding your internal team.

Addressing Information Overload

With a world’s worth of information available at our fingertips, learning how to parse and interpret it becomes all the more important. Using OSINT as part of a holistic approach to threat intelligence and monitoring can help your organization reduce risk and keep employees safe during emergencies and other critical events. Learn more about how AlertMedia’s Threat Intelligence helps customers monitor, analyze, and quickly communicate threats to their people. Request a demo to see for yourself.