Using open source intelligence software for cybersecurity intelligence

Open source intelligence (OSINT) is a method of using open source tools to collect information from publicly available sources and then analyse it in order to make a decision or take some action. OSINT can be helpful in the right hands, but harmful when hackers use it to learn more about your organisation.

Download this free guide

Computer Weekly's buyer's guide to GDPR compliance

This 12 page buyer's guide gives you the tools you need to get up to scratch with GDPR compliance and to understand the mythology surrounding the new rules.

Start Download

• Corporate E-mail Address:

  You forgot to provide an Email Address.

  This email address doesn’t appear to be valid.

  This email address is already registered. Please login.

  You have exceeded the maximum character limit.

  Please provide a Corporate E-mail Address.

• • I agree to TechTarget’s Terms of Use, Privacy Policy, and the transfer of my information to the United States for processing to provide me with relevant information as described in our Privacy Policy.

  Please check the box if you want to proceed.

• • I agree to my information being processed by TechTarget and its Partners to contact me via phone, email, or other means regarding information relevant to my professional interests. I may unsubscribe at any time.

  Please check the box if you want to proceed.

By submitting my Email address I confirm that I have read and accepted the Terms of Use and Declaration of Consent.

Getting to know these open source tools and using them to collect data on your own organisation will enable you to tailor your remediation efforts to further secure your organisation.

In my work as a penetration tester, I am amazed at the amount of information one can garner about a particular organisation. During social engineering tests, the more information I can gather about the organisation I am testing, the more persuasive the results will be. Collecting employee names, job roles, organisational hierarchies, systems/software in use, even friend lists, likes/dislikes and favourite topics help me (as an ethical hacker) build convincing data to test “the human firewall.” I am able to show the organisation that hired me that, if I can do it, so can an attacker.

Information collection tools
To begin defending against OSINT-based attacks, it is first important to understand the information that can be gathered about an organisation and the open source intelligence software that can be used to gather it.

Collating technical information on an organisation’s public-facing systems is the first step. Internet registries, coupled with services such as Shodan or VPN Hunter, can highlight and identify an organisation’s Web servers, mail servers, remote access endpoints and many other Internet-facing devices.

Once the applicable systems have been identified, the Google Hacking Database (GHDB) can be used to detect vulnerabilities within systems without actually coming in contact with them. GHDB contains hundreds of “Google dorks” (search terms which can be used to identify system vulnerabilities from their cached Google pages), which can be accessed with a handy search function. These can prove to be an absolute treasure trove if used correctly.

Tools also exist to harvest information from all the major social networks. There are many ways to harvest data from social networks but, depending on how the tool performs the data collection, it may break the sites' terms of service. Therefore, I won’t discuss the tools to do this. Just bear in mind the information they harvest, including names, online handles, locations, jobs, friends, pictures, etc. can be a goldmine for a malicious social engineer. In my work as an ethical pen tester, I replace these tools with manual searches through the sites, which reveals the same information, although the process tends to be slower.

Other tools exist to collate data on people, such as pipl.com, which helps aggregate information about individuals into one place. There are even tools, such as Creepy, which collect geolocation information from Twitter, FourSquare and various image-hosting sites to paint a picture of particular users’ movements or even their current locations. These tools are passive, in that the attacker does not “touch” any of the systems or people involved in the intelligence-gathering process and simply collates all the available information.

Plugging the intelligence leaks
With the right tools and knowledge, collecting this information is relatively simple, but what can be done to defend against attackers gathering this cybersecurity intelligence? Unfortunately, there isn’t a silver bullet that will prevent the collection of this information. However, from a physical point of view, there are some defensive steps that can be taken, including:

• Keeping systems patches up-to-date;
• Stripping metadata from any corporate document posted to the Internet; and
• Using robots.txt or metatags to stop Google and other search engines from indexing sensitive pages.

Changes in user perception are even more vital to stem the flow of information leaking into social networks, image-sharing sites and the like. Training on the dangers of revealing this information, and how to spot suspicious emails, friend requests, social messages and phone calls, should be conducted regularly.

Getting to know these open source tools and using them to collect data on your own organisation will enable you to tailor your remediation efforts to further secure your organisation. Once the risks are understood, bringing in the relevant training and procedures will help reduce the overall attack surface your organisation presents. Inevitably, and necessarily, this will be a gradual and evolving process, but one which is essential to maintain a reasonable level of assurance that the information you give away will not come back to harm you.

About the author:
Mike McLaughlin is a penetration tester working for First Base Technologies, an information security consultancy in the UK. Mike's daily work consists of both internal and external network based penetration testing, Web application penetration testing, and social engineering.