Osint examples

Osint examples DEFAULT

8 Healthcare Cyber Attacks You Should Know About

Juniper Research estimates that there will be 83 billion IoT connections by 2024 — that’s 10 times more IoT devices that will be generating data than the estimated number of people who will be alive on Earth at that time

Wondering “what is OSINT?” The term, which stands for open source intelligence, refers to a system of gathering data from freely available public resources. (So, if you’re also wondering “what is open source information,” this article will answer that question, too.) But what makes the process of using open source intelligence different from general data collection is that it goes beyond querying search engines using different permutations of the same phrase.

OSINT is a term that originated with the U.S. military in the 1980s. They needed to find a way to keep up with dynamic information to keep a tactical frontline advantage. Nowadays, professionals across various industries use OSINT data to achieve different functions. For example, marketing and sales teams use it to increase conversions, whereas cybersecurity teams use it to conduct investigations and mitigate threats.

Today we’re going to discuss what OSINT is, cover some of the most popular OSINT tools and techniques, the OSINT framework, and much more. But before we dive into that, let’s start by defining open source intelligence and open source information.

What Is Open Source Intelligence (OSINT) & How Does It Relate to Open Source Information?

OSINT is a term that refers to a framework of processes, tools, and techniques for collecting data passively from open or publicly available resources (not to be confused with open-source software). Open source intelligence historically referred to open source information gathering via conventional channels such as newspapers, radio, TV, etc. Nowadays, to extract specific intelligence, we use:

  • Blogs,
  • Discussion boards,
  • Social media,
  • The dark web (accessible through TOR), and
  • Deep web (pages not indexed by Google like a people search database).

In some cases, such as with social media, OSINT has developed into a prominent subset of its own called SOCMINT, which is short for “social media intelligence.”

Examples of open source intelligence gathering include:

  • Searching for information about a competitor’s employees or services,
  • Law enforcement agencies gathering intelligence using online public resources to prevent crimes,
  • Identifying vulnerabilities to exploit at a later stage on a target system or network, and
  • Collecting information to conduct a social engineering attack.

Advantages of Using OSINT

Open source intelligence gathering comes with several benefits. Let’s take a look at some of them:

  • If you’re on a budget, conventional information collecting techniques and tools may not be an economically viable solution. One of the main benefits of employing OSINT to gather intelligence is that it involves a minimal level of financial investments.
  • The information obtained is not classified and has been divulged freely, hence it is legal to obtain any such information.
  • Because it relies on public resources, users frequently share and update the information regularly.
  • Business owners and decision makers can gain perceptivity through OSINT data that can facilitate building long-term strategies for a variety of business goals.
  • OSINT can also be an invaluable tool in matters of national security.

Disadvantages of OSINT

Now that we understand the convenience of using OSINT, what are its drawbacks? Like you’ve probably guessed, as easily as you can utilize OSINT to gather intelligence, an adversary can also use it to collect information about you or your business. Besides that, here’s a list of a few other disadvantages you may run into while using OSINT:

  • Finding information means very little unless you can put it to use in some meaningful way. Filtering out junk data from valuable information can be challenging based on the volume of data you find.
  • Once you have filtered out usable data, you need to validate that the information is reliable. Organizations and individuals may deliberately post false information to mislead potential attackers.
  • The information gleaned is not consumable as it is, and there is a considerable amount of analysis work involved.

How OSINT Relates to Cybersecurity

In cybersecurity, OSINT techniques aren’t a one-size-fits-all approach. Depending on the purpose of your research, your end goal, and what you’re trying to find, the techniques you deploy will vary as will the tools used. Once you determine who your target is and the steps you’ll take to conduct your research, then you can choose the appropriate tool and approach.

While gathering INT data, the basic idea is to:

  • Connect the dots,
  • Pivot to a new source of information as needed to build a deeper intelligence profile, and
  • Validate every assumption that’s made along the way.

If you’re conducting OSINT as a part of an engagement, generating a report at the end with screenshots attached is an indispensable part of the process.

OSINT is frequently used to profile a target, and it’s done by conducting passive reconnaissance to glean information without actively engaging with the individual or corporation. However, there are some obstacles to gathering intelligence. An account created specifically to conduct OSINT on, for example, Facebook, might end up looking like a fake account. Some websites actively delete such accounts that don’t look legitimate. Moreover, the sheer volume of available data that you need to analyze and process to gain useful insights can be overwhelming.

OSINT data is valuable in the later stages of an attack because it adds credibility if and when there’s direct contact with the target. It allows virtually anyone to craft a customized attack that exploits weaknesses in people, processes, or technologies.

A screenshot of OSINT data from Greynoise.io

Using OSINT as Counterintelligence in Organizations

Counterintelligence refers to activities concerned with detecting and neutralizing threats to an organization’s security against any opposition’s intelligence service. The first move any hacker makes when planning an attack is to gather as much information as possible. This can be information about the target organization, specific employees, and any other available data that’s useful. The next step is to take all of this information they gather from various sources and turn it into intelligence through analysis and correlation.

By utilizing OSINT capabilities, your organization can take steps to identify all publicly disclosed information. You can use this intelligence to scrub the data to prevent disclosure of sensitive information or to train your employees to be aware of it. Having a dedicated team that identifies correlatable data to form intelligence is invaluable. They can help you avoid reputational damage by identifying and then attempting to obscure or censor any publicly disclosed information of a sensitive nature where possible. Additionally, these insights may also prevent or buy you time to mitigate the risk of any potential data breach due to such exposure.

What is the OSINT Framework?

The OSINT framework provides a collection of OSINT tools, classified into various categories, that pentesters and hackers alike can use for reconnaissance. The OSINT framework has a web-based interface and is primarily focused on listing free resources.

For instance, the first entry “username” can be explored in our OSINT research if we focus on discovering usernames utilized by a target across various accounts on the internet. On clicking the entry, it’ll display a list of all the tools that can be employed to accomplish this goal.

6 OSINT Tools That Can Enhance Your Cyber Security Efforts

Scouring the internet manually to profile your target organization or individual can be time consuming. Thankfully, the new generation of “OSINT-ware” removes this obstacle for attackers and pentesters alike. These tools help them to quickly determine, with very little effort, the finer details about a target’s network.

DuckDuckGo, Google Maps, Pastebin, and social media sites are good places to start and are commonly used. However, there are several additional tools that might help you gather intelligence more efficiently:

1. Shodan

The first OSINT tool we’ll discuss is Shodan, which stands for Sentient Hyper Optimized Data Access Network. This search engine for interconnected devices allows you to search for IoT/SCADA devices, routers, traffic cameras, and more.

Shodan search attempts to grab data such as the service, software, version number, or other information from the ports it scans. The tool comes with filters such as country, port, operating system, product, version, hostnames, etc. that helps narrow down the results. It displays a vast amount of insecure information that’s freely available and access to web interfaces of IoT devices with weak or default passwords, devices like webcams at people’s homes, and other unsecured appliances.

Pentester can use Shodan to find insecure web services while conducting vulnerability assessments. The tool comes with a free plan that offers a limited number of scans, or you have the option of using a paid version. However, organizations can request to block Shodan from crawling their network.

2. Maltego

Maltego is an aggregator of interfaces to several OSINT databases and covers infrastructural reconnaissance. This tool can harvest a wealth of sensitive information about any target organization, such as:

  • Email addresses of employees,
  • Confidential files that have carelessly been made publicly accessible,
  • DNS records, and
  • IP address information.

You can also use Maltego for personal reconnaissance to collect individual-specific data. Maltego communicates with search engines on the internet to gather all this information in one convenient location.

3. Metagoofil

Metagoofil is another OSINT tool that utilizes the Google search engine to extract metadata from publicly available files (.pdf, .doc, .xls, .ppt, etc.) belonging to any target company. After downloading the documents onto the local disk, it pulls out the metadata using various libraries like Hachoir, PdfMiner, etc. and generates a report.

4. GHDB

Hacker and cybersecurity Expert Johnny Long developed the Google Hacking Database (GHDB) for pentesters in 2000. It’s a list of search queries that reveal interesting information that was, in all probability, made public unintentionally. An example of unintended disclosure could be a search engine crawling a web document that contains a link that holds sensitive data. The search engine may subsequently follow it and index any information on it.

Some of the information that you can query using GHDB includes verbose error messages that contain sensitive information like:

  • Directory paths,
  • Files with sensitive data, passwords, or usernames; and
  • Information about web servers.

5. SpiderFoot

SpiderFoot automates collecting information about IP addresses, domain names, e-mail addresses, usernames, names, subnets, etc. and comes with an opensource version. This tool allows you to examine any suspicious IPs, phishing scam e-mail addresses, and HTTP headers (which can be parsed to reveal OS and software version numbers, etc.). It’s also useful to organizations for monitoring any information that’s been made public inadvertently.

6. Foca

A network infrastructure mapping tool, Foca, can extract and analyze metadata from different types of files (pdf, doc, etc.) fed at a time or given together. It also can enumerate users, e-mail addresses, software being used, and other useful information.

Need Additional OSINT Tools and Resources? Look No Further…

We’ve put together a list of other resources that may assist you in carrying out any OSINT research includes (but is not limited to) the following:

  • Plugins — Passive Recon is a Firefox plugin that searches through several public databases and look-up services. It passively gathers information about a domain so long as you’re on the page or one linking to it.
  • Search engines — Apart from Shodan, attackers and pentesters frequently rely on search engines like Censys, ZoomEye, Greynoise, BinaryEdge, among others.
  • E-mail harvester — Some tools like the hunter, theHarvester, Prowl, and a few others can help you detect email information for a target organization’s employees.
  • DNS Enumeration — Tools like DNS dumpster, Sublister, etc. are useful for enumerating valid subdomains.

Additional resources include haveibeenpwned, Recon-ng, CheckUserNames, Creepy, Nmap, etc.

Final Thoughts on Open Source Intelligence Gathering

Frankly, there’s a lot to know when it comes to answering the question, “what is OSINT?” As such, I hope this article provides clarity about open source information, open source intelligence, the OSINT framework, and showcases the types of OSINT tools that are available to you.

Although you can use OSINT techniques to cyberstalk or conduct other nefarious deeds, you can also use them for good purposes like fuddling information and misleading attacks to protect privacy. Any data that’s made publicly available will be accessible in bits and pieces to anyone with or without the knowledge of OSINT. With it, an individual or an organization has the tools necessary to assess what’s out there and, at the very least, obfuscate the narrative.

Sours: https://sectigostore.com/blog/open-source-intelligence-what-is-osint-how-does-it-work/

The blog

With the democratisation of OSINT, more and more ordinary people began using it. But did you know some of them have been able to forecast economic and geopolitical events and investigate crimes better than professionals? 


 

As people in all fields are becoming more familiar with OSINT methods and techniques, the tradecraft is used to aid many different research works. It's helping people make the world a better place by serving researchers, educators, forecasters, and those who want to make a change.

For this blog post, we've gathered 10 examples of OSINT in action. The common ground between these investigations is their use of open-source intelligence with the objective of helping and educating.  

 

 

  1. 1. Locate Centre for Missing People Investigations

     

  2.  

Locate Centre for Missing People Investigations is a company dedicated to helping the families of missing people whose cases remain unsolved. They combine the competence of experienced investigators, data analysts, and open source experts. Their work consists of reviewing all available evidence and seeing if they can get any new leads on the cases. They have also developed a model that involves the help of local volunteers. Students from the universities of South Wales and Central Lancashire take part, too. The organisation has formed a talent development program to provide real-life experience to students and help them develop their skills.

This community grants resources to families, the general public, agencies as well as the police. They recognise police are under great pressure both internally and externally. However, their limited capacity is not always sufficient for a proper review of unsolved cases. That’s why the dedicated investigators and the volunteers review cases independently. They try to be objective and make sure that no measures to be taken were missed. Finally, they assist the flow of information between parties to help families get closure and hopefully reunite with their loved ones.

 

 

 

 

  1. 2. Trace Labs

  2.  

 

Trace Labs is a not-for-profit organisation that has turned finding missing people into a Capture the Flag contest. “Helping people on the worst day of their lives” is their mission and they are employing a few approaches to make that happen.

Firstly, they are holding local and national contests in CTF format. OSINT enthusiasts with all levels of expertise take part in these to gather new leads on missing people cases. The global community of Trace Labs is also constantly working on crowdsourcing new intelligence. They are in collaboration with some law enforcement agencies who review the reports to take action for any cold or new cases.

Secondly, they believe in the power of education. That’s why they have made it a priority to educate their community. Trace Labs are working together with the best OSINT experts to provide proper training to their public and ensure more and better results.

Finally, their partnerships with law enforcement agencies intend to bring awareness. They are spreading the issue of missing people and are helping other non-profit organisations in the field.

 

 

 

3. Trace an Object



  1.  

In 2017, Europol started the crowdsourcing project “Stop Child Abuse - Trace an object”. They own millions of pictures of child sexual abuse some of which have undergone all possible examinations and still provided no clues. That is precisely why this project aired. Out of the belief that more pairs of eyes can recognise small details that can turn out to be essential in the long run.

Censored information is often posted on their website for the public to review. Europol is usually looking for the country of origin or the approximate location of these clues so they could advance in their cases.

By far, more than 24 thousand people have submitted tips. That helped identify ten victims and prosecute two offenders.

 

 

 

  1. 4. Innocent Lives Foundation 

  2.  
  3.  

The Innocent Lives Foundation (ILF) is a nonprofit organisation with the mission to assert a change on behalf of sexually abused children. They fulfill their purpose to protect children online in a few ways.

Their team is formed by cybersecurity and technology experts who use online investigation to identify pedophiles, sexual predators, and human traffickers. What they do next is to provide law enforcement with that information and aid the prosecution of these criminals.

These crimes have an enormous impact on innocent lives and that fact is what keeps the project going. ILF is also providing some educational resources on how to make children aware of the risks and hopefully keep them safer.

 

 

 

  1. 5. Operation Safe Escape 

  2.  
  3.  

Domestic abuse is a widespread problem with horrendous consequences for its victims. The people behind Operation Safe Escape realise that and are striving to help.

Their program aims to aid the escape from the abuser. They are working in close collaboration with the victim and their support system to create an escape plan and make sure it’s carried out successfully.

What’s more, Operation Safe Escape is empowering people through education. They make sure that the escaped person is safe physically and digitally. They help victims stay protected online by teaching them privacy-focused practices.

The security and cybersecurity experts in the organisation deliver instructions on how to leave the least possible digital footprint. That is a must because it's crucial that the victim stays safe and undetectable as soon as they flee from their abuser.

 

 

 

  1. 6. Irdeto against wildlife crime 

 

Irdeto is a company focused on the cybersecurity of digital platforms. However, they’ve decided to partner with the African Wildlife Foundation (AWF) to fight another type of crime.

Illegal trade with wild animals’ parts is the fourth biggest illicit industry after drugs, counterfeit goods, and human trafficking. This is a low risk but high-profit business which kills animals, ruins ecosystems, and leaves people unemployed.

The African Wildlife Foundation has been trying so hard to find ways to preserve the wildlife. They’ve been equipping wildlife rangers, deploying sniffing dogs, training law enforcement, creating awareness and partnerships across the globe with the sole purpose of stopping wildlife crime. However, criminals still manage to find ways and go unbothered.

Irdeto’s cyber intelligence department employs OSINT to tackle this issue. They are using their capabilities to monitor the illegal trade of animal parts online to identify and locate the people behind them. Then law enforcement takes the wheel to do what needs to be done further.

 

 

 

 

  1. 7. The Good Judgement Project 

 

 

The Good Judgement Project is “harnessing the wisdom of the crowd” to help organisations make better decisions.

Almost a decade ago, three Pennsylvania University professors created The Good Judgement Project. They were namely Philip Tetlock, Don Moore, and Barbara Mellers.

In 2011 the Intelligence Advanced Research Projects Activity (IARPA) launched The Aggregative Contingent Estimation (ACE). The goal was to identify the best methods for prediction of geopolitical events. The Good Judgement Project (GJP) turned out to be the most successful participant.

For the following four years of forecasting tournaments, the GJP outperformed every other competitor. They even got ahead of intelligence analysts with access to classified information.

During that time also emerged the so-called super forecasters, people particularly good at making predictions about future outcomes.

 

After the ACE tournaments ended, the Good Judgement Project went on with their activities. They recruit forecasters in search of the best and they cultivate and nurture their skills to turn them into a helpful part of society.

There are three important conditions for identifying superforecasters - training, teaming, and tracking. Bias, noise, and information are also taken into account when investigating the results.

Superforecasters excel at what they do because they are open-minded and treat forecasting as a skill to cultivate. Teaming such people together is also an important factor in their performance. Apart from that, their prediction-making process involves looking up open-source information on the internet.

There are about 260 superforecasters identified so far with hopefully many more to come in the future.

 

 

 

 

  1. 8. IHS Markit Conflict Monitor 

 

 

IHS Markit is an information company headquartered in London. They provide a bigger picture by delivering actionable insights to their clients and are operating in multiple sectors from education to retail and technology.

 

During the Syria civil war, they developed the Conflict Monitor project to provide their clients with a very granular situational awareness of developments on the ground. Using OSINT techniques, they monitor social media, events, armed actors as well as numerous other indicators. 


By systematically gathering and organising this information, they were able to provide media and national security clients with a granular mapping of the many belligerents’ activities and their evolution over time. In doing so, they were able to identify patterns and make accurate forecasts about the evolution of the conflict.

 

  1. 9. The cat killing murderer 

 

 

This one is the story of a group of OSINT enthusiasts who decided to bring justice to a few kittens. They went after Luka Magnotta, a Canadian citizen whose actions inspired one of the most popular Netflix crime documentaries in 2019.

It all started in 2010 when a man posted a graphic video of himself killing two kittens on Facebook. He then went on to upload two more but little did he know that he was already under investigation. Deanna Thompson and John Green had already created a Facebook group to look into the perpetrator. All members analysed the videos' details to identify him and gather more and more information.

That turned out to be extremely useful when the cat killer struck again but this time his victim was not an animal. He murdered and mutilated the body of the Chinese student Lin Jun and had it all recorded.

When the police came into the picture, the community of cat lovers had already gathered extensive intelligence on the criminal and provided them with leads to follow.

He was discovered in Europe in 2014 and is currently serving a life sentence.

 

 

 

10. Crowdsourced knowledge production  

 

 

Crowdsourcing is using the knowledge of many people on a certain issue to gather information. It is still debated whether crowdsourcing is helpful or harmful to the intelligence community. However, it is a fact that places for crowdsourcing exist on the internet and people are taking advantage of them. Some examples of crowdsourced knowledge available for anybody interested are the Fatal Encounters database, The Gun Violence Archive, and the subreddit RBI (Reddit Bureau of Investigations).

Fatal Encounters has a straightforward goal. To create a comprehensive and searchable database of all people killed during interactions with the police.

The gun violence archive is a complete archive of all violence that involved guns in the United States.

The Reddit Bureau of Investigation is another type of source. Instead of an archive or database for particular information, people in this subreddit can ask for anything and see what the community has to offer. The requests vary from geolocating images to asking for advice on dog’s strange behavior. What connects them all is that they are real-life problems. Solving them through the public’s knowledge is the reason for this subreddit to exist.

 

 

 

If you think we've left out other important causes or OSINT investigation examples worth sharing, don't hesitate to contact us at [email protected] 

 

We are also proud to announce that our solution to online research, the Digital Investigative Board, is now available for purchase. Click hereto see how it might aid your research process. 

Sours: https://blog.reknowledge.tech/blog/examples-of-osint-for-good
  1. 22re coolant hose
  2. Lee industries chaise
  3. Ancient lettering font

This is part 2 of our series of articles on OSINT. Find all articles here.

OSINT is the practice of gathering intelligence from publicly available sources to support intelligence needs. In the cybersecurity arena, OSINT is used widely to discover vulnerabilities in IT systems and is commonly named Technical Footprinting. Footprinting is the first task conducted by hackers – both black and white hat hackers – before attacking computer systems. Gathering technical information about the target computer network is the first phase in any penetration testing methodology.

In this article, I will demonstrate how various OSINT techniques can be exploited to gain useful intelligence from public sources about target computerized systems.

Technical Investigation of Target website

By knowing the type of programming language, web frameworks, content management system (CMS) used to create the target website, we can search for vulnerabilities that target these components (especially zero-day vulnerabilities) and then work to exploit any of these vulnerabilities instantly, once discovered.

There are different online services to examine the type of technology used to build websites. To use such service, all you need to do is to supply a target domain name, to have a full list of technical specifications and online libraries/programming languages used to build a subject website. These services also reveal the hosting provider of the target website, SSL certificate register name in addition to email system type. The following are some popular services to use:

  1. https://builtwith.com
  2. https://www.wappalyzer.com

In the following screen capture, I use builtwith service to investigate the technical specifications of a target website. This reveals different technical information (see Figure 1) and opens the door to more examination for each technology used to build the subject website. Now, I need to check the list of technical specifications to see if there is unpatched operating systems or outdated content management system with known vulnerabilities that I can exploit to gain entrance to target system.    

For example, large numbers of ASP.net websites, use Telerik Controls (https://www.telerik.com) to enrich their design. To find security vulnerabilities associated with Telerik Controls, you can go to https://www.cvedetails.com and search for Telerik security vulnerabilities (see Figure 2).

There are many websites that list security vulnerabilities of operating systems, software and other web applications. The following are the most popular one that we can use to search for common security vulnerabilities and exposures:

  1. https://vulmon.com
  2. https://sploitus.com
  3. https://www.saucs.com
  4. https://www.shodan.io

Analytics and Tracking

Most websites use Google services to analyze traffic and serve advertisements. We can use this feature to capture all linked domain names. For example, I can find all websites that use the same Google AdSense or Analytical accounts. Dnslytics (https://dnslytics.com/reverse-analytics) is a free online service that finds domains sharing the same Google Analytics ID (see Figure 3).

Target website previous History

In many instances, checking the old version of the target website can reveal important information. For example, an old website version of a corporation may reveal top managements’ email addresses and phone numbers before they got removed from the new version. Wayback Machine (https://archive.org/web) is a good place to start your search for old versions of websites (see Figure 4).

Sub-domain name Discovery

Finding a target website sub-domains is important and can reveal sensitive information about the target such as the VPN portal, email system and FTP server address where some files may have left unprotected. To find all sub-domain names of a target indexed by Google, use the following Google search command (see Figure 5).

Type and versions of IT infrastructure of the target company

Job websites – and any job announcement posted on the target website – should be analyzed to discover the exact IT infrastructure used by the target organization. For example, I conducted a simple search on employee resumes on job websites and was able to capture important information about target organization security systems (e.g. Firewalls and Intrusion Detection Systems), server operating system type, email system, networking devices, types of backup systems and much more (see Figure 6).

Harvest digital files hosted on the target domain name

Using advanced Google search engine techniques (also known as Google dorks) can reveal a great amount of information about the target organizations’ IT systems in addition to confidential files left on the public server. There are thousands of Google dorks and you can practice creating yours. A comprehensive list of Google dorks can be found in the Google Hacking Database (https://www.exploit-db.com/google-hacking-database).

I will experiment using Google dork to locate all PDF files posted on the target website (see Figure 7):

In the above example, I searched for PDF files, however, you can change the file type to something else as you want (doc, docx, xls, txt).

Information contained within files metadata

For each file found on the target website, we should investigate its metadata. Metadata is data about data. In technical terms, it contains hidden descriptive information about the file it belongs to. For example, some metadata included in an MS Office document file might include the author’s name, date/time created, comments, software used to create the file in addition to the type of OS of the device used to create this file. (see Figure 8).

From Figure 8, I found the following facts about the subject PDF file metadata:

  1. Installed PDF reader Version on the creation device: 1.5
  2. Application used to create the report: MS PowerPoint 2010 (using the “Save As” function)
  3. Type of OS used on the target device: Windows
  4. File creation date/time:  July 2017
  5. Author Name (The person who creates the file).

If the file contains an author name, an additional search could be conducted to lock up more details of the file’s author using specialized people data collection websites. The following lists some popular people search engines:

  1. Spokeo (https://www.spokeo.com) (see Figure 9)
  2. Truepeoplesearch (https://www.truepeoplesearch.com)
  3. Truthfinder (https://www.truthfinder.com)
  4. 411 (https://www.411.com)

Email naming criteria

To predicate the naming criteria used by the target organization when creating new email accounts, we should investigate the naming of current email addresses. For example, many organizations use the following naming criteria:

  • Most common patterns of naming new emails: {first}(DOT){last first three characters}@exampleWebsite.com
  • Other naming criteria include: {first}@exampleWebsite.com

I usually use this website https://www.email-format.com to find the email address formats in use at thousands of companies.

Leaked Credentials

Leaked accounts credentials are spread everywhere online, especially in the darknet. For example, pastebin websites (see Figure 10) contain a vast amount of leaked credentials. Anonymous file sharing websites, such as https://anonfile.com (see Figure 11) also contain large numbers of leaked credential files with billions of records.

Conclusion

In this article, I tried to give a brief overview of OSINT capabilities and how to use it to gather useful intelligence about different entities.

In today’s information age, having OSINT skills is something great to have, however, there are many things – or prerequisites – you should master in order to make your OSINT search rich and effective. For instance, before you begin your OSINT search, you should learn how to conceal your digital identity and become anonymous online. This is essential to prevent threat actors from discovering your search activities. OSINT is strongly related to Digital Forensics and knowing basic information about digital forensics operations will also prove useful when conducting OSINT gathering activities. 

In the next article, I will cover how to assure your online privacy, I will talk about the different tracking techniques – currently employed – to track and profile Internet users and how to avoid them, I will also explore web layers and teach you how to access the Darknet in addition to using anonymity networks such as the TOR network to surf the ordinary web anonymously.

The first part of this series, an introduction to OSINT, can be found here.

Dr. Varin Khera

Chief Strategy Officer ITSEC Group / Co-Founder ITSEC Thailand c|Website

Dr. Khera is a veteran cybersecurity executive with more than two decades worth of experience working with information security technology, models and processes. He is currently the Chief Strategy of ITSEC Group and the Co-founder and CEO of ITSEC (Thailand). ITSEC is an international information security firm offering a wide range of high-quality information security services and solutions with operation in Indonesia, Malaysia, Philippines, Singapore, Thailand and Dubai.

Previously the head of cyber security Presales for NOKIA, Dr. Khera has worked with every major telecom provider and government in the APAC region to design and deliver security solutions to a constantly evolving cybersecurity threat landscape.

Dr. Khera holds a Doctor of Information Technology (DIT) from Murdoch University, a Postgraduate Certificate in Network Computing from Monash University and a Certificate of Executive Leadership from Cornell University.

Dr. Khera was one of the first professionals to be awarded the prestigious Asia Pacific Information Security Leadership Awards (ISLA) from ISC2 a world-leading information security certification body under the category of distinguished IT Security Practitioner for APAC.

Sours: https://cyberprotection-magazine.com/open-source-intelligence-osint-a-practical-example/
OSINT - Open Source Intelligence Overview

OSINT: What is open source intelligence and how is it used?

Long favored by spooks and spies, OSINT is also a powerful weapon in the security pro’s armory

OSINT: What is open source intelligence and how is it used?

In July 2014, Malaysian Airlines Flight MH17, from Amsterdam to Kuala Lumpur, crashed some 50km from the Ukrainian-Russian border. All 298 passengers and crew on board the Boeing 777 lost their lives.

In the aftermath of the incident, separatists from Donetsk claimed to have shot down a Ukrainian transport aircraft, claims they later withdrew. The Ukrainian authorities said separatists had downed the airliner. The country’s president called the incident an act of terrorism.

Several independent and official investigations followed. One of the most comprehensive was conducted by Bellingcat, the investigative and citizen journalism group.

Bellingcat, along with other media organizations, used open source intelligence (OSINT) techniques and methodology to build a timeline of the incident, and to expose Russian claims and counterclaims as fabrications.

What is OSINT?

OSINT is intelligence “drawn from publicly available material”, according to the CIA. Most intelligence experts extend that definition to mean information intended for public consumption.

OSINT is information that can be accessed without specialist skills or tools, although it can include sources only available to subscribers, such as newspaper content behind a paywall, or subscription journals.


Read more of the latest open source security news


The CIA says that OSINT includes information gathered from the internet, mass media, specialist journals and research, photos, and geospatial information. Most of these sources were used in the Bellingcat MH17 investigation.

OSINT does not require its exponents to hack into systems or use private credentials to access data. Viewing someone’s public profile on social media is OSINT; using their login details to unearth private information is not. In intelligence agency terms, OSINT is also information drawn from non-classified sources.

When were OSINT techniques first used?

Open source intelligence predates the internet. Governments have long used newspapers, and later broadcasts, to track potential adversaries’ military, political, or economic plans and activities.

OSINT is low risk, cheap, and often highly effective, as corporate intelligence consultant Cameron Colquhoun has written in a Bellingcat article on the history of OSINT.

As Colquhoun suggests, OSINT fell out of fashion after World War Two, with intelligence agencies instead focusing on the more glamorous and dangerous world of HUMINT – human intelligence or spying – and SIGINT: signals and electronic intelligence.

But with the rise of the internet and social media, and online tools that can sift through vast amounts of information, OSINT is now more relevant than ever.

Not just for spies: OSINT and cybersecurity

Intelligence agencies use OSINT to track events, equipment such as weapons systems, and people. These are the ‘targets of interest’ (ToIs).

But hackers use OSINT to identify technical vulnerabilities as well as human targets for phishing and social engineering attacks. As a result, pen testing and security teams deploy similar techniques to find and close down weaknesses.


Read more of the latest social engineering news


“When explaining OSINT and how damaging it can be to clients, I do like the analogy of putting up a big poster in your front window with all your information on,” Liam Follin, penetration tester and web application security consultant at Pentest People, told The Daily Swig.

“We know where to look and we have the tools. But the nefarious side of hacking, the black hats, also know where to look.”

OSINT helps security teams unearth clues that individuals leave in the open that compromise security. Like using a vulnerability scanner to find flaws in systems, OSINT tools pick up on problem data, such as dates of birth, Social Security numbers, family members or even hobbies that could help attackers compromise an account.


OSINT techniques can be turned inwardly to gain information about your company's exposureOSINT techniques can be turned inward to gain information about your company’s exposure

Common OSINT techniques

There is no single playbook for OSINT: most pen testers have their own methods and preferred tools.

This often starts with manual reconnaissance, and reading up on the target subjects, including using non-technical sources such as an organization’s annual report, financial filings, and associated news coverage, as well as content on its websites and YouTube and similar services.

“Anything you can obtain via online as well as by traditional media research can be used for OSINT. And one of the simplest and yet most effective tools is using search engines,” Anita Bielicka, a cyber threat intelligence researcher at Orpheus, told The Daily Swig.

A hacker will also search for information on employees, cross checking against external social media and professional profiles.

“If I can identify the likely members of the development team and identify what they have done in public source code repositories, the technology meetups they have participated in, their blog posts, I can start to build a profile for how strong their skillset is and also an understanding of the types of mistakes they are likely to make,” Tim Mackey, principal security strategist at the Synopsys’ Cybersecurity Research Centre, told The Daily Swig.

“If I know what mistakes they’ve historically made, I can potentially build a targeted attack against that particular weakness in their coding… that all becomes part of the corpus of information we can use to define a potential attack.”

And although OSINT is often associated with social media and human tooIs, it can equally be used alongside security scanners to find problems with physical assets and IT systems, for example via a Shodan search.


RECOMMENDEDShodan founder John Matherly on dual-purpose hacking tools, and information overload


And researchers will look at GitHub, Stack Overflow, Reddit, vendor support forums, and even job listings for insights on a target’s technology.

“Since almost everyone uses LinkedIn, there is a massive amount of information there,” Andreas Georgiou, security consultant at Trustwave Spiderlabs, told The Daily Swig. “If you can find out how an email or username is constructed, you can look at past credential leaks, and have a good chance of finding a way into a network.”


The world of OSINT is a vast placeThe number of OSINT tools and services is constantly growing (image via osintframework.com)

OSINT in the open – examples of open source intelligence

Pentest People’s Follin recalls an OSINT engagement that found floor plans of a sensitive location online, and another where an online photo contained enough information to copy a keycard. Both could compromise the physical security of an organization.

This shows why OSINT is a valuable tool for raising security awareness, as well as a technical tool for identifying security risks.

“Organizations are potentially enabling cyber-attacks against themselves through the information they publish online,” James Dale, penetration testing and red team lead at PA Consulting, told The Daily Swig.

“OSINT is harvesting data from legitimate sources such as online search engines, websites, and professional social networks. But our cybersecurity experts have conducted client OSINT assessments and discovered information such as versions of software, names of devices used to print documents, and email addresses.

“Along with obvious sources, such as a company website and LinkedIn, this information can also be gathered through metadata stored within files created and published by an organization.”

Even fairly trivial information can have big security consequences, warns Dale. “A pet’s name, or the version of Office used to create a document, may seem insignificant, but it can be used to inform a potential cyber-attack,” he says.

Other increasingly important OSINT sources are open data feeds and geospatial information, from Google and other mapping tools.

And the use of OSINT can go even deeper, potentially right into the code of a company’s web applications.

If a security team can find those OSINT weaknesses first, they can move to close them down.


Social media channels are often used in open source intelligence (OSINT) gatheringSocial media has become a readymade OSINT channel for intel gatherers

Social media and OSINT

Although online media and search engines make OSINT quicker and easier, social media has been the most effective medium for gathering information on individuals with a view to defrauding them or stealing their identities.

“Social media has made [conducting] OSINT on people super easy,” Drew Porter, president and founder at security firm Red Mesa, told The Daily Swig.

"People willingly put their info out there. If a person is in scope, looking at their social media pages for 15 minutes will tell us more about the person than one hour of OSINT will most the time.”

Hackers, pen testers, and intelligence agencies view social media profiles on sites such as LinkedIn, Facebook, and Instagram as fair game for OSINT. Some firms use scraping tools, although these are against most sites’ terms, including LinkedIn for example.

This, of course, will not deter malicious actors. And although security teams have to be careful with social media, trawling these platforms can quickly reveal serious security flaws.


Read more of our latest deep dives into security topics and trends


“Social media analysis (or SOCMINT) is a subsection of OSINT, although its value can be hampered by privacy and platform restrictions,” Louise Taggart, manager for cyber threat detection and response, and Kirsten Ward, senior associate for threat intelligence, at PwC UK, told The Daily Swig.

“Sources can include social networking sites, professional networking sites, video sharing or vlog sites, or microblogging sites, for example.

“Social media platforms can betray a considerable amount of information on a person or organization, whether it is through a seemingly innocent picture, which could be the answer to a secret question, or a colleague wishing someone happy birthday and thereby inadvertently disclosing their date of birth.”

Security teams can use social media as an entry point for social engineering, or for physical site penetration. But the breadth of social media, including images and video, means it can all too easily provide malicious actors with information about security systems and IT, often without the business realizing. With no system compromise to detect, OSINT recon stays well below the radar.

“Social networks have long been used in pen testing and red team engagements,” Jordan Cheal, senior security consultant at Bridewell Consulting, told The Daily Swig.

“They can provide a host of information for performing passive reconnaissance or enumerating data that can be weaponized and used in phishing campaigns or for performing password sprays.”

Is OSINT legal or ethical?

In the US and the UK, OSINT is legal, but security teams need to stay within a clearly defined framework, which is agreed with their clients in advance of conducting OSINT.

Much will depend on where target information resides. OSINT that gathers information where there is a reasonable expectation of public access – a blog post or a LinkedIn profile, for instance – is generally considered legal. But where data are password-protected, obtained by deception, or anonymized and aggregated, the legality is less clear-cut.


RELATEDethicsFIRST: Maintaining ethical behavior across the cybersecurity industry


It often comes down to intent, says Red Mesa’s Porter. “When we are hired to do it for companies or high net worth individuals it is 100% ethical. When someone is doing it to stalk an ex, it’s not ethical.”

At PwC, Taggart and Ward point to the Berkeley Protocol, which sets out a framework for conducting open source investigations into war crimes and human rights violations, as a useful ethical standard for cybersecurity-related OSINT, too.


OSINT, or open source intelligence, entails the use of readily available services to gather information on targetsOSINT practitioners must ensure they are operating on the right side of the law

The right choice?

Should, then, security teams use OSINT?

If they are not already, they are overlooking a vital means of spotting, and removing, sensitive information from the public domain that could be abused by malicious actors to compromise an organization. And, unlike many other methods and tools, OSINT is largely free.

Set against these benefits are the legal and ethical considerations, and the fact that security teams must exercise a certain level of skill and caution to use OSINT effectively, ethically, and legally.

“It’s also important to bear in mind, that whatever a ‘good guy’ can find through OSINT, so can the ‘bad guy’,” warn PwC’s Taggart and Ward.

OSINT tools: An expanding list

Pentesters use a wide range of tools for OSINT, with consultants often using their own tools. Some of these, such as Pentest People’s Athena, are available on GitHub. Here are some other popular OSINT tools:

  • Scrapesy: Scrapes both the clear web and dark web for exposed credentials
  • O365 Squatting: Generates typosquatting permutations and cross-references them against Office 365 infrastructure to find potential phishing websites
  • ZMap: Network scanner that discovers devices and services exposed to the internet
  • Ghunt: Finds information associated with a Google ID
  • Intel Owl: Pulls together threat analysis tool feeds into a single API
  • ReNgine: Open source tool for aggregating recon feeds
  • Shodan: IoT device search engine used to find unsecured equipment on LANs and other hardware-based weak spots
  • Social Mapper: Developed by Trustwave Spiderlabs, Social Mapper uses facial recognition, as well as usernames, to track targets across platforms
  • Spiderfoot: OSINT automation tool, available in open source and commercial versions
  • Sublist3r: Python-based sub-domain enumerator
  • theHarvester: Helps to “determine a company's external threat landscape on the internet” by gathering “emails, names, subdomains, IPs and URLs”
  • Google dorking: Less a tool than a technique, Google Dorking involves using specialist search terms to find results not visible to natural language search

The SANS Institute has also published a detailed list of OSINT tools. Check out our Latest Hacking Tools page to keep track of the latest open source releases.


READ MORE Suspected Vietnamese cyber-spies targeting dissidents in Germany

OSINTDeep DivesHacking ToolsOpen Source SoftwareResearchAnalysisHacking NewsNetwork SecurityDatabase SecurityCloud SecurityEmail SecurityPhishingSocial EngineeringVulnerabilitiesOrganizationsEnterpriseGovernmentNetherlandsEuropeUSGitHubHacking TechniquesPasswords

WhatsAppFacebookRedditLinkedInEmail

Sours: https://portswigger.net/daily-swig/osint-what-is-open-source-intelligence-and-how-is-it-used

Examples osint

What is open-source intelligence, or OSINT? The fourth meetup of the Future Female x Helsec Cyber Security Essentials training program went online as a precaution to stop the coronavirus from spreading, and the OSINT hands-on workshop had to be postponed. However, we at Nixu thought that we could arrange some reading material for self-paced learning for the participants of this free course, who are women interested in working in cybersecurity or gaining more in-depth technical knowledge. Let's have a glimpse of how we can use open-source intelligence, gathering information from public sources, in cybersecurity and what tools are useful.

What is OSINT, and why is it useful?

Open-source intelligence (OSINT) means collecting information from public sources, analyzing it, and using it for intelligence purposes. The information sources can be anything from television and print newspapers to blogs and websites, social media, research papers, business and sales documents, and anything you can find online or offline. OSINT is one of many intelligence collection types. The main categories are human intelligence (HUMINT), measurement and signatures intelligence (MASINT), signals intelligence (SIGINT), and imagery intelligence (IMINT). Sometimes HUMINT and SIGINT can overlap with OSINT.

Open-source intelligence (OSINT) means collecting information from public sources, such as social media and web pages, analyzing it, and using it for intelligence purposes.

Traditional uses of open-source intelligence lie in national security, investigating crime and cybercrime, and researching threat intelligence or investigating malware campaigns and advanced persistent threat (APT) groups. However, OSINT is also useful for regular companies, cybersecurity consultants doing penetration testing, or red teaming and privacy-aware people.  Everyone who browses and shops online and uses social media can have a surprisingly large digital footprint.

In cybersecurity, specialists mine data from open sources, combine pieces of information, and create a map or a profile of the target. The target might be an organization and its network infrastructure and services they use, a person, or a group of employees that play a vital role in the organization. The information gathered with OSINT is useful in several ways:

You can identify the attack surface: This is especially important if you are offering online services. What kind of version information is available? Are you exposing only the necessary services and information? What about your office network: are there any printers, WLAN controller management interfaces, or other unexpected hosts accessible from the internet? Can you identify potential persons who to target with social engineering attacks? This information-gathering phase is a typical stage in penetration testing and red teaming as well. However, it’s essential to understand that while passive and semi-passive data collection from open sources is considered legal, active information gathering methods, such as port or vulnerability scanning, are considered illegal if you don’t have permission. 

You can identify security gaps: Have you hardened the operating systems and applications that are exposed? Are there known vulnerabilities or weaknesses? What kind of ways there are to contact you or your organization, and are they all well-protected? 

You can fight information leaks: Accidents happen: somebody might accidentally publish material that wasn't supposed to be released yet. Maybe another online service suffered a data breach, and the email addresses and passwords are now in Pastebin – including your employees' passwords that might work for your web services as well. If you notice problems early, you have more time to react.

Outside security, open-source intelligence techniques, and tools can be useful for investigating market opportunities and checking what your competitors are doing.

Information sources and tools

There are numerous publicly available sources, both online and offline, that you can use for gathering information. Have you ever thought about how much data your social media profiles reveal when you take into account all your connections, as well? There's also more than meets the eye. File metadata often reveals interesting information about the author of the document and the tools and the operating system used for creating the file. Image metadata may contain the location where the picture was taken and thus your whereabouts.

The mind map below shows some common information sources, their uses, and what tools you can use for getting and analyzing the data. The mindmap has been from the point of view that an organization is a target. If the goal would be to gather information from an individual, say for social engineering or phishing attacks in penetration testing, many of the same information sources and tools apply.

Mindmap about OSINT tools and information sources

One of the biggest challenges in OSINT is to handle the loads of information with which you will very typically end up. Tools come handy pretty quickly so you can automate data collection, organize the data, and find links between individual pieces of information. Tools like Maltego visualize the info so you can examine it more easily.

It's also good to understand that all information sources may not be reliable, so you may need to filter out some of it. You also need to pay attention to securely storing the data and respecting privacy, since the data may point out significant weaknesses in an organization and contain personal data.

The dark side of OSINT

All information that is out there is also available for cybercriminals. When you look at the mind map showing possible information sources and tools, you can begin to imagine all the potential malicious uses of that data. Identity frauds, social engineering, CEO frauds, targeted attacks with customized malware, … open-source intelligence can be a gold mine if somebody is interested in breaking into your company's IT systems. That's why it's better to understand your exposure. You cannot wholly remove data that has been published on the internet once, but you can make it more difficult to find. And what's more important, you can ensure that you don't have vulnerable services running and otherwise minimize your attack surface.

Want to learn more about OSINT?

If you want to learn more about open-source intelligence, take a look at the following material:

Want to keep track of what's happening in cybersecurity? Sign up for Nixu Newsletter.

Sours: https://www.nixu.com/blog/open-source-intelligence-its-incredible-what-you-can-find-public-sources
OSINT At Home #4 – Identify a location from a photo or video (geolocation)

15 top open-source intelligence tools

OSINT definition

Open source intelligence (OSINT) is the practice of collecting information from published or otherwise publicly available sources. OSINT operations, whether practiced by IT security pros, malicious hackers, or state-sanctioned intelligence operatives, use advanced techniques to search through the vast haystack of visible data to find the needles they're looking for to achieve their goals—and learn information that many don't realize is public. Open source in this context doesn't refer to the open-source software movement, although many OSINT tools are open source; instead, it describes the public nature of the data being analyzed.

OSINT is in many ways the mirror image of operational security (OPSEC), which is the security process by which organizations protect public data about themselves that could, if properly analyzed, reveal damaging truths. IT security departments are increasingly tasked with performing OSINT operations on their own organizations to shore up operational security.

OSINT history: From spycraft to IT

During the 1980s, the military and intelligence services began to shift some of their information-gathering activities away from covert activities like trying to read an adversary’s mail or tapping their phones to discover hidden secrets. Instead, effort was put into looking for useful intelligence that was freely available or even officially published.

The world at the time was changing, and even though social media had not yet made the scene, there were plenty of sources like newspapers and publicly available databases that contained interesting and sometimes useful information, especially if someone knew how to connect a lot of dots. The term OSINT was originally coined to refer to this kind of spycraft.

These same techniques can now be applied to cybersecurity. Most organizations have vast, public-facing infrastructures that span many networks, technologies, hosting services and namespaces. Information can be stored on employee desktops, in legacy on-prem servers, with employee-owned BYOD devices, in the cloud, embedded inside devices like webcams, or even hidden in the source code of active apps and programs.

In fact, the IT staff at large companies almost never knows about every asset in their enterprise, public or not. Add in the fact that many organizations also own or control several additional assets indirectly, such as their social media accounts, and there is potentially a lot of information sitting out there that could be dangerous in the wrong hands.

Why is OSINT important?

OSINT is crucial in keeping tabs on that information chaos. IT needs to fulfill three important tasks within OSINT, and a wide range of OSINT tools have been developed to help meet those needs. Most tools serve all three functions, though many excel in one particular area.

Discovering public-facing assets

Their most common function is helping IT teams discover public facing assets and mapping what information each possesses that could contribute to a potential attack surface. In general, they don’t try to look for things like program vulnerabilities or perform penetration testing. Their main job is recording what information someone could publicly find on or about company assets without resorting to hacking.

Discover relevant information outside the organization

A secondary function that some OSINT tools perform is looking for relevant information outside of an organization, such as in social media posts or at domains and locations that might be outside of a tightly defined network. Organizations that have made a lot of acquisitions, bringing along the IT assets of the company they are merging with, could find this function very useful. Given the extreme growth and popularity of social media, looking outside the company perimeter for sensitive information is probably helpful for just about any group.

Collate discovered information into actionable form

Finally, some OSINT tools help to collate and group all the discovered information into useful and actionable intelligence. Running an OSINT scan for a large enterprise can yield hundreds of thousands of results, especially if both internal and external assets are included. Piecing all that data together and being able to deal with the most serious problems first can be extremely helpful.

Top OSINT tools

Using the right OSINT tool for your organization can improve cybersecurity by helping to discover information about your company, employees, IT assets and other confidential or sensitive data that could be exploited by an attacker. Discovering that information first and then hiding or removing it could reduce everything from phishing to denial-of-service attacks.

Following (in no particular order) are some of the top tools used for OSINT, what areas they specialize in, why they are unique and different from one another, and what specific value they might be able to bring to an organization’s cybersecurity efforts.

  • Maltego
  • Mitaka
  • SpiderFoot
  • Spyse
  • BuiltWith
  • Intelligence X
  • DarkSearch.io
  • Grep.app
  • Recon-ng
  • theHarvester
  • Shodan
  • Metagoofil
  • Searchcode
  • SpiderFoot
  • Babel X

Maltego

Maltego specializes in uncovering relationships among people, companies, domains and publicly accessible information on the internet. It’s also known for taking the sometimes enormous amount of discovered information and plotting it all out in easy-to-read charts and graphs. The graphs do a good job of taking raw intelligence and making it actionable, and each graph can have up to 10,000 data points.

The Maltego program works by automating the searching of different public data sources, so users can click on one button and execute multiple queries. A search plan is called a “transform action” by the program, and Maltego comes with quite a few by default that include common sources of public information like DNS records, whois records, search engines and social networks. Because the program is using public interfaces to perform its searching, it’s compatible with almost any source of information that has a public interface, so adding more searches to a transform action or making up a whole new one is easily possible.

Once the information is gathered, Maltego makes connections that can unmask the hidden relationships between names, email addresses, aliases, companies, websites, document owners, affiliations and other information that might prove useful in an investigation, or to look for potential future problems. The program itself runs in Java, so it works with Windows, Mac and Linux platforms.

There is a free version of the program with limited features called Maltego CE. Desktop versions of Maltego XL run $1,999 per instance. Server installations for large-scale commercial use start at $40,000 and come with a complete training program.

Mitaka

Available as a Chrome extension and Firefox add-on, Mitaka lets you search over six dozen search engines for IP addresses, domains, URLs, hashes, ASNs, Bitcoin wallet addresses, and various indicators of compromise (IOCs) from your web browser. sharma osint 1Ax Sharma

The extension saves up your time by acting as a shortcut to various online databases that can be queried with a click.

For those who prefer a focused, more limited set, an alternative extension Sputnik is also available.

Spiderfoot

Spiderfoot is a free OSINT reconnaissance tool that integrates with multiple data sources to gather and analyze IP addresses, CIDR ranges, domains and subdomains, ASNs, email addresses, phone numbers, names and usernames, BTC addresses, etc. Available on GitHub, Spiderfoot comes with both a command-line interface and an embedded web-server for providing an intuitive web-based GUI.

The application itself comes with over 200 modules making it ideal for red teaming reconnaissance activities, to discover more information about your target or identify what you or your organisation may be inadvertently exposing on the internet.

Spyse

Spyse describes itself as the “most complete internet assets registry” geared toward cybersecurity professionals. Relied on by projects like OWASP, IntelligenceX, and the aforementioned Spiderfoot, Spyse collects publicly available data on websites, their owners, associated servers, and IoT devices. This data is then analyzed by the Spyse engine to spot any security risks in and connections between these different entities.

A free plan is available, although for developers planning on building apps using the Sypse API, paid subscriptions may be required.

BuiltWith

As the name implies, BuiltWith lets you find what popular websites are built with. Different tech stacks and platforms power different sites. BuiltWith can, for example, detect whether a website is using WordPress, Joomla, or Drupal as its CMS and provide further details.

BuiltWith also generates a neat list of known JavaScript/CSS libraries (e.g., jQuery or Bootstrap) that a website uses. Further, the service provides a list of plugins installed on the websites, frameworks, server information, analytics and tracking information, etc. BuiltWith can be used for reconnaissance purposes.

What’s more? Combine BuiltWith with website security scanners like WPScan that, for example, integrate with WordPress Vulnerability Database API to spot common security vulnerabilities impacting a website.

For those looking to identify mainly the tech stack makeup of a site, Wappalyzer may be better suited as it provides a more focused, concise output. Try both BuiltWith and Wappalyzer for yourself and see which suits your needs better.

Intelligence X

Intelligence X is a first-of-its-kind archival service and search engine that preserves not only historic versions of web pages but also entire leaked data sets that are otherwise removed from the web due to the objectionable nature of content or legal reasons. Although that may sound similar to what Internet Archive’s Wayback Machine does, Intelligence X has some stark differences when it comes to the kind of content the service focuses on preserving. When it comes to preserving data sets, no matter how controversial, Intelligence X does not discriminate.

Intelligence X has previously preserved the list of over 49,000 Fortinet VPNs that were found vulnerable to a Path Traversal flaw. Later during the week, plaintext passwords to these VPNs were also exposed on hacker forums which, again, although removed from these forums, were preserved by Intelligence X.

Previously, the service has indexed data collected from email servers of prominent political figures like Hillary Clinton and Donald Trump. Another recent example of the media indexed by on Intelligence X is the footage from the 2021 Capitol Hill riots and the Facebook’s data leak of 533 million profiles. To intel gatherers, political analysts, news reporters, and security researchers, such information can be incredibly valuable in various ways.

DarkSearch.io

While frequent visitors to the dark web may already be familiar with where to look for what, for those who may be new, DarkSearch.io can be a good platform for starting with their research activities. Like another dark web search engine Ahmia, DarkSearch is free but comes with a free API for running automated searches. Although both Ahmia and DarkSearch have .onion sites, you don’t need to necessarily go to the .onion versions or use Tor for accessing either of these search engines. Simply accessing darksearch.io from a regular web browser will let you search the dark web.

Grep.app

How do you search across half million git repos across the internet? Sure, you could try individual search bars offered by GitHub, GitLab, or BitBucket, but Grep.app does the job super efficiently. In fact, Grep.app was recently used by Twitter users and journalists on multiple occasions to get an idea of approximately how many repositories were using the Codecov Bash Uploader:

sharma osint 2Ax Sharma

Grep.app can also be useful when searching for strings associated with IOCs, vulnerable code, or malware (such as the Octopus Scanner, Gitpaste-12, or malicious GitHub Action cryptomining PRs) lurking in OSS repos.

Recon-ng

Developers who work in Python have access to a powerful tool in Recon-ng, which is written in that language. Its interface looks very similar to the popular Metasploit Framework, which should reduce the learning curve for those who have experience with it. It also has an interactive help function, which many Python modules lack, so developers should be able to pick it up quickly.

Recon-ng automates time-consuming OSINT activities, like cutting and pasting. Recon-ng doesn't claim that all OSINT gathering can be conducted by its tool, but it can be used to automate much of the most popular kinds of harvesting, leaving more time for the things that still must be done manually.

Designed so that even the most junior Python developers can create searches of publicly available data and return good results, it has a very modular framework with a lot of built-in functionality. Common tasks like standardizing output, interacting with databases, making web requests and managing API keys are all part of the interface. Instead of programming Recon-ng to perform searches, developers simply choose which functions they want it to perform and build an automated module in just a few minutes.

8 pitfalls that undermine security program success

Sours: https://www.csoonline.com/article/3445357/what-is-osint-top-open-source-intelligence-tools.html

You will also be interested:

What is OSINT?

As information becomes more available from a vast number of sources, skilled researchers can often find nearly any type of data they’re looking for, provided they know where to look. These sources include both public and private databases that hackers, journalists, spies, and ordinary people all use to do the work of collecting information every day.

Much of this information simply isn’t able to be retrieved through search engines and requires knowledge of where the right database is located to get the right answers. Fortunately, there are a multitude of tools that both beginners and seasoned investigators can take advantage of to use OSINT sources while conducting research. 

Get the Free Pen Testing Active Directory Environments EBook

“This really opened my eyes to AD security in a way defensive work never did.”

What Should I Do for OSINT to Protect My Organization?

Every organization creates data, and much of this data may be public if it’s produced or recorded by a local or state government. Aside from business data, technical data from registration or development of websites or products can expose information about the internal workings of an organization that might otherwise be impossible to find.

The first thing to know about OSINT is that it cuts both ways, and the same data used to learn about a competitor can also be used to infer information about your company or organization. It can be incredibly rewarding to discover what information about you might be more public than you expect and understand how an outsider could know internal details about your organization that make it easier to attack.

How Do I Conduct an OSINT Investigation?

An OSINT investigation starts with answering an answerable question. Because there is no shortage of data out there, it’s easy to get lost in all the noise if what you’re looking for isn’t clear and answerable in the first place. An example of this might be questioning “is New York safer than Los Angeles?” versus “Have reports of police misconduct increased in Los Angeles county since 1999?” One question calls for an opinion and is difficult to answer, where the second can be easily answered by finding the right record holder.

After establishing the question to answer, the next step is to identify the most likely owner of the best quality data. Primary source information like government or corporate data are the best sources, followed by information created by trade-based organizations or non-governmental agencies (NGO’s) related to the industry or trade your question is related to. Finally, third-party aggregators of data can provide useful links back to a primary source of data, although this “tertiary” data cannot be used as evidence. 

Generally, an investigation will develop an understanding of a subject by asking answerable questions and using the best quality answers to paint a picture. While third-party reports like a newspaper quoting another source are useful, it’s important to be critical of the sources you find during an OSINT investigation. Because there is so much information, much of it inaccurate or published by unreliable sources, OSINT researchers must be extremely critical of the sources of data used, preferring only to use verified primary source data to draw conclusions. 

What OSINT Tools & Frameworks Are There?

Because OSINT covers so many different types of data, there are many different types of investigations that can be conducted. This ranges from social media investigations using free tools to comb through vast amounts of Tweets, to geospatial investigations using satellite imagery to locate where photos were taken. 

There are many different resources for investigators looking for more OSINT tools, including the osintframework.com website which lists a huge number of data sources for different types of OSINT investigations. There are also many fantastic free OSINT tools on Github, many of them curated into this “Awesome-OSINT” list – https://github.com/jivoi/awesome-osint

What OSINT Techniques Are There?

OSINT tactics can be divided into active and passive techniques, with active tactics involving some sort of actual contact with the target, and passive tactics avoiding any contact with the target. Active techniques always involve some small risk of the target detecting you are investigating them, whereas passive tactics usually involve querying a database maintained by someone else, and usually do not involve any risk of being detected.

 

An active OSINT tactic could be as simple as scanning a website or web server owned by a target, or registering to download a competitor’s product catalog. While this small contact probably won’t blow your investigation, matching a download to your organization’s IP address could tip off a subject that your organization is investigating. Comparatively, a passive technique like using a search engine like Shodan to examine services a company is running without scanning them yourself would be nearly impossible for the target to detect.

How Does OSINT Relate to SOCMINT & HUMINT?

HUMINT, or human intelligence, is old-school intelligence collection done using human sources to collect information. With the explosion of data available to OSINT investigators, HUMINT has been supercharged by the ability to learn nearly everything about the person you need to interview before you even meet them. This gives the investigator a huge advantage when interviewing a source, and can make HUMINT a natural and extremely valuable add-on to any OSINT investigation.

OSINT is the secret power behind many HUMINT collection wins, providing the context to get a person to do something they’re not supposed to do. Often with the right OSINT information, it’s possible to get someone who is aware they hold sensitive information to either grant you access because they think they are supposed to, or because they think you already know the information they have access to.

By taking advantage of the information about a company’s employees on social media as well as internal policies leaked by documents easily found through passive search techniques, it’s simple to understand who has access to the information an investigator needs. This can cut down the time needed to get answers substantially by getting the exact right answer from the right source earlier. 

How Do I Find OSINT on Phone Numbers?

Phone numbers are often linked to individuals through phone books, social media accounts, and data leaked by businesses who have worked with someone before. When phone numbers turn up in an OSINT investigation, third-party aggregators like www.opencnam.com, thatsthem.com, and truecaller.com can begin matching the number to businesses or people.

Once you have an initial match, you’ll want to pull details like addresses, names, and social media names to expand the scope of your discovery to other web accounts or documents related to your target. Matching each of these details to even more information is often how a phone number can lead to a business license or other more concrete details.

How Do I Find OSINT on Names?

Names often appear in documents related to business filings, so the best quality results for name searches related to business owners or powerful people will often be on websites like lilsis.org or OpenCorporates. For normal people, third party aggregators will often attempt to stitch together lots of information about people by name, especially if you know the general area they live in.

While third-party aggregators can’t be used as answers to your investigation, they often point to better sources of data. To start an investigation, sources like pipl.com, beenverified.com, and peekyou.com can get you started, but should only be used to point to other accounts or searchable details like past addresses or phone numbers.

How Do I Find OSINT on Businesses?

Businesses are some of the easiest entities to find information on, as they need to register a lot of paperwork with public entities to exist. This paperwork is often entirely public and searchable, giving investigators a trove of information full of details which help to expand an investigation in the early stages. 

To get started, it’s best to search the secretary of state website of whichever state a particular organization does business in. This should provide a starting point of data, but can often be expanded on by searching additional states as well. Because there are tax and legal incentives to register in certain states in the US, it’s often worthwhile to check the secretary of state databases in Nevada, Delaware, California, and Wyoming as well.

 

After locating primary source data, third party aggregators like OpenCorporates give you the ability to search all secretary of state databases indexed by the service, often locating valuable primary source information somewhere you might have missed. Lilsis.org is also a good source of information about business people including stock options and tax information. 

How Do I find OSINT on Websites?

Websites represent a lot of information about a business, both technical and in the way the website has changed messaging and branding over time. For OSINT investigations, websites are often looked at for their technical information, like who registered it, what servers are in use, and what software is maintaining it. This can be done through services like Shodan, which allow you to profile an organization’s technical infrastructure without actually scanning it yourself.

Another source of information is the actual contents of the website, which can include files left unintentionally public, or information that may have been removed from the website in the past. Tools like the Internet Archive can show an investigator how the website has changed over time by comparing snapshots taken years earlier, often pointing to organizational changes or showing who has left or joined the company and when.

Google Dorking is also a powerful way of finding files and other details left open to the internet by accident. By structuring Google search queries to look for certain types of web pages or files, it’s easy to find any parts of a target’s website that might be leaking confidential information.

What Resources Are There For OSINT Research?

There is a huge community around OSINT investigations that loves to share tricks and techniques for investigations. In particular, Mike Bazell hosts an amazing website and podcast about OSINT that constantly points to new and innovative investigation techniques, and both Twitter and Github are full of new OSINT tools being developed and released by the OSINT community.

What Dangers Are There in Conducting OSINT Research?

While OSINT is based on public data, there are always risks when doing any sort of investigation that involves either making direct contact with the subject, or a third party who may sell your search to the third party directly. In the example of LinkedIn, you may find a link to a subject’s LinkedIn profile and forget to sign out of your own, prompting LinkedIn to offer the target the information that you were looking at their profile as part of their product. 

This kind of contact can blow an investigation, as can accessing a web resource belonging to a target from your organization’s IP address. If you work for an organization that does not want to be identified while investigating a target, you must always use a VPN to hide the IP address of the place that you’re working from, or risk making your attention to any particular part of the target’s infrastructure obvious. 

Is Data Collected in an OSINT Investigation Regulated Under Privacy Laws Like GDPR, CCPA?

Thanks to the many loopholes provided by CCPA, OSINT investigations are generally not something California residents need to worry about. GDPR, however, is more strict with these rules, meaning it’s important to take basic steps during an investigation to ensure you don’t expose personal information of a subject and that you’re storing it properly. This means using encryption to store your investigation notes and not leaving them where they can be accessed by a third party.

What Are Some Examples of OSINT Investigations?

Bellingcat has produced some of the most dramatic OSINT investigations in recent memory, in particular blowing the lid off denials by the Russian government of involvement in various operations by digging through databases to prove their involvement directly. 

Memorable examples include tying Russian spies to their secret employment by digging through car registration databases (https://www.bellingcat.com/news/2018/10/04/305-car-registrations-may-point-massive-gru-security-breach/comment-page-5/), and linking social media photos to prove Russian soldiers were operating in areas the government claimed they weren’t. Many OSINT investigations cover topics like war crimes in areas too remote or dangerous to access remotely and was used extensively in the investigation into the downing of Malaysian Airlines Flight 17 (MH17). (https://www.bellingcat.com/news/uk-and-europe/2017/12/08/russian-colonel-general-delfin/)

Who Makes Use of OSINT Reports?

OSINT is a critical part of both public and private intelligence, arming businesses, governments, and individual investigators with a vast amount of high-quality information to base and make decisions on. Whether conducting an investigation for research, business intelligence, or threat analysis, OSINT can allow anyone to have access to some of the best available data in the world. 

 

 

Kody Kinzie

Kody Kinzie

Kody Kinzie is a security researcher who specializes in open-source intelligence and Wi-Fi security. He teaches cybersecurity to beginners on two popular YouTube channels called Hak5 and Null Byte, as well as organizing cybersecurity training and outreach events in Los Angeles.

Sours: https://www.varonis.com/blog/what-is-osint/


1985 1986 1987 1988 1989