Article

Last Updated on Sun Jul 24, 2022

Top 10 Awesome Open-Source Adversary Simulation Tools

Written by Hardik Manocha

Co-founder @ FourCore

Open-Source Breach and Attack Simulation

Cyberattack simulation, aka Threat Simulation, is an emerging IT security technology that can help discover gaps, vulnerabilities, and misconfigurations in your security infrastructure. We will take a look at the need for adversary simulation and the top ten open-source adversary simulation tools.

What is Adversary Simulation?

Adversary simulation/emulation is the process of mimicking the attacker's behaviour. It offers the capability to test an organisation's resilience against an advanced attacker in a situation known as Assumed Breach, i.e. all simulations/tests are run by the system willingly. Although adversary simulation sounds much like automated penetration testing on the surface, this type of simulation covers a broader specturm of the security infrastructure. The job is not always to get to the crown jewels; instead, identify the different attack paths an adversary might take, mitigate the most impactful threats and prioritise with an actionable remediation plan to maximise security resources and reduce cyber risks.

Simulation refers to the ability to mimic malicious actors' techniques, procedures and tactics (TTPs). Most attack simulation tools and platforms provide an automated or semi-automated means of attaining the attacker's perspective of the victim's network.

Why is Adversary Simulation helpful for you?

As a CISO, you must maintain a proactive security program that defends your organisation against all sorts of cyber attacks. However, even maintaining a security baseline is difficult, let alone having a mature security program, to begin with.

It is a constant war against the attackers to stay ahead of the game. The answer, however, lies in maintaining a healthy security posture by continuously validating your security controls against all sorts of adversary behaviours.

As a CISO, you may find yourself in a situation where you have to answer the board on how you will allocate the cyber security fund to have the maximum return on investment (ROI). It has been challenging for CISOs to supply metrics to board members for years. It is often hard to give additional context and explain their security program's effectiveness in more detail and in C-Suite level language. However, the answer lies in continuous security validation.

Recently, companies have had a more aggressive approach to testing security controls continuously through Breach and Attack Simulation platforms. With continuous security controls validation, companies can simulate different types of cyber-attacks, including insider threats and lateral movements by attackers. This new proactive approach of continual testing has given companies the feedback they need to be successful, benchmarked against the MITRE ATT&CK framework.

Adversary simulation is different. It focuses on the customer's ability to deal with an attack, post-compromise. These assessments look at incident response and provide a valuable "live fire" training opportunity for the analysts who hunt for and respond to incidents each day. Recognising that conventional penetration testing requires more skills, resources and time than most companies have available, many have turned to breach and attack simulation (BAS) tools.

Open Source Adversary Emulation Tools

Here's an overview of the top open source adversary emulation tools:

MITRE CALDERA

MITRE CALDERA is a cybersecurity framework developed by MITRE that empowers cyber practitioners to save time, money, and energy through automated security assessments. It offers an intelligent, automated adversary emulation system that can reduce resources needed by security teams for routine testing, freeing them to address other critical problems.

Caldera leverages the ATT&CK model to identify and replicate adversary behaviors as if a real intrusion is occurring. It empowers cyber teams in three main ways:

Autonomous Adversary Emulation allowing teams to build a specific threat (adversary) profile and launch it in a network to see where you may be susceptible. This helps with testing defenses and training blue teams on how to detect specific threats.
Autonomous Incident Response enables your team to perform automated incident response on a given host, allowing them to find new ways to identify and respond to threats.
Manual Red-Team Engagements helps your red team perform manual assessments with computer assistance by augmenting existing offensive toolsets. The framework can be extended with any custom tools you may have.

Atomic Red Team

Atomic Red Team is a library of simple tests that every security team can execute to test their defenses. Tests are focused, have few dependencies, and are defined in a structured format that can be used by automation frameworks. The ART maps small and highly portable detection tests to the Mitre ATT&CK Framework. This framework is not automated, yet supports Microsoft Windows, MacOS & Linux flavours.

It’s used for many purposes, including but not limited to:

Validating assumptions about security controls
Testing detection coverage
Learning what malicious activity looks like

DumpsterFire

The DumpsterFire Toolset is a modular, menu-driven, cross-platform tool for building repeatable, time-delayed, distributed security events. Easily create custom event chains for Blue Team drills and sensor / alert mapping. Red Teams can create decoy incidents, distractions, and lures to support and scale their operations. Turn paper tabletop exercises into controlled "live fire" range events. Build event sequences ("narratives") to simulate realistic scenarios and generate corresponding network and filesystem artifacts.

The toolset is designed to be dynamically extensible, allowing you to create your own Fires (event modules) to add to the included collection of toolset Fires. Just write your own Fire module and drop it into the FireModules directory. The DumpsterFire toolset will auto-detect your custom Fires at startup and make them available for use.

firedrill from FourCore

firedrill is an open-source library from FourCore Labs to build malware simulations quickly. We have built a set of four different attack simulations for you to use and build on top of Ransomware Simulations, Discovery Simulation, UAC Bypass, and Persistence Simulation. Download them now from the firedrill GitHub repo. In addition, you can use the firedrill binaries to find out how effective your antivirus is or if your sandbox is detecting the proper signatures. To learn more about Firedrill, read the linked blog by FourCore.

Mordor

The Mordor project provides pre-recorded security events generated by simulated adversarial techniques in the form of JavaScript Object Notation (JSON) files for easy consumption. The pre-recorded data is categorized by platforms, adversary groups, tactics and techniques defined by the Mitre ATT&CK Framework. The pre-recorded data represents not only specific known malicious events but additional context/events that occur around it. This is done on purpose so that you can test creative correlations across diverse data sources, enhancing your detection strategy and potentially reducing the number of false positives in your own environment.

The name Mordor comes from the awesome book/film series "The Lord of the Rings", and it was a place where the evil forces of Sauron lived. This repository is where data generated by known "malicious" adversarial activity lives, hence the name of the project.

Roberto Rodriguez created Mordor to help analysts who might not have extensive testing or red team experience simulate adversary behaviors to test their ability to detect or prevent.

The Mordor Projects fulfils the following goals:

Provide free portable malicious datasets to expedite the development of data analytics.
Facilitate adversarial techniques simulation and output consumption.
Allow security analysts to test their skills with real known bad data.
Improve the testing of hunting use cases and data analytics in an easier and more affordable way.
Enable data scientists to have semi-labeled data for initial research.
Map threat hunter playbooks to their respective pre-recorded data for validation purposes.

Sharing a wonderful article by Roberto employing Mordor Datasets: https://medium.com/threat-hunters-forge/threat-hunter-playbook-mordor-datasets-binderhub-open-infrastructure-for-open-8c8aee3d8b4

Infection Monkey from Guardicore

Infection Monkey is an open-source breach and attack simulation (BAS) platform that helps you validate existing controls and identify how attackers might exploit your current network security gaps. The Monkey uses various methods to self propagate across a data center and reports success to a centralized Monkey Island server. It is also coded in Python and is environment agnostic. On-premises, containers, public and private clouds are all supported. It allows continuous testing to regularly run the platform to test your security strategy and specific controls. It also provides actionable data using reports on your network’s performance against a broad set of attacker behaviors.

Red Team Automation

Red Team Automation provides a framework of scripts designed to allow blue teams to test their detection capabilities against malicious tradecraft, modeled after MITRE ATT&CK. RTA is composed of python scripts that generate evidence of over 50 different ATT&CK tactics, as well as a compiled binary application that performs activities such as file timestopping, process injections, and beacon simulation as needed.

Where possible, RTA attempts to perform the actual malicious activity described. In other cases, the RTAs will emulate all or parts of the activity. For example, some lateral movement will by default target local host (though with parameters typically allow for multi-host testing). In other cases, executables such as cmd.exe or python.exe will be renamed to make it appeas as if a Windows binary is doing non-standard activities.

A great article by Elastic: https://www.elastic.co/blog/introducing-endgame-red-team-automation

Stratus Red Team

As the developers put it, Stratus Red Team is Atomic Red Team for the Cloud. It comes with a variety of common attack techniques mapped to the MITRE ATT&CK matrix that attackers can use to exploit your AWS environment. You can use the single Stratus Red Team binary available for free on GitHub.

Some of the attack techniques you can simulate:

Find all the techiques availalble here.

Metta

Uber open sourced this adversarial simulation tool, which was born out of multiple internal projects. Metta uses Redis/Celery, python, and vagrant with VirtualBox to perform adversarial simulation, which allows you to test your host based security systems. This also may allow you to test other network based security detection and controls depending on how you set up your vagrants. Metta is compatible with Microsoft Windows, MacOS and Linux endpoints. More details on Metta here.

Encripto Blue Team Training Toolkit

Encripto Blue Team Training Toolkit (BT3) is software for defensive security training, which will bring your network analysis training sessions, incident response drills and red team engagements to a new level. The toolkit allows you to create realistic computer attack scenarios, while reducing infrastructure costs, implementation time and risk.Blue Team Training Toolkit. It is written in Python and includes the latest version of Encripto’s Maligno, Pcapteller and Mocksum. It also includes multiple malware indicator profiles that ensure a “plug & play” experience, when planning and preparing a training session, incident response drill or red team engagement.

Final Thoughts

Network security professionals have long used open-source BAS tools such as Atomic Red Team, Caldera, and Metasploit to run tests against their infrastructure on an ad hoc basis. Nonetheless, growing awareness of the need for regular testing has driven a surge in automated BAS services that increase the frequency and consistency of testing.

At FourCore, we believe in simplifying security for everyone. As a result, we have launched a breach and attack simulation platform, FourCore ATTACK, to assess your organisation against the most sophisticated threats continuously so that you can discover gaps in real-time.