Threat Simulation with Miter Caldera
What is Miter Caldera?
Based on an ATT&CK project, Caldera; generates plans during the operation using an attacker's technical, tactical, and attack patterns. These features allow Caldera to operate on a range of systems using variable behaviors. This better represents how attackers perform transactions than systems that “follow a defined set of events” before.
The working logic of Caldera is based on sending attack scenarios to the relevant agent from the server where Caldera is installed, and generating reports on whether the attacks made within the scope of the scenarios are successful or not.
https://pentesttools.net/caldera-automated-adversary-emulation-system/
Server:
- Planner: The main engine that allows the caldera to choose actions
- Attacker Model (Attacker Model): current actions based on ATT & CK
- World Model: Model of the Environment
- Execution Engine: It provides the application of techniques and updating the database.
- Database: As it learns the environment, it saves the data here.
- HTTP Server
(Client) Endpoints:
- Agent: Client on endpoint systems used for communication
- RAT: Remote access tool used to mimic attacker behavior during operation
Setup
Caldera is pretty simple to install, https://github.com/mitre/caldera by downloading from:
git clone https://github.com/mitre/caldera.git –recursive –branch xxx
pip install -r requirements.txt
python server.py –insecure
It can be installed using commands. Here, the version/release number should be written in the place indicated by xxx. If we got the output “All system ready” after the Python server.py start command98 then everything is fine and we can now reach the caldera.
http://localhost:8888 If we go to the address, our login screen will welcome us here. We can login with red-blue as username and admin as password.
Server configuration
Path to configuration file conf/default.yml
The YAML configuration file contains all the necessary configuration of CALDERA to boot. An example configuration file is below:
Port: The port number served by CALDERA
Plugins: A list of all installed plugins. An add-on must be in this list to be available while CALDERA is running. Adding a plugin to this list causes the plugin's hook.py file to be called when CALDERA is opened.
Users: Username / password information of all accounts you want to access CALDERA. Users can be in red/blue groups.
Reports_dir: The directory where all reports will be saved when the server is shut down
app.contact.http: The http location where you want HTTP agents (such as Sandcat) to connect to.
app.contact.tcp: TCP socket where you want reverse-shell agents (like Manx) to connect to.
app.contact.udp: UDP socket where you want UDP agents (like Manx) to connect to
app.contact.websocket: websocket port agents
Plugin Library
Sandcat (54ndc47): In other words, the Sandcat plugin known as 54ndc47 is the default agent that CALDERA ships with. 54ndc47 is written in GoLang for cross-platform compatibility. 54ndc47 agents request network connectivity to CALDERA on port 8888.
- Deploy: To deploy 54ndc47, use one of the built-in deployment commands that allow you to run the agent on any operating system. Each of these commands downloads the compiled 54ndc47 executable from CALDERA and runs it immediately. You can find the commands in the Sandcat plugin tab.
- Options: When deploying a 54ndc47 agent, there are optional parameters you can use when you start the executable:
- Server: This is where CALDERA is located. The agent must have a connection to this host/port.
- Group: This is the group name you want the agent to join when it starts. The group does not need to exist. If no group is migrated, the default group my_group is used.
- v: Use -v to see detailed output of sandcat. Otherwise, the sandcat will run silently.
Mock: The Mock plugin adds a set of emulation tools to CALDERA and allows you to perform full operations without plugging any other computers into your server. These agents conf/agents.yml is created inside the file. They are editable and you can create as many as you want. An example agent looks like this:
Terminal: The Terminal extension adds reverse-shell to CALDERA along with a TCP-based agent called Manx. When this plugin is installed, you access a new GUI page that allows you to drop a reverse-shell on target hosts and interact with them manually.
Compass: Create visualizations to explore TTPs.
Caltack: Plugin is public MITER ATT&CK™ framework adds the website to CALDERA. This is an operator MITER ATT&CK™ framework useful for CALDERA deployments where it cannot access the Internet to refer to its matrix.
SSL: Adds HTTPS to Caldera
Human: The Human plugin allows you to create “Humans” that will perform user actions on a target system as a way to hide Caldera's red-team actions.
Training: This plugin allows the user to earn a “User Certificate” that measures their ability to use CALDERA.
Access: This add-on lets you assign tasks from the database to any agent with any skill. It also allows you to perform first-access attacks.
Atomic: Atomic Red Team is an open source software that allows you to perform attack tests developed in accordance with Miter ATT&CK Framework. Each test is designed to implement a specific tactic. Atomic Red Team thus provides a very suitable environment for blue team members to immediately start testing their defenses against a wide range of attacks.
Caldera Agent Creation
This is the screen that welcomes us after entering the system. First, to add an agent to a system here, we select an agent under the campaigns section on the upper left.
Here is a list of active agents that we have added. To add a new agent “Click here to deploy an agent” We have to click the button. Here we are presented with 3 options. The first of these is the sandcat plugin. Another is to communicate with the agent using a reverse-shell connection. Preferably, we continue with the sandcat, which we think will work more stable.
After selecting Sandcat, it will be enough to take a command line code suitable for the client or server that is running on the client or server that we want to install the agent, and run it on the opposite computer.
In order to install the agent on a Windows operating system, it will be sufficient to take the code seen in the screenshot below and run it on powershell on the Windows system.
Plugins
In this section, we'll take a look at how the plugins described above work in practice.
Access
Thanks to the agents we mentioned here in Access, there are tools that we can use directly on the system. First, on the left, we select which system we want to run the tools on. Under that, we can select and train tactics, technique and finally abilities. As an example, the outputs of the system that has run a tool are below. Of course, the conclusions drawn from this are also important with what authority your agent is connected to the system.
Thanks to the agents, the outputs of the commands we have run are separated as success/fail. We can see the output of successfully running commands by clicking * under the relevant process. A few sample outputs are as follows.
WMIC Process Enumeration
Install PStools
game board
The GameBoard plugin allows you to monitor both red and blue team actions. The game tracks points for both sides and determines which one will "win". In short, it is a plugin that shows how much of the red team activities can capture the blue team teams and take action accordingly.
You can get a score for both the red and blue teams by adding the operations you will perform through the operations that we will talk about later on the left side.
manx
The Manx tool, written in GoLang, connects to the presentation over the TCP port. This raw TCP socket connection allows Manx to establish a permanent connection between the host and the server. Along with Manx is a reverse-shell administration tool called terminal that allows you to create a local shell through an agent. In order to use this, you need to install the agent on the relevant system using manx.
Compass: Create visualizations to explore TTPs. You can save the visualization you have created or upload another file here.
Training
This is an area where you can measure your Caldera skills. There are questions on the right in the picture below. You are trying to solve all the questions in a CTF-like structure. It notifies you when it's last done.
advanced
When we first enter the advanced settings, the first thing that welcomes us is the configuration. After the above installation, I mentioned the configuration file again, when you need to make a setting, not that file, you can perform them via caldera.
The part under it is Plugins. In the configuration file, we had to write the plugins we wanted there, there is no need for it here, you can activate and use these plugins by clicking the enable button on the right side.
Then the part under the plugins obfuscators part has. Here is the section where you can obfuscate the commands you use through the agent.
next setting contacts part has. A record of this traffic is kept with the server (http, udp, tcp, etc.) from where the agents you have established are connecting from. In other words, what you have run on a machine, when you have run it, is included here, up to the command line outputs. You can review the links by clicking "download report" on the right. An example report is as follows.
Operations
We can think of this part as a simple operation for red/blue teams. It consists of two parts, Add/View, you can add a new operation in the add part, and in the view, you can see the results of the works you have added and run. Here, you can see how many of the actions are blocked in the Gameboard or how many processes the blue team has solved with the scores.
In the screenshot on the left, we first need to give a name to this process. 
Then we choose which users we want to perform these tests on.
There are profiles in the Adversaries section, we can create our own profiles for threat actors. We can choose any of them from the profile section.
Then we specify whether it should work for a certain period of time or work until we intervene.
In the Stealth section, we can choose one of the obfuscations methods that we have seen above.
Schedule part is the part of scheduling the process to a future date.
After making the necessary adjustments and saying start, the following screen welcomes us. Here, there are buttons on the top, such as a start/stop, and there is a progress bar under it, with warning texts in different colors below it. These represent the results of the given operations.
On the lower right “+potential linksThere is a section. Here we will choose what we want to do. here 
After opening this place, there are 3 fields at the top, the first one is in the system where the agent is installed, we want to perform these tests, we choose it, then we specify the tactics and techniques, and a list appears before us. Again, the techniques and tactics in the Miter Att&ck framework are included here with detailed explanations. Whichever ones you want to use, you can add them to the test list by saying add on the right side of the green button.
After adding the necessary techniques, tactics, they will automatically start working on the counter system and as a result, a green ring will appear at the beginning of each line when they successfully complete their task. If the result was not successful it would have been red if it had been the fail. Here, you can see how successful the blue team teams are in defending the automatic actions you make.
When the processes are finished, it is a nice feature that it presents you the last thing you have done as a report. When you say download report, it presents you a report in json format. An example is below.
Adversaries
Finally, in this part, we can now create our own custom profiles for threat actors. We can customize this part, which includes ATT&CK TTPs.
On the left, there is a button that we can choose as add/view, the part we call view already contains the profiles that come by default. When we open your profile list and select any profile, the TTPs listed here will welcome us. We choose the add part to create our own profile file. Our priority is to enter a profile name and additional information. After this process, on the right “+add ability” section and a screen like this welcomes us.
Here, we determine the technique and tactic we want to add to the profile. Example:
After making the necessary choices, he gives us a brief summary here, with name, a brief summary of what he does, what tactics and technique are used. Just below it, there is information on which operating system it is running and what the application is running.
A little below, it shows how it is on the command line, and finally, there is a button where we can save the changes we made and add this process to the profile.
If a script file or a PE file is required for the technique you have chosen to work, it adds them to the Payloads section when adding them. Again, it shows how it looks on the command line, and there is a command line code that shows how to delete it from the system after the last run.
From now on, our profile starts to create like this, the image starts to look like this as we add it. After the last additions are finished, when we save the profile from the left side, we can use this profile whenever we want.
