Bad magic: when patient zero disappears without a trace

Imagine the scene... Your organisation has been targeted by a sophisticated ransomware attack, and you have multiple encrypted systems that are all accompanied by a ransom note. You have managed to contain the situation, but you have no idea what the attacker did before they started the encryption process, or how they breached your systems in the first place. In an ideal world, the potentially relevant data would be collected at a very early stage, enabling forensic investigators to see the systems as they were at the time of the attack. But in reality, it can be days or weeks, or even months, between the attack and the incident response team beginning their investigation. This is because it takes time to realise that an attack has occurred, to liaise with different internal teams to remediate the immediate threat, to contact cyber insurers, and to make a start on the investigation. This means that vital evidence is often lost.

Has anyone seen patient zero?

One of the key questions to answer during an investigation is “who is patient zero?”, i.e. what was the first compromised machine? Analysing patient zero is critical because it can help to determine the initial compromise date, the method of compromise, and how the threat actor then moved to other systems. This can present challenges to an investigation because in some cases, patient zero is not even known to have been compromised.

Inevitably, after an attack, a company needs to make changes to systems on their network to remediate the immediate threat, such as installing new software, rebuilding or restoring a device from backups, changing security settings, or removing user accounts. Having a copy of the data prior to these changes helps to avoid the loss of important evidence and enables the investigator to distinguish malicious behaviour from legitimate activity (e.g. the addition of a new user account). It is essential to proceed cautiously in the early stages of incident response, particularly when handling devices that could hold vital information about lateral movement within the network, such as domain controllers.

A company may choose to carry out an internal investigation of the incident, often before an external forensic team is engaged. This can lead to work being conducted on live systems, which may destroy critical data. It can also lead to systems being left running unnecessarily, or allowing repeated connections from remote devices, which may overwrite important evidence.

Unfortunately for forensic investigators, the accidental deletion of critical data is common and can result in important questions never being answered.

Not quick enough… when log data vanishes

Even if systems are identified and preserved, it can sometimes be too late. In a forensic investigation, time is of the essence. Limits on log retention periods and log file sizes are commonplace, so preserving evidence as soon as possible is crucial. It is also important that the right types of data are preserved, so that the incident response team can view all of the artifacts they need to help them piece together what has happened.

For example, most organisations that use Microsoft 365 have a 90-day retention period for audit logging. If you consider the steps outlined above that need to be followed before the investigation begins, and factor non-working days into this, it can take a week or more before the logs are collected. It is not unusual for the team carrying out the forensic investigation to be instructed months after the incident, at which point the most useful logs may well have been lost. In addition, an attacker may have had access to the network or account for weeks prior to an incident being spotted, so this extra log data could be the difference between understanding an attacker’s activities and not.

The right data, the right way

Data should be preserved by suitably skilled people, to avoid any issues with the collection and subsequent investigation. Depending on the nature of the investigation, it might be necessary to collect live memory data from one or more machines, which must be carried out before the machines are powered down. In other situations, it might be necessary to keep a device that has been hit by ransomware powered on so that full disk encryption does not prevent it from being booted up again.

If the organisation’s IT team or IT provider is planning to preserve the evidence, they should as a minimum be confident that they are compliant with the four “ACPO Principles” (Association of Chief Police Officers) for collecting digital evidence (as explained in the ACPO Good Practice Guide for Digital Evidence). Not only does this mean that they are complying with best practice, but it also means that if an investigation ever did make its way into a courtroom the evidence would be admissible. In summary, these are as follows (quotation uses our italics):

1. No action taken should change data held on a computer or storage media which may subsequently be relied upon in court.
2. In circumstances where it is necessary to access original data, that person must be competent to do so and be able to explain the relevance and implications of their actions.
3. An audit trail of all processes applied to computer-based evidence should be created and preserved.
4. The person in charge of the investigation has overall responsibility for ensuring that these principles are adhered to.

It is essential that the team preserving the data know which artifacts need to be collected – collecting the wrong ones or missing out key artifacts is as bad as destroying the evidence.

Tricks of the trade

How can deleted data be avoided? Although the methods for preserving data will depend on the type of system and the type of data, the worst examples of disappearing data can be avoided by following these suggestions:

• Do not install new software on any endpoints under investigation.
• Preferably turn off any endpoints under investigation and consider taking a memory capture before doing so. The current guidance when switching off machines to preserve evidence is to simply pull the power cord out (and remove the battery for laptops) for Windows systems and shut down properly any MacOS systems. If ransomware is known to have affected the network, endpoints must not be powered down or restarted.
• If you need to power on an endpoint, take copies of any logs that have a finite size and might be overwritten, such as Windows Event logs and the USN Journal.
• Try to keep connections to affected endpoints to a minimum, but if this is necessary, record the connecting users and IP addresses.
• Record any action taken on live endpoints including the time and date this was carried out.
• Work from copies of log files and other collected data, rather than on the live files.
• Record any action carried out as part of the incident remediation, such as password resets, security settings updates and user account changes.
• Don’t wipe, rebuild or restore affected systems that are likely candidates for patient zero or that would have been priority targets for the threat actor.

In summary

In our forensic investigations, we routinely find that critical data has been accidentally deleted in the very early stages of incident response. When critical systems are affected, such as patient zero, it becomes much more difficult to answer the important questions such as "how were we compromised?" and "was data taken?". However, these problems are easily avoidable, simply by following the suggestions in this article, and by ensuring that the correct data is being collected by someone who knows exactly what is required. In our cases, we regularly provide guidance to IT teams and third party providers on what data is needed, where to find it, and how to safely preserve it. With broader awareness of the traps to avoid, more unintended vanishing acts could be avoided.

Find out more

If you are currently experiencing a cyber attack, please request emergency assistance immediately.

To find out more about any of our services, please contact us. To start a conversation or report any errors or omissions, please feel free to contact the author directly.