Perched | Security Education, Consulting, and Support
Security Solutions

Perched Blog

Importing ".evtx" files into HELK or Elastic

Introduction

At some point in your career, there will come a time when you need to collect Windows event logs in EVTX format. Likely, these were acquired from a Windows event archive, during forensics investigations, or someone sent you log samples/files. This is especially true when you have collected these logs from offline assets.

You can easily view these logs using Microsoft's Event Viewer or a tool such as WEFFLES (side note, anything Jessica Payne develops/discusses is worth your time). However, you probably want to leverage your existing Elasticsearch database… or better yet, use HELK.

Previous solutions have involved converting EVTX to XML to JSON however, those processes are often subject to parsing issues and incorrectly use the event manifest. In turn, this may cause issues with field names and or missing values or they may choose to outright use their own field names. These implementations are not necessarily wrong, but you may spend your time massaging data back to its original state.

This post will walk you through how to use Winlogbeat 8.0 (beta) and it's new capabilities for importing EVTX files into HELK (option 1) or your own Elasticsearch instance (option 2). We would like to thank Brandon DeVault for finding this hidden gem and performing our first iteration of testing!

Using Winlogbeat 8 with HELK is possible as compatibility was recently added for both Winlogbeat 7 & 8 and yet is still (backward) compatible with 6!

Download, Configure, and Run Winlogbeat

Downloading Winlogbeat 8

First, we need to download Winlogbeat 8.0. There are a few ways to do this, but the easiest way is to use the direct link:
https://storage.googleapis.com/beats-ci-artifacts/snapshots/winlogbeat/winlogbeat-oss-8.0.0-SNAPSHOT-windows-x86_64.zip

Now, unzip the download to a directory of your choice.

Alternatively, you can find more specific versions via Elastic’s public repository. In addition, you can compile the latest version from GitHub, but then you'd have to spend your time learning and or compiling Go... So let's press the easy button and move forward.

From here we are going to bypass the standard procedures for installing the Winlogbeat service because we only need to use the standalone executable.

Import to HELK (Option 1)

Create a new YML file called `winlogbeat-evtx.yml` inside the directory that contains winlogbeat.exe.

**Note**
The executable may be located within a subdirectory of the unzipped folder. For example, my `winlogbeat.exe` was located in `C:\Users\root\Downloads\winlogbeat-8.0.0-SNAPSHOT-windows-x86_64\winlogbeat-8.0.0-SNAPSHOT-windows-x86_64`

winlogbeat.event_logs:
 - name: ${EVTX_FILE}
   no_more_events: stop
winlogbeat.shutdown_timeout: 60s
winlogbeat.registry_file: evtx-registry.yml

#----------------------------- Kafka output --------------------------------
output.kafka:
  # initial brokers for reading cluster metadata
  # Place your HELK IP(s) here (keep the port).
  # If you only have one Kafka instance (default for HELK) then remove the 2nd IP that has port 9093
  hosts: ["<HELK-IP>:9092","<HELK-IP>:9093"]
  topic: "winlogbeat"
  ############################# HELK Optimizing Latency ######################
  max_retries: 2
  max_message_bytes: 1000000

Import directly to Elasticsearch (Option 2)

Create a new YML file called `winlogbeat-evtx.yml` inside the directory that contains winlogbeat.exe.

**Note**
The executable may be located within a subdirectory of the unzipped folder. For example, my `winlogbeat.exe` was located in `C:\Users\root\Downloads\winlogbeat-8.0.0-SNAPSHOT-windows-x86_64\winlogbeat-8.0.0-SNAPSHOT-windows-x86_64`

winlogbeat.event_logs:
 - name: ${EVTX_FILE}
   no_more_events: stop
winlogbeat.shutdown_timeout: 60s
winlogbeat.registry_file: evtx-registry.yml

# Make sure to replace localhost with your Elasticsearch instance
output.elasticsearch.hosts: ['http://localhost:9200']

Output the Logs!

Output a single log

Using the CLI, we will now import our .evtx file.

.\winlogbeat.exe -e -c .\winlogbeat-evtx.yml -E EVTX_FILE=C:\path\to\log.evtx

Output multiple logs at once

Ok, so what kind of blog post would this be if we left you without an automation piece? So let's take a look at a simple PowerShell script that will loop through a directory full of .evtx files!

# Change this line to the path containing your evtx/raw logs
$Path1 = "C:\path\to\logs\"

# Now we filter for just the .evtx files
$Dir1 = Get-ChildItem -Path $Path1 -filter *.evtx

# The for loop to import all the logs!
foreach($file in $Dir1){
   $filePath = $Path1 + "\" + $file
   Write-Host $filePath
   .\winlogbeat.exe -e -c .\winlogbeat-evtx.yml -E EVTX_FILE="$filePath"
   Sleep 2
}

**Note**Importing using either scenario above will display a lot of output. As long as you don’t get any errors you should be good to go :)

Conclusion

Success! Now you can view all of your Windows logs and leverage your existing visualizations and dashboards.

Imported logs shown in HELK (Option 1)

Imported logs shown in HELK (Option 1)

Imported logs directly (Option 2)

Imported logs directly (Option 2)

Happy hunting!

Nate GuagentiComment