Perched | Security Education, Consulting, and Support
Security Solutions

Perched Blog

Detecting Client Library Using HTTP Headers

Introduction

This post describes a way passively, using Zeek(Bro) + the Elastic Stack within RockNSM, to detect the library used to make a web request using HTTP headers.
Using the described method has positive and negative implications for both Blue Teams (defenders) and Red Teams (attackers).

Detecting applications using passive network methods is nothing new. There are methods for operating system fingerprinting using packet TTLs or Window sizes (see: p0f), SSH libraries (see HASSH), SSL/TLS libraries (see JA3).

However, when it comes to HTTP the main focus has always been on using the layer 7 application details of the HTTP User-Agent header. Although this may be true, these can be spoofed and typically are replaced with any User-Agent of choice.

Technical Background

Chrome, Firefox, cURL, Python, and others will place the HTTP HOST header before the User-Agent header regardless of operating system (Windows or Linux).

Windows applications such as Internet Explorer, BITS, MSHTA, PowerShell, CryptoAPI and Microsoft Office use a library called WinHTTP or WinINET. When one of these libraries makes an HTTP request the HOST header is always after the User-Agent header.

There is one exception, Wget will place the HOST header after the User-Agent. However, the User-Agent always seems to be the first header..Therefore it is a bit easier to distinguish versus actual use of WinHTTP or WinINET.

Note: Also, in my testing even behind a proxy it did not change the header ordering.

rabbit-hole.jpg

To oversimplify:
HOST before User-Agent == NON Windows Library Use
HOST after User-Agent == Windows Library

Examples

HOST Before User-Agent (NON Windows Library Use)

Firefox on Windows

Firefox on Windows

Chrome on Windows

Chrome on Windows

Firefox on Linux

Firefox on Linux

HOST after User-Agent (Windows Library)

Microsoft Office on Windows

Microsoft Office on Windows

PowerShell on Windows

PowerShell on Windows

Internet Explorer on Windows

Internet Explorer on Windows

Use Cases

Blue Team

Use Cases cases include, but are not limited to, detecting:

  • Spoofing legitimate windows applications (PowerShell, BITS Client, etc) to look like a normal web request (ie: using Chrome/Firefox)

  • cURL, Python, Firefox, Burp Suite, Chrome, or other mimicking Internet Explorer for inbound web server requests

As an example. when an attacker is using one of these applications to alter requests to aide in performing web server attacks like XSS or SQLi. Many will use these applications because of their extended functionality/plugins. Especially if you combine this with additional query parameters like suspicious URI/URI and request method (PUT/POST)

Red Team

Use cases include, but are not limited to, detecting:

  • Researchers contacting public infrastructure spoofing Internet Explorer or Microsoft office web requests

  • Tracking image requests (similar to the one used against our CEO outlined here) that are made by a spoofed Microsoft Outlook User-Agent header using Firefox, Chrome, cURL, Python, or other

It would be interesting to see if you could block or redirect clients based on their HTTP header ordering through the use of a web proxy, like NGINX (or similar).

Detection Method Proof of Concept

Our detection method consists of querying the order of the HTTP client headers HOST and User-Agents.

In RockNSM, version 2.4 or later, we collect the HTTP client headers using Zeek(Bro) and stored in Elasticsearch...Perfect.
Finding the ordering of two strings (terms) is easy starting in Elasticsearch 7 (included in RockNSM) where Elastic introduced a new query type called “Intervals queries”.

First, we need to copy and paste the following into Kibana Dev Tools (The Wrench icon on the left inside Kibana). This allows us to use the intervals query, which requires analyzed text.

PUT /_template/bro-http_headers
{
  "order": 100,
  "version": 2019050701,
  "index_patterns": [ "bro-network-*" ],
  "settings": { },
  "mappings": {
    "properties": {
      "http": {
        "properties": {
          "client_header_names" : {
            "ignore_above": 4096,
            "fields": {
              "text": {
                "type": "text"
              }
            },
            "type": "keyword"
          },
          "server_header_names" : {
            "ignore_above": 4096,
            "fields": {
              "text": {
                "type": "text"
              }
            },
            "type": "keyword"
          }
        }
      }
    }
  }
}
Screenshot of what pasted code, with acknowledgement after pressing ctrl+enter.

Screenshot of what pasted code, with acknowledgement after pressing ctrl+enter.

Second, we need to import the saved searches used for the detection scenarios that we will perform later.

  1. Download the saved searches http-ordering-blog-searches.json

  2. https://gist.github.com/neu5ron/6297f3aa4fed0b30a00be48fc8d14a01/archive/3bd017586f2b9d29d702802b79624fad790041b3.zip

  3. Go to Kibana, and select the Management Tab (gear icon on the left side)

  4. Select "Saved Objects"

  5. Select "Import" (in the top right)

  6. Browse to the file "http-ordering-blog-searches.json", you saved from step 1, and select import

Now we can run the scenarios!

Detection Scenario 1: Powershell spoof of (Google) Chrome

  1. Create the PowerShell request.

    # If running within CMD/Command Prompt:
    powershell.exe -command "& Invoke-RestMethod -Uri \"http://www.perched.io\" -UserAgent \"Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36\""
    
    # If running within PowerShell
    Invoke-WebRequest -Uri "http://www.perched.io" -UserAgent "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36"
  2. Wait a minute for the data to load into Elasticsearch.

  3. Then load the search/discover "(HTTP Ordering Blog) Windows HTTP Library Spoofing Chrome/Firefox".

Detection Scenario 2: cURL Spoof of Internet Explorer]

  1. Create the cURL request with Internet Explorer 11 User-Agent

    curl google.com -A "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
  2. Wait a minute for the data to load into Elasticsearch.

  3. Then load the search/discover "(HTTP Ordering Blog) Spoof Windows InternetExplorer".


After you try the dashboard and visualizations, we would love to hear any feedback of false positives, additional use cases, and or recommendations you may have.

Happy hunting!