Building Gophish Healthcheck: Part One

One of the questions I see most often from Gophish users is “how do I get past my spam filter?” Generally, my answer to this is something along the lines of “just whitelist the IP address,” since it’s my opinion that phishing simulations should be a test of the people and processes, not the email infrastructure.

But what if we do want to test the email infrastructure?

This post is the first in a two-part series about how I’m creating the email healthcheck service for Gophish. This post talks about how I handle DNS programmatically, and the next post will describe the actual architecture being used.

It’s All Just DNS

The first version of the project is simple. I want users to be able to send themselves emails that have either invalid or valid settings for:

SPF Records
DKIM Records
DMARC Policies
MX Records

All of these are handled through DNS. When a mail server receives an email, it will perform DNS queries to fetch things like the SPF record for the sending domain and use that record to see if the email is properly authenticated.

The big question I had to answer is: how do I set up DNS to support all the possible options?

The initial answer might be to try and set up subdomains for all the possible combinations. Then, I could use the right subdomain when sending the email. The problem is that this gets overwhelming fast, and I plan to support many more types of healthchecks in the future.

We need a different plan.

Say Hello to CoreDNS

Right now, each message we send out will get a unique 16-byte hex-encoded ID that looks like this: 0d5750699dc7544052930c0a90fdd273.

This sparked an idea: What if we could use this message ID to somehow dynamically return DNS responses based on the settings the user chose?

Basically, we’d send a message that came from, say, no-reply@0d5750699dc7544052930c0a90fdd273.mail.healthcheck.getgophish.com. When the mail server received this message, it would send various DNS queries for the domain d5750699dc7544052930c0a90fdd273.mail.healthcheck.getgophish.com. If I had a DNS server I could control dynamically, I could extract the message ID, look up what tests the user requested, and send back the right DNS response like magic.

But first, I needed to find a DNS server that can build responses on-the-fly. Fortunately for us, this exact DNS server exists, and it’s written in Go!

CoreDNS is a DNS server that works by running each query through plugins. It’s built on the foundation of the awesome Caddy project and has a familiar syntax.

This is perfect for us. We can write a CoreDNS plugin that will catch the DNS requests and send back the right response - no gigantic list of hardcoded subdomains needed!

Writing a CoreDNS Plugin

Going into this, I hadn’t written a CoreDNS plugin. In fact, I hadn’t done much DNS work in Golang at all. Fortunately, CoreDNS has a great introductory blog post showing how to build a plugin from scratch.

I won’t repeat all the steps here, so let’s just cover the important stuff. There’s two things we need to do:

Register the plugin using caddy.RegisterPlugin and set a few options
Implement the ServeDNS() and Name() methods

Registering the Plugin

This is the easiest part. We’ll just make a new file called setup.go and create a new plugin we’ll call healthcheck. This starts with the init function:

func init() {
	caddy.RegisterPlugin("healthcheck", caddy.Plugin{
		ServerType: "dns",
		Action:     setup,
	})
}

This expects to call a function called setup when the plugin is created. This function will set up our local app config and establish a connection to the database so we can look up messages later. Finally, it registers an instance of HealthCheckPlugin (we’ll define that later) to the global list of plugins:

func setup(c *caddy.Controller) error {
	c.Next() // healthcheck
    
 	// Load our configuration file
	err := config.LoadConfig("./config.json")
	if err != nil {
		return err
	}

	// Set up the database, running migrations if needed
	// and get a handle to the database so we can look
 	// up messages later.
	err = db.Setup()
	if err != nil {
		return err
	}

	// Register the plugin with CoreDNS
	dnsserver.GetConfig(c).AddPlugin(func(next plugin.Handler) plugin.Handler {
		return HealthCheckPlugin{
			Next: next,
		}
	})

	return nil
}

Piece of cake.

So now CoreDNS knows that our plugin exists, but it still doesn’t know what our plugin actually does. To build out the plugin, we can create a file (uncreatively) named plugin.go.

The first step is to make our plugin struct:

// HealthCheckPlugin is a CoreDNS plugin that emulates various email
// authentication states.
type HealthCheckPlugin struct {
	Next plugin.Handler
}

The real magic happens in the ServeDNS function. This function takes a DNS request and a DNS response writer. The general idea is that, if the request is something we’re interested in, we can return a response. Otherwise, we can just move on to the next plugin (or in our case, fail).

For the rest of this section, we’ll be implementing a function that looks like this:

func (hc HealthCheckPlugin) ServeDNS(ctx context.Context, w dns.ResponseWriter, r *dns.Msg) (int, error) {

	// TODO: ✨MAGIC✨ HAPPENS

	// Return a status telling CoreDNS we processed this request
	return dns.RcodeSuccess, nil

The general structure of the ServeDNS function is pretty simple, we can just check to make sure the request type is one we support and, if so, we can process it. For our case, we only care to support MX and TXT requests. We’ll also be nice and support the deprecated SPF request type. Here’s an example of the type of switch statement we can do to process the request based on it’s type:

a := new(dns.Msg)
// Fill in the response based on the result of us processing the request
switch state.QType() {
case dns.TypeTXT:
	a.Answer, err = hc.processTXTRecord(state)
	if err != nil {
		return plugin.NextOrFailure(hc.Name(), hc.Next, ctx, w, r)
	}
case dns.TypeMX:
	a.Answer, err = hc.processMXRecord(state)
	if err != nil {
		return plugin.NextOrFailure(hc.Name(), hc.Next, ctx, w, r)
	}
// This is really only supported for odd legacy issues. Per RFC 7208, SPF
// records must be TXT records
case dns.TypeSPF:
	a.Answer, err = hc.processSPFRecord(state)
	if err != nil {
		return plugin.NextOrFailure(hc.Name(), hc.Next, ctx, w, r)
	}
}

The flow of each process*Record function is simple. It goes something like this:

Parse out the message ID and look it up from the database
Create a DNS response based on what options were set up for that message
Return that DNS response

Let’s use processMXRecord as the example, since it’s the shortest. While MX records are technically for receiving mail, some mail servers look this up as an input to their reputation engines. In this case, we have three options:

No MX record - Don’t return an MX record
Invalid MX record - Return invalid.mail.healthcheck.getgophish.com
Valid MX record - Return the hostname of our email service, mail.healthcheck.getgophish.com

func (hc HealthCheckPlugin) processMXRecord(state request.Request) ([]dns.RR, error) {
	rrs := []dns.RR{}
	// Get our message
	messageID := strings.Split(state.QName(), ".")[0]
	message, err := db.GetMessage(messageID)
	if err != nil {
		return rrs, err
	}
	// Create our DNS response
	rr := new(dns.MX)
	rr.Hdr = dns.RR_Header{Name: state.QName(), Rrtype: dns.TypeMX, Class: state.QClass()}
	rr.Preference = 10
	// Fill in the response based on the requested configuration
	switch message.MessageConfiguration.MX {
	case db.None:
		return rrs, nil
	case db.HardFail:
		rr.Mx = dns.Fqdn(fmt.Sprintf("invalid.%s", config.Config.EmailHostname))
	case db.Pass:
		rr.Mx = dns.Fqdn(config.Config.EmailHostname)
	}
	rrs = append(rrs, rr)
	return rrs, nil
}

Compiling the Binary

After implementing the other functions, we’re ready to compile our instance of the coredns binary. The first step is to edit the plugin.cfg file in the github.com/coredns/coredns directory to include our healthcheck plugin:

healthcheck:github.com/gophish/healthcheck/dns

In that same directory, we can build our new binary using make. We can move this binary into our github.com/gophish/healthcheck directory.

Finally, we need to write a Corefile, which is the configuration used by the coredns binary. Our Corefile is simple, and can be found here:

. {
    healthcheck
}

And that’s it! We can run our coredns binary, and it will automatically find and use our Corefile:

$ ./coredns -dns.port=1053
goose: no migrations to run. current version: 20180620193535
.:1053
2018/09/02 18:27:05 [INFO] CoreDNS-1.1.4
2018/09/02 18:27:05 [INFO] darwin/amd64, go1.10.2, 9c2dc7a1-dirty
CoreDNS-1.1.4
darwin/amd64, go1.10.2, 9c2dc7a1-dirty

Making a request, we can see the valid MX record being returned:

$ dig @localhost -p 1053 mx 2ee7320f5339b71d8b953e73b025d4ad.mail.healthcheck.getgophish.com

; <<>> DiG 9.10.6 <<>> @localhost -p 1053 mx 2ee7320f5339b71d8b953e73b025d4ad.mail.healthcheck.getgophish.com
; (2 servers found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18914
;; flags: qr aa rd; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;2ee7320f5339b71d8b953e73b025d4ad.mail.healthcheck.getgophish.com. IN MX

;; ANSWER SECTION:
2ee7320f5339b71d8b953e73b025d4ad.mail.healthcheck.getgophish.com. 0 IN MX 10 mail.healthcheck.getgophish.com.

;; Query time: 2 msec
;; SERVER: ::1#1053(::1)
;; WHEN: Sun Sep 02 18

Moving Forward

Getting programmatic DNS to work was a huge hurdle to overcome, and I’m really excited with the results. With this in place, I can start building out the various handlers, making sure the right responses are returned.

In the next post, I’ll talk about the architecture of the project, showing how the overall Gophish infrastructure is organized. There’s a lot of moving parts to this project, so expect more magic coming soon!