2023-09-27

Go for cybersecurity - tools

After getting a basic idea of what TLS is in the previous post, let’s write a tool the will help us finding out the TLS version of a server.

NOTE: You can read this post also on github.

The first idea might be to range over the arguments that should be TCP addresses. For each address we’ll get and print the TLS version. We might start with a pseudo-code like:

for all IP addresses supplied as CLI arguments:
    get the the TLS version

This look simple enough to implement:

package main

import (
    "crypto/tls"
    "fmt"
    "os"
)

func main() {
    for _, addr := range os.Args[1:] {
        ver, err := getTLSVersion(addr)
        if err != nil {
            fmt.Fprintf(os.Stderr, "tlsver: %v", err)
            continue
        }
        fmt.Printf("%s\t%s\n", ver, addr)
    }
}

Now we need to write the GetTLSVersion function. Fortunately there’s the standard library package tls:

func getTLSVersion(addr string) (string, error) {
    conn, err := tls.Dial("tcp", addr, &tls.Config{InsecureSkipVerify: true})
    if err != nil {
        return "", err
    }
    defer conn.Close()
    return tls.VersionName(conn.ConnectionState().Version), nil
}

Let’s run the program (saved under filename tlsver.go):

$ go run tlsver/1/tlsver.go example.com:443 example.net:443 wall.org:443
TLS 1.3 example.com:443
TLS 1.3 example.net:443
TLS 1.2 wall.org:443

As you can see, Larry is a bit behind :-).

See the whole program at https://github.com/jreisinger/docs/blog/gosec/tlsver/1.

Adding concurrency

The code above works fine. But imagine that we want to check the TLS version of a thousand hosts. Connecting to the hosts one after another might take some time. Concurrency means organizing the program in a way that multiple processes can execute independently. Even at the same time if you have multiple processors (which you most certainly do nowadays). Go has an excellent support for doing this.

Basically we want to run the getTLSVersion function and forget about. Then run the next one and forget about it (like when you background a shell command with &). And so on. Obviously we need to feed some input (function parameters) into the function and collect its output (return values). We use in and out channels for this. The channels are like typed shell pipes ($ ls | wc -l). The type in our case is host - it holds all the necessary information. When we fire up goroutines we usually don’t want to allow for an unlimited number of them because we might exhaust computing resources (like open sockets or file descriptors). So we run only 30 goroutines. We also want to know when the goroutines are done. For this we use the WaitGroup, which is kind of a concurrency-safe counter.

type host struct {
    addr     string
    tlsVer   string
    insecure bool
    err      error
}

in := make(chan host)
out := make(chan host)
var wg sync.WaitGroup

for i := 0; i < 30; i++ {
    wg.Add(1)
    go func() {
        for h := range in {
            h.tlsVer, h.err = getTLSVersion(h.addr, h.insecure)
            out <- h
        }
        wg.Done()
    }()
}

We want to get the list of TCP addresses from command line arguments or from standard input. We don’t want to wait (block) on the input so we run it in a goroutine as well. When there’s no more input we close the in channel and decrease the WaitGroup counter.

go func() {
    if len(flag.Args()) > 0 {
        for _, addr := range flag.Args() {
            in <- host{addr: addr, insecure: *insecure}
        }
    } else {
        s := bufio.NewScanner(os.Stdin)
        for s.Scan() {
            in <- host{addr: s.Text(), insecure: *insecure}
        }
    }
    close(in)
    wg.Done()
}()

And we just print the output coming from the out channel. We close the channel when the WaitGroup counter is zero, i.e. all goroutines are done.

go func() {
    wg.Wait()
    close(out)
}()

for h := range out {
    if h.err != nil {
        fmt.Fprintf(os.Stderr, "%s: %v", h.addr, h.err)
    }
    fmt.Printf("%s\t%s\n", h.tlsVer, h.addr)
}

Now we can quickly check many hosts:

$ go install tlsver/2/tlsver.go
$ cat ~/Downloads/top1000domains.txt | tlsver -insecure -concurrency 10
TLS 1.3 facebook.com:443
TLS 1.3 youtube.com:443
TLS 1.2 bing.com:443
<...SNIP...>

See the whole program at https://github.com/jreisinger/docs/blog/gosec/tlsver/2.

Tips for designing programs

Design iteratively. No one designs a program top to bottom in a linear, systematic fashion.

Try out alternatives. Good design involves a lot of trial and error. When you look at someone’s code, it’s finished work, not the process they went through to get there.

Keep it simple. Don’t design in extra complexity until it is really needed.

Solve one problem at a time, don’t be overwhelmed by everything.

2023-09-26

Go for cybersecurity - learning

I think cybersecurity practitioners should be able to program. If they do, they can understand computer technologies better and they can automate tasks by building tools. This is something I want to demonstrate a bit in this post and the next one.

NOTE: You can read this post also on github.

But why Go

I think no one really doubts it’s a good thing to be able to program. But why Go and not some other language, like Python? I think you should also learn Python and Bash and Javascript, if you can. Following are some qualities of Go I like.

Simplicity. Hoare said that “There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies.” Go language and (many) Go programs follow this idea. You want simplicity because there’s already enough technological (and organizational) complexity. Simpler systems are easier to understand and thus tend to have fewer bugs and are easier to modify.

Security. Go is a relatively new language (version 1.0 was released in 2012) built with safety and security in mind. This is not true of languages created in the pre-Internet era that was more innocent (Python appeared in 1991).

Stability. Go maintainers intend not to break Go compatibility so you don’t need to worry that the programs you write will stop working. I also think that Go has a future because it was created and is maintained by very experienced and skilled people (like Rob Pike, Ken Thompson and Russ Cox). It’s an open source language, supported by Google and used by many other big companies, like Cloudflare or PayPal. A lot of important open source software is written in Go, for example Kubernetes or Terraform. It has a first class support on all cloud providers and most of the Cloud Native Computing Foundation (CNCF) projects are written in Go.

Typed, cross-compiled to a single binary. If you are (as I used to be) familiar only with dynamic scripting languages like Python or Perl, Go will help you to really understand what are the large-scale systems languages like. What is cool is that you can build your program to run on any supported computer (CPU) architecture and operating system. For example, if you are on a Mac and want to run your tool on a Linux based Raspberry Pi:

$ GOOS=linux GOARCH=arm64 go build mytool.go
$ scp ./mytool user@raspberry.net:
$ ssh user@raspberry.net ./mytool

To list all supported platforms:

$ go tool dist list

TLS version

Imagine you (or your boss :-) read somewhere that TLS 1.2 is not fast and secure enough. Everyone should be using TLS 1.3! Are we?

Let’s have look. As usual we need two generic steps to solve this puzzle. First of all we need to know what TLS is. Second of all we check what versions are we using for our services.

What is TLS - learning by reading

TLS (Transport Layer Security), formerly known as SSL, is a protocol to encrypt, authenticate and check the integrity of data that is transferred over network. You can think of it as secure TCP. The nowadays omnipresent HTTPS is an extension of HTTP that uses TLS underneath. As you can see in the picture below in yellow, a TLS connection is initiated via TLS handshake. (The blue stuff is the standard three-way TCP handshake).

TLS handshake

One of the things negotiated during the handshake between the client and the server is the version of TLS to use. There are several TLS versions. TLS 1.3 is the latest version, that also happens to be the fastest and most secure. You should be using TLS 1.3.

TLS versions

What is TLS - learning by doing

There’s a difference between knowing the path and walking the path. Or, as Evi Nemeth said: “You don’t really understand something until you’ve implemented it”.

TCP

Let’s start with TCP because we said that TLS is kind of a secure version of TCP. All we need to implement a TCP server and client is inside the net standard library package.

The server code has three parts: listening, accepting and handling.

First we need to start listening for incoming TCP connections on some address (host:port):

ln, err := net.Listen("tcp", "localhost:8000")
if err != nil {
    log.Fatal(err)
}
defer ln.Close() // execute when surrounding function (main) returns

Then we enter an infinite loop in which we accept and handle incoming connections:

for {
    conn, err := ln.Accept()
    if err != nil {
        log.Print(err)
        continue
    }

    go handle(conn) // handle connections concurrently
}

The handle function is running in a goroutine which means the program doesn’t block waiting for the function to return. It continues running the loop handling multiple connections concurrently.

The connection handling in this case is really simple, we just copy back whatever we receive:

func handle(conn net.Conn) {
    defer conn.Close()
    io.Copy(conn, conn)
}

The client code has also three parts: connecting, writing and reading.

First we need to connect to the address of a TCP server:

conn, err := net.Dial("tcp", "localhost:8000")
if err != nil {
    log.Fatal(err)
}
defer conn.Close()

Next we send out some bytes (converted from a string):

_, err = io.WriteString(conn, "Hello from client.")
if err != nil {
    log.Fatalf("client write error: %s", err)
}

And then we receive some bytes (printed as a string):

buf := make([]byte, 256)
n, err := conn.Read(buf)
if err != nil && err != io.EOF {
    log.Fatal(err)
}
fmt.Printf("client read: %s\n", buf[:n])

Let’s run the server and client:

$ go run tcp/server/echo.go

# from another terminal
$ go run tcp/client/main.go
client read: Hello from client.

You can find the whole code here.

TLS

As we’ve learned, TLS adds encryption, authentication and integrity checking to TCP connections. Let’s see.

The server code will be very similar to TCP server only we’ll use the tls package instead of the net package:

ln, err := tls.Listen("tcp", "localhost:4430", config)
if err != nil {
    log.Fatal(err)
}
defer ln.Close()

Ok, we need some configuration for TLS to work. What’s that? TLS uses public key cryptography. The TLS server needs a certificate to authenticate to the client and a private key to decrypt and sign data:

certFile := flag.String("cert", "cert.pem", "certificate file")
keyFile := flag.String("key", "key.pem", "private key file")
flag.Parse()

cert, err := tls.LoadX509KeyPair(*certFile, *keyFile)
if err != nil {
    log.Fatal(err)
}
config := &tls.Config{Certificates: []tls.Certificate{cert}}

Let’s run the server:

$ go run tls/server/echo.go
2023/09/26 21:02:33 open cert.pem: no such file or directory
exit status 1

Right. We don’t have the certificate or the private key. Let’s create them using mkcert and re-run the server:

$ mkcert localhost
<... snip ...>
$ go run tls/server/echo.go -cert localhost.pem -key localhost-key.pem

X509 certificates contain server’s public key, along with its identity and a signature by a trusted authority (typically a Certificate Authority). We can have a look:

$ openssl x509 -in localhost.pem -text -noout

The TLS client differs from the TCP client in a that it needs a trusted CA certificate that can be used to verify the server:

certFile := flag.String("cert", "cert.pem", "trusted CA certificate")
flag.Parse()

data, err := os.ReadFile(*certFile)
if err != nil {
    log.Fatal(err)
}
certPool := x509.NewCertPool()
if ok := certPool.AppendCertsFromPEM(data); !ok {
    log.Fatalf("unable to parse certificate from %s", *certFile)
}

config := &tls.Config{RootCAs: certPool}
conn, err := tls.Dial("tcp", "localhost:4430", config)
if err != nil {
    log.Fatal(err)
}
defer conn.Close()
$ go run tls/client/main.go -cert localhost.pem
client read: hello from client

You can find the whole code here.

More

2023-01-02

Practical cloud security overview

IT industry has been undergoing seismic shifts regularly. The latest one goes by the name cloud computing. However, the security goals, principles and areas don’t change with the adaption of a new paradigm. Terminology, methods and tools change, though.

Why we need security

The need for security stems from the fact that people make suboptimal decisions and mistakes - in design, implementation, configuration, operations - creating vulnerabilities in systems. And some other people (threat actors or groups) try to exploit these vulnerabilities for various reasons.

In case you don’t know or don’t care about your vulnerabilities and threats, you can run into trouble. Your data can be stolen, altered, deleted or encrypted for ransom. Your infrastructure or applications can be shut down, misused for crypto-mining or for launching attacks against other targets. Your customers or employees can get scammed.

This creates all sorts of issues. Your customers/users can’t use a service at all or the service is not working correctly. Operational problems, when employees have to handle security incidents instead of normal business. Loss of reputation and consequently of employees and customers. The company can get fined or someone (like a CISO) can even go to jail. All this gets reflected also in terms of financial losses. Obviously, you want to avoid this at least to a certain degree.

Security goals

It’s impossible to not create any vulnerabilities and you can’t make the bad actors disappear. So what can we do then? Well, we can avoid at least some of the vulnerabilities and we can handle or contain at least some of the exploits.

More formally, cybersecurity is a never ending effort to manage risks of computer systems by increasing their resiliency to threats. A threat is the possibility of something bad happening; it’s the combination of a vulnerability and a threat actor. A risk is the quantified refinement of a threat; it’s defined by its likelihood and impact.

CIA triad represents the traditional (since 1977) security goals: “The protection of system data and resources from accidental and deliberate threats to confidentiality, integrity, and availability.”

  • confidentiality: no unauthorized (read) access to data or resources
  • integrity: no unauthorized changes to data or resources
  • availability: data and resources available when needed

Security principles

OK, so how do we achieve the security goals? First, there are some principles everybody should keep in mind and apply whenever possible:

  • simplicity - the worst enemy of security is complexity because it makes defense harder
  • minimal attack surface - minimize possible ways a system can be attacked
  • least privilege - deny by default to limit the blast radius of a compromise
  • segmentation - create boundaries between systems to limit the blast radius of a compromise
  • defense in depth - since any security control can fail have multiple overlapping layers of controls
  • secure by design - security shouldn’t be an afterthought because then it’s much more expensive

Security areas

Second, you have to care about many things because the weakest link in the chain of interconnected components can get exploited (security is a systems property). Not to get overwhelmed one might create some abstractions in the form of distinct areas to cover.

Governance and risk management

If you know the enemy and know yourself, you need not fear the result of a hundred battles. – Sun Tzŭ: The Art of War

You should get at least a rough understanding of your organization’s business and products. Find out your responsibility boundaries and what data and compute/storage/network resources you need to protect. This depends on the service model you use or provide (IaaS, PaaS or SaaS). Get some idea who is most likely to cause problems and how.

Risk level is the likelihood of a future problem times its impact. You can approach each risk in one of these ways:

  • avoid it - don’t build the system in the first place or turn it off if benefits are lower than risks
  • transfer it - pay someone else to manage the risk (e.g. SaaS, insurance)
  • mitigate it - apply some security measures (controls)
  • accept it - if benefits are higher than risks (this should be conscious)

You might need to prove your security to a 3rd party; this is called compliance.

Identity and access management

If an attacker gets credentials all patches and firewalls won’t help. Manage user and program identities (authn) and access rights (authz) in as few places as possible. Have process of removing users that left the company. Make sure that strong passwords and MFA are used. Use a password manager (1Password) and don’t commit unencrypted passwords or API keys to repositories (gitleaks). Access rights (roles, policies) should follow the least privilege principle.

Vulnerability management

Detect and remediate security bugs and misconfigurations in application (SonarQube, ZaP, trivy) and infrastructure (tfsec) code, systems and networks (Nexpose). Before (SAST, SCA) and after (DAST) deployment. Important point to emphasize here is to make sure that the vulnerabilities found by the scanners are also remediated not only reported. So detect them as soon as possible in the process of developing and deploying code and infrastructure. First handle only the critical ones. Code reviews and penetration testing is helpful but expensive. Regularly upgrade (patch) your systems and dependencies.

Security monitoring

You want to know what’s going on and then do something about. Detect threats and security incidents, and respond to them. You do this by first collecting and parsing logs and metrics in a central place (Splunk, Graylog, Datadog). Then you create alerts (log/metric queries with a threshold) and handle them when they get triggered. Find a good balance between too many and too few alerts. Prefer simplicity and quality over cleverness and quantity to avoid alert fatigue.

Network security

If you can’t talk to a component, you can’t compromise it. Use network policies, ACLs, VPNs, WAFs, antiDDoS, IDS/IPS when it makes sense. Try to create trust boundaries. Anything inside a trust (or security) boundary can trust, at least to some level, anything else inside that boundary but requires verification before trusting anything outside that boundary. Also (almost always) encrypt data in motion using TLS.