The Perimeter is a lie – The Container Layer (part 3)

The perimeter is a lie

Following on from previous posts (part 1, part 2) I wanted to drill down a bit more into the components from the container cluster node in the reference architecture as is shown on the image below.

1. The App

This is the most important component – it’s where you do the ‘real’ work. You should be building this to best security practices. This implies:

  • Your app developers are trained on how to code securely – to get started, OWASP have some great guides
  • You’ve picked a programming language and framework that makes it easy to code securely – typically this means a higher level, type safe language with memory management built in. Ideally you’d also get protection for the common attacks, systematic input validation and all of the ‘security’ code out of the box
  • You’ve tested your application for security, and you’ve built the testing into your release process. Security is not a one time check – it should be continuous
  • You’ve reviewed your architecture and understand what it is you’re trying to protect and from what

I recommend looking at OWASP guides and the Samm framework – it’s a really good thing to encourage your team to plan security in as part of their design!

2. The load balancer/TLS layer

It’s a common requirement to have a load balancer/TLS offload layer in front of applications. Ideally this would encrypt the connection all the way to the app – but for performance or logistical reasons it often doesn’t. Regardless, you still can ensure no traffic is exposed by running it 1:1 with app containers on the same container host. You can enforce strict network security so only the load balancer and the target node can communicate.

3. Logging

All apps need to log, but often those logs are stored in an ad-hoc fashion and are not analysed. Logs should be sent to a local logging agent that ‘stores and forwards’ the logs to a central server, and then those logs can be analysed both automatically in real time and manually after the event.

The reason remote logging is important is that the first thing an attacker will do is disable or modify the logs. If you have them feeding centrally then that becomes much, much harder. Stopping them should trigger alerts, and modifying a remote server is much harder than editing a file on the local disk.

4. Container Integrity Verification at Container Run-time

Assuming you are running in containers – make use of the built-in container signing/verification features of tools like Docker Image Signing. This will ensure that when you start a container, the image you are running has not been modified. This is important as for a persistent attack to succeed they need to modify either the Docker image, or the host OS.

Container verification on its own however is not enough – it does nothing to stop the running OS being modified. There are a number of approaches that can be applied here – including ‘locking down’ the image. However, they tend to be only partially effective unless you have something like Irdeto’s Cloakware Secure Environment which ensures only signed software can run. This means that even if your app has a vulnerability it can’t be easily exploited. Many teams don’t use run-time verification as they assume their servers are secure. However if you look at recent history and the complexity of computers, you’ll find every system has had some 0-day vulnerabilities – and having a guard dog to catch intruders is a good idea!

5. OS Integrity Verification and Hardening

Hardening and protecting the container isn’t enough – you also need to harden the OS that’s running them. This implies you need to ensure you’re running from your image (boot time verification). The OS should be hardened to best practices; there are several guides online or you can get hardened images from several vendors (including Irdeto).

As with the container, run-time protections are important:

  • you need file system monitoring (Linux AuditD for example)
  • logging integration (as with the app)
  • if you’re handling files you may need AV – especially if you allow file uploads
  • the file system should be locked down and Docker containers should not have permissions to modify the OS
  • patching mechanism must be in place to rapidly patch the host

6. OS/VM Firewall

The final piece of container puzzle is to ensure the network is locked down – this could be using OS firewall features, Docker container networking or cloud hosting security groups. Locking ports makes it a lot harder for an attacker (assuming they can get access) to ‘pivot’ and then reach other servers. It’s important to lock both inbound and outbound connections to ensure and attacker can’t download tools and extract data easily if they do get access to the host.

A Question for you!

The above architecture ideas are what we think is needed, but we are interested in what threats you’re concerned about. Let us know and we’ll make sure to address them in future blog articles!