Sean Hamlin
|Aug 22, 2023
Aug 22, 2023
|5 min read
Powered by Lagoon: Managing countless CSP violations using amazee.io's platform flexibility
If you've ever dealt with web security, you've probably heard of CSP (Content Security Policy). It's the unsung hero of the web, silently guarding sites against malicious attacks. However, imagine managing tens of millions of CSP violations every month. Sounds like a monstrous task, right? Let’s dive into how we supported a customer tackling this challenge in a lightweight, super scalable way.
What even is a CSP?
For those new to the CSP world, let’s quickly demystify it. Content Security Policy (CSP) is a web standard designed to control what resources a user’s browser can load on a page. Think of it as a security guard, looking for suspicious activities and preventing malicious attacks.
Content Security Policy (CSP) is an added layer of security that helps to detect and mitigate certain types of attacks, including Cross-Site Scripting (XSS) and data injection attacks. These attacks are used for everything from data theft to site defacement to malware distribution.
Our Australian CSP Adventure
Enter the challenge. A significant government agency in Australia uses CSP on its site. Sounds pretty straightforward, but here's the twist: no matter how tuned a CSP is, it can only partially prevent violations. Why? Unpredictable elements, like browser extensions, often trigger some violations.
The agency's site, built on Drupal, could log CSP violations using the Seckit module. But with the astronomical number of violations, it was like we were asking a household vacuum to clean an entire beach! And before you ask, third-party tools were off the table due to the usual government security and procurement processes.
We needed to roll something lightweight and scalable.
The Road to Golang and Lagoon
So, how did we solve this mountain of a challenge? First, we ditched Drupal for logging these violations. Drupal is just too heavy for this volume.
Instead, we picked a zippy dedicated Golang binary that snugly fits into its own container within the amazee.io project and is easily deployed by Lagoon. The stars of this setup?
Firstly the go-CSP-collector streamlined the collection like a charm.
And secondly, Lagoon for deploying the application on the amazee.io platform.
Hosting this as a service in our Lagoon project was relatively quick. Lagoon is more than just the application deployment component for amazee.io. It’s also not just a dream for developers, ensuring congruent environments, 100% open source transparency, and top-notch web application delivery. It is also incredibly flexible!
We settled on using the `basic` service type from Lagoon, explicitly crafted for extended services like this scenario required. More about Lagoon service types can be found on the lagoon.sh website.
So the amazee.io team got to work lending a hand to our customers, helping them master the art of multi-stage docker builds to keep their image lean and mean.
What’s going on under the hood?
While the story of solving a scale problem is fascinating, for those technical folks reading, we thought we’d break down the mechanics of our solution. Let’s look at the magic behind the curtains!
Docker Compose Setup
To our `docker-compose.yml`, we add the following:
services: … csp: build: context: . dockerfile: .docker/csp.dockerfile labels: lagoon.type: basic ports: - '3000:3000'
This segment is essentially how we tell Lagoon to build our Golang container. The `lagoon.type: basic` is important to note - this is the directive for Lagoon, telling it to treat this service as a base type which by default listens on port 3000. Lagoon will test port 3000 when the container is deployed to confirm the service is running.
The CSP Dockerfile
Now, let's take a quick tour of the CSP service Dockerfile:
FROM golang:alpine as builder
ARG VERSION=0.0.10
RUN apk update \
&& apk add curl \
&& mkdir /tmp/csp \
&& cd /tmp/csp \
&& curl -L https://github.com/jacobbednarz/go-csp-collector/archive/refs/tags/v${VERSION}.zip > csp.zip \
&& unzip csp.zip \
&& cd go-csp-collector-${VERSION} \
&& go build csp_collector.go \
&& mkdir /csp \
&& mv csp_collector /csp
FROM alpine
COPY --from=builder /csp /csp
WORKDIR /csp
CMD ["/csp/csp_collector", "--port", "3000"]
We start from a base image, `golang:alpine`, which is lightweight and equipped with a Go installation. We use this in the first stage of our multistage build to build our application.
Once we have downloaded the source and compiled the application (based on the open source jacobbednarz/go-csp-collector), all we need from the builder is the compiled CSP binary, so we switch to the next stage in the build and copy the binary into a standard thin alpine image which will run the binary.
We do this for at least two reasons:
- We save ourselves a lot of storage space by using the lightest possible image
- By limiting what we have in the container, we explicitly reduce the attack surface of any application in the container by, essentially, limiting what is in the container to be attacked.
With everything in place, we tell the CSP application to start listening on port 3000 on startup. The Lagoon `basic` service type also monitors port 3000, so if, for any reason, our application stops responding, this will alert Kubernetes to take corrective actions.
With this setup, our CSP collector service is ready to be built and deployed by Lagoon. When deployed, it's isolated yet part of the bigger picture, efficiently catching the tens of millions of violations
Below we see our container, scanned by Trivy, and only 7.2MB in size!
The Result
Fast forward to now, the system is handling a whopping 38 million CSP violations every month, all without breaking a sweat.
And the best part? The main Drupal site remains unaffected, singing its merry tune and performing content delivery as it was designed to.
Below we see an example of a logged CSP violation (with names changed to protect the innocent!)
timestamp="2023-08-06T21:21:38Z" level=info blocked_uri="https://redacted.redacted.redacted/script/5525/webReader/webReader.js?pids=wr" disposition=report document_uri="https://redacted.redacted.redacted/contact-us?context=64107" effective_directive=script-src-elem metadata="" original_policy="default-src 'self'; connect-src 'self' *.google-analytics.com *.googletagmanager.com *.nr-data.net *.readspeaker.com translate.googleapis.com; font-src 'self' data: fonts.googleapis.com fonts.gstatic.com themes.googleusercontent.com; frame-src 'self' *.azurewebsites.net *.googletagmanager.com redacted.redacted.redacted *.readspeaker.com redacted.redacted.redacted.vudoo.io youtube.com *.youtube.com; img-src 'self' bam.nr-data.net chart.apis.google.com data: *.google.com *.google-analytics.com *.googletagmanager.com *.gstatic.com i.ytimg.com translate.googleapis.com; object-src 'none'; script-src 'self' 'unsafe-eval' api.microsofttranslator.com bam.nr-data.net *.google-analytics.com *.googleoptimize.com *.googletagmanager.com js-agent.newrelic.com https://redacted.redacted.redacted.azurewebsites.net https://redacted.redacted.redacted.azurewebsites.net https://redacted.redacted.redacted.azurewebsites.net; script-src-elem 'self' 'unsafe-inline' *.google-analytics.com *.googleoptimize.com *.googletagmanager.com *.newrelic.com *.nr-data.net translate.googleapis.com https://redacted.redacted.redacted.azurewebsites.net https://redacted.redacted.redacted.azurewebsites.net https://redacted.redacted.redacted.azurewebsites.net; style-src 'self' 'unsafe-inline' fonts.googleapis.com *.readspeaker.com f1-oc.readspeaker.com; style-src-attr 'self' 'unsafe-inline' 'unsafe-hashes'; style-src-elem 'self' 'unsafe-inline' f1-oc.readspeaker.com fonts.googleapis.com *.googletagmanager.com translate.googleapis.com; base-uri 'self'; form-action 'self' https://redacted.redacted.redacted.com; frame-ancestors 'self'; report-uri https://redacted.redacted.redacted" referrer="https://redacted.redacted.redacted/" script_sample="" status_code=200 violated_directive=script-src-ele
This journey showed us that with the right approach and tooling choices, a pinch of innovation, and a healthy dose of Lagoon flexibility, large-scale CSP can be achieved for large enterprises and government customers without relying on third-party tools and platforms.
Open Source Wins Again!
This story definitely underscores the capabilities of Kubernetes, Lagoon, and amazee.io, showing how they handle large-scale tasks without missing a beat. So, the next time you're facing a tech conundrum and starting to reach for third-party tools, remember there may be a simple, secure, and scalable open source solution deployable by Lagoon on amazee.io.
If you’re interested in learning how amazee.io’s ZeroOps application delivery platform could help solve your technology challenges and achieve your strategic goals, 🗓️ schedule your free technical demo below.
Writer