How I Almost Won a National Cyber Competition

Last Time on Cyberforce…

For the 10 of you that read this blog consistently, you’ll remember that one of the biggest issues I had with last year’s event was the conflict of interest between external pentesting and the “assume breach” scoring. For the uninitiated, Cyberforce features 2 primary ways to do red team scoring:

1. Traditional Pentesting - This is the classic Red v Blue format most people are familiar with. Red team hacks blue team, blue team stops red team from doing that and shutting down systems. Simple.
2. Assume Breach - Here, red team is already given access to a machine, and is tasked with executing various, predetermined attack chains. You aren’t punished for being attacked, rather, you’re scored for how well you can identify indicators of compromise and follow incident response protocols.

Last year, we were tasked with hardening all 6-7 machines on our network, which was great for the Traditional Pentesting scoring. But when it came time for assume breach, chat logs were basically:

redteam: Hey could you disable antivirus so we can do this assume breach?
redteam: Hey could you disable the firewall so we can do this assume breach?
redteam: Oh did you patch this? Well I guess we skip this one then.

Admittedly, we had pretty bad incident response when it came to actually doing it, but you see the problem here. This time, however, machines were specifically split into Traditional and Assume Breach, where we would only be allowed to harden the former and could only enumerate the latter.

This was a very, very good change. Some teams were confused, but I’ll call that a reading-comprehension issue since it’s pretty clear what is what from the rules. Outside of the red team scoring, there was scoring in a few other areas as well:

• White Team Scoring - Security Documentation had to be filled out with the following information: Statement of Purpose and Scope, Network Diagram, a full list of vulnerabilities found on all machines with remediation steps, and a detailed writeup as to how we hardened our machines.
• Orange Team Scoring (C-Suite) - A 5-7 minute presentation had to be recorded for a fictional C-Suite panel detailing an initial security risk assessment, risks of integrating new networks, and a summary of the hardening.
• Green Team Scoring - We had to develop a full stack website from scratch for the fictional company we were representing for volunteer end users to access throughout the event. It was out of scope for Red Team to attack, but with some of the required integrations (e.g. SQL, Mail, LDAP), Red Team attacks could easily impact this.
• Anomalies - Basically a mini 8-hour CTF happening at the same time as everything else

To clarify:

The worst part about all of this is that we only had two weeks to get all of the technical things done (i.e. hardening, green team), but only one week to get the White Team documentation and the Orange Team presentation turned in. So, naturally, I neglected all life responsibilities committed myself to trying to complete everything to its fullest extent. Teams were made up of 4-6 people, although I’ve heard it almost a million times now, working in a team is hard. It’s especially hard when you have people at varying levels of expertise. I would love to have 6 clones of me take care of everything, but there are things that I need to focus on, and things that I need to trust other people to do, which I found very difficult. But, such is life and we move on.

Here was the network that we had because I couldn’t find a better place to put it.

• TaskBox (Windows Server 2022) - Administrative server hosting internal services such as email and file uploads
• ICS CNC (Windows Server 2016) - Hosts the HMI for Solar systems and runs the sunpartners.local domain
• ICS PLC (Ubuntu 18.04) - Acts as a modbus master for Industrial Control System operations (ICS)
• PublicDB (Debian 10) - Runs a MariaDB server for, uh, things
• WebServer (CentOS 7) - Hosts a website for a fictional iSolr mobile app (no mobile security here though)
• WebSolar (Ubuntu 20.04) - Hosts the company website for Sole-Zon-Solis which Vita Vehiculum recently acquired

The Game Plan

After the most stressful two weeks of my life a busy two weeks, and a (honestly soothing) late night work session, we were ready for the day of. So, how did we approach this?

Short Answer: Poorly.
Long Answer: All of us had fairly good fundamentals when it came to hardening machines, that was not the hard part. Here’s some of the tools that we used for scouting out vulnerabilities and misconfigurations:

• OpenVAS is an open source vulnerability scanner that we used instead of Nessus, mainly because I didn’t want to install Nessus and open source is pog.
• PEASS-ng is technically designed for penetration testers to find quick wins when trying to escalate privileges on Linux and Windows machines, but it’s also just as useful when trying to find vulnerabilities.
• BloodHound is an absolutely goated tool for Active Directory enumeration. Since the domain controller was set up using BadBlood, I was able to witness such beautiful graphs like this one for AS-REP Roastable users.

^{Yes. Every one of these users was AS-REP Roastable. I couldn’t believe it either, trust me.}

After scanning, we got to work documenting, and documenting, and documenting, and then maybe some hardening. Talking about hardening and putting the C-Suite Brief together is like talking about code, it’s very boring. However, implementing monitoring tools is something I’d like to dedicate an entire section to.

“This is Fine”

It is the day before the competition. I’m stressed. Green team site is not done. Windows 2022 has not been hardened. I still don’t know how Splunk works. We do our red team check-in where they give us a test scenario to make sure everything is working, and we choke very hard. Morale, at least for me, is slowly trending downward.

^{We only found Trevor because they told us the name was Trevor :(}

But then we leave the venue for the hotel, and the air begins to change. We ate at some Asian place because we like getting sushi before events (food was a 5/10) and it was fun. At the hotel, I’m stressed and start working on the green team site right away because I know it’s going to take me a while. The others go for a pool party. Once everyone gets back, however, we all meet in one of our rooms, and we just get to work. You have Lofi Girl in the background, a bunch of laptops out on a table that’s way too small, the smell of burnt popcorn because someone left it in the microwave for too long, a carton of eggnog, and most importantly, determined, yet calm vibes. It’s probably a top 3 college moment for me, but there was something really peaceful in that moment. It’s just being with a group of people you enjoy being around all working toward a common goal, and I don’t think I’ve really had the opportunity to feel that since the pandemic started.

We make cheatsheets, we harden machines, we finish web dev and we go to sleep at 12:30 am ready for a 10 am - 6 pm day ahead of us.

Wait, This is Actually Fine

Fast forward past breakfast, we’re greeted with the sheer volume of college students all vying for first place. Not everyone was there when we were at the venue for check-in for the day before, but when everyone is here, the scale of the whole thing becomes much more real:

^{This does not include the other ~120 teams that were fully online.}

After a grueling very informative 30 minutes of talking from National Labs people giving the usual “oh no we need people in cyber”, the day begins. Documentation is long behind us. Our objectives are now (1) defend against external pentesting, (2) solve CTF challenges, and (3) incident response.

^{Red teamer, if you’re reading this, please reach out to me. You never accepted my discord friend request. :(}

I spent most of my time flipping between incident response and CTF solving depending on my ability to focus on either. Our score started off in a pretty good place as well; we scored ~1875/2000 points on documentation and the C-Suite combined, which is solid, and put us in the top 20 to begin with. Not bad.

We go through our first incident response chain. We choke the bag pretty hard, only scoring 30 of the 150 possible points.

Not a great start. But then we start understanding what we need to look for. If we don’t have solid logging, we better be catching this stuff in the moment. With our eyes glued to TCPView, Process Hacker, and some custom scripts, our incident response starts to get kind of nice. We get 120 on the next chain. Then we get 60 (this one was also a choke). Then a 150. Then two more 90’s. I also attempted to social engineer the red teamer to no luck but that’s aside the point.

Was this the best incident response in the world? No. As I said earlier, you need good logging and monitoring if you want to be able to respond to incidents. We’re somewhat lucky we’re being told when things are going down. However, what we couldn’t do in incident response, we could make up for in anomaly points.

Now, I keep saying anomalies are like CTF challenges, but they’re not like the “Bypass strict CSP into an 0day into reflected XSS via bad error handling into prototype pollution + DOM clobbering” type of challenges. These ones were all on the easy to maybe medium side of things, but the main problem is that you’re trying to secure a network and help your teammates all over the course of 8 hours. On top of this, the challenges are supposed to align to the NIST NICE framework, so there’s a decent amount of policy, administrative, etc., related challenges (read as: reading challenges). Again, not very easy trying to read an intel report while your friend is asking you what a random Windows binary is because they’re all named weirdly. All in all, we got about 80-90% of the anomalies done.