Intro

It’s been a hell of a month. If you follow me at all (if you do, thank you), you’ll know that last year I “almost won” a national cyber competition, or that I’ve been doing this for a while. For the unaware, every November, the Department of Energy runs a collegiate cyber competition called Cyberforce, where teams from across the US compete to secure infrastructure related to some energy sector related service. This year, the event was exclusively in person, so while it had less attendance (~150-160 last year to ~100 this year), it was far and away the hardest it’s ever been.

With it being the third time I competed in this event, and the fifth time I’ve done a Cyberforce-affiliated event, it’s become tradition here to go over the event, the highs, the lows, and everything in between, and let me tell you, there was a lot of all of those things.

Part 1: From a Sprint to a Marathon

Last Time on Cyberforce…

Cyberforce is not like any other Attack-Defense/Red v Blue/Cyber event. For one, since it’s run by the Department of Energy, the fictional company you represent is in the energy sector, which means a couple of things. For one, unlike competitions like CCDC or DEFCON CTF, you’re not only protecting any old network, but you have Industrial Control Systems (ICS) to take care of. Additionally, there’s more than just attack and defense. Cyberforce features scoring in 6 different areas, those areas being:

If words aren’t your thing, here’s a picture: diagram

The network we got was six machines, three of them Windows, three of them Linux. Below is a picture of our topology.

Pasted_image_20231105182108.png

The increased Active Directory usage, as well as requiring services such as SNMP or VNC that I would normally just not use in a network make managing everything even harder than it’s been in past years. Speaking of things that got harder, let’s talk about how things changed.

Becoming a Salaryman (Minus the Salary)

Another unique aspect of Cyberforce is that everything is slightly different every year, for better or for worse. Normally, we get two to three weeks of prep time, where security documentation is due in two weeks, but everything can basically be done at the team’s leisure and you just have to be ready for game day.

Not this year!

Work was trickled out on a per-week basis. For the first week, we were given a short blurb about the business (DER8.9), and then had a week to prepare a risk assessment presentation without knowing the network topology at all. The next week, we finally got access to our machines in AWS, so we had to then write the documentation and finish that in a week (ours was 18 pages*). Finally, for the last week leading up to the competition, we actually finally had time to not only harden and configure machines, but to also set up logging and monitoring software (a lesson we learned from last year).

If words aren’t your thing, here’s a picture that’s as equally professional as the diagram from before: Pasted_image_20231105180516.png

* This doesn’t include a bunch of Active Directory vulns we forgot to add because one of the machines had zero disk space and we then forgot to check stuff after that got fixes.

Professors, if you’re reading this and wondering why the quality of my work tanked (or I was showing up to research meetings with basically nothing), this is why.

I’m not going to say it was a bad thing to have the work stretched out like this. For a competition to be as ambitious and give people the full, all-round experience, there is inherently going to be a lot of work. The trickiest part of this is that it’s really only brutal for mid to advanced level teams. The beginner teams don’t know enough to realize how much work there is to do, and the advanced teams usually have a pretty good spread of skill where everyone can contribute and be efficient. If your team has three beginners and three experienced people, it becomes way more difficult to coordinate and delegate. We definitely got through it in the end, but not without me wanting to hunt people down a few times.

Still, we managed to get through it. There’s a reason this section is called “From A Sprint to a Marathon”, though. We had spent so much time trying to hone in the C-Suite presentation and documentation that by the time we got around to hardening machines, we were all so exhausted from it already. If you wanted to win, you had to be locked in. And if you wanted to be locked in, time management was harder than it had ever been.

No Stone Unturned

In my reflection last year, I shared some tools last year that our team used to automate the process of checking for vulnerabilities. Those were still useful, however, a change I’d like to note is the decreased use of CVEs for vulnerabilities. In previous years, boxes had SLMail 5.5, the pwnkit vulnerability I did a whole blog on, and more. This year, unless we missed something obvious, there were definitely a lot less of that, which meant spotting misconfigured services was key.

Here are some highlights:

Pasted_image_20231105185620.png everything is fine, this is fine :lemonthink:

One thing that irks me, and it’s entirely my fault, is that when we got the Windows 2016 machine was given to us, the disk was almost full with SQL data. Since it was Assume Breach, we weren’t allowed to modify any data that would be used in Assume Breach chains, and we weren’t told until about half of the week in how to save some space without getting in trouble. Because of the low disk space, I couldn’t run SharpHound, and I didn’t want to query LDAP, so we just forgot to audit the second domain altogether, which significantly hurt our documentation score. It’s absolutely our fault for not remembering, but that disk issue was irritating and I’m airing it now because I know someone who has the power to fix it will read this.

Aside from the rant, there were a lot of vulnerabilities, so many so that I found a backdoor 20 minutes before documentation was due. Regardless, once documentation was done, it was time to actually build up defenses.

rake gif

Part 2: Learning From Mistakes (and making new ones)

In last year’s post, I made a point to say that our logging and monitoring was terrible last year, which it was. We used Sysmon, and basic Linux logging, and that was it. Our responses were slow, because we spent so much time trying to search for things that we didn’t know happened. As such, we made it a point to improve.

Thankfully, our work did pay off and we had a much better time scoring assume breach points than we did last year. We picked up Active Directory attacks like it was nothing, and managed to catch a number of ICS related issues when one of our teammates stepped up to dig into how that worked a bit more. Still, we later realized we left ourselves open to a number of issues.

Overall though, there was significant improvement. With the amount of work we put in prior to the event, some assume breach exercises went down like it was nothing.

Pasted_image_20231105192242.png sometimes you just know

Other times, even with all of the logging, we just did not know.

Pasted_image_20231105192608.png he was very upset about this on the drive home

But we’re getting ahead of ourselves, what happened on game day?

Part 3: Ocean Wide, Canyon Deep

This is Fine

Maybe I don’t have that dog in me, but I was absolutely stressed in the days leading up to the competition. Last year it was because we were so far behind, this year it was because things were actually going fairly well and on schedule. As a result, I was extremely paranoid. Did we miss vulnerabilities in our documentation? Does everyone know enough that I don’t need to be asked questions over and over again? Did we patch all of the vulnerabilities? Are we going to crash Splunk?

asdf

Once we got to the venue, though, those concerns had to be thrown out the window. We were back. Same venue. Many same faces, many new. After hanging out and catching up with some people, we headed back to our AirBnB ready to prepare for the event.

Admittedly, the AirBnB was packed. Nine guys in a one floor house with two bedrooms and one bathroom is not something that I was expecting to happen a month ago. Although I wanted to get to work right away, after everyone trying a Reaper-spicy chicken sandwich, playing some rounds of Smash Ultimate, and struggling to get into the ping pong room, we were back to the grind. No eggnog this time because someone decided not to pick it up, but the feeling of determination was in the room once again, albeit a bit spread out with both teams having different priorities. If you’re someone who’s just trying to get into any kind of computer field, I will tell you right now that very few things beat the feeling of having a LAN party with the common goal being to do great things.

Pasted_image_20231105194203.png If you’re ever in Elburn, IL, Paisano’s Pizza is pretty good (not sponsored).

We installed Suricata, Splunk, figured out ICS, and called it a night at 2 am.

Round 2

We ate breakfast at the Q Center (great food btw, any place with lox and cream cheese bagels is great in my book), and it was time to do the thing, just way more packed this year.

1699132376280.jpg

In fact, it was so packed, that when the event finally started, the bandwidth was completely eaten up. Our primary goals were to (1) handle assume breach exercises, (2) solve anomalies, and (3) defend traditional infrastructure, but we could barely do (1) or (3) because of the internet problems. I don’t know if issues ever got better, because one our team members was basically unable to access AWS infrastructure consistently over the course of the whole competition, which sucked.

Eventually, internet issues became tolerable for the rest of us, and it was time to game. If the story of last year was struggling to do Assume Breach, then the story of this year was to solve anomalies. I have previously gone on record to say that the anomalies are usually pretty easy and/or guess-y, but I guess they took great offense to that because this year’s were brutal.

It felt as if most of the challenges were either reverse engineering or steganography, and I would have been more than happy to do these if I wasn’t bogged down trying to do assume breach. As it turns out, when you know the most about Active Directory and reverse engineering on your team, you’ve got a ocean’s worth of work cut out for you and get spread so thin in the process. A friend of mine who wrote anomalies for the event had an entire four-part Nim reversing challenge for ~200 points, and I was barely able to touch it because of how busy I was trying to coordinate other things.

Pasted_image_20231105202107.png Do you think we could form a union to demand less steg puzzles?

That said, it was clear that these anomalies were brutal. Last year, we got about 1400 points doing anomalies, which is ~75-80% completion. The winning team last year had ~95% anomaly completion. This year, there were only maybe 3 or 4 teams that cracked 1000/2000 points on anomalies, my team getting 814 points. The winning team only got 1588 points from anomalies, which says a lot.

Needless to say, the mental stack this year was hard. Bouncing around between anomalies, answering questions, assume breach, and back to anomalies is the most intense multitasking I’ve had to do in a while. I’m writing this the Sunday after the competition, and to be totally honest, the events of those 8 hours between 10 am and 6 pm were a blur.

Part 4: Fin.

The moky of Cyberforce

But when the dust finally settled, it appeared that things did not go how I wanted them to.

Pasted_image_20231105203220.png

Our peak placement was 6th, and we just barely made it by to stay in the top 10 at the very end. I’m trying to get this out as soon as possible, so I don’t have the specifics on how our sub-scores compared to others.

For most people who came to this event, getting top 10 would be something to celebrate about, call a job well done, and be excited for the next one (which, not to discredit anyone, 10th is a high placing that is well above the average). However, there’s a huge difference between wanting to place high, and wanting to close it out and win the whole thing, and I wanted the latter.

The reason I was proud of a 10th place finish in 2021 and a 5th place finish in 2022 was because both felt like Cinderella runs. 2021 was the first year I really began contributing and participating in Red v Blue competitions, and seeing that work pay off felt nice. In 2022, we were woefully underprepared until literally hours before the event, and managed to clutch a higher position than I thought we ever could with how we prepared. This year, with our team’s varied strengths, I thought, and knew, we could do so much better than 10th.

Am I happy about the consistency over the last few years? Yeah, I’m glad we didn’t have a year where we completely got owned and lost it. Am I really happy with 10th? No. Being happy with 10th feels like settling. I came here to win, and we just didn’t close.

So what went wrong? My thoughts:

Shout out to the rest of Order of the Purple Flamingo. Regardless of how I felt about the event, I’m happy people who were new to cyber competitions got a good experience from this and have a much better understanding on what to improve on. Also shouts out to the Cyber Flyers team. We had enough sign ups for Cyberforce this year that we sent two teams, and I put most of the good people on my team. Even though they got 58th (I think), I know many of them outperformed what they thought they could do, which is a great thing.

Final Thoughts

This is (probably) the end of me competing as a blue teamer for Cyberforce, unless I decide to go to graduate school, which I don’t even know if I want to be in the US for. It’s a bittersweet end, and no matter how I feel about how it went, I have to thank the organizers for putting on a solid event yet again. I keep making bulleted lists of things, but I know a few people actually read this part last year, and I was a little tilted at 6:00 pm when we had to fill out post surveys and filled mine with fake, useless answers, so here’s some extra stuff:

Pasted_image_20231105214401.png That Club Penguin black hoodie pfp is the only acceptable use of a black hoodie person on Discord or any social media.

That about sums up my thoughts on the event. Like last time, no regional awards or anything, just recognition of the top 3. It’s okay though, we make do.

Pasted_image_20231105214812.png listen I’m just trying to save the environment, saying I’m a thief is just a government psyop

ggs we go next