Well-designed CTFs expose individuals and teams to operational challenges, novel attack paths, and creative scenarios that can be later applied in their work both as offensive and defensive security professionals.
Not all CTFs are created equal, and there's a lot more that goes into designing a successful CTF competition than just coming up with the challenges.
Fun and practical was the mindset that Hwong brought to the DefCon CTF, a massive multi-day affair that had over 400 individuals and teams trying their hands at the challenge and a team of 20 working under him to run the event.
A veteran researcher and seasoned CTF participant, Hwong had never run a CTF before this event.
One of his biggest hopes for his first try at the job was to level up the relevancy and realism of the challenges in the event, which can sometimes be a bugaboo in CTFs today.
As he dove into the project one thing he found especially challenging is how little information there is out there about running CTFs. Most write-ups are from participants who rate an event and explain how they solved challenges, but there's rarely information offered on best practices in running an event.
In that spirit of security community sharing, he shares some important lessons that his team picked up along the way so that others in charge of CTF design can learn and understand from the process.
Take a Software Development Approach CTF creators should definitely take a software development approach to designing the technical elements of their challenge, Hwong recommends.
One of the big lessons he learned is that CTF designers need to bring software development rigor to the table that goes all the way through testing and viability work.
The crew running a CTF also needs some serious operational rigor as well.
He and his team are trying to learn from the experience to figure out a practical method-from time, effort, and expense perspective-to give participants a truly isolated environment without making the whole CTF less viable because things break or take forever to execute.
Finally, Hwong says that on the operational front CTF show runners also have to be mindful of the constant communication that they'll need to facilitate between their team and participants.
Designing Different Difficulty Levels Is Hard Getting the difficulty levels of challenges right and creating a fair scoring system may be harder than a newbie CTF organizer may initially think, warned Hwong.
There's also the issue of normalizing and balancing out the advantage that big CTF teams have in racking up challenge points-an issue that one of the participants provided him feedback about after the event.
One possibility is making challenges sequential, but the downside of that is it could make the CTF too rigid and linear, and it could create a bottleneck or dependencies that could blow up one or more challenges.
Hwong says he'd also love to see more CTFs reward participants on techniques like how stealthily they operate in an environment or dock points if they leave too many footprints and fingerprints, and that's an area he'd like to explore as he designs future events.
Blue Teams Need More Fun CTF Challenges After working through his first CTF, Hwong also increasingly believes these events don't do enough to challenge and really engage blue team participants.
Those kind of scenarios are harder to do but they're more realistic for defenders and will make CTFs more valuable for them, he says, explaining that is on his radar for next time.
Hwong also challenges CTF designers-and himself-to incorporate more fresh exploit and vulnerability information into their challenges.
CTF 'Building Blocks' to Improve 'Reusability' Finally, one of the biggest lessons Hwong says he learned is that the industry needs to find more ways to create reusable components for CTF just like software developers do for applications.
This Cyber News was published on www.darkreading.com. Publication date: Thu, 11 Jan 2024 13:20:15 +0000