CDDC21: How Not To Organize a CTF

CDDC 2021 was nothing short of a disaster.

Before I say anything more scathing, it must be said that I do not seek to slam DSTA and BSW who organized this CTF. This post, however strongly worded, is also meant to be constructive criticism.

Between all the technical hiccups, lacklustre moderation, and underwhelming challenges, many participants were overall unsatisfied and even livid at having wasted their time to partake in a fiasco.

I initially decided not to write anything in order to spare time to study for my upcoming Block Tests. Yet I believe that future organizers stand to gain from the constructive criticisms that my seniors, friends, competitors had to share for this CTF in general.

So here we are.

Background

Seeing as to how DSTA was a major cybersecurity organization (and had in fact organized relatively successful CTFs before), many CTF players had come to anticipate a good CTF.

In bullet points, these are what CTF players have come to expect of DSTA’s events:

• A plethora of challenging tasks
• Servers with good uptime
  • More importantly, accountability should they encounter technical hiccups.
• A well-engaged community
• Order within the Discord server

Unfortunately, this event managed to break most if not all of said expectations…

What went so wrong?

Problems

Platform Failures

In general, the standard the organizers held themselves to for the challenge platform was abysmal.

The fruits of their efforts would have paled in comparison to a simple git clone https://github.com/CTFd/CTFd.git. (Fun fact, their platform is largely based off of RootTheBox)

Scoreboard Issues

Their scoreboard saw teams ranked not by solve time, but in lexicographical order:

Though arguably a Quality-of-Life gripe, this caused some confusion and frustration between contestants.

CTF Delays

Brushing unfamiliarity with scoreboard conventions aside, the organizers also had to delay the CTF for the Senior category by 1 day due to technical difficulties.

This is in and of itself not a terrible issue.

However, the organizers had planned the CTF to take place on a weekday. In any case, they should have planned for contingencies that might inconvenience university students and especially working/NSF contestants, who have to take time off their studies, internships and jobs just to participate in the CTF.

Such contingency measures could have, but did not include:

• Reducing the CTF timeframe and challenge scope, seeing as to how multiple Junior category contestants had already finished the first wave of challenges.
  • This would minimally inconvenience the Senior category participants, while salvaging their CTF experience.
• Announcing platform fixes in an organized and scheduled manner.
  • As evidenced by the chaos in the chatrooms, this was not adequately done, and indeed, many announcements were drowned in the deluge of spam messages until @Corliss (DSTA) placed some of the messages into the #announcements channel.

Sowing Discord in Discord

In an ironic break from convention, this was the first local CTF in which the Discord server was thrown into utter disarray.

The organisers seemed to be entirely unacquainted with Discord’s workings:

• The #announcements channel was not set to read-only, and participants had a field day spamming the chat.
  • This was equal parts risible and jaw-dropping, since virtually all online events should have clear information dissemination channels.
  • Lesson: Properly configure your Discord server before any participants join.
• The Discord invite link was also distributed freely, in contrast to last year’s more restrictive distribution of Slack invites. This might have contributed to all the bot spam the server received.
  • Lesson 1: Refrain from disseminating Discord invites publicly.
• Also, Discord permissions were not set as high as they could have been. This most definitely enabled spam bots like those below to vandalize the server.
  • Lesson 2: At the very least, establish basic protections for your event Discord server. (e.g. Discord’s inbuilt phone verification system)

The Result?

Within the first day alone, spammers sent all sorts of junk, absolutely flooding the server.

(Such spam included, most abjectly, ASCII porn, in a server filled with adolescents, and copypasta.)

Bans were eventually meted out by the admins to combat the spam, but only after the damage had already been done.

Interestingly, this patent negligence contrasts very well with the man hours CPG Support put into eliminating (most) mentions of BSW (the organizers’ names) and criticism from participants.

Given that the censor seemed to be on and off for extended periods of time, some moderator on the Discord must have spent hours looking for occurrences of the keyword to censor.

All in all, due to the organizers mismanagement of the Discord server and ignorance/negligence of basic practices, the Discord server was in such shambles that they had to nuke it immediately after the CTF:

Not a good look.

Support

CPG Support’s relevance in support, much unlike in Discord moderation, was a lot less consistent. While some members of the staff were helpful (and we are always grateful for that), other questions were routinely ignored, or just dodged.

Instead of creating a ticketing system as with most other CTFs, support staff simply invited participants to DM them.

• Suggestion: Streamline support requests by establishing a ticketing system. This would also curb the problem of spam drowning requests for assistance.

Most of these requests were pertaining to challenge clarifications and logistics assistance, each of which had their own problems:

• After clarifications that resulted in the organizers making changes to the challenges themselves, only the person seeking the clarification was notified.
  • Not only that, the organizers often acted as if they were constantly besieged by insult after insult to their dignity. On occasion, they claimed that challenges were “working as intended”(Another Example), before surreptitiously pushing hotfixes to said challenges, publicising the changes only under public scrutiny.
  • This is childish behavior and shoddy communication on the part of the organizers, and led to much frustration among participants.
  • Lesson: Communicate any changes to challenges properly and expediently.
  • Suggestion: Give participants ETAs so they have something to look forward to. It is difficult to keep to said ETAs, but at least you don’t leave them hanging.

(Above) An example of the organizers’ poor communication.

(Below) Challenge authors changing challenges after a user found that the attached link gave a dysfunctional archive. This message was NOT relayed to others by the organizers.

• During the tiebreaker especially, a request for help with logistics was simply ignored, acknowledged by none of the organizers. This was absolutely infuriating as a tiebreaker contestant.
  • Lesson: There are no lessons to be had.
    • Rule #1 of Support: Don’t ditch those in need.
  • (I AM rather salty. But of course, I still do congratulate burden_bear+’s excellent efforts at beating us in the tiebreaker.)

Challenge “Design”

Many challenges were designed in the most literal sense of the word. That is, they were thought of, roughly drafted, and then left sitting in a folder until the day of the CTF came.

We have strong reason to doubt that, for example, many of the Linux challenge boxes were tested rigorously, which led to many humorous situations (and countless others less so):

Reports of Linux box passwords being tampered with, preventing competitors from even attempting the challenges to begin with.

Upon entering a pwn Docker container. Someone had altered the .bashrc file to play a low-resolution Rickroll.

Some contestants seem to have had their fill of vexation and aggravation. Not a good sight in a CTF.

Or, that the flags were even present in some of the challenge environments in the first place.

So one day, we were doing a Web challenge and our teammate discovered a directory traversal vulnerability…

After 1 hour of searching our butts off for the flag to no avail, the flag spontaneously manifested in the /etc/passwd file…

And it appears we weren’t the only people who lost our marbles at this foul play by the organizers…

In and of themselves these were not unredeemable mistakes. The occasional unintended solution and unforeseen bug is very common in CTF events, and no one can be expected to make watertight challenges.

What WAS exasperating was the organizers’ absolute lack of transparency and communication whatsoever.

For starters, in changing the document in the Microbes challenge without first announcing this change to the participants (see N00bcak’s conversation in #support)

Or, condoning bots that repeatedly leaked flags into the chat room:

Perhaps most appallingly, the lax security measures on multiple servers that led to:

• Challenge passwords being altered after an unintended privilege escalation, robbing other participants of access to flags.
• The overriding of flags, to the same effect of the previous point.
• The ability of some participants to exfiltrate the flags for all Web challenges after completing just one of the challenges:

Overall, their challenge creation process appeared to be extremely sloppy, riddled with a lack of accountability and unresolved errors.

Challenges (Or, the lack thereof)

DISCLAIMER: This was written by a Junior category player.

It must be said that we are understanding of the DSTA’s aims to promote information security and cybersecurity awareness among young Singaporeans. So we will ignore the fact that many of the challenges were very different in nature compared to what one would expect from, say, CTFTime.

Nonetheless, the difficulty disparity between this event and that of other CTFs was not only jaw-dropping, it was absolutely egregious.

To further elucidate our points of contention, we will highlight several issues with the challenges in this event in general:

Point 1: Full Solves

Admittedly, it is a reasonable aspiration for cybersecurity agencies to make their CTF challenges on the easier side so as to engage newer CTF players and to retain their interest. Even so, they would usually try to make tougher challenges for veterans who seek more learning experiences in their CTF journey.

However, the concept of “challenge” seems to have flown over the heads of this year’s organizers.

A cursory glance at some beginner CTFs such as the recent HSCTF 8 and TAMUCTF 2021 would show that CDDC’s challenges were far easier and significantly less demanding than is expected of an average CTF for beginners.

Since many of the challenges were also much simpler than even that of local CTFs, an interesting phenomenon occurred, virtually unheard of in “normal” CTFs.

Many teams in both the Junior and Senior category managed to solve ALL the provided challenges before the stipulated time was up.

Point 2: Lack of Flexibility

In spite of the repeated, in-your-face feedback that the organizers were provided with, with respect to challenge difficulty, the organizers took no measures to adapt to their unexpected situation.

• Instead of releasing new challenges from the Senior Category (thereby sidestepping the need to make new challenges), the organizers stuck to their original list of challenges.
  • Lesson 1: Be prepared to adapt to unexpected situations.
  • Lesson 2: Be aware that other people also have lives to lead…
• Rather than adjusting their challenge release schedule to keep the multiple top teams engaged, the organizers stubbornly clung onto their pre-determined release schedules.
  • This artificially protracted the competition, much to the dissatisfaction of participants.
  • Lesson: Develop (and execute) contingency plans so you don’t have to infuriate everyone when you hit a roadblock.
  • Suggestion 1: Ask your participants what THEY would like. Understand their concerns and preoccupations and plan accordingly. Give them options, so everyone can be happy.
  • Suggestion 2: And please, try not to do Suggestion 1 the other way around…

Point 3: “Skidding”

This is, admittedly, my personal gripe with CDDC 2021. Not many share my vehement hatred for this practice. Nevertheless, it is a point all CTFs should consider.

And because this is my pet peeve I will expound on it for a bit, (because I can >:)):

My own conviction is that CTFs should be a setting where one is constantly challenged to think critically and creatively of ways to break an app, a web service, a binary, or a cryptosystem, etc.

The practice of skidding, or, the abuse and at times **plagiarism** of pre-made scripts in the hopes that the flag would be handed to you on a silver platter, chucks this vital function of CTFs out the window, in favor of ungratifying rote learning that propagates a disgusting impression of cybersecurity as a discipline and a job field.

With this in mind, it is abhorrent and disheartening that an overwhelming majority of the challenges could be completed solely through “skidding”,

For a prominent example, the challenge named “Default Password”, a memory forensics challenge, can be completed simply by invoking volatility2’s lsadump command. This is the challenge with the greatest number of points in that category.

This should NEVER, EVER be the mindset that organizers encourage or expect in CTFs:

If not, the CTF is an utter failure and the organizer should pack their bags and be sent rolling home.

Suggestions:

• Make CTFs engaging and interesting.
  • Take the effort to make a one or two challenges from scratch.
  • If you are short on time, introduce some tweaks!
• Take inspiration from real life!

• At the very least, don’t plagiarize challenges like a certain cough cough…

Looking to the Future

I offer our condolences to everyone left with a sour taste in their mouths by this year’s CDDC. This is nowhere near the best that local CTFs have offered us, and deserve and should expect better.

To be fair to DSTA and BSW, the organizers of this CTF, they have also endeavored to make CDDC 2021 an enjoyable experience. I applaud them for their efforts and where they have fallen short, I hope they will learn from participant feedback and continue to perfect their CTF organization.