Developer War Games - Capture The Flag With Danny Grander

“Guy Podjarny: That's the reason for the name is Capture the Flag. In all these cases, you're trying to somehow find a secret, some digital treasure.

Danny Grander: There are two types of Capture the Flag competitions. One is called jeopardy-style. The other type of CTF is the attack-defence one.

Guy Podjarny: Whoever conceived of this idea was already trying to find flaws in it as they were conceiving it. That's what I thought about, how can they reduce the risk? Do you need to be a super security expert to tackle them?

Danny Grander: There are five different categories for challenges. These include crypto, forensics, web, binary reverse engineering, protocol analysis. The different challenges, they have different difficulties. You could also pick on the simpler ones and see how it goes from there.

[INTRODUCTION]

[0:00:47] Guy Podjarny: Hi, I'm Guy Podjarny, CEO and Co-Founder of Snyk. You're listening to The Secure Developer, a podcast about security for developers, covering security tools and practices you can and should adopt into your development workflow.

The Secure Developer is brought to you by Heavybit, a program dedicated to helping startups take their developer products to market. For more information, visit heavybit.com. If you're interested in being a guest on this show, or if you would like to suggest a topic for us to discuss, find us on Twitter @thesecuredev.

[INTERVIEW]

[0:01:18] Guy Podjarny: Hello, everyone, and welcome back to The Secure Developer. Today, we actually have a slightly unusual episode. We're not going to talk necessarily about best practices, but rather about Capture the Flag, CTF, these types of security competitions. To do that, I actually have my co-founder here from Snyk, Danny Grander. Danny, thanks for coming to the show.

[0:01:35] Danny Grander: Thank you and hello.

[0:01:37] Guy Podjarny: Actually, I guess, before we dig into CTFs, Danny, can you maybe say a little bit about your history, about how you got into this whole world of security?

[0:01:44] Danny Grander: Nowadays in Snyk, I'm responsible for the security side of things, mainly dealing with vulnerability database and collection of dead data. In my past, I've been doing development for about 10 years. Later, I switched to research, where I mostly been doing reverse engineering and hunting for vulnerabilities in different systems, but mostly embedded ones.

[0:02:09] Guy Podjarny: We started this chapter about CTFs, because you've participated in a bunch of them. Before we go deep in, can I ask you to say a few words about what is CTF? What is this Capture the Flag?

[0:02:20] Danny Grander: Yeah, so CTF is essentially a hacking competition between teams or individuals. They compete against one another and measured by their skill in different areas. All are related to security. This can be cryptography, technography, finding vulnerabilities, various engineering web challenges.

[0:02:44] Guy Podjarny: This is all different aspects of, I guess, information security, so it's mostly digital, but all sorts of digital security?

[0:02:50] Danny Grander: Yeah. I haven't seen CTFs that do physical security thing, but I could guess that there are some.

[0:02:58] Guy Podjarny: Yeah, that exists. I guess, if they're going to do lock-picking sessions at DEF CON, then there's no reason why you wouldn't include those in the CTF competition as well.

[0:03:05] Danny Grander: Yeah. Actually, now that you mentioned that in the last DEF CON conference, in the car hacking village, there was a car-hacking CTF. I haven't participated in that one, but I know there was one.

[0:03:17] Guy Podjarny: It's one of the car-hacking villages. Every conference would have a car hacking village, just for kicks. CTF is this hacker competition, and people come in. I guess, what's the typical format? Is it timebox? How does it work?

[0:03:32] Danny Grander: Yeah. Basically, there are two types of capture deflect competitions. One is called jeopardy style, where there are usually dozens of standalone challenges that each has a score, usually related to the difficulty of the challenge. The team that wins is the team that solves the biggest number of challenges. By the time the competition ends, which usually it depends, but it can run for a day, or for two.

The other type of CTF is the attack defence one, where unlike the jeopardy style, each team has to protect their systems, their services, but also, attack others. It's a much more dynamic in a way and represents the reality better in that sense. The skill set is also a little bit different that is needed for that CTF. You both need to find vulnerabilities and attack the other teams with these vulnerabilities, with the exploit you create from finding these vulnerabilities, but also make sure that you are defended against those.

[0:04:37] Guy Podjarny: How does the – I guess, let me split the two apart, just to take a little bit into them. For the jeopardy-style CTF, what's an example of a question? What's an example of a challenge?

[0:04:48] Danny Grander: The one I personally participated for several times was CCC, the Chaos Communication Congress Capture the Flag. It's a conference, and during that conference there is a 48 hours Capture the Flag competition. There are about five different categories for challenges. This includes crypto, forensics, web, binary reverse engineering, protocol analysis. Challenges in any of these are pretty much different. Classic, I would say, reverse engineering challenge would include a binary that, well, you have to reverse engineer, find a vulnerability, and possibly, you also get an IP address and a port where that service that you are looking at is listening on. Then you actually need to develop an exploit and attack that system, getting the flag, which is most often, is a string, or a hash, or something like that, where you submit the solution to the competition website and get the point.

A crypto challenge could be breaking some crypto algorithm that, again, might be in a form of a binary that you are given, or just some ciphertext. You need to figure out what are you looking at and how to tackle that.

[0:06:07] Guy Podjarny: Cool. I guess, that's the reason for the name is Capture the Flag, right? In all these cases, you're trying to somehow find a secret, some digital treasure flag that's either hidden behind a crypto algorithm, or hidden behind a server.

[0:06:21] Danny Grander: Right. The attack-defence type of Capture the Flag competition, it's similar to the Aldur’s game, the Capture the Flag game, where every team has a flag on their system and each team should attack the others and Capture the Flag by compromising other’s systems. Unlike in the jeopardy style, where you should have solved each challenge at a time and the flag is somewhere there, either encrypted inside the data, if it's a crypto challenge, or a file, or some string that is – you can get to only after you attack and exploit the system.

[0:07:00] Guy Podjarny: For these different types of challenges, do you need to be a super security expert that controls all these different attributes to tackle them? How many people, I guess, on a typical CTF team?

[0:07:11] Danny Grander: Yeah. Usually, each person on a team has some specialty. There are some people that do all the challenges, but typically, in our team, we have about 20 members, but not in every Capture the Flag competition all of us actually compete. Just an example, for the last DEF CON Capture the Flag, it was an attack-defence style competition. We were nine people on site and three others were helping from back home.

[0:07:43] Guy Podjarny: Well, cool. Actually, not everybody had to necessarily be there in person to participate in this game.

[0:07:47] Danny Grander: Right. That’s for the attack-defence. Usually, the attack-defence is limited by the number of teams and the number of members that can take part in the competition. In this kind of Capture the Flag, there are qualifying rounds, because in the end, in the finals, there is only limited number of teams that can take part and have their system being protected. Unlike the jeopardy styles, usually is open to unlimited number of players. They don't have to be on-site, on the conference, or whatever is taking place. It can be, yeah, anyone could just sign up and join and play the game.

[0:08:29] Guy Podjarny: Yeah, cool. I guess, the jeopardy-style ones sound pretty straightforward, right? It's basically a quiz. It happens to be a challenge. You go and you do it. It's cool. I guess, in today's digital world, you can just stand-up sandbox environment with as many of those as you want, or some environment where people can go and just get that flag. There's no limit at how many people can attempt a riddle at the same time. The attack-defence, you go against one another. If there's a thousand teams, it becomes pretty impractical to work?

[0:08:59] Danny Grander: Right. Something to add to your previous question is that in the attack-defence one, there is, beside the technical skill of finding the vulnerability and writing an exploit, there is also a big operational part of timing the exploits and trying to figure out, like game theory, things like, figure out what the other team might be doing. There's a whole set of things opens up for you, because you can steal other exploits. You can just wait for your service to be attacked, sneeve the traffic. You could just try to, I don't know, send random traffic to the other team.

One good example of these kind of things that I've seen in the last DEF CON CTF, where a team would backdoor their binary, their service and other teams, including mine, would assume that it's a fixed binary, that one that is protected against a vulnerability, would just take it as is. But sometimes it would actually fix the vulnerability, but it would also include a backdoor that allowed that team that created the binary just to easily exploit it anytime and only by them, because they know about existence of the backdoor. This is the kind of things that can happen in attack-defence. It's an open world there.

Yeah, and the jeopardy style one is just stand-alone challenges and they are really more suitable for a person to take on and try to solve and usually, really can be limited to a person that has a specific skill that solves the challenge.

[0:10:31] Guy Podjarny: Yeah. I guess, in the attack-defence environment, it sounds like some things are visible. I guess, in any one of these cases, you need to set some context, so you can see that, I guess, another company, another team has patched their binaries, so you can see it and you can just use it. What's to keep you from just taking your systems offline, right? It's not like say, “Hey, I gained a good number of points. I don't want anybody to hack my servers. I'll just unplug.”

[0:10:56] Danny Grander: Yeah, so it's a good question. The answer is that for the competition, the organisers, they constantly check for the availability of your services. If your services are down, if you just went and added some firewall rules that prevent any communication to your services, you're basically losing points and you're penalised for that. Same goes for, suppose you patched your service and it's now protected against the vulnerability, but its performance went down, also you get penalised for that. Again, there's a lot of real-time decision-making you need to do about what's good for you, what you can live with, what you should fix. Unlike, again, in the jeopardy-style that is more simpler and defined.

[0:11:41] Guy Podjarny: I guess you can trust that in a hacking competition, everybody's trying to hack the rules and have done so many times in the past, so it's probably pretty bulletproof to find those steps, or shortcuts and factor them into the counts. In the last DEF CON and CTF, there was a bit of an unusual CTF added into it, right? I think it was this DARPA CTF. What was interesting about that one?

[0:12:03] Danny Grander: This year, before the actual CTF, there was another Capture the Flag game played by computers, by machines. This was one that DARPA, it's called the Cyber Grand Challenge, DARPA's attempt to improve and advance the automated vulnerability, discovery, and protection field.

[0:12:25] Guy Podjarny: Or, at least that's what they say.

[0:12:26] Danny Grander: Exactly. Actually, during the last year, there were different qualification rounds between different systems and the final event, the finals, there were eight different machines, different systems that were competing one against the other. The winner would join the main CTF event. There were 15 teams on the CTF. One of them were a completely automated computer system competing against us, 14 human teams. It actually did pretty well, surprisingly or not. It was really interesting to see how a computer could find vulnerabilities, create an exploit for them and protect against them.

Obviously, some of the things, the machine was doing really well. For example, protection is something that is – it comes with a cost, but it's in a way, easier for a machine to do. The cost is usually performance. But for the vulnerability side, usually, the human is better. Again, the machine could fuzz, could try different inputs for discovering vulnerability in a software, but it, of course, could do it really quickly compared to humans. They were seeing all these human teams competing against a machine and machine is not being the last. It was really nice and interesting.

[0:13:47] Guy Podjarny: I mean, it's crazy. It sounds super cool from a technology perspective, seeing the level of technology and seeing this work and the fact that an automated attacker, defender, security team can actually stand up to a reasonable place in a competition amongst already really good security people and hackers. But it's also scary. It's entirely scary when you think about that type of technology just floating around the Internet and people in cybercrime, or people that are well-funded having access to these types of automated machines, maybe it means we need some machines on our side as well on the protecting side for us to have a bit of a shot at defending ourselves.

[0:14:32] Danny Grander: It's a really good point. Actually, one of the things, special things about the cybercrime challenge is that they created a special environment for the challenges and the competition. In the end, it's a Linux system, but they created different binary format, different from an ELF, something that has some reduced functionality, a reduced number of system calls, and that it supports. That, again, was done for both simplification and gaining determinism in all the, again, attack and defence sides. But also, to prevent the easy use, or abuse of the achievements of the different system in the real world.

[0:15:12] Guy Podjarny: Yeah, interesting. I guess, that comes back again to that breaker's mindset, which is probably whoever conceived of this idea was already trying to find flaws in it as they were conceiving it. It's a thought about, how can they reduce the risk? I mean, I think CTF is cool and I think there's a lot of good stuff to read about it. We talked about conferences. I think DEF CON is the most popular CCC being another good one. I guess, are those the only places where CTF competitions run? I mean, if I want to participate in the CTF, do I need to find one of these conferences and go there?

[0:15:44] Danny Grander: Right. Almost in every security conference nowadays, there is a CTF. The big one in Europe is the CCC Conference, the Chaos Communication Congress Conference. It has the CCC CTF. It's a jeopardy-style. The big one in the US would be DEF CON. It's an attack-defence one. But there are dozens of other conferences and most of them have CTFs. Also, companies nowadays run their own CTF game. Google was, this year actually, was having a CTF game, where our team, Pasten, won the first place.

[0:16:20] Guy Podjarny: Congrats.

[0:16:22] Danny Grander: Thanks. It's interesting. All their challenges were – it was a jeopardy style and the challenges were around Google services. It's interesting to see, and for us, first of all, to learn about the different services and things in Google's product. For them, I'm sure it was useful to see how a bunch of teams get to play an attack system that are based on Google's infrastructure, or languages, or services.

There is quite a lot of CTFs going around. For somebody who want to give it a try, I would suggest just signing up for a jeopardy-style CTF. These are open for everybody. You don't have to be in the conference. The next Capture the Flag competition just can play it. Another thing is to go back and look at previous competitions. Usually, definitely for the major CTFs, there are quite a lot of write-ups and all the challenges are open. The information is there. You just could try and take one challenge and try to solve it and of course, see the different solutions and learn from that. Yeah, there's quite a lot of opportunities there.

[0:17:38] Guy Podjarny: Yeah, that makes sense. I guess, the jeopardy-style ones, the bar is actually substantially lower than I originally thought. It sounds like, the jeopardy-styles have many teams. You don't need to pay, or even - or travel, or do anything like that to join them and they're available. You can even practice a little bit ahead of time with some of these older ones. Yeah. I mean, sounds very useful. Also, it's an interesting play around the companies doing the CTFs. I guess, on one hand, it's a recruiting one.

[0:18:05] Danny Grander: Yeah. There is obviously a recruiting angle there. Also, one thing I want to add is that the different challenges in the jeopardy style CTFs, they have different points. Well, the difficulties. You could also pick on the simpler ones, smaller ones and see how it goes and then progress from there. That's also a good place to start.

[0:18:27] Guy Podjarny: Yeah. I guess, one thing to note that happened sometime earlier this year was that Facebook open-sourced their CTF platform. I think they were using it in universities and the likes, probably at the end of the day for a similar purpose of recruiting awareness, making people aware of the services. The fact that they've open-sourced, that implies that maybe at a certain size of either an event, or a company that you can just choose to do it, to even raise education and awareness amongst your employees, or amongst a certain community and just choose a bunch of challenges. You can probably tailor the level of complexity of the challenges to the audience that you have, so not everything has to match the top security conference tiers. That same format could happen for things that are more specific, just attuned to maybe more of a developer audience, or more an audience that's a little bit newer to security.

[0:19:20] Danny Grander: Right. A good example is also in the CCC and CTF, the focus mainly on reverse engineering and exploitation, while other CTFs can be more focused on web.

[0:19:32] Guy Podjarny: Danny, thanks for the review of CTFs. I mean, I think they're a really fun thing. I've not yet participated in one myself and I think I should make the time and go and join one of those. I guess, maybe before we take off, can you share maybe one example of the most interesting, or fun moment, or learning that you've had from your CTFs? Do you have an example like that in mind?

[0:19:57] Danny Grander: One fun story I remember from two years back in a CCC CTF, there was a web challenge, which required exploiting a cross-site scripting vulnerability. There was a blog platform where, what we were supposed to do is to leave a comment on the blog post and for the admin to later visit and click on a link and, well, that's where we were supposed to steal the session and get that mean credential and all that. But because we were trying to solve this challenge on the second day, while all the other teams did that in the first day, during the first day, the automation around clicking on the link on the organiser's part wasn't working. We would understand the vulnerability, we would create a special comment and all that and nobody clicked.

We didn't realise that that's a problem on the organiser's side. After a lot of different attempts of finding different vulnerabilities, eventually, we found a vulnerability in the whole system and we popped a shell on the box that is running the challenge. It wasn't the intention. It wasn't there. It actually didn't help us too much, because, well, we ended on a box. We got root and we started looking for what's going on, which is much harder when you have all the different components that you need to figure out what was the challenge and design for.

Eventually, we contacted the organisers and they fixed the problem. But the funny part is that one year later, they actually liked the challenge, the vulnerability and both the vulnerability and our exploit that they created another challenge that was exactly like that. It was fun, because for us, that one was really easy to solve, because we already have been there. Yeah.

[0:21:47] Guy Podjarny: Even an organiser of a CTF competition is not bulletproof to having vulnerabilities that they did not plan in the system itself. Cool. Well, this has been super fun. Thanks, Danny, for coming, joining us. And, I guess, good luck in whatever future CTFs again, and you’ll keep on winning them.

[0:22:04] Danny Grander: Thank you and thank you for having me.

[END OF INTERVIEW]

[0:22:06] Guy Podjarny: That's all we have time for today. If you'd like to come on as a guest on this show, or want us to cover a specific topic, find us on Twitter @thesecuredev. To learn more about Heavybit, browse to heavybit.com. You can find this podcast and many other great ones, as well as over a hundred videos about building developer tooling companies, given by top experts in the field.