Application Security With Omer Levi Hevroni

Levi Hevroni: “If you have a tool, [inaudible 0:00:02], you cannot assume that a developer can undermine you say, “Hey, [inaudible 0:00:06].” I think this is something that, like, you should try to think about it all the time on how to make security usability, and not just security for security. It's hard. It's not simple, but this is where we should go. Everyone [inaudible 0:00:22], but it also can come with known security vulnerabilities, and you want to make sure you scan your packages to make sure it doesn't [inaudible 0:00:31]. [Inaudible 0:00:32], for example, Equifax.”

[INTRODUCTION]

[0:00:36] Guy Podjarny: Hi, I'm Guy Podjarny, CEO and Co-Founder of Snyk. You're listening to The Secure Developer, a podcast about security for developers covering security tools and practices you can and should adopt into your development workflow. The Secure Developer is brought to you by Heavybit, a program dedicated to helping startups take their developer products to market. For more information, visit heavybit.com. If you're interested in being a guest on this show, or if you would like to suggest a topic for us to discuss, find us on Twitter, @thesecuredev.

[EPISODE]

[0:01:07] Guy Podjarny: So, welcome back to the show, everybody. Glad to have you back. Today, we have with us Omer Levi Hevroni who is a DevSecOps engineer at Soluto. Welcome to the show, Omer.

[0:01:17] Omer Levi Hevroni: Hi, thank you for having me.

[0:01:18] Guy Podjarny: So, there's a lot of interesting topics for us to talk about, Omer. But before I dive in, why don't you start by just telling us a little bit about yourself, what you do, and and maybe a little bit of how you got here, how you got into security into this specific role?

[0:01:33] Omer Levi Hevroni: So, hi. As guy said, my name is Omer. For the first three and a half years, I’m working at Soluto. For the past two years, I'm doing mainly OpSec, DevSecOps at Soluto. I started as a developer and then shifted to OpSec, and something that has an insight in Soluto, and I think we will talk about it a bit later.

This is a bit about Soluto, just in case you didn't hear about Soluto. Soluto is an easily managed company, four years ago, purchased by Asurion, a US company. Our mission is to help people with the technology devices they own, or the devices they have at home, and how to get the most out of them. So, this is Soluto, this is me. I like to think about my job as helping the developers to ship more quality product, which is main security, but not only, because I'm part of the DevOps team, and this is what we are trying to do. So, security, but also other things.

Finally, also important is OWASP. I'm in love with OWASP, involved in many projects, and leading one of the OWASP projects which is called OWASP Glue. Glue is a tool that is the integration of CI/CD tools into the CI/CD pipelines. Security tools into the CI/CD pipelines. This is me, this is OWASP.

[0:02:50] Guy Podjarny: Yes. I think, OWASP is for, if you're not familiar with it, is the Open Web Application Security Project, which I know probably most of our listeners know what it is. But it's sort of I guess, a community and a gathering have all sorts of projects and initiatives that relate to security tooling, security knowledge, application security, more specifically.

[0:03:07] Omer Levi Hevroni: Yes. If you're familiar with OWASP, check out the website. It's a really good source to getting started with security and find resources and all that.

[0:03:14] Guy Podjarny: How is it structured indeed, inside Soluto? So, we're going to dig in a little bit into Soluto and security. But you mentioned, you are a part of the DevOps teams, and you also mentioned your DevSecOps. How does it work? What does the team look like?

[0:03:28] Omer Levi Hevroni: So, Soluto worked with [inaudible 0:03:30], which mean that teams are built according to business values. So, for example, we have a team that is responsible for the first few days of the user within the application. It’s called first experience. The team can contain developers for the activity, conduct ideas, whatever the team needs to be full operative, and teams value independent, and they can decide whatever technologies they need. We work a lot with microservices, so each team develops its own microservices, and if they need something from other times, they just add code.

So, this is all the [inaudible 0:04:06] out of a bit different team, which are the DevOps team. The DevOps team is, it's a bit non-traditional to have DevOps team, and we know it. So, we try to find the right spot to have a DevOps team which will do things like operating command to standstill, so the rest of the teams can use it. And on the other hand, we're not the ops team, but inspire the entire company to build more quality product. I think AppSec is a really good fit into this theme, because it's another part of building more quality product. So, this is about how Soluto operates. It means, that the developers, and actually, everyone at Soluto has a lot of freedom and a lot of responsibility and a lot of power to do whatever they want, and to get us to a better place.

[0:04:52] Guy Podjarny: Yes. I think there was a lot of logic to it. I think there's a difference between perceiving or creating a DevOps team in the notion that you're still not sharing ops responsibility with the team. But rather, suddenly this team is the one that needs to be DevOps, sometimes goes against the very premise of what DevOps is about. But I think title aside, there's a lot of sense in saying that there's a team inside the company and we see this a lot, and we do this in a little bit at Snyk, whose customer is the internal developer, kind of the prospect is there, and kind of the team's goal is to indeed, make that.

To an extent, SRE is also like the service, the reliability engineer, kind of that concept. Again, a lot of different tweaks and different elements to it. But what I really like is that security is part of that. You deal with security, you focus on security, you work on security, but you're not like some disparate independent security team. You work as the same part that empowers the developers to get, as you said it, better quality and security be an aspect of that. How, I guess, like one more bit of context on it. So, Soluto is a specific company that is a part of Asurion?

[0:05:59] Omer Levi Hevroni: Yes. Asurion is a very big insurance company located in the United States. Asurion focused a lot on insurance. That's what they did in the past and they are still doing and we are part of Asurion R&D. So, Asurion have a lot of other R&D’s all around the globe. So, we are part of them.

[0:06:18] Guy Podjarny: Got it. How does the – we talked about this a little bit before. How do you work from a security perspective? You gave us the context of Soluto, how do you work, like your DevOps, the security practices that you do? How does that interact with sort of the broader security perspective from Asurion?

[0:06:37] Omer Levi Hevroni: So, this is what they started with. I started at Soluto as a developer. I think it was two years ago. Yes, it was more than two years ago. Asurion started an initiative called – it comes from insurance security. They understood that they have more and more developers all around the world, and they can no longer maintain security from sitting in Nashville, or in San Mateo, or wherever they're sitting, introducing the entire R&D all over the globe.

So, they started a program called the Security [inaudible 0:07:07] which is basically like other companies called the Security Champions. It's basically the same thing, which is, instead of going and hire external security experts in each R&D, and started to embed them into the R&D, they choose developers from around the globe, give them security education. For example, I did a SANS course, SANS pen testing course. It was one week of a course and then a test. So, we did this course and we started to work together as a maintenance. We are now a [inaudible 0:07:41]. We have a Slack channel we're talking about. This is how I get into all this security OpSec world.

Currently here today, we have two mavens, and we started to add more, because we understand we need more. This is how it all started. We are now interacting, there is someone in insurance security team that is responsible on mavens, so we're communicating a lot with him for guidance and help, and to learn more of what we want. Here at Tel Aviv, all of the people here have lot of freedom to add things and try to think about new things that can impose the security here.

[0:08:18] Guy Podjarny: Do the mavens, typically, like yourself and the other maven, in the – the name is catchy. Just roll with that.

[0:08:27] Omer Levi Hevroni: The source of the word maven is English, actually.

[0:08:30] Guy Podjarny: Oh, I did not know.

[0:08:30] Omer Levi Hevroni: Yes. It comes from the word [inaudible 0:08:32], which is Hebrew [inaudible 0:08:33]. It’s very funny.

[0:08:35] Guy Podjarny: I didn’t know that. I guess, because of the history. Yet another reason to have two mavens in the Tel Aviv team. Do the mavens typically have as a part of their day job? Is your day job security? Or is your day job, a bunch of other things and security, and you're just sort of passionate and therefore find yourself doing security? I guess, how is that similar or different to other mavens across Asurion?

[0:09:01] Omer Levi Hevroni: That's a good question. One of the main challenges of being a maven, and I've seen that within time events, is that you need to be a maven, besides of your regular day job. I very first understood that this is what I want to do full time and I think there are not a lot of mavens who are doing it full time. I did it full-time for one and a half year. And then I joined the DevOps team. So now, I do part of regular DevOps tasks like running Kubernetes cluster or stuff like that, and security tasks, try to combine them. But it's definitely very, very hard to find the right spot between doing your regular job and doing the maven stuff, because it's not easy. You have tasks that you need to complete and you need to somehow think about security things. This is a challenge of being a maven. I am lucky to not have it.

[0:09:52] Guy Podjarny: Yes. But the versatility also of the program where is relevant, then people would have that as a full-time job versus not. Very cool. So, I think like the maven program, I want to spend a couple more minutes on that one still, just because it sounds like it's working well, and I think it's a program that many people tried to do. Can you tell a little bit about how is knowledge shared within the maven? Like you mentioned the Slack channel, internally. What other good tooling or methodologies work to share information between mavens?

[0:10:23] Omer Levi Hevroni: We have a few on meetings. One [inaudible 0:10:26]. We also try to meet like in big conference, for example, in two months some of us are going to be at the AppSec California. I'm going to give a talk there. So, we will be there. We will talk together and hang out. Last year, we did an internal con for the mavens, right after California, and we might do it again this year.

For example, do it at DEF CON Black Hat USA. So, this is an example as to where we can meet, share ideas, and learn from what others are doing. Sharing is hard, but we try to do it as much as we can. These conferences have different challenges. So, this is also interesting to see what they face, what we are facing. There is someone from insurance security team who is responsible for all the migrants. So yes, so help us like to say, okay, they face the issue you're facing. Maybe if you talk with them. So, there's a coordination.

[0:11:20] Guy Podjarny: Yes. It makes sense. Basically, a component of it is just natural social interactions. You're on the Slack meeting, you meet up, you naturally go to the same events, and then you meet up and share knowledge there. And some of it is a little bit more coordinated by the person in the in the central security team whose job it is to make this program successful.

[0:11:38] Omer Levi Hevroni: Part of his job, yes.

[0:11:38] Guy Podjarny: Or part of his job.

[0:11:39] Omer Levi Hevroni: His main job is responsible for application security. So, it makes sense that that mavens is part of his job.

[0:11:45] Guy Podjarny: It’s one of his primary tools, I guess. Primary means of succeeding. Cool. So, thanks for sharing that. Indeed, let's maybe veer off from the broader program into, like, practically speaking. So, can I can ask you, to describe a little bit, how is security embedded today? You work in this very modern, it sounds like a very modern surrounding, and I empower teams. They run it. You have your continuous deployment pipelines. Use Kubernetes, quite clearly. How was security embedded into it and the sort of the practical day-to-day?

[0:12:17] Omer Levi Hevroni: So, I already mentioned that we have very independent teams, and we don't have one central mechanism that control what teams are doing. Say to teams, okay, you can't do that, or you need to do it that way. This is something that is very challenging as a maven, to come and engage the developers, help them understand why they need their security. Not only developers, but also product managers and everyone else. This is a main challenge I'm facing for the last two years. We are trying to do a few things to handle it. It's still a challenge we want to succeed solving it. But we try different things.

For example, I'm doing a monthly security awareness session each time on a different topic, trying to encourage development developers during the session themselves, so they can learn about something. We also try to do [inaudible 0:13:05] workshop when we do like, a quick overview of what to start modelling and give them a hands-on experience. Of course, CI/CD where we try to integrate the static analysis tool. So, just do that dynamic analysis with OWASP. [inaudible 0:13:19] of course, Snyk, but this is also a place where things can get really complex, because it's not enough to have tools. You need someone to integrate them into the pipeline.

So, we need somehow make the destined product understand that, for example. Now, it's the issue, when they get a new project, they should spend some time on adding Snyk to the project, and look on Snyk’s finding and fix them if they need, because it's acquire some time and they need to want to do it. Otherwise, they will just ignore it and literally not be helpful. So, it's not a simple task.

[0:13:53] Guy Podjarny: Indeed. I actually like digging into that and I think you've already veered a little bit. You've actually written a good post about this, that just summarises a bunch of these tools. Maybe let's get detailed a bit, and just say, “Hey, across these tools, for the different categories, what's your view of the tools wants you to consider? The ones you use? And like the elements of them that help or hurt embedding them into that type of development process?”

[0:14:18] Omer Levi Hevroni: First, I think that when talking about tools, in general, the most important thing is that the tour has been activated. I mean, if you have a tour that just generate a quote, you cannot assume that, a developer kind of mind you say, “Hey, you haven't fought there. It's might be interesting [inaudible 0:14:33] you can see it.” No one will do it. I can say about myself. We have a static analysis tool that just create a report in the build. I was like, “Okay, there is a folder. I have no idea what – I have other things.”

So, this is the most important thing, back to build, or it doesn't [inaudible 0:14:47]. This is why I'm so invested in Glue. A lot of time, you can find the guide security tools, but when you try to integrate them into the pipeline, things get messy because the tool don’t support the CI tool you’re using. For example, they don't know how to output in a format that teams know and love. They don't have a good way to ignore false positive, which is also very important. You can work a bit, but you need some others, for the developer to say this is false positive. I want to ignore it. Glue can help with that. And this is why I have so much invested in this stuff. For me, it's like, it's the basic requirement for any security at all. If you don't do that, there is no meaning to adding the tool at all. Besides having like a checkbox saying, “I want a static analysis”, which is also important.

[0:15:33] Guy Podjarny: I guess, it's a tricky action. On one hand, I'm entirely with you, which is, you have to be realistic with your expectations. The fact that you've provided information off band, just in some other repository, which is not – nobody's going to notice that you're on it. So, you have to kind of get real and break the build or get in line with the developer’s actions, not expect them to kind of go off and read and work off a list that is stored somewhere else. On the flip side, breaking the build is like a pretty aggressive action. It's a sledgehammer. So, I guess what you're saying in that context is, the security person should feel empowered to break the build. But to achieve that, you need to ensure that the developer has concrete actions they can take, without waiting on that security person move forward.

[0:16:20] Omer Levi Hevroni: Yes. This is what I said about engagements. For example, how to it our works, developers have more control and they can decide that a specific finding is a false positive. It's not that I'm saying that they need to come and talk with me and they get an improvement. So, you need to understand that the pipeline is there. It's not yours. If you add a tour to their pipeline, you should expect that it's not your pipelines, and they should have the full ownership of the tour. They should have the full power of disabling it, marking it false positive or do whatever they want, if the tool is not behaving. So, it's a good point. This is something you should take into consideration. Again, this is why you must make sure that you talk with the devs and they want to edit and they know what the tool is doing and evaluate it, which is not a simple thing.

[0:17:06] Guy Podjarny: So, I guess like an interim solution between like, it's in the builds textual output, or whatever it is, or even in like some other third-party tool’s portfolio, and breaking the build, oftentimes, what's being discussed is like log a Jira issue, right? Or like, get it into my issue management. Do you find that effective? Do you do any of that?

[0:17:25] Omer Levi Hevroni: I think it's even more violent, because as I say, it’s the team responsibility to decide if they want to log into find an issue. It's their decision, if they want to open an issue on GitHub, [inaudible 0:17:37], or whatever they want to do. It's a totality, team decision. What I want to do is [inaudible 0:17:42] and then let them – Glue let you do that. Glue basically lets you say, “I want to ignore it. I want to postpone it, because I want to handle it like in two months”, and then you can file an issue and take care of it later. It's all kind of from the same place that lets the developers all the information, then let them choose what action to take. But give them the power.

[0:18:04] Guy Podjarny: Yes. It just has to be a part. It's legitimate that on one hand, breaking the build is hard action. On the other hand, it's a transient action. So, like the build broke, you've done something, you went away. If this was a false positive, there's never any like false record of it being an issue that you have to now acknowledge. It's like you broke the build. You said it was a false positive. You can order it accordingly and you continue. While if it was logged as an issue, then suddenly it gets into like your metrics and those elements and that might actually be more aggressive, depending on how the team operates.

[0:18:38] Omer Levi Hevroni: Yes. You make a decision for them that it's an issue they should look at. Of course, you can also support it as a GitHub check on [inaudible 0:18:45]. For me, it's the same. If it’s a check on PA or work in the bit, same thing.

[0:18:51] Guy Podjarny: It's about the same because it's in the pipeline. At Snyk, and in general, and like myself, I feel like the commits status tests on pull requests are pretty happy medium between them, because on one hand, they are very visible, as an individual test. And on the other hand, you can choose whether they're breaking or informational. You can kind of make them, so they're a little bit more flexible around allowing a team to decide whether these breaks or not.

What you can do in the build as well. It's just the build is a little bit of this, like one big unit. The other aspect of it is that pull request test a delta, right? They test the change between this change as well. Builds test a state. So, especially in the context where suddenly some new vulnerability can come from the outside, pull requests have some advantages there. But I agree. Conceptually, it's the same as what you're describing. It's just about putting in the pipeline.

I love that in the blog, you actually talk, just again, practical because I believe kind of the listeners would like that. Just go through the specific classes of tools that you have, and maybe say a few words about how you treat them.

[0:19:50] Omer Levi Hevroni: What I try to do on the blog is going over about different security tests that you can use different kinds of tests, that you can use to test the quality of your code. This is why I called, want to write a good code start using security dev. I try to mention a few open-source, free security tools that you can start using today. So, for example, I started with static analysis and there are a lot of tools with a value and quality out of them, especially commercial one. Not all of them are good enough. There is a nice tool by Microsoft called DevSkim. It's a nice tool. What I really liked about it is the IDE integration. There is an example on the blog post where I can say that by using DevSkim with Visual Studio code. You can just, with a few clicks, fix the issue. DevSkim can fix it for you. You don't need to think. You just need to click twice in two places and issue is fixed.

It's a really powerful evolution of static analysis. It's feeding you with a spoon, and you don't need to think, “Oh, I [inaudible 0:20:50].” Or, “What do I need to do?” You can just click, “I have an MD five, which is insecure, which has [inaudible 0:20:56].” It does the decision for you and it's really nice. It makes it a lot more easier.

[0:21:02] Guy Podjarny: Very cool. DecSkim, right?

[0:21:04] Omer Levi Hevroni: DevSkim. Yes. By Microsoft. It has support for a lot of languages and some IDEs. Another tool to notice is that it's easy to write tools for DevSkim, which is not always the situation with all the static analysis. In DevSkim, it’s very simple and you can even write the history, which is amazing. Because if you cannot customise the static analysis tool, you'll miss a lot of the power. He can do it and it's simple. So, this is DevSkim.

[0:21:31] Guy Podjarny: Okay. Very cool. The second thing is dynamic analysis, which is static analysis look on code. Dynamic analysis try to look on live application requests and responses and see what's going there. Here, I mentioned, OWAPS ZAP, which is a security tool by OWASP. You can use it to proxy your application and it can like find various security issues in the app. Very nice, very powerful. You can check out to see how you can do it. It's actually relatively simple to take existing black box or end-to-end test and put them closer. So, you can get dynamically, a security test for free.

[0:22:08] Guy Podjarny: How do you set up? Like static analysis, you look at the code. There's a question about scan times and the likes. Although, it sounds like DevSkim is more about the IDE. So, it's more like just the inline. You don't put in the build. But –

[0:22:20] Omer Levi Hevroni: It works also both CLI and IDE

[0:22:23] Guy Podjarny: Okay. But for ZAP, one of the objections oftentimes to put black box testing or dynamic testing, is that you also need to set up an environment. You need to set up some system to run and the length of it. What have you seen with ZAP? What would you recommend doing there around it?

[0:22:41] Omer Levi Hevroni: So, why am I doing here today, we have a lot of – we call it black box tests, but some other places called end-to-end tests, which is a test where you put [inaudible 0:22:49] with Docker Compose, and then you send like HTTP request to it and see how it reacts. So, once you have such tests, you can just proxy the request and response from the black box. Then, you get four dynamic tests without the need to have another enrolment and start mocking requests and responses. It is the entire solution. Of course, if you have good coverage of the black box, or end-to-end test, you will also have good coverage of the dynamic analysis. So, it's nice.

[0:23:18] Guy Podjarny: So, it leverages, it builds on the invocations of inputs and things like that, that your natural unit tests would invoke, and just extends them to introduce security tests, like, manipulate, basically.

[0:23:33] Omer Levi Hevroni: Yes. Leveraging the fact that we're already using something to send request and response to the application, yes.

[0:23:37] Guy Podjarny: Yes. To an extent it sounds like the unit tests are the crawling phase, because a lot of these black box testing tools start by crawling the application to discover inputs, and then continue to manipulate them in attack. To an extent, like what that does is uses these existing tests as the crawling bit, and then just goes on to do it.

[0:23:56] Omer Levi Hevroni: Yes. Then, you don’t need to guess what your – where's the existing application. Again, you have good coverage, because the end-to-end test, usually try to cover as much as possible on the application.

[0:24:06] Guy Podjarny: Got it. How much, like what's your experience been around the duration over the time? How much would that add to my build time, if I introduce this type?

[0:24:14] Omer Levi Hevroni: Not a lot. The effect on the build time is relatively small. It's a bit of a challenge to add the app to existing test. It’s acquired not a lot of work. I try to make it as simple as possible and there is a full guide linked on the blog. But it does require some effort and some changes to our view on your test, because you need to interfere with the request and response from the test to that.

[0:24:38] Guy Podjarny: Got it. Yes. So, there's some natural. You're going to send some requests, you need to proxy some components.

[0:24:44] Omer Levi Hevroni: Exactly.

[0:24:44] Guy Podjarny: Cool. So, we've got SAST and DAST, kind of the static and dynamic application security testing. What else do you do?

[0:24:51] Omer Levi Hevroni: So, to complete that, if we talk about our code, the next part is talking about the packages we are using. Everyone likes to use packages, and packages is a guide. But also, can come with non-security vulnerabilities. After all, it's a code that someone [inaudible 0:25:04]. And you want to make sure you scan your packages to make sure it doesn't contain viruses that everyone knows about for example, Equifax, which we all I think –

[0:25:13] Guy Podjarny: Yes. The infamous struts vulnerability.

[0:25:15] Omer Levi Hevroni: It's always good to mention them, to make sure – it's now even part of their OS top 10. A9 is about using components with known vulnerabilities. So, there are a lot of tools out there that can be used. I think, the most mature one, which is also open source is [inaudible 0:25:33] by OS defensive attack and check. It has a nice UI, and it has support for some languages. But here, the tricky part is how do you know which packages contain vulnerabilities? I think this is something you can talk a bit about more than me. But this is part of the reason that if you can pay for a tool, Snyk might be a better option, because – and again, this is something you can share more than anything. But they usually have, can detect issues that other packages didn't due to [inaudible 0:26:05]. This is about packages, and here, the hotspot is like how we need to upgrade, and this is another place where Snyk do a bit better job than others. They really face the issue [inaudible 0:26:18]. This is about packages, if you want to add something.

[0:26:24] Guy Podjarny: Now, I appreciate I knows. It's great and it's probably worth for the listeners to know that, Omer has been a super helpful user as well, in the world of Snyk. Kind of giving us a great insights and input around kind of making the tooling better, and helping indeed, whether it's commenting and helping kind of build-out have a better vulnerability database, have this better automated remediation, those elements, helping kind of build up the container scanning capability that we launched earlier this year and those capabilities.

We're going to put this aside a little bit, because I'm always a little bit uncomfortable with sort of the – not to make this podcast the Snyk plug. But package scanning should absolutely be a part of that stack that you put together, and I hope Snyk ends up serving your needs. But you definitely should consider if you want OS dependency checker as a free open-source tool there. Again, just evaluate your database and usability there as compared to what commercial offerings like Snyk can do.

[0:27:17] Omer Levi Hevroni: Both those [inaudible 0:27:17] and Snyk supporting notifications. So, if you're using a package, which is not vulnerable now, but I don’t know in a month, someone would have thought the vulnerability. Both of the [inaudible 0:27:28] send notification to you. And we integrated the Synk with a single-use monitoring tool. If you're not familiar with the thing, [inaudible 0:27:37].

Basically, what you're going to do is when someone adds to Snyk, he can also add an alert. If in the future, a new vulnerability will be reported, the team will get an alert like any other layout, like any other production, and will tell you have any vulnerability, please fix it. It's nice. It's all using the existing flow of how we manage our production. So, it's suddenly, the developer is familiar with, and it's really important to use this notification. Otherwise, if we don't publish something, I don't know, for a month or so, you might miss new vulnerabilities.

[0:28:08] Guy Podjarny: Indeed.

[0:28:08] Omer Levi Hevroni: Yes. I think the next priority is talking about Docker, like you need to put your code in something. Usually, we're using Docker, and Docker images has pretty much the same images as packages. There are a lot of packages installed there [inaudible 0:28:24] both issues. First is how you find all the packages. For example, there are some featured image which I will not mention, who use [inaudible 0:28:31] to install things and not via normal package management. You have the same problem as before, to know which packages are vulnerable.

There are good free open-source tool here like [inaudible 0:28:44] which you can give certain Docker image and scan it and put it in a report. The report is where things can get messy, because it's not very actionable. I gave one example of a vulnerability [inaudible 0:28:58]. And this is something that you can't give to developer. I myself have no idea what is [inaudible. If I should fix it, and I will feel a bit scary to fix it. This goes back to what we talked about that we need to engage developer and we need to help them fix the issue. We can't just talk about them like, “Hey, your bridge is broken. You have all these issues.” No, [inaudible 0:29:20]. We just say, “Okay, I don't understand what's going on here. We need to help them fix it.” Here's another case where Snyk is doing a really good job. But I think we can keep talking about Snyk all the time. So, I think it last time I mentioned Snyk. But this is another thing that you should consider when you choose a security tool, see if the tool can help you understand what to do, and not only what the issue is, because it's really important.

[0:29:46] Guy Podjarny: Yes. I think that's consistent throughout this conversation. We've been talking about how the actionability, if you put it in the developer team, like in the dev team’s pipeline, it's their pipeline. They need to have that actionability and understand it. If it's not realistic for them to have the time or the expertise, or if you do not empower them to do so, that tool is not going to stay in the pipeline in a long – the best-case scenario in those cases that they will move it out. The worst-case scenario is that they will just, like agility would grind to a halt.

[0:30:17] Omer Levi Hevroni: Yes. Exactly. You also need to think about the amount of features. For example, if you start reporting like a lot of issues, no one will take a look.

[0:30:26] Guy Podjarny: Indeed.

[0:30:25] Omer Levi Hevroni: No one can go and list, I don't know, more than 50 issues. Also, it's like impossible to have something that is not reasonable. So, we need to make sure that your expectation is reasonable, and not just say, “Okay, I'm going to back the bed and just throw every issue I can find.” This is a serious issue with Docker scanner, where the open-source does not give good solutions. I hope it will improve in the future.

The last thing is a Kubernetes [inaudible 0:30:52]. If you're familiar with Kubernetes, Kubernetes is a way to run Docker containers in production. You will have find that control how your Docker is running in production. Now, you can think about it. Again, please do Kubernetes. Don’t nitpick with me, but you can think about it like how you define your VM, and you can understand from this description that you can do severe security issues. For example, you can mount or the root filesystem. Then, if the container is compromised, the attack surface is a lot bigger. There's actually only one tool that I was able to find out here, which called [inaudible 0:31:28]. The only problem is that it's an online service. You need to send all your files to an online service that are owned by a third party. This is something I think most of enterprise will not feel comfortable to do. Something to know, there are not a lot of good solution here. I hope we're seeing more tools, or at least things that run only internally.

[0:31:49] Guy Podjarny: Yes, absolutely. I guess to an extent, I like to think about this aspect as if you have infrastructure as code, that code can have security flaws in it as well. Basically, it's almost like a static analysis to your infrastructure as code, to say, which bad security decisions have been made there. Or even decisions that were not a problem at the time, but new knowledge coming to fore, implies now that there's a better option.

[0:32:12] Omer Levi Hevroni: Yes. It’s the same for telephone or for a tablet, or whatever you're using. It's again, going back to the developers and let them know. Say to them, “Hey, you open SSH port to the Internet, maybe it's not a good idea.” So, they now have an informed decision to make and they don't just [inaudible 0:32:31] and don't even know that there might be an issue here.

[0:32:36] Guy Podjarny: This is awesome, Omer, like a lot of info. I think this is like very practical, here's how we'll use, and we'll put the link to the blog post in the podcast notes for people to use. How did you find when you kind of rolled out? It’s the other way around, right? This is your learnings and understanding and working with those, Glue. I guess maybe two questions for you. One is maybe just sort of give us a quick rundown on like, what OWASP Glue is and how it helps there. Second, just like what was your experience of doing this inside? If I wanted to start with one of these, for instance, where would you start?

[0:33:08] Omer Levi Hevroni: So, I start with the Glue one because it's a lot easier. As I said before, Glue came to fill the gap of a lot of security tools, are not just good enough to run in the CI. For example, again, they don't know to output in the format that you'll see I understand. For example, they don't support data check API. So, Glue is like the bridge. It's like helping you glue your security thrown into the CI/CD. It has a few nice features which allow you to say this is a false positive. I want to postpone it. It has a lot of format output. So, you can output to file, CSV, Jira, or even TeamCity.

Recently, we added a way to like dynamically add security tools. If your security tour can output JSON, you can provide a file to Glue that say, “Map this thing to this thing.” And then you don't need even to add code to Glue if you want to add new security. So, this was Glue. The second question is a bit more tricky. We did a few alterations and this is actually something I think to talk about somehow in the future and the conference. I just need not to be the full doc. But we did a few iterations when we try to embed security tools into the pipeline from like starting with the static analysis tool and just add it manually with like a skate to all the builds all of us at Soluto. This gives us the benefit of static analysis is now running all over the code. But it has a few downsides.

First, people just have a new thing in the background, which they don't always know what it's doing or why it's broken. And it makes them some time to just say I don't know what it did and it's like my build. I'm ignoring it. This this is one issue. The second issue was that when people created new pipelines, they're not using the regular templates we provided. They don't always have the static analysis, because, again, they didn't know what it is. They didn't understand the value and it was just there. I just need to go over and say, Hey, you have a new pipeline. Please edit.” It does not always happen.

So, this was the first iteration, which was not a great success. With ZAP, we tried a bit combined way, like it exists in some template. And if you create a project with this template, you will get ZAP out of the box. But it does not exist in all template and the adoption is very low. Lastly, with Snyk, where we try to just say to people, “Hey, edit, please.” Again, the adoption is very slow, but the people who actually add it, understand what it does. So, there is nothing I can say that work without issues and it's all going back and back to the same point of talking with the developer, talking with products, trying to understand, trying to extend to the product. “Hey, you added a new feature. If you don't add this tool, here is what is that.” It's also important for them to understand. So far, it's working slowly, and lastly, is having champions on mavens. This is something we're trying to expand the mavens. We have now in Soluto. Hopefully, I think more mavens will allow us to get to more teams in Soluto and engage with them all.

[0:36:18] Guy Podjarny: Got it. Just spread the local champions, indeed. Very cool. Well, thanks a lot for all the info. Before I kind of let you go here, I like to ask every guest just if you had one bit of security advice, or a security pet peeve, or some piece of knowledge that you'd want to share with the team looking to level up their security, what to start doing, stop doing, what would that be?

[0:36:44] Omer Levi Hevroni: So, I think I mentioned Avi [inaudible 0:36:46]. I'm going to mispronounce his name. Avi [inaudible 0:36:50]. Sure, I mispronounced his name. As a sector guy, he likes to say that if security is not usable, no one will use it, which is basically what we talk about all the session. Make sure that you build things that are usable that developers and users can use. For example, don't add a login page that is breaking usability. I think this is something that, like you should – I try to think about it all the time on how to make security usability and not just security plus security. It's hard. It's not simple, but this is where we should go.

[0:37:23] Guy Podjarny: I'm 100% with you. It's not easy. We don't do it. We're all lazy at the end of the day and we all – not just about laziness, but we all have a million things to do. Omer, this has been great. Thanks a lot for coming on to the show.

[0:37:34] Omer Levi Hevroni: Thank you very much for having me. It was very nice talking with you.

[0:37:36] Guy Podjarny: Thanks, everybody for tuning in and I hope you join us for the next one.

[OUTRO]

[0:37:41] Guy Podjarny: That's all we have time for today. If you'd like to come on as a guest on this show, or want us to cover a specific topic, find us on Twitter, @thesecuredev. To learn more about Heavybit, browse to heavybit.com. You can find this podcast and many other great ones, as well as over a hundred videos about building developer tooling companies, given by top experts in the field.