InSpec Insights With Mandi Walls

[00:01:17] Guy Podjarny: Hello, everybody. Welcome back to the Secure Developer. Today we have another special edition version of the Secure Developer and we have with us Mandi Walls from Chef. Welcome to the show, Mandi.

[00:01:27] Mandi Walls: Hey, thank you.

[00:01:29] Guy Podjarny: Mandi, before we kind of dig in to the topics of the talks and all that, tell us a little bit about yourself. What is it that you do?

[00:01:33] Mandi Walls: Yeah. I am a developer advocate, technical community manager at Chef Software. I’ve been at Chef a number of years doing different things. Chef, if you’re not familiar with the company, we started out doing infrastructure security and automation and now we have products to do security automation and application automation and some other things too.

[00:01:53] Guy Podjarny: Yeah. Well, definitely one of the pioneers of sort of infrastructure as code. 

[00:01:56] Mandi Walls: Yeah.

[00:01:57] Guy Podjarny: And kind of modern, I guess, a release management, right?

[00:01:59] Mandi Walls: Absolutely. Yup.

[00:02:01] Guy Podjarny: Mandi, what is your talk about? You talk about InSpec?

[00:02:03] Mandi Walls: Yeah. We have a product called InSpec that we originally purchased a company a number of years ago and they had been working with some larger companies in Germany to do infrastructure security testing for like code, the infrastructure as code part. 

But the tool itself is pretty flexible so that my talk was mostly about using it in isolation to actually to dig into your hosts, your infrastructure hosts, whether they’re virtual machines or whatever they are and dig around in there and see what they’re doing. If they’re violating your security policies or any of these kinds of things. So, files that are owned correctly or packages that shouldn’t be installed or this kind of mundane, boring security things that like every sysadmin has to do and probably has like a set of shell scripts to do them, but InSpec tooling gives you something more robust to deal with. 

[00:02:56] Guy Podjarny: Got it. Is this something you’d like installed on the different machines? So, like you’d put the InSpec agent, I guess?  

[00:03:02] Mandi Walls: Yeah, you can put an InSpec agent on. You can run it inside your other configuration manager and software or you can run it remotely like as on-demand sort of scan. So, it’s pretty flexible that way.

[00:03:13] Guy Podjarny: This would work – In what surroundings would it work versus not. Is it like a more VM machines, or is it more containers?

[00:03:19] Mandi Walls: Yeah, you can run it against VMs. You can run it against software containers. It’s focused on Linux and Windows. There’s like stuff right off-the-shelf for most major Linux distributions as well as sort of the modern Windows releases.

[00:03:33] Guy Podjarny: Okay. Let’s kind of go up a little bit at the picture. You’ve gone along, you’ve put InSpec in all the different set of portions of your servers, right?

[00:03:41] Mandi Walls: Yeah. 

[00:03:42] Guy Podjarny: Or is that in a good enough portion to care? Then what happens? So, you have those agents and then how does it work? Does it funnel data or what does it –

[00:03:48] Mandi Walls: Yeah, you can funnel data places. It has exporters. You can run it in JSON and collect it yourself or you can hook it into some of our other products that allow you to sort of centrally manage it. It gives you a variety of ways to like if there’s a new vulnerability you can push out a new test and just sort of automatically collect information back off of your estate to say, “All right. Well, now I know this thing came out yesterday. Here is how I tested it. I know that 45% of my fleet is vulnerable to this,” and you can sort of schedule from there what you want to do to remediate that. It gives you a lot of faster ways of implementing a security check across stuff at that sort of level.

[00:04:25] Guy Podjarny: Got it. So, I have a whole bunch of tangents of like thoughts to take down with questions. Let’s try to sort of shoot then down one at a time. First of all, maybe let’s sort of back to maybe the value propositions. What types of questions? Now you have this sort of new data perspective at your fingertips, and you mentioned one example, which is you want to answer what vulnerable component or what component where might it be present. 

I guess can you give us maybe some other examples of what’s done there, and in general, do you see this as more kind of more inspection or more action or am I even asking the right question? 

[00:04:59] Mandi Walls: Yeah. We’re kind of like – It partners really well with configuration management software such that you can establish a baseline and then work towards fulfilling that baseline. Kind of like you can sort of take a test-driven development approach to it. Say, “Here’s my security requirements based on these tests,” and it functions the spec part of it. The name kind of gives it away a little bit. It’s based off of like spec testing from Ruby in particular, right? Something like RSpec, which gives you this sort of predictable behavior-driven sort of test model where I can say, “I want this package to not be installed or I want this service to not be running,” and you can sort of push that stuff all out across all of your platform. 

It gives you that sort of capability to say, “Here’s my baseline. Whether I am working against the CIS profiles or NSA profiles,” or any of these sort of recommendations that when you look at them, they’re massive pieces of information, like CIS for Rails, like 500 pages of PDF and like distilling that down into code that my security engineers understand it. It fulfills their requirements. I can put enough comments in it, do descriptions that my compliance people who may not necessarily be as technically savvy, like their job is more on the compliance side, they can figure out what’s going on there. Then my operations folks, my DevOps, my developers and other engineers can say, “All right, I can see what this is doing, what it’s testing against, so I know I can’t violate these particular sets of rules.” 

It gives sort of everybody one place to document and have their sort of requirements met in one set of tools and I can save them on GitHub and I can pass them around and people can consume them whenever they need them. In that way, it gives you sort of unified but a very flexible approach to the infrastructure layer by the way. 

[00:06:54] Guy Podjarny: Yeah. Kind of being able to just asking the code. So, you can ask at that scale. How does this work in containers? Containers, I guess two differences come to mind. One is just the short-lived nature of – Containers are these machines that don’t stick around for that long if you do it well. Yeah.

[00:07:11] Mandi Walls: You hope, right. 

[00:07:13] Guy Podjarny: Then the second is the definition area, like where are they defined? Often times it’s at the hands of developers, right? You edit some Docker file. You change it, as supposed to those ops people and also the DevOps teams that run it. How is InSpec used in that context?

[00:07:26] Mandi Walls: A couple of ways. You can actually attach directly to a running container and dig around in there the same way you would with a running file. You can also attach a – Is it Docker containers, or what the – Sidecars, to also run in your instance if you’re inspecting that container to be a little bit longer-lived, a same sort of thing. It’s like some of the tests can be a little bit more abstract. It’s not necessarily the lowest level stuff. You can also run tests against the configuration of a service. You can interrogate the http.com file or your nginx.com file, and those are things that they’re still going to be running – You can run them in your container. That’s perfectly acceptable. You still want to be able to interrogate those and make sure that they’re configured correctly even if they’re going to be in the container. 

[00:08:10] Guy Podjarny: Yes. 

[00:08:11] Mandi Walls: Yeah.

[00:08:11] Guy Podjarny: Okay. Got it. Fundamentally, this is one area where containers are very much VM-like.  

[00:08:18] Mandi Walls: Absolutely, for those sort of services.

[00:08:20] Guy Podjarny: Does it even make sense to think about this in the context of serverless?

[00:08:24] Mandi Walls: I’m not really sure that it does. I mean, somewhere somebody is running a platform underneath that. So, your provider you would hope would have some kind of –

[00:08:30] Guy Podjarny: Yeah. The underlying, not for the functions, but the –

[00:08:32] Mandi Walls: Exactly. On the substrate somewhere.

[00:08:34] Guy Podjarny: Yeah.

[00:08:35] Mandi Walls: Yeah.

[00:08:35] Guy Podjarny: Often times containers once again, right?

[00:08:36] Mandi Walls: Yeah, absolutely.

[00:08:37] Guy Podjarny: Yeah. But in that case, it might need to be so that the ops teams that are still running kind of that containers. 

The other thought that comes to mind is more about people. Who gets the alerts? If you have a noncompliant system, it doesn’t correspond to the spec, who is told about this?

[00:08:55] Mandi Walls: Yeah. I mean, that’s something that we kind of see people do all kinds of varieties of things depending on how they’re handling their reporting, whether it’s coming to one of our components or if they’re shipping it off to something else, like different subscriptions can happen. To something that highlights into the automate server. It will give you red-green. That’s our product that sort of pulls this stuff in. Anybody who has sort of access to that server can investigate that and see what’s going on. 

In other places we’ve seen folks do – It still goes to the operations people primarily as the first sort of report. Then sort of rolls back up through whoever requested the test or sort of responsible for it in the first place to see, because you can remediate something right away or you can say, “We accept this risk and we’re okay with this right now,” and you can sort of skip it so it’s still in the code but you’re not reporting on it anymore. You can say, “Well, we’re going to accept this for now,” and move on.

[00:09:50] Guy Podjarny: Do you think that over time this is something that would come into sort of more the realm of developers? I mean, as you think about more and more under the mantra really of DevOps, but also just in practice, more responsibility around operating your system falls to the developers. Do you think more of these alerts should come to the developers as the org matures?

[00:10:13] Mandi Walls: I think I would hope in an ideal maturation, right? A lot of these stuff I think would get caught during integration testing, because some of these stuff on the infrastructure, the things that you’re sort of violating are going to be like ports that shouldn’t opened or they’re doing the wrong thing or services that aren’t supposed to be functioning or are misconfigured or things like that. Because of the flexibility of InSpec as a tool itself, you can run it pretty often. It’s fairly quick. 

What we would hope to see is that it’s going to run in production just to make sure there’s no drift on configuration and things like that. But at the same time, we run it in integration so your developers can say, “Oh, this change I requested for this particular configuration violates the security policy.” They can see that before their stuff even gets to production. Some of it can be seen like almost as soon as you check-in the code depending on what they’re working on. Maybe they’ve changed the JMX setting or something weird that folks aren’t allowed to do. I mean, you can catch that right away. 

[00:11:10] Guy Podjarny: Got it. All of these is really independent of sort of Chef, the configuration manager so that they’re –

[00:11:14] Mandi Walls: Absolutely. Yeah, it runs independently. 

[00:11:16] Guy Podjarny: Yeah. I mean, clearly, that’s a good way to sort of address the remediation.

[00:11:20] Mandi Walls: Absolutely. 

[00:11:21] Guy Podjarny: [crosstalk 00:11:21]. Which is fun and on the patch system, “Can I trigger some automated patching?” But it’s not. It’s independent so that you can do it anyway. I believe that Netflix, like in one of their sort of Simian Army has this thing called a conformity monkey that basically goes around and finds –

[00:11:37] Mandi Walls: Drift, I think. Yeah. 

[00:11:39] Guy Podjarny: Yeah, Drift from Conformity. Is this along the same lines? Are the best practices from there, things that you could apply with InSpec as well?

[00:11:45] Mandi Walls: I would expect so. I mean, I think the key parts that we were sort of going after were the readability and the flexibility to sort of allow everyone who has a stake in your security process be a participant in the whole process. Part of the stuff that we see with infrastructure security is a lot of it is very boring and a lot of it is very repetitive. So that’s great stuff to automate. But your systems administrators on the frontline don’t necessarily know 100% of the coverage that they should be doing.

[00:12:16] Guy Podjarny: Yeah. Right. I mean, I think just knowing what’s around. It’s one of those problems where doing it on a single server is easy. Doing it on 10,000 of them –

[00:12:22] Mandi Walls: Put your laptop in production. Let’s do it. Yeah.

[00:12:24] Guy Podjarny: Yeah. Not an enticing proposition. I guess when you think about this from – When you cast your eyes to the future and you think about maybe more containerized or orchestrated surroundings, how do you see InSpec fitting into that future reality? Does it just work the same? There’s still an underlying machine and it conforms it? Should there be an InSpec at the app level is the OS is lower?

[00:12:55] Mandi Walls: I mean, I think we don’t have any current plans for that, because as far as like the actual underlying technology under InSpec, the stuff that’s under there is Ruby-based and there’s a lot of built-in matures. There’s already a lot of libraries there for looking at the components that you would normally investigate. Over time, it’s not on the realm of possibility that it would be able to help you checking for have you checked all of your inputs and certain functions? That additional absec sort of things, whether it would be the best tool for that based on what’s already out there, it’s hard to say. 

But for anything that has an actual running operating system on it, whether it’s your container or there’s enough stuff going on in the container to sort of approach that or looking forward into like smaller footprint for IoT and like all of those kinds of things, like there’s definitely a place there to continue to investigate these sort of infrastructure issues even when the infrastructure is very tiny from that sort of perspective.

[00:14:01] Guy Podjarny: This is sort of about the future. One of the things I like about sort of using InSpec in this context is that it’s the same tool that really grew up in the ops world, but we’re talking about a security utilization to it. I like to geek out or sort of – I always like it when you talk about the future of security in DevOps terms. Do you see in practice like when people write ASpec or when they run through it, is there a difference between a security need and an operational need? Is it the same person that generally gets it?

[00:14:34] Mandi Walls: Actually, what we see is a lot more cross-team collaboration on these sorts of things where – one of the speakers mentioned this morning that you often see teams where you have maybe 100 developers and 10 ops people and one person whose job is primarily security. That’s not unusual. We see that with customers all the time, right? Part of that becomes your ops folks have a certain realm that they’re super familiar with. If their [inaudible 00:15:00] for any given length of time, they know – They’re probably familiar with some of the CIS standards and they know sort of baseline, the things that they want to check when you instantiate a new system or a new container or whatever. You’re looking for specific things. 

Then the job of the dedicated sort of security person sometimes the things that fall to them are like vulnerabilities that come from your upstream providers. They’re maybe watching mailing lists or cert releases or things like that for specific vulnerabilities and they sort of apply their expertise in that fashion. The good part about an InSpec profile that you can sort of buildup overtime is that it can include all of those requirements and sort of like work for everyone in that sort of space. 

We see a lot more – What we wanted out of configuration management I think years ago when we first started doing these things was that engineers, application developers, operations people would all sort of collaborate on the configuration management software. That didn’t really happen in most places. There was never enough of an impetus for the application engineers to sort of participate in that. 

But we see a lot more security and operations folks sort of collaborating on these profiles to sort of get everything met. There’s so much and so much of it changes sort of all the time that you want to be up on as much as possible and bring in everybody’s expertise when you can.

[00:16:24] Guy Podjarny: It makes perfect sense. And I love that very much. I think a very repeated theme across sort of the episodes here as we’ve been sort of talking to people running modern security teams is this notion of reuse of the tech, but really reuse of the tech is just exemplifies the reuse of the practice and the knowledge and embedding security question and security practice into just of daily operational or development depending on the sort of security threats.

[00:16:49] Mandi Walls: Absolutely. You want to make it as easy as possible. If it’s going to be part of your build process to check your substrate before you deploy. It has to be superfast. You don’t want to be waiting on it. You don’t want to be blocking your builds, any of those kinds of things. Yeah.

[00:17:02] Guy Podjarny: Well, this sounds super useful for people to check out. I assume kind of like most stuff out of Chef, there is an open source path and there’s a commercial path if you prefer?

[00:17:10] Mandi Walls: This particular product in particular has a really lively open source community. They’re global. There are folks all over the world using this and working on it. In addition to our employees, there’re a number of folks in the open source world and they’re always sort of open for new –

[00:17:25] Guy Podjarny: Building more of those specs and evolving this. 

[00:17:26] Mandi Walls: Yeah. There’s a lot of stuff out there. 

[00:17:28] Guy Podjarny: Very cool. Well, Mandi, thanks a lot for coming in the show and sharing these perspectives and we’ll put links to the YouTube video of the talk. 

[00:17:34] Mandi Walls: Oh, excellent. Great.

[00:17:35] Guy Podjarny: Thanks everybody for tuning in and I hope you join us for the next one.

[END OF INTERVIEW]