Understanding the developer attitude to security

They do it because they should

The best approach for getting development teams to take on security is to make it an expected part of their role and goals. This gives development teams a mandate and a means to prioritize security. With this model, clear ownership is important so that teams aren’t assuming another team has taken the responsibility for the security of a shared or legacy project.

Having people know what they should do isn’t enough. It’s just as important to have processes in place that help teams meet their security goals. For example, visibility into testing results, or an explanation of the additional levels of exposure and risk that exist after a developer has written a new feature. The visibility and transparency benefits of a scorecard can help provide the accountability that individual developers need to ensure it gets done as part of the workflow. This technique also works particularly well with outsourced teams that tend to float across projects and have less ownership over (or pride in) a project.

They do it because they should

It’s such a good reason, we gave it twice! There’s another driver behind why a developer should develop securely, and that’s because it’s the right thing to do. A proportion of your developers care about security or take great personal or professional pride and responsibility in coding in a sustainable and secure manner.

Make the right path the easy path (a.k.a. paved road)

Every obstacle that stands in the way of a developer performing a security task has the potential to stop them from completing that task. These obstacles can range from a lack of understanding to a painfully slow workflow to being overwhelmed with issues and not knowing where to start. Making security an easy add-on to existing workflows is key to the adoption of security practices.

Encouraging the use of standard practices via a paved road approach is a great way to provide helpful support and proven best practices. Additionally, there is a greater opportunity to create accurate and up-to-date documentation and advice that can be reused and updated by others regularly.

"We call that concept a paved road. You could certainly bushwhack and make your way through the woods. But if you have this nice smooth paved road that gets you to your destination, you're likely to opt in there."

Jason Chan, VP Security at Netflix on The Secure Developer talking about developer empathy

Think about what the developers need in order to make progress. Scanning, testing, and identifying issues is a relatively easy step in comparison to fixing. A developer doesn’t want to know about the ten vulnerabilities in their dependency graph — they want to know that if they perform a minor upgrade to one dependency, they will fix their issues that are blocking their release. So think about solutions rather than problems, as the list is usually a lot smaller!

Also be very conscious of where developers want to spend their time, and how they want to consume security results and actions. Anything outside of their usual workflow is a distraction that they’ll need to remember to do. Integrations into existing workflows, which have been implemented by people who understand the existing processes and technology underneath, can help create a great developer experience. Be intentional about making results useful, feedback clear and actionable, and keeping the workflow fast and painless. In short, implement with empathy for your developers.

Automate, automate, automate

The strongest DevOps pipelines are well-automated. In a recent study, Snyk found that automation also had a strong correlation with successful shift left security programs. The data showed that organizations with fully automated deployment pipelines are twice as likely to adopt static applications security testing (SAST) and software composition analysis (SCA) tooling into their SDLC. Additionally, those with full automation were over four times more likely to fix security issues in a day, and over twice as likely to fix them within a week.

It’s clear that the more we automate, the more tests we can run, and the more issues we can catch. And with careful consideration, we can add security guardrails and policies on our pipelines to give us visibility and awareness of issues that need our attention and action. This provides a nice dynamic, as it’s very familiar to a developer that already runs automations for things like integration testing.

Additionally, automating security also reduces the friction between the security team and the development team, because it’s a process — not a team or an individual — saying you need to fix an issue. This allows the security team to support the development team and help with issues that the development team needs expertise with, rather than being a bottleneck or the bringer of bad news.

"I would say the thing that they need to do is automate all of the security tools (that make sense) into their DevOps pipeline. That’s something that I wish we had done earlier than we are doing it and have done it. Being able to, in an automated way, let a developer know as early as possible about a problem in their code is the thing that you have to do. If you could even do it at the code check-in point, so that they understand it then, or even with a tool as they’re writing their code that flags something as a problem in their IDE immediately. Getting that automation in place is critical. Trying to fix an issue just before a product goes out the door is so much harder."

Ryan Ware, Security Architect and director of the Intel Products Assurance and Security Tools team

Education and secure development knowledge

Education can be a hit-or-miss activity among development teams, particularly when there are so many irrelevant training courses and materials that teams are often mandated to take (and spend time taking a short exam on afterwards), which all likely ends up as a tick in a checkbox on an education compliance checklist.

Making education a relevant part of a developer’s day, such that it helps them do their job, fix a bug, or develop more securely than they would have, increases the speed of security adoption. The impact of the course or education module needs to be of value to the developer, make them more secure, and be felt quickly.

When thinking about education, consider how it helps your teams focus on secure development. Think about the actions they will take away and actually apply to their processes or workflows. Think about how you can be targeted with your education to provide relevant assistance to them at the point in time when they need it most. Consider how your education goes beyond technology and also focuses on process and culture.

Here are some tips to get the most out of your security education:

• Training should be engaging enough that they want to do it. An easy way to boost the quality of your education is to run pilot programs, gather feedback, and make adjustments before rolling out to the masses.

• Recognize that development teams could have significant differences in their attack vectors, and provide relevant lessons accordingly. For example, directory traversal attacks may apply more to your backend developers.

• When training around vulnerabilities, ensure you train on those vulnerabilities that affect the team (based on applicable language, framework, libraries, etc.). Stories about incidents that affected other companies using similar ecosystems could help show the potential for disaster.

• While it’s good to show real world examples of security issues, don’t use fear as a main motivator. It might help you get movement in the short term, but it’s not a long-term strategy. The exception being security issues that are truly scary — like ones that could devastate a business!

• Add education into scorecards to hold teams accountable to completing their education.

• Make sure your education focuses on how they can prevent tangible attacks. If you’re explaining a vulnerability, show the vulnerability as exploitable. If you’re explaining how to harden part of a pipeline, explain the attack surface that’s being mitigated. Make it real and tangible.

• Ensure that documentation exists for any tasks that you want developers to carry out. Make sure developers validate the effectiveness of the docs, or better, write it themselves for the next developer or team. Remember, documentation isn’t a replacement for guardrails, but you should also document your guardrails.

• If a red team finds an incident, use it as an opportunity to create relevant and realistic education. Providing deep dive information into the incident brings real risk to life. Bring the red team in for a Q&A session if possible.

Why do developers avoid security?

We’ve discussed why developers do take the time to build secure software. But it’s equally important, if not more so, to understand why they push back and avoid taking additional steps to ensure they deliver secure code.

Friction

Depending on the stage of security adoption in your teams, and their maturity in modern development, the barriers that teams identify can vary. For teams that are early in their security onboarding, we see pushback when the reasons and value behind process changes aren’t communicated clearly. The new process needs to be non-invasive and empathetic to the development team and their workflows. To drive adoption, over communicate, keep friction to a minimum by integrating with existing processes, and where possible, consolidate tools to reduce complexity.

"I think one of the biggest challenges we’ve had is getting key stakeholders in the tech org and developers to really appreciate the importance and value of security. In an ideal world, there’d already be that understanding. A developer isn’t really going to have much motivation to do something if they don’t see the value in it. That’s the same for a product owner or a dev manager or even a software engineering director. f they don’t empathize with the value of security, they don’t really have any impetus to prioritize [over] their feature work."

Nicholas Vinson, DevSecOps Lead at Pearson

Mistrust

Providing value and maintaining or gaining trust is needed for a long-term, sustained process that will be followed by teams. When they lose trust in the data, tooling, or process, they will consider it a blocker that they will try to work around. For example, persistent false positives from a tool or process will not only dishearten or annoy the developer, but they will be more likely to ignore or write off the results of real vulnerabilities or issues than with data that they believe in.

Complexity

Similarly, when tools add complexity to existing problems and don’t help to find or offer a solution, they just create more work for developers. Knowing a vulnerability exists is just the start of the problem. The developer still needs to identify all the ways and places that this vulnerability is accessible in the app (for example, the various dependency graph paths). Developers need to know instantly whether it can be fixed or not, and if it can, what the remediation process is. Developers want intuitive and instructive tooling so they can implement solutions quickly.

Ownership

Another reason why work might fall off one person’s plate and not land on someone else’s is ownership. With newer projects or features, the ownership might be clear, in that nobody else touches the code with the issue other than that one team. However, what if a part of the application was a common service that many teams used and contributed to, but nobody wholly owns? Who would pick up a bug or a security issue that doesn’t affect them directly? Another common example would be a container image used by many different teams. When every team is scrambling to deliver their own team’s feature work, shared common code is often a place where these issues aren’t claimed.

Time

Of course, even where ownership is clear, there’s still the challenge of finding time to include it into the workflow. Even if the automation is there to test your code, that’s just the easy bit. The real work is in the developer consuming the results and performing the fix. The more that the tooling gives bad results or isn’t prescriptive or autonomous with the remediation, the opportunity to drop it increases. Similarly, if the organization doesn’t prioritize security fixes and activities, the developer will push back or ignore security teams asking them to test or fix.

Accountability

Accountability is a core pillar of why a developer should do security work, but who they are accountable to is equally important. For example, a developer would likely feel it more compelling to do work if their team or a security champion (developer) in their team or their management chain is holding them accountable to doing the work. Whereas. if it’s just the security team holding them accountable, with no clear recourse, they’ll feel less pressure to be secure.

Lack of role models

Humans model their behavior after those around them. If we don’t see others developing securely, or notice other teams prioritizing security into their projects, it’s easy to conform by ignoring security tasks, similarly. There are a couple of good ways to counteract this, including adding security visibility into code reviews so that security questions are being asked that require some thought or attention to the topic. Another way is to make role models!

Celebrate individuals for behavior that you want to see more of from the organization. Highlight where teams have excelled or made forward steps in integrating security into their pipeline or reducing their security backlog. Make sure the celebrations of security achievements are visible, done by the engineering leadership teams, and repeatable by others through documentation, automation, etc.