How the Air Force and SSA Navigate the (Sometimes Bumpy) Flight to DevSecOps

Rapid software development allows government agencies and military organizations to keep pace with innovation while effectively accomplishing their missions and delivering services to constituents. In order to stay ahead of adversaries and remain secure amidst an increasingly-sophisticated cyber threat landscape, agencies need to deploy application software updates at record speed by streamlining application security solutions within development cycles. Organizations are expediting the software development process by employing new development methodologies and tools that embrace:

• microservices for smaller yet faster, easier, and more targeted updates.
• advanced development tools like containers.
• platforms that make deployments smoother and more seamless.

Learn more about how Checkmarx Delivers Containerized AppSec Solution to DoD’s Platform One to Secure DevOps Initiatives. However, the necessary switch to new approaches like Agile Development and DevSecOps for accelerated ATO has proven difficult for many of today’s government agencies and military organizations.

Addressing the Need for Security in Speed and Functionality

During a webinar sponsored by Checkmarx and hosted by the cybersecurity thinktank, Institute for Critical Infrastructure Technology (ICIT), entitled “DevSecOps: Analyzing Legacy Apps for Agile Development,” presenters outlined the challenges that organization face when trying to embrace Agile Development and DevSecOps. The Webinar, moderated by Nick Sinai, former Obama Administration U.S. Deputy CTO, explored the common deep-seated, underlying organizational issues that keep government and military agencies from employing Agile Development and DevSecOps best practices in the modernization of legacy IT systems. Joining Nick were Kendra Charbonneau, a lead engineer and enterprise agile transformation coach at U.S. Air Force Business Enterprise Systems, and Rajive Mathur, the former CIO at the Social Security Administration (SSA). Both Mathur and Charbonneau made it clear that should agencies fail to embrace Agile Development and DevSecOps, there’s a lot at stake: “Agile is important to responding to the need of the warfighter more quickly,” Charbonneau said. “Back when we were developing software with a waterfall methodology, it could take two to three years to get that functionality out to the end-user.” The speed of DevOps is essential for the Air Force. All DoD organizations and branches of the Armed Forces are under pressure to stay ahead of the adversaries’ development and deployment. Read more about how agencies can take advantage of DevSecOps and automation to accelerate ATOs. However, accelerating software updates isn’t always about functionality. It’s often a matter of security. Vulnerabilities in enterprise applications and software remain among the most exploited attack surfaces to gain entry to networks. Identifying and fixing those vulnerabilities are increasingly essential for today’s government agencies, as Charbonneau explained: “…cyberattacks are at an all-time high. They’re happening rapidly. We have to be positioned to change quickly and address [those vulnerabilities]. If we’re to continue doing business the way we’ve been doing business in regards to application development, then we’re going to have a hard time adjusting and addressing these cyberattacks.” For the SSA, the impetus to innovate quickly wasn’t directly tied to cybersecurity requirements or keeping pace with adversaries. Instead, it was about helping a 60,000 employee-strong agency provide better services to American citizens at every stage of their lives. “For us, it was all about service,” Mathur said. “How do you deliver more service, better service, faster - and not necessarily just through phones or field offices, but through any way possible?” While there was a strong demand in the SSA to embrace Agile Development and DevSecOps, shifting to these software development approaches wasn’t easy and was sometimes met with developers’ resistance. “…there were so many resources [within the Air Force] that simply didn’t understand what it meant to go faster with Agile,” said Charbonneau. “It seemed silly to a lot of them, honestly, because they had only known waterfall development. That’s what they had done for years…”

Turbulence on the Flight to Agile Development

Charbonneau painted a daunting portrait of the application development environment that she inherited at U.S. Air Force Business Enterprise Systems in the panel discussion. A decade before her arrival, her organization was mandated to embrace Agile Development and DevSecOps best practices, yet, upon her arrival, her organizational audit revealed an underwhelming result: “My findings were significant….of the 90 programs that had been assessed - we had 23 percent in the Infancy [category], which means they haven’t even begun their Agile journey for one reason or another. There was 46 percent in the Fall [category]. That means that they had just started [embracing] Agile and were starting to implement the Agile Development terminology and principles…The Walk [category] had 21 percent. That means they had established a disciplined approach to Agile, were looking at different metrics, and were starting to think about the automation of the application development process. And then we had 11 percent in the Run [category] and zero in the Fly [category].” In order words, 69% of application development teams had failed to make any progress towards embracing Agile Development and DevSecOps or having just started their journey. But why? 

Six Hurdles for Agile Development

Charbonneau found that six distinct factors determine a team’s inability to move towards an Agile Development culture. These included:

1. Technical debt: stuck with outdated legacy systems, many of which remained mission-critical and shared resources across multiple applications.
2. Product owner involvement: having a product owner that was simply a “bill payer” and not in open communication and collaboration with the end-user or familiar with end-user requirements.
3. Contracts: having legacy contracts built around a waterfall approach to application development.
4. Training: not having the right training, appropriate training, or failing to apply training quickly enough.
5. Tooling: having existing tools suited to a waterfall approach to development and not for Agile Development.
6. Resources: considerable constraint in funding, environment, skillsets, and other resources.

Unfortunately, government readers may find themselves facing some – if not all – of these six challenges, which boil down to people, processes, and technology, within their organizations. Luckily, both Mathur and Charbonneau offered advice for agencies making the transition to Agile Development and DevSecOps.

Agile Program Management Best Practices for AppSec

Both Mathur, who spearheaded a successful shift towards Agile Development at SSA starting in 2014, and Charbonneau identify the need for portfolio analysis, organizational buy-in, and transparency. These three aspects of a successful transition to Agile methodologies and DevSecOps maturity were particularly essential to the SSA. For the Air Force, assessing their current organization status and challenges for adoption was vital in identifying the steps, tooling, and training necessary to make swift progress. “[Developers] wanted to do what…senior leaders in our government were asking. However, they just hadn’t been armed with the proper resources, money, tooling, etc., to do it,” said Charbonneau. “But…we know what we’ve got now, and we know what [senior leadership] has to do at this point - start having those hard discussions and finding budget dollars to make it happen.” Building a coalition of stakeholders invested in making the change to Agile development is as important as knowing the current status of an organization’s journey and an actionable plan to complete the transition. “Regarding consensus across the agency and across business lines - there’s no doubt that everyone’s heart is in the right place. There’s no doubt that everyone wants to do the right thing. Recognizing that and finding common ground has to be point number one,” Mathur explained. “I had great partners at the SSA and great partners in IT who helped in that regard. You can’t do it alone. You can’t do it unless you have that sort of partnership.” Watch the full webinar recording here. >> _{Article originally published on GDSOH.}