Blog

What, How, and Where Open Source Gets Pulled into a Codebase

5 min.

December 7, 2021

The vast majority of software developers in the industry today are paid to solve business problems. Regardless of whether they work for small independent software vendors or Fortune 500 companies, solving such problems is now one of their primary responsibilities. Given the time and the opportunity, many software developers would write as much functionality into their applications as they possibly could from scratch. However, that can be very time consuming: first, they have to debug and fix it, and then, they have to maintain it (or better yet, enhance it).

Third-Party Extensions Are the Answer

To increase productivity and save a great deal of time, developers often use code written by third parties rather than rebuilding the same generic functionality across multiple applications. While there has always been a market for commercially licensed and supported extensions (including modules, packages, libraries, and frameworks), the vast majority of the third-party code used today is open source. This means that there is no marketplace and no purchase order; rather, a few extra lines of someone else’s code are simply imported into the software.

This can cause problems with licensing and disclosure if it is not accurately tracked and monitored. That is why software composition analysis (SCA) products are worth their weight in gold. These solutions find all of the third-party packages that are in use, then identify the corresponding licenses. They can even show if they are out of date or if known security vulnerabilities have been reported against them.

It’s All Based on Open Source

Even the lowest level of an application stack (the language and runtime engine) is often open source. The most popular languages in use these days are all open source, or at least have open source distributions. Go, Python, PHP, Ruby, and JavaScript are all open source by default, and even languages that are traditionally commercially supported have open source distributions like OpenJDK (for Java) and gcc (for C/C++).

After you’ve chosen your language, you’ll likely want to ensure that you have some structure in place so that you won’t need to declare all the basic functionality like dependency management and data management. Well over half of all Java applications use the Spring framework as their starting point. PHP uses Laravel, while JavaScript uses React and Bootstrap, among others.

Frameworks and languages form a solid foundation for any application, but the bulk of open source influence can be found in the staggering number of modules that are available as packages and libraries which can be easily integrated into applications.

How Easy Is It to Find Open Source Modules and Libraries?

Any web search for any type of functionality will often return results that link to places like GitHub, GitLab, PyPI, and many other sites. So how do you find what you need?

Let’s say that you want to make a particular form a little more secure by including a CAPTCHA. If you don’t know where to start, just head over to your favorite search engine and enter, “captcha library for Python.” In our case, the first result is an open source library that can be installed via pip (pip is the standard utility used in the Python ecosystem to install modules).

Installing this module is as simple as typing, “pip install captcha.”

Now, with just a couple lines of code, a whole new set of tested and proven functionality is added to the application in minutes.

from captcha.image import ImageCaptcha
image = ImageCaptcha(fonts=['/path/A.ttf', '/path/B.ttf'])
data = image.generate('1234')
image.write('1234', 'out.png')

For another, more real-world example, let’s say that you have a web application that needs to be able to pick a date from a calendar. To show you how to do this, we will use the jQuery library, which has a great deal of functionality and is easy to use.

The first step is to add the jQuery modules to the web page in question. There are two stylesheets and two script files that need to be imported. These are added between the head tags. The next step is to define the datapicker function, which activates the appropriate pieces of the jQuery library. The final step is to define where to put it on the page using an input field.

The code looks like this:

<!doctype html>
<html lang="en">
<head>
 <meta charset="utf-8">
 <meta name="viewport" content="width=device-width, initial-scale=1">
 <title>jQuery UI Datepicker - Default functionality</title>
 <link rel="stylesheet" href="//code.jquery.com/ui/1.12.1/themes/base/jquery-ui.css">
 <link rel="stylesheet" href="/resources/demos/style.css">
 <script src="https://code.jquery.com/jquery-1.12.4.js"></script>
 <script src="https://code.jquery.com/ui/1.12.1/jquery-ui.js"></script>
 <script>
 $( function() {
   $( "#datepicker" ).datepicker();
 } );
 </script>
</head>
<body>
<p>Date: <input type="text" id="datepicker"></p>
</body>
</html>

The finished web page looks like this:

When you select the date input, it will present a calendar. You can stylize it, of course, but this example shows the simplicity that open source libraries can provide:

Conclusion

Without intentional effort, you will not find a single modern microservice or web site that doesn’t have open source somewhere in the components that it relies on or ships. The question isn’t whether you can use open source, but whether it provides a full view. Different open source licenses have different restrictions around distribution. GPL requires the disclosure of all source code, whereas Apache and BSD licenses simply require proper copyright attribution. This can determine which open source libraries and modules you are able to include in a given application.

In any case, rather than just trusting the development team to document everything they do (and we all know how much developers love to document things), a better and more viable long-term solution would be to build a pipeline to catch all of the open source code early, before it could introduce known security vulnerabilities or licensing complications. Better yet, you could integrate SCA into your source code repository and let tools like Checkmarx Software Composition Analysis do the heavy lifting for you.

Vince Power is an Enterprise Architect with a focus on digital transformation built with cloud enabled technologies. He has extensive experience working with Agile development organizations delivering their applications and services using DevOps principles including security controls, identity management, and test automation. You can find @vincepower on Twitter.

Download our Ultimate Guide to SCA Here.

Table of Contents

  • Third-Party Extensions Are the Answer
  • It’s All Based on Open Source
  • How Easy Is It to Find Open Source Modules and Libraries?
  • Conclusion