IoT DevOps For Continuous Integration

DevOps Plays A Key Role In IoT Security
June 15, 2017
Securing the Identity and Access Management of ‘Things’
June 29, 2017
Show all

This is the sixth in a series of Arrayent blog posts on IoT security. The series is written for employees of companies who sell connected products, especially those new to IoT. With this series, we hope to bring a basic level of awareness and understanding of key issues that face everyone who develops connected consumer products. We also hope to stimulate an ongoing dialog that helps move the conversation about connected products security forward.

A major impetus for manufacturers is wanting to have connected products that are ‘always on.’ This allows them to innovate faster and deliver product improvements, new functionality, and services to existing as well as new customers. However, this also requires structuring the manufacturer’s organization around rapid development and frequent product updates. It also means that applications and code revisions are updated on a daily or weekly basis, versus only sporadically. Plus, updates in the cloud infrastructure, and on the connected devices, must all happen at the same time—all without compromising product and service uptime or quality.

The industry calls it ‘continuous integration’ for good reason. With ‘continuous integration’ there is a blurring of lines, where infrastructure, configuration, and code all blur and—in essence—become the same thing. The implications for security are enormous. Whether a change is minor or major, in the complex IoT environment, continuous security testing becomes critical across the spectrum from development to deployment – in every aspect from checking code to pushing it out into the production environment, to validating updates on devices.

We believe there are two fundamental DevOps processes for IoT platforms that are critical for security:

1. REPEATABILTY OF DEPLOYMENT AND SETUP

Deployment is unarguably the most important component of DevOps. Does your IoT platform have a crisp and repeatable deployment process, and why is that important?

Ultimately, an IoT platform needs to be able to extract and replicate the configuration of an environment, whether it’s because of a problem (i.e. disaster recovery), the need to scale up and clone into another region, or to rebuild the environment. For whatever reason, is your platform prepared to go back and figure it all out. You want to have something that is always the same; it cannot be made up or be slightly different every time, or you will have security holes.

Here are two key questions about repeatability to ask when evaluating IoT platform operators:

  • Is manual intervention required for deployment and configuration?
  • Can they answer, “How was this configured last week?” and “How do you know?”

2. REVISON MANAGEMENT OF CONFIGURATIONS

Version control is the foundation of revision management, and can serve as a ‘single source of truth.’ While it’s obvious we should carefully manage revisions of source code, perhaps less obvious with IoT is the need to expand revision management to configuration, tools and infrastructure. If I’m looking at a bug that happened yesterday morning, I need to know how the system was configured and deployed. What infrastructure it was running at that time. And whether there is a historical log (journal) of the activity so that we could run forensics and analysis on the past. We also need revision management of configuration, tools, and infrastructure so that you can map them to show the whole picture.

Here are key questions to ask your platform team or provider:

  • How do they approach configuration management?
  • Is everything automatically scripted and stored in a standard version control repository?
  • Do they log actions?
  • Who can make changes? Can an intern deploy code the same as the chief architect? You need to manage identities with different roles and separate accounts to protect the architecture.
  • Is versioning built into the tools, environment configurations, and infrastructure?

By automating processes from end-to-end, errors will be minimized with less human intervention. As important, however, is ensuring the tools we design cannot do what humans aren’t supposed to do. You always want the same behavior. You need to understand that when you deploy an environment, it’s not automated only for efficiency, but for security. You want it to be done 100% the same way, every time. That’s how IoT platform operators can sleep at night.

LEARN MORE ABOUT CONTINUOUS INTEGRATION

Here are Seven Best Practices for Continuous Delivery Success excerpted from the DevOps education resource, Dev.Com.

  1. Version-Control Everything. Every change a developer makes must be documented in the source control repository or it must not included in the build process.
  2. Build Binaries Once. Every build process in a single organization must be completely consistent. Each build version should happen exactly the same way and result in unique binary artifacts.
  3. Deploy The Same Way Every Time. For a reliable process in all environments, the same set of steps must be repeated from start to finish.
  4. Smoke Test. Non-exhaustive software testing does not provide the same fine-grain comprehensiveness as full test suites; however, it can be run frequently and quickly. This allows for a much quicker turnaround on what can become very time-consuming and basic issues.
  5. Deploy Into A Production-Like Environment. Each new deployment should be made in an environment that mimics the actual final production environment as closely as possible. We use different cloud accounts for protection of real customer data for a staging account vs. an actual production account.
  6. Instant Propagation. Development teams merge developed code and get feedback from automated test tools many times a day, for a more efficient deployment and update process.
  7. Include the Database. The database is actually a repository of the most valued and irreplaceable asset—the data—and preserving it accurately is imperative to continuous delivery. 

On a final note, IoT platform teams should continuously evolve and improve their DevOps processes. Every time there is a problem, there should be a post-mortem where everyone learns how to improve. Because of the continuous nature of the IoT DevOps ‘beast’ two of the most important questions to ask IoT platform operators are: “How often has you DevOps process changed over the past year?” And, “What DevOps process have you improved in the last twelve months.” If the answer is zero, that’s a big red flag, for operational efficiency, and security.