Continuous Monitoring

Organizations find value in identifying performance deviations, availability issues, and functional correctness problems before they effect end users. While each aspect of a release is carefully crafted and tested, bugs and performance problems still slip between the cracks. Proactive monitoring is a way to identify the issues that were missed by running monitors in the background while teams continue to propel forward with new releases.

Verifying that deployed features didn’t break on their way into production is an important pre-release quality check. However, teams also need to ensure they can measure a feature’s performance and value over its entire useful life. Teams cannot stop checking for quality now—before a feature has been used. That would undermine the whole intent of the Continuous Delivery Pipeline, which is to test solution ideas (hypotheses) in a real business setting to make quick and informed judgments about how best to enable strategic business outcomes. The insights that drive this critical feedback loop come primarily from robust monitoring capabilities, which must be in place prior to release.

Effective monitoring requires that full-stack telemetry is active for all features deployed through the Continuous Delivery Pipeline. This ensures that system performance, end-user behavior, incidents and business value can be determined quickly and accurately in production. That information allows tracking and monitoring of each feature, which increases the fidelity of the assertions about business value delivered, as well as increased responsiveness to production issues.

While some business-value metrics cannot be collected until release, teams need to make sure they know how to measure them once the decision to release occurs. Three skills help support this: Full-stack telemetry – the ability to monitor for problems across the full stack that a system covers Visual displays – tools that display automated measurements Federated monitoring – consolidated monitoring across applications in the solution that creates a holistic view of problems and performance

• Full-stack telemetry – the ability to monitor for problems across the full stack that a system covers
• Visual displays – tools that display automated measurements
• Federated monitoring – consolidated monitoring across applications in the solution that creates a holistic view of problems and performance

Monitoring Practices

• Reuse Assets – Leverage design specifications or test scripts to create monitors for your applications and APIs. AlertSite supports reusing OpenAPI specification (formerly known as Swagger Specification) files, SoapUI test cases, Selenium scripts and more to create new monitors instantly. Not only does this save time, effort, and money, but reusing assets also ensures consistency and standardization across multiple departments in an organization.
• Monitor Earlier – Monitoring in pre-production provides a plethora of benefits. Get started monitoring in pre-production environments by running a monitor in your test environment. This monitor can run in conjunction with the test cases that are being tested in those environments, or monitor different user journeys that are an essential part of the application or APIs Success.
• Leverage REST APIs – Create monitors automatically using REST APIs. For example, AlertSite’s REST API empowers teams to create monitors automatically in the build process in conjunction with their code deployments. By shifting the monitor creation responsibilities left of production, it ensures that the monitors are running to capture the information that is most important for that application.

Monitor in Pre-Production Environments

By consistently monitoring applications in pre-production environments you gain a view of performance from an on-going perspective. Monitoring provides a heartbeat of how your apps are performing as you’re deploying new versions of your code in various environments. Catching issues earlier in the process empowers teams to quickly remedy the issue, and continue to test and monitor the subsequent changes. With real-time validation that your new versions are performing as expected, you can release new deployments confidently.

Monitoring in pre-production also offers the ability to ensure that any new updates or changes will not break the monitors in production. Monitoring continuity is essential for agile teams and DevOps initiatives because without it, the ebb and flow of iterative improvements is halted by a bottleneck monitor. Teams will have to back track to rectify the false alerts and blind spots caused by the broken monitor, wasting time and resources fixing an issue that could have been avoided earlier in the development lifecycle. Ensuring that test teams and operations teams are in alignment about what and how they are monitoring is vital to sustaining visibility into application health.

Additionally, by monitoring pre-production environments continuously, you glean insight into the state of your test environments themselves, so you can maintain the environment stability and continue to test applications unaffected by environmental problems.  Finally, monitoring will drive continuous traffic through your testing environments. Oftentimes pre-production environments have low traffic or sporadic traffic, and by constantly monitoring in your pre-production environments, you keep the environments primed for executing tests.

Continuous Monitoring and DevOps

DevOps refers to a culture and practice in which Development and Operations teams maintain an agile, collaborative relationship in order to benefit the software development lifecycle. The goal is to break down departmental barriers to shorten development cycles, increase deployment frequency, and enhance the success and reliability of each release. For teams to successfully embrace a DevOps culture organizations need to make it easy to –

• Collaborate – Monitoring earlier in the lifecycle provides teams with the ability to collaboratively work to optimize performance and availability. By reusing artifacts from design and test phases to create monitors that can run in pre-production and production environments, teams can streamline standardization and ensure everyone is on the same page.
• Automate – By nature, proactive monitoring is an automated process. In fact, the more you do not interact with your monitoring tool the better – this means that your applications are up and running as expected, and that the monitoring tool itself is functioning properly. From a team perspective, monitoring earlier and more often helps reduce the amount of manual work dedicated to ensure that the application is running properly.
• Configure – Proactive monitoring tools should adapt to you. A good proactive monitoring tool will provide configurability measures for monitor creation and alert set up, so that you receive data that matters to you when you most need it. Whether that’s setting up a blackout period, changing a recipient group, or monitoring from a specific location, a monitoring tool helps make your processes tailored and your MTTR more efficient.
• Share – A keystone factor of collaboration is the ability to share information. Monitoring can help provide teams with a common data set that can be shared across different departments to improve your application. Alignment is a foundational element of DevOps culture, and sharing data is the lifeblood of this element.

Monitoring Your Applications

Monitoring applications is a key factor in identifying issues (performance, regression, or otherwise) and fixing them quickly as part of a team’s iteration. Three of the more popular application performance monitoring (APM) tools are New Relic, Dynatrace, and AppDynamics. Apart from letting teams monitor and manage their software, they also allow for end-user monitoring, which is crucial to ensuring an application is delivering the best experience.

Monitoring Your Infrastructure

The other side of the coin to application performance monitoring is infrastructure monitoring. There are many, many tools for this, including SaaS solutions such as DataDog, LogicMonitor, and SignalFx, as well as hybrid solutions such as Zenoss. Though it’s a favorite of teams transitioning from legacy development processes, Nagios alone isn’t enough for your DevOps monitoring strategy.

Monitoring Your Logs

Analyzing logs is a crucial part of the iterative improvement cycle and is a critical part of system troubleshooting, debugging, and security incident response. Popular log analysis tools include Splunk, LogEntries, and Loggly. They allow you to study user behavior based on log file analysis. They also let you collect a fast amount of data from various sources in one centralized log stream. This is a convenient way to maintain and analyze log files. Apart from debugging, Log analysis plays a key role in helping you comply with security policies, and regulations and is vital in the process of auditing and inspection.