DevOps (Development & Operations)

DevOps isn't a tangible product. "You can't buy DevOps in a box," explains Kevin Behr, author and Chief Science Officer at Praxis Flow. The culture that can foster the DevOps mindset has to exist throughout a company, not just within a single department, so people can collectively contribute all their skills to solving problems. Otherwise, DevOps can become just a local optimization.

DevOps is all about:

1. culture
2. automation
3. monitoring metrics
4. sharing

However, DevOps is NOT solving a technical problem, it's trying to solve a business problem and bringing better value to the end user at a more sustainable pace.

For the DevOps' best practices, we may want to check the Accelerate book and the State of DevOps report. This includes practices such as using Continuous Integration and Continuous Delivery, using trunk-based development, and using the cloud well.

These days, using the cloud well means using a container management platform such as Kubernetes, which automates the running and load balancing of containers on any cloud infrastructure. It also means scaling the nodes and containers elastically, all of which can be managed by your cloud provider.

Picture from Using Kubernetes, Spinnaker and Istio to Manage a Multi-cloud Environment

Here is a DevOps snapshot (deployment stats for production hosts & environments) for a month at amazon (data from Keynote: Why We Need DevOps):

1. 11.6 seconds : Mean time between deployments (weekday)
2. 1,079 : Max # of deployments in a single hour
3. 10,000 : Mean # of hosts simultaneously receiving a deployment
4. 30,000 : Max # of hosts simultaneously receiving a deployment

Although devops is not about tooling, there are a number of open source tools out there that will be able to help you achieve your goals. Some of those tools will also enable better communication between your development and operations teams.

Most successful DevOps organizations automate using tools in a few core categories, using a variety of specific tools (DevOps Best Practices: Finding the Right Tools):

1. Provisioning/Configuration management.
   

   When DevOps aficionados throw around phrases like automated infrastructure, infrastructure as code, and programmable infrastructure, they're talking about provision or configuration management. That's the tracking and controlling of changes to the software code base and the archiving of all file versions into a central configuration management database (CMDB), which enables multiple developers to work on the same code base while avoiding version-control issues.

   Provisioning tools include:

   1. Terraform - cloud agnostic
   2. AWS Cloudformation AWS only
   3. Pulumi - similar to Terraform but we can use Python, Go etc. instead of HCL DSL.
   
   

   Configuration management tools include:

   1. Chef - Ruby/Erlang, Knife, recipe, cookbook
   2. Puppet - Ruby, Puppet manifests, Factor, catalog
   3. Ansible - Python, Playbook, inventory
   4. CFEngine
   5. SaltStack - Python, YAML, Jinja2, salt master / salt minion
   6. Fabric - Python




2. Application deployment.
   

   Application deployment tools enable the automation of releases, and are at the heart of continuous delivery, one of the primary tenets of DevOps.

   For continuous integration and continuous deployment, we need a number of tools to help us there. We need to be able to build reproducible artifacts which we can test. And we need a reproducible infrastructure which we can manage in a fast and sane way. To do that we need a Continuous Integration framework like Jenkins:

   1. Capistrano, a deployment library, is the most popular standalone tool in this category.
   2. Other popular tools for automating application deployment include Ansible
   3. Fabric,
   4. and Jenkins.

   Again, the key is to find a tool that tracks behavior-from history to change logs-in ways that is meaningful for both dev and ops.

3. Monitoring.
   

   DevOps requires two distinct types of monitoring. Application performance monitoring tools:

   1. nagios
   2. zabbix
   3. icinga
   4. New Relic APM enable code-level identification and remediation of performance issues and at the infrastructure level, server monitoring tools like New Relic Server provide visibility into capacity, memory, and CPU consumption so reliability engineers can fix issues as soon as they appear.
   5. pingdom
   6. appdynamics
   7. datadog - though listed here near the bottom, datadog is one of the widely used monitoring tools that provides good integrations with other services: host integration (aws, github), local endpoint integration(postgres, nginx, docker), notification integration (pagerduty, webhooks, slack), and framworks, libraries (Go, Python, Ruby), APIs and Trace integration.
   8. splunk - I haven't used splunk but it could be listed at the top probably along with datadog.
   9. sumologic - It has very good query capabilities and provides lots of PCI compliant templates. I could list sumologic as top 5 monitoring tools.
   
   

   The key is to make sure that everyone can see the data so they can make better decisions.




4. Version control.
   

   To achieve the benefits of DevOps, it's essential to version not just our application code but our infrastructure, configurations, and databases. This requires scripting all of our source artifacts, but the payoff should be a single source of truth for both our application code and our IT systems and databases, allowing us to quickly identify where things went wrong, and recreate known states with the push of a button. No more having to play Sherlock Holmes to figure out which versions of our application code goes with which environments or databases. While commonly used version-control tools include

   1. Git
   2. Perforce
   3. and Subversion

   they differ widely on how well they support DevOps-style collaboration.

5. Test and build systems.
   

   These tools automate common developer tasks including compiling source code into binary code, creating executables, running tests, and creating documentation. Tools in this category include

   1. Ant
   2. Jenkins
   3. Maven
   4. Gradle

6. Storing Metrics
   

   Graphite is one of the most popular tools to store metrics. Plenty of other tools in the same area tried to go where Graphite is going, but both on flexibility, scalability and ease of use, not many tools allow developers and operations people to build dashboards for any metric they can think of in a matter of seconds.

   A timestamp and a value to the Graphite platform provides us with a large choice of actions that can be done with that metric. We can graph it, transform it, or even set an alert on it. Graphite takes out the complexity of similar tools together with an easy to use API for developers so they can integrate their own self service metrics into dashboards to be used by everyone.

7. Logstash
   

   Initially the logstash was just a tool to aggregate, index and search the log files of our platform, it is sometimes a huge missed source of relevant information about how our applications behave. Logstash and it's Kibana+ElasticSearch ecosystem are now quickly evolving into a real time analytics platform. Implementing the Collect, Ship+Transform, Store and Display pattern we see emerge a lot in the #monitoringlove community. Logstash now allows us to turn boring old logfiles that people only started searching upon failure into valuable information that is being used by product owners and business manager to learn from on the behavior of their users.

8. Elasticsearch
   

   Elasticsearch is a distributed and scalable search engine based on Lucene (free open source information retrieval software library). It provides a distributed, multitenant-capable full-text search engine with a RESTful web interface and schema-free JSON documents. Elasticsearch is developed in Java and is released as open source.

   Elastic search hides Lucene complexity by exposing all services. It's a search engine! Not a serach tool.

9. Redis
   

   Redis is a in-memory key-structure database. Redis maps to five data structures: Strings, Lists, Sets, Hashes, Sorted Sets (Zsets). It also supports publish/subscribe, server-side scripting, and replication.

   It keeps data persistency via snapshot or append-only file. HA and automated failover via "Redis sentinel" daemons.

10. Mesos - Data center Orchestration
    

    Image source: Apache Mesos

    Mesos is built using the same principles as the Linux kernel, only at a different level of abstraction. The Mesos kernel runs on every machine and provides applications (e.g., Hadoop, Spark, Kafka, Elastic Search) with API's for resource management and scheduling across entire datacenter and cloud environments.

    

    Image source: Mesos Orchestrates a Data Center Like One Big Computer

11. Kubernetes CI/CD
    
    1. Spinnaker:
       Originally developed at Netflix, Kubernetes-native CD (cloud driver plugin) for Kubernetes it allows for the direct monitoring and management of ReplicaSets and Pods in our destination cluster.
       Spinnaker uses third party build tools like Jenkins for CI. These plugins with the component-based architecture of Spinnaker and its fine-grained configuration options make it extremely versatile, but also very complex to operate.
       
    2. ArgoCD
       Developed at Intuit, Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes.
       It takes the GitOps model that features a controller that constantly reconciles the contents of a git repository with resources running in the cluster.
       Argo CD follows the GitOps pattern of using Git repositories as the source of truth for defining the desired application state. However, ArgoCD is not CI tool and there is no test or build step when syncing.
       
    
    
    
    3. Flux
       Flux - the GitOps family of projects

    4. Jenkins-X:
       

       Image source: Jenkins X - Accelerate

    5. Helm
       Helm helps users to have a more consistent deployment by packaging up all of the needed resources needed for a Kubernetes deployment.
       Helm uses a packaging format called Charts. A Helm Chart is a collection of files that describe a set of Kubernetes resources.
       Like the image-to-container relationship, a running instance of a Chart is called a Release. In the end, a Helm Chart is an executed template converting Chart definitions to Kubernetes manifests.
    6. Harness:
       The Harness software delivery platform has Continuous Delivery, Continuous Integration, and Cloud Cost Management modules, allowing us to build, test, deploy, and verify on-demand.

Though the Kubernetes getting popular out of orchestration tools (Kube, officially won the battle as of 2018: almost every cloud vender provides Kubernetes), we may still want to look other orchestration tools: CONTAINER ORCHESTRATION : DOCKER SWARM VS KUBERNETES VS APACHE MESOS

The right tool chain for DevOps will automate IT services, provide real-time visibility into system and application performance, and give us a single source of truth. More important than an individual tool's capabilities, though, is how closely the all match our organization's strategic goals. That's the way to maximize our chances of achieving DevOps goodness.

Of course, tools are only part of the DevOps equation. We also need to create a culture that gets dev and ops working together towards the same goals.

1. Chef: Chef is an automation platform that transforms infrastructure into code.

   This is a configuration management software. Most of them use the same paradigm: they allow you to define the state you want a machine to be, with regards to configuration files, software installed, users, groups and many other resource types. Most of them also provide functionality to push changes onto specific machines, a process usually called orchestration.

2. Vagrant: Create and configure lightweight, reproducible, and portable development environments.

   It provides a reproducible way to generate fully virtualized machines using either Oracle's VirtualBox or VMWare technology as providers. Vagrant can coordinate with a configuration management software to continue the process of installation where the operating system's installer finishes. This is known as provisioning.

3. Docker: An open source project to pack, ship and run any application as a lightweight container

   The functionality of this software somewhat overlaps with that of Vagrant, in which it provides the means to define operating systems installations, but greatly differs in the technology used for this purpose. Docker uses Linux containers, which are not virtual machines per se, but isolated processes running in isolated filesystems. Docker can also use a configuration management system to provision the containers.

4. OpenStack: Open source software for building private and public clouds.

   

   Picture source: OpenStack 101 - What Is OpenStack?

   While it is true that OpenStack can be deployed on a single machine, such deployment is purely for proof-of-concept, probably not very functional due to resource constraints.

   The primary target for OpenStack installations are bare metal multi-node environments, where the different components can be used in dedicated hardware to achieve better results.

   A key functionality of OpenStack is its support for many virtualization technologies, from fully virtualized (VirtualBox, VMWare), to paravirtualized (KVM/Qemu) and also containers (LXC) and even User Mode Linux (UML).

To present these products as components of an specific architecture, we should define our needs with regards to the environment we need (Chef, Puppet, Ansible, ...), then be able to deploy it in a controlled fashion (Vagrant, Docker, ...) and finally scale it to global size if needs be.

How much of all this functionality we need should be defined in the scope of our project.

- From Relationship between Vagrant, Docker, Chef and OpenStack (or similar products)?.

1. What is DevOps and where did it come from?
2. How does DevOps differ from Agile?
3. How does DevOps differ from ITIL or ITSM?
4. How does DevOps fit with VisibleOps?
5. What are the underpinning principles of DevOps?
6. What are the areas of DevOps patterns?
7. What is the value of DevOps?
8. How does Infosec and QA integrate into a DevOps work stream?
9. My DevOps Favorite Pattern #1
10. My DevOps Favorite Pattern #2
11. My DevOps Favorite Pattern #3

Top 11 Things You Need To Know About DevOps
- from Thttp://itrevolution.com/pdf/Top11ThingsToKnowAboutDevOps.pdf

Picture source : wiki

1. Iaas : Amazon Web Services (AWS), Microsoft Azure, Google Compute Engine (GCE)
2. Paas : AWS Elastic Beanstalk, Cloud Foundry, Google App Engine, Heroku, OpenShift
3. Saas : Salesforce, Workday, New Relic, Microsoft Office 365

The distinction between the two is cloudy and there are some debates on it. But here is the difference by some of the DevOps:

Continuous Delivery => Software can be deployed to customers at any time with the "push of a button" (i.e. by running a deployment script).

Continuous Deployment => Software is automatically deployed to customers once it passes through the continuous integration system.

What is the difference between continuous delivery and continuous deployment?