Docker & Kubernetes : Istio (service mesh) sidecar proxy on GCP Kubernetes - 2020

bogotobogo.com site search:

Introduction

In this post, we'll add Istio support to services by deploying a special sidecar proxy to each of our application's Pods.

The sidecar proxy for each application has all the non-business logic. So, it lets developers can focus on application/service itself not worring about security or monitoring etc.

Istio is a Service Mesh that manages communications between microservices.

Picture source: Using Kubernetes, Spinnaker and Istio to Manage a Multi-cloud Environment

The proxy intercepts all network communication between microservices and is configured and managed using Istio's control plane functionality. Sometimes, it is called a Service-Side Load Balancer, and it allows us to skip extra hops that required when we use other types of LB.

Picture source: Istio : Integrating Virtual Machines

Picture source: Introduction to Service Management with Istio Service Mesh (Cloud Next '18) where Pilot: Control plane to configure and push service communication policies. Mixer: Policy enforcement with a flexible plugin model for providers for a policy. Citadel: Service-to-Service auth[n,z] using mutual TLS, with built-in identity and credential management.

We'll learn how to install and configure Istio on Kubernetes Engine, deploy an Istio-enabled multi-service application, and dynamically change request routing.

Istio is an open source framework for connecting, securing, and managing microservices. It lets us create a network of deployed services with load balancing, service-to-service authentication, monitoring without requiring any changes in service code.

Setup GKE cluster

Google Cloud Shell is loaded with development tools and it offers a persistent 5GB home directory and runs on the Google Cloud. Google Cloud Shell provides command-line access to our GCP resources. We can activate the shell: in GCP console, on the top right toolbar, click the Open Cloud Shell button:

In the dialog box that opens, click "START CLOUD SHELL".

gcloud is the command-line tool for Google Cloud Platform. It comes pre-installed on Cloud Shell and supports tab-completion.

Set our zone:

$ gcloud config set compute/zone us-central1-f
Updated property [compute/zone].

Run the following command to create a Kubernetes cluster:

$ gcloud container clusters create my-istio --num-nodes 4
...
kubeconfig entry generated for my-istio.
NAME      LOCATION       MASTER_VERSION  MASTER_IP      MACHINE_TYPE   NODE_VERSION  NUM_NODES  STATUS
my-istio  us-central1-f  1.11.7-gke.4    35.238.72.145  n1-standard-1  1.11.7-gke.4  4          RUNNING

Now, grant admin permissions in the cluster to the current gcloud user. We need these permissions to create the necessary cluster permissions to install Istio:

$ kubectl create clusterrolebinding cluster-admin-binding \
    --clusterrole=cluster-admin \
    --user=$(gcloud config get-value core/account)

...
clusterrolebinding.rbac.authorization.k8s.io "cluster-admin-binding" created

Install Istio

Now, install Istio. Istio's control plane (istiod) is installed in its own Kubernetes istio-system namespace, and can manage microservices from all other namespaces. The installation includes Istio core components, tools, and samples.

The Istio release page offers download artifacts for several OSs. In this post, we'll use this convenient command in Cloud Shell to download and extract the 0.5.1 release automatically:

$ curl -L https://git.io/getLatestIstio | ISTIO_VERSION=0.5.1 sh -
...
Downloaded into istio-0.5.1:
bin  install  istio.VERSION  LICENSE  README.md  samples  tools
...

The installation directory contains the following files which we'll use:

Installation .yaml files for Kubernetes in install/kubernetes
Sample applications in samples/
The istioctl client binary in the bin/ directory. Similar to kubectl for Kubernetes, this is the tool used to manage Istio, including network routing and security policies.
The istio.VERSION configuration file

Let's add the istioctl client to our PATH:

$ cd ./istio-*

$ export PATH=$PWD/bin:$PATH

Next, we'll install Istio's core components and the optional Istio Auth components, which enable mutual TLS authentication between the sidecars:

$ kubectl apply -f install/kubernetes/istio-auth.yaml
namespace "istio-system" created
clusterrolebinding.rbac.authorization.k8s.io "istio-pilot-admin-role-binding-istio-system" created
clusterrolebinding.rbac.authorization.k8s.io "istio-sidecar-injector-admin-role-binding-istio-system" created
clusterrolebinding.rbac.authorization.k8s.io "istio-ca-role-binding-istio-system" created
clusterrolebinding.rbac.authorization.k8s.io "istio-ingress-admin-role-binding-istio-system" created
clusterrolebinding.rbac.authorization.k8s.io "istio-sidecar-role-binding-istio-system" created
clusterrolebinding.rbac.authorization.k8s.io "istio-mixer-admin-role-binding-istio-system" created
configmap "istio-mixer" created
service "istio-mixer" created
serviceaccount "istio-mixer-service-account" created
deployment.extensions "istio-mixer" created
customresourcedefinition.apiextensions.k8s.io "rules.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "attributemanifests.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "circonuses.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "deniers.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "fluentds.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "kubernetesenvs.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "listcheckers.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "memquotas.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "noops.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "opas.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "prometheuses.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "rbacs.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "servicecontrols.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "stackdrivers.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "statsds.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "stdios.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "apikeies.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "authorizations.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "checknothings.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "kuberneteses.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "listentries.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "logentries.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "metrics.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "quotas.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "reportnothings.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "servicecontrolreports.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "tracespans.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "serviceroles.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "servicerolebindings.config.istio.io" created
configmap "istio" created
customresourcedefinition.apiextensions.k8s.io "destinationpolicies.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "egressrules.config.istio.io" created
customresourcedefinition.apiextensions.k8s.io "routerules.config.istio.io" created
service "istio-pilot" created
serviceaccount "istio-pilot-service-account" created
deployment.extensions "istio-pilot" created
service "istio-ingress" created
serviceaccount "istio-ingress-service-account" created
deployment.extensions "istio-ingress" created
serviceaccount "istio-ca-service-account" created
deployment.extensions "istio-ca" created
...

As we can see from the output, it creates the istio-system namespace along with the required RBAC permissions, and deploys the four primary Istio control plane components:

Pilot: Handles configuration and programming of the proxy sidecars.
Mixer: Handles policy decisions for your traffic and gathers telemetry.
Ingress: Handles incoming requests from outside your cluster.
CA: the Certificate Authority.

Just to make sure, let's check if the following Kubernetes services are deployed: istio-pilot, istio-mixer, and istio-ingress by getting the services in istio-system namespace:

$ kubectl get svc -n istio-system
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                                                            AGE
istio-ingress   LoadBalancer   10.23.254.98    35.222.16.117   80:30497/TCP,443:31582/TCP                                         6m
istio-mixer     ClusterIP      10.23.254.230   <none>          9091/TCP,15004/TCP,9093/TCP,9094/TCP,9102/TCP,9125/UDP,42422/TCP   7m
istio-pilot     ClusterIP      10.23.241.41    <none>          15003/TCP,8080/TCP,9093/TCP,443/TCP                                6m

Also, we need to make sure that the corresponding Kubernetes pods are deployed and all containers are up by running:

$ kubectl get pods -n istio-system
NAME                             READY     STATUS    RESTARTS   AGE
istio-ca-7c7894c898-dhbmd        1/1       Running   0          58s
istio-ingress-68868ccb77-6rvtr   1/1       Running   0          59s
istio-mixer-6cdf78c8f5-qvchl     3/3       Running   0          1m
istio-pilot-f464f5cfb-xkxrt      2/2       Running   0          59s                           6m

Application deploy

Now that Istio is installed and verified, we can deploy one of the sample applications provided with the installation - BookInfo.

This is a simple mock bookstore application made up of four microservices - all managed using Istio. Each microservice is written in a different language, to demonstrate how we can use Istio in a multi-language environment, without any changes to code.

The microservices are:

productpage: calls the details and reviews microservices to populate the page.
details: contains book information.
reviews: contains book reviews. It also calls the ratings microservice.

Reviews v1 doesn't call the ratings service.
Reviews v2 calls the ratings service, and displays each rating as 1 - 5 black stars.
Reviews v3 calls the ratings service, and displays each rating as 1 - 5 red stars.

ratings: contains book ranking information that accompanies a book review.

We find the source code and all the other files used in this example in our Istio samples/bookinfo directory.

Let's have a look at the samples/bookinfo/kube/bookinfo.yaml which describes the bookInfo application:

apiVersion: v1
kind: Service
metadata:
  name: details
  labels:
    app: details
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: details
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: details-v1
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: details
        version: v1
    spec:
      containers:
      - name: details
        image: istio/examples-bookinfo-details-v1:1.5.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
##################################################################################################
# Ratings service
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: ratings
  labels:
    app: ratings
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: ratings
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: ratings-v1
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: ratings
        version: v1
    spec:
      containers:
      - name: ratings
        image: istio/examples-bookinfo-ratings-v1:1.5.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
##################################################################################################
# Reviews service
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: reviews
  labels:
    app: reviews
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: reviews
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: reviews-v1
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: reviews
        version: v1
    spec:
      containers:
      - name: reviews
        image: istio/examples-bookinfo-reviews-v1:1.5.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: reviews-v2
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: reviews
        version: v2
    spec:
      containers:
      - name: reviews
        image: istio/examples-bookinfo-reviews-v2:1.5.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: reviews-v3
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: reviews
        version: v3
    spec:
      containers:
      - name: reviews
        image: istio/examples-bookinfo-reviews-v3:1.5.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
##################################################################################################
# Productpage services
##################################################################################################
apiVersion: v1
kind: Service
metadata:
  name: productpage
  labels:
    app: productpage
spec:
  ports:
  - port: 9080
    name: http
  selector:
    app: productpage
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: productpage-v1
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: productpage
        version: v1
    spec:
      containers:
      - name: productpage
        image: istio/examples-bookinfo-productpage-v1:1.5.0
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9080
---
###########################################################################
# Ingress resource (gateway)
##########################################################################
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gateway
  annotations:
    kubernetes.io/ingress.class: "istio"
spec:
  rules:
  - http:
      paths:
      - path: /productpage
        backend:
          serviceName: productpage
          servicePort: 9080
      - path: /login
        backend:
          serviceName: productpage
          servicePort: 9080
      - path: /logout
        backend:
          serviceName: productpage
          servicePort: 9080
      - path: /api/v1/products.*
        backend:
          serviceName: productpage
          servicePort: 9080

Note that there are Deployments, Services, and an Ingress to deploy the BookInfo application, however, there is nothing Istio-specific here at all. If we were to deploy the application as it is, it would work, but it would not have any Istio functionality.

We will use the following Istio command to inject the proxy sidecar along with each Pod that is deployed. istioctl kube-inject takes a Kubernetes YAML file as input, and outputs a version of that YAML which includes the Istio proxy server:

$ istioctl kube-inject -f samples/bookinfo/kube/bookinfo.yaml

Look at one of the Deployments. Now it contains a second container, the Istio sidecar, along with all the configuration necessary.

We can take the output from istioctl kube-inject and feed it directly to kubectl to create the objects with their sidecars:

$ kubectl apply -f <(istioctl kube-inject -f samples/bookinfo/kube/bookinfo.yaml)
service "details" created
deployment.extensions "details-v1" created

service "ratings" created
deployment.extensions "ratings-v1" created
service "reviews" created
deployment.extensions "reviews-v1" created
deployment.extensions "reviews-v2" created
deployment.extensions "reviews-v3" created
service "productpage" created
deployment.extensions "productpage-v1" created
ingress.extensions "gateway" created

Let's confirm that the application has been deployed correctly by running the following commands, and when all the pods have been created, we should see five services and six pods:

$ kubectl get services
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
details       ClusterIP   10.23.254.193   <none>        9080/TCP   2m
kubernetes    ClusterIP   10.23.240.1     <none>        443/TCP    30m
productpage   ClusterIP   10.23.241.147   <none>        9080/TCP   2m
ratings       ClusterIP   10.23.246.53    <none>        9080/TCP   2m
reviews       ClusterIP   10.23.247.61    <none>        9080/TCP   2m

$ kubectl get pods
NAME                              READY     STATUS    RESTARTS   AGE
details-v1-65dfd4dd95-z84nm       2/2       Running   0          3m
productpage-v1-846f956479-wtzl4   2/2       Running   0          3m
ratings-v1-59486f4787-7wms8       2/2       Running   0          3m
reviews-v1-686c54676c-2rxtk       2/2       Running   0          3m
reviews-v2-84dbbd6dbc-ln4n2       2/2       Running   0          3m
reviews-v3-7b8bb94fff-kwfxq       2/2       Running   0          3m

Application

Let's get the ingress IP and port:

$ kubectl get ingress
NAME      HOSTS     ADDRESS        PORTS     AGE
gateway   *         35.184.41.14   80        3m

Copy and paste the ingress IP into the our gateway IP environment variable:

$ export GATEWAY_URL=35.184.41.14

Once we have the address and port, check that the BookInfo app is running with curl:

$ curl -o /dev/null -s -w "%{http_code}\n" http://${GATEWAY_URL}/productpage
200

Refresh the page several times. Notice how we see three different versions of reviews! If we refer back to the diagram, we will see there are three different book review services which are being called in a round-robin style - no stars, black stars, and red stars. This is normal Kubernetes balancing behavior.

Istio can also be used to control which users are routed to which version of the services.

Dynamic request routing

When we were accessing the application, we saw the three versions of reviews. This is because without an explicit default version set, Istio will route requests to all available versions of a service, in a round-robin fashion.

Route rules control how requests are routed within an Istio service mesh. Requests can be routed based on the source and destination, HTTP header fields, and weights associated with individual service versions.

Now we'll use the istioctl command line tool to control routing.

Static Routing

Let's add a route rule that says all traffic should go to v1 of the reviews service.

First, confirm that there are no route rules installed:

$ istioctl get routerules
No resources found.

This example of a route rule will route all traffic for a service named reviews to Pods running v1 of that service, as identified by Kubernetes labels:

apiVersion: config.istio.io/v1alpha2
kind: RouteRule
metadata:
  name: reviews-default
spec:
  destination:
    name: reviews
  precedence: 1
  route:
  - labels:
      version: v1

BookInfo includes a sample with rules for all available services. Install it by running:

$ istioctl create -f samples/bookinfo/kube/route-rule-all-v1.yaml
Created config route-rule/default/productpage-default at revision 4534
Created config route-rule/default/reviews-default at revision 4535
Created config route-rule/default/ratings-default at revision 4536
Created config route-rule/default/details-default at revision 4537

Confirm that four rules were created:

$ istioctl get routerules
NAME                    KIND                                    NAMESPACE
details-default         RouteRule.v1alpha2.config.istio.io      default
productpage-default     RouteRule.v1alpha2.config.istio.io      default
ratings-default         RouteRule.v1alpha2.config.istio.io      default
reviews-default         RouteRule.v1alpha2.config.istio.io      default

Go back to the BookInfo application (http://$GATEWAY_URL/productpage) in the browser. Refresh a few times. Do we see any stars? We should see the book review with no rating stars because the service reviews:v1 does not have any stars.

Dynamic Routing

As the mesh operates at Layer 7, we can use HTTP attributes (paths or cookies) to decide on how to route a request.

In this example, a rule which routes certain users (in this case, Jason) to a service (v2) based on a cookie, looks like this:

apiVersion: config.istio.io/v1alpha2
kind: RouteRule
metadata:
  name: reviews-test-v2
spec:
  destination:
    name: reviews
  precedence: 2
  match:
    request:
      headers:
        cookie:
          regex: "^(.*?;)?(user=jason)(;.*)?$"
  route:
  - labels:
      version: v2

Create the rule from the sample file:

$ istioctl create -f samples/bookinfo/kube/route-rule-reviews-test-v2.yaml
Created config route-rule/default/reviews-test-v2 at revision 5227

List the rules and make sure it's there:

$ istioctl get routerule
NAME                    KIND                                    NAMESPACE
details-default         RouteRule.v1alpha2.config.istio.io      default
productpage-default     RouteRule.v1alpha2.config.istio.io      default
ratings-default         RouteRule.v1alpha2.config.istio.io      default
reviews-default         RouteRule.v1alpha2.config.istio.io      default
reviews-test-v2         RouteRule.v1alpha2.config.istio.io      default

Use -o yaml to see the full output:

$ istioctl get routerule reviews-test-v2 -o yaml
kind: RouteRule
metadata:
  creationTimestamp: null
  name: reviews-test-v2
  namespace: default
  resourceVersion: "5227"
spec:
  destination:
    name: reviews
  match:
    request:
      headers:
        cookie:
          regex: ^(.*?;)?(user=jason)(;.*)?$
  precedence: 2
  route:
  - labels:
      version: v2

Now we have routed requests from the user "jason" to use the reviews:v2 service.

Test how the page behavior changes with this new rule.

Log in as user "jason" on the product page web page by clicking the Sign in button at the top of the screen, typing jason as the user name - we don't need a password - then clicking Sign in.

Refresh the browser. We should now see black ratings stars next to each review.

If we try logging in as any other user (log out as Jason and sign in as Kylie), or don't log in at all, we will continue to see reviews: v1.

Run these commands to remove the routing rules:

$ istioctl delete -f samples/bookinfo/kube/route-rule-all-v1.yaml
Deleted config: route-rule/default/productpage-default
Deleted config: route-rule/default/reviews-default
Deleted config: route-rule/default/ratings-default
Deleted config: route-rule/default/details-default

$ istioctl delete -f samples/bookinfo/kube/route-rule-reviews-test-v2.yaml
Deleted config: route-rule/default/reviews-test-v2

We can go back to the tab with the web app to see that when we refresh the page, we're back to cycling through the 3 types of Reviews available: