Some time ago, at my company, we had the problem that certain parts of our users should use some different versions of our services depending on the subscription plan that they have at that moment. As we are running microservices using Docker containers and Kubernetes we had to come with some different ideas on how to implement this.
The solution needed to be robust, as thousands of users could use the services every second, and what’s more important, it needed to be secure and establish these connections between PODs in a safe way.
After some investigation, the winning idea was to use Envoy for this purpose. Given all its benefits, our Architects, Technical Leads, Developers, Testers and me as a Security Lead were happy with the idea.
A little more investigation brought us to Istio which makes easier the use of Envoy.
For those who don´t know what is Istio, I will explain more or less its purpose and some of the things that we can do with it, but probably if you reached this article, you already know what is it and you can simply jump to the examples part.
Istio is an open platform to connect, manage, and secure microservices. It supports Kubernetes so it makes it easier to integrate into your project if you already use this orchestrator. With Istio we can control traffic between services and we can make it more robust and reliable. It can show you dependencies and the flow between services and it also provides access policies and authentication within your service mesh.
So let´s take a look at the architecture of Istio:
On top, we have the control plane with the Pilot who sends the configuration to every Envoy running in our services to be able to communicate between them as we specify, the Mixer which receives and collect telemetry data and also does policy checks and the Istio-Auth which is responsible for the authentication part with TLS certificates.
On the data plane, we can see Service A and Service B. Every service runs an instance of Envoy in its own Kubernetes POD which communicates between them and with his own service acting as a “proxy” let’s say. The communication happens in HTTP/1.1 or HTTP/2, gRPC, TCP with or without TLS (which is the authentication part of Istio that can be enabled or disabled so our data is encrypted between these communications if we wish).
Now that we understand the big picture, I can show you some examples and use cases for this technology. And for not making this article very long we’ll concentrate only on the Pilot part which apart from sending the configuration is also responsible for service discovery, traffic management, and resiliency. For example A/B tests and canary deployments.
The pilot does this by converting Istio rules to Envoy configurations. So we don’t really need to know how Envoy works. The only thing we need to do is to create Istio rules and it will convert them for us.
HOW TO DO TRAFFIC MANAGEMENT WITH ISTIO
Let’s imagine that we have a website where we run a Service A that is using Service B v1, but then we create a new version of Service B that we want to test before releasing to everyone or maybe we want that only a certain percentage of users use it before we roll out to everyone (this scenario happens a lot for example in the videogames industry).
So one of the most interesting things to do with Istio is traffic management. We can forward some of the users of our website to the Service B v1 and other users who have different cookies in their browser, for example, forward them to Service B v2. In this case, they will see different versions of the website.
And this is what I’m gonna show you now! 🙂
For the examples, I’m using some YAML files from the official documentation as they are quite good to understand the concept.
Let’s start by creating the rules for routing all users to the Service B v1. For that we can create the following file:
FILE: route-rule-all-v1.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: productpage
spec:
hosts:
- productpage
gateways:
- bookinfo
- mesh #TODO remove this if not needed
http:
- route:
- destination:
name: productpage
subset: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- route:
- destination:
name: reviews
subset: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: ratings
spec:
hosts:
- ratings
http:
- route:
- destination:
name: ratings
subset: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: details
spec:
hosts:
- details
http:
- route:
- destination:
name: details
subset: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: productpage
spec:
name: productpage
subsets:
- name: v1
labels:
version: v1
---
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: reviews
spec:
name: reviews
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
- name: v3
labels:
version: v3
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: ratings
spec:
name: ratings
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
- name: v2-mysql
labels:
version: v2-mysql
- name: v2-mysql-vm
labels:
version: v2-mysql-vm
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: details
spec:
name: details
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
---
Let’s have a look what’s in this file:
To make the traffic control works, first, we need to create VirtualServices (one for every microservice that we run). In our case, we have one for “productpage”, one for “reviews”, one for “ratings” and one for “details”.
We also need to create DestinationRules where we specify the name of the VirtualService, for example, “productpage” and we specify also the subsets, adding the label “v1” that our POD running the Product Page service should have.
If we look at the DestinationRule of the “reviews” we can see that this is one of the services running several versions: “v1”, “v2” and “v3”.
Back to the VirtualServices, each one of them has specified to which destination it should connect, using the label “v1” in the subset parameter.
Now run this command in the same directory of the file for creating the Envoy configurations:
istioctl create -f <your-directory>/route-rule-all-v1.yamlFrom this point, all users should see only the v1 of Service B.
Now let’s change this for them to see v2. For that we are going to create another file:
FILE: route-rule-reviews-test-v2.yaml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- match:
- headers:
cookie:
regex: "^(.*?;)?(user=jason)(;.*)?$"
route:
- destination:
name: reviews
subset: v2
- route:
- destination:
name: reviews
subset: v1
Here we only have a VirtualService with a special rule, using the http, match, headers, cookie, regex we can route to the destination that we wish. If the cookies of the user match with the regex, they will be connected to v2 BUT if not, they will still see v1.
Now we can run this command and NOTE that this time we are using “replace” as some rules are already existing (it will replace only this rule):
istioctl replace -f <your-directory>/route-rule-reviews-test-v2.yamlIf the user that login into the website is “jason” then the magic will happen.
So this is a very interesting way of having multiple applications deployed and redirecting traffic based on a rule. This allows developers to do a lot better testing before a new version hits all the users.
There are much more use cases where for example we could redirect only the 10% of the users to the new version and see how it behaves, or we can do some A/B testing as I mentioned before so we can see in an online shop which version gives us more selling (changing the color of some buttons or changing the size and the place of some image links to see which provides better user experience).
The possibilities are infinite 🙂
Let me know in the comments how do you use Istio or Envoy, or any question that you may have I will be glad to help!
Hi! My name is Adrián and I´m Security Lead of a Telecommunications company. One of my passions is videogames so I spend some of my time trying to learn how to make my own ones.
Have fun here! 🙂