Skip to content

Latest commit

History

History
262 lines (193 loc) · 10.2 KB

3.04.Canary-Release.md

File metadata and controls

262 lines (193 loc) · 10.2 KB
Raw

Canary Release With Cloudflare

Background

Acanary releaseis a software testing technique used to reduce the risk of introducing a new software version into production by gradually rolling out the change to a small subset of users, before rolling it out to the entire platform/infrastructure.

Advantages

  • Service Evaluation:You can evaluate multiple service versions side by side using canary release in real-world environments with actual users and use cases.
  • Zero downtime:Canary release does not cause downtime
  • Simple Rollback Mechanism:You can always easily roll back to the previous version.

Key point

  • Traffic tagging:To categorize traffic based on various business attributes such asuser devices,geographic locations,or user attributes(e.g. age, gender, level, etc.), we can apply different labels or tags to the traffic. It's important to note that IP addresses should not be used as user tags as they can be inaccurate and inconsistent.
  • Traffic scheduling:Once traffic is tagged, we can use traffic rules to schedule specific traffic to a designated Canary. For instance, traffic from an iPhone user in Beijing could be directed to a new version of a service.

Why use Easegress and Cloudflare

  • For traffic tagging, UsingCloudflarewe can easily label the traffic.
    • Geographic Location:Cloudflare offers a feature calledManaged Transforms,which can be used to identify the geographic location of a user.
    • User Device:UsingWorkers,we can extract information about the user's device, operating system, and web browser from theUser-Agentfield of the HTTP request.
    • User's Customized Label:Using CloudflareRules,it is easy to extract the user's customized business label from HTTP request headers (such as: Cookies, etc.)
  • For traffic scheduling, Easegress can be used to route traffic to different canary releases.

Based on geographic location

case 1

Suppose we run an e-Commerce service, and we recently upgraded our Order Service from version 1 to version 2. To ensure that our users have the best experience, we want to route the users who are accessing our service from Beijing to use the new Order service (version 2).

Download the latest Service code fromcanary-case-1

1. Traffic tagging

To determine the geographic location of a user, we can leverage theCF-IPCityfield in Cloudflare's HTTP header. We need to first enable the "Add visitor location headers" option in the Cloudflare dashboard. (Cloudflare Dashboard->Rules->Transform Rules->Managed Transforms->enabled “Add visitor location headers”)

2. Traffic scheduling rules:

Save the following YAML code toorder-service.yamland make sure to replace theserversconfiguration with your own.

name:order-pipeline
kind:Pipeline
flow:
-filter:order-service
filters:
-name:order-service
kind:Proxy
pools:
-servers:
-url:http://0.0.0.0:5002#new version
filter:
headers:
cf-ipcity:#Cloudflare geographic location header
exact:Beijing#the traffic from Beijing should be directed to the new version
-servers:
-url:http://0.0.0.0:5001#old version

Then create the order pipeline with the command:

egctl create -f order-service.yaml

Save below YAML toecommerce-server.yaml.

kind:HTTPServer
name:ecommerce-server
port:8888
https:false
keepAlive:true
keepAliveTimeout:75s
maxConnection:10240
cacheSize:0
rules:
-paths:
-pathPrefix:/order
backend:order-pipeline
-pathPrefix:/notify
backend:notify-pipeline

Then create the HTTP server with command:

egctl create -f ecommerce-server.yaml

Send a request to e-Commerce service:

curl http://127.0.0.1:8888/order -H"cf-ipcity: Beijing"
{"order_id":"5245000","role_id":"44312","order_status":1,"order_version":"v2"}

curl http://127.0.0.1:8888/order -H"cf-ipcity: Shanghai"
{"order_id":"5245000","role_id":"44312","order_status":1,"order_version":"v1"}

Based on user devices

case 2

In real-world scenarios, services are usually interconnected and depend on each other to provide a seamless experience for the customer. In our case, we recently upgraded our Notify Service from version 1 to version 2. To ensure that our users have a holistic experience, we want to route users who are accessing our service from a Mac device to use the new version of our Notify Service (version 2).

Download the latest Service code fromcanary-case-2

1. Traffic tagging

To determine the user's device and system, we can extract it from theUser-Agentfield in HTTP request byWorkers.

We offer an easy way to parseUser-Agentineasegress-rust-uaparserlibrary.

git clone https://github /megaease/easegress-rust-uaparser.git

Go to theeasegress-rust-uaparser/cloudflaredirectory and deploy the worker by using thewranglercommand.

npx wrangler publish

Go to the Cloudflare website configuration page and configureWorkers Routes.

2. Traffic scheduling rules

Save the following YAML code toorder-service.yaml,and make sure to replace theserversconfiguration with your own.

name:order-pipeline
kind:Pipeline
flow:
-filter:order-service
filters:
-name:order-service
kind:Proxy
pools:
-servers:
-url:http://0.0.0.0:5003#order service

Then update the order pipeline with the command:

egctl apply -f order-service.yaml

Save the following YAML code tonotify-service.yaml,and make sure to replace theserversconfiguration with your own.

name:notify-pipeline
kind:Pipeline
flow:
-filter:notify-service
filters:
-name:notify-service
kind:Proxy
pools:
-servers:
-url:http://0.0.0.0:5002#new version
filter:
headers:
x-ua-device:
exact:Mac#the traffic from Mac device should be directed to the new version
-servers:
-url:http://0.0.0.0:5001#old version

Then create the notify pipeline with the command:

egctl create -f notify-service.yaml

Send a request to e-Commerce service:

curl http://127.0.0.1:10080/ecommerce -H"user-agent:Mozilla/5.0 (Linux; Android 4.4.2; GT-I9505 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.117 Mobile Safari/537.36"
{"order_id":"5245000","role_id":"44312","order_status":1,"order_version":"v2","notify_status":1,"notify_version":"v1"}

curl http://127.0.0.1:10080/ecommerce -H"user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36"
{"order_id":"5245000","role_id":"44312","order_status":1,"order_version":"v2","notify_status":1,"notify_version":"v2"}

Based on user OS

case 3

Suppose both the Order service and Notify service have been upgraded simultaneously. The user who is accessing our service from a MacOS should be routed to the new version of the Order service (v2), and from Mac device should be routed to the new version of the Notify service (v2).

You may think Cloudflare's Workers is not suitable for your needs, there is another easy way to parseUser-Agentwith Easegress's WASM.

Download the latest Service code fromcanary-case-3

1. Traffic tagging

Using Easegress's WASM to parse theUser-Agent.

Downloadeasegress-rust-uaparser

git clone https://github /megaease/easegress-rust-uaparser.git

Go to theeasegress-rust-uaparser/binarydirectory and just use theeasegress.wasmfile directly. It will parse theUser-Agentand add thex-ua-deviceandx-ua-osheaders to the HTTP request.

2. Traffic scheduling rules

Save the following YAML code toorder-service.yaml,and make sure to replace theserversconfiguration and wasm file path with your own.

name:order-pipeline
kind:Pipeline
flow:
-filter:wasm
-filter:order-service
filters:
-name:wasm#parse user-agent, add `x-ua-device` and `x-ua-os` headers to the HTTP request.
kind:WasmHost
maxConcurrency:2
code:/<Path>/easegress.wasm#easegress.wasm file path
timeout:100ms
-name:order-service
kind:Proxy
pools:
-servers:
-url:http://0.0.0.0:5002#new version of order service
filter:
headers:
x-ua-os:#Easegress's user OS header
exact:MacOS#the traffic from MacOS should be directed to the new version
-servers:
-url:http://0.0.0.0:5001#old version of order service

Then update the order pipeline with the command:

egctl apply -f order-service.yaml

Send a request to e-Commerce service:

curl http://127.0.0.1:10080/ecommerce -H"user-agent:Mozilla/5.0 (Linux; Android 4.4.2; GT-I9505 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.117 Mobile Safari/537.36"
{"order_id":"5245000","role_id":"44312","order_status":1,"order_version":"v1","notify_status":1,"notify_version":"v1"}

curl http://127.0.0.1:10080/ecommerce -H"user-agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36"
{"order_id":"5245000","role_id":"44312","order_status":1,"order_version":"v2","notify_status":1,"notify_version":"v2"}