Gateway¶

What is Kubernetes Gateway?¶

The Kubernetes Gateway API, like Kubernetes itself, is an open source project managed by the SIG-NETWORK community. It is a collection of resources that model service networking in Kubernetes.

Gateway is intended as an architected extension of Ingress. In Kubernetes, and in the LiteSpeed Ingress Controller, you can run both Ingress and Gateway resources together. Gateway is not currently scheduled to replace Ingress, but when you use Gateway resources like Gateway, and HTTPRoute you are using them instead of the Ingress resource of Ingress. Both utilize the lower level Kubernetes resources of Services, Deployments and Pods.

Because you can run Gateway with Ingress we encourage you to experiment with Gateway on your network knowing that the controller will still handle Ingress traffic at the same time.

Why use Gateway?¶

Gateway has more complexity than Ingress, so why use it in an environment with lots of complexity already? Because it's worth it. The goals of a role based environment as described in the Kubernetes Gatewap API Introduction and Concepts guides gives you details into the reasons why Gateway is worth considering.

From our perspective, the best part of Gateway is its consistent, well-documented interface which allows fine-detailed control over your environment. For example, you can specify HTTPRoute resources to examine and route traffic based on not just domains and paths, but also headers and query strings; you can modify headers in all sorts of ways; you can specify HTTP error return codes; and so many features too numerous to enumerate here.

So if you're reaching the edge of what Ingress can do for you or just wish it had more architected features for what you need, Gateway is the place to turn.

Using Gateway in the LiteSpeed Ingress Controller¶

The Gateway API is in the beta phase. In Kubernetes, beta is actually a fairly stable phase of development as it describes a set of features which are unlikely to change in the final stable release.

The Gateway API is installed using a set of CRDs (Custom Resource Definitions) and is currently installed and used using kubectl. The LiteSpeed Kubernetes Ingress Controller uses the latest beta release, and the installation for that can be found in the scripts ./gateway-load.sh and ./gateway-unload.sh in the samples. You can install the samples following the instructions here

$ ./gateway-load.sh

When the LiteSpeed Ingress Controller starts, it attempts to install Gateway support. If it sees the CRDs installed, it will create the GatewayClass if it does not exist, and begin listening for and processing Gateway requests. The helm install automatically includes Gateway support. And it does this without losing any of the load balancing, monitoring, security and other features which makes the LiteSpeed Ingress Controller a standout in the Kubernetes sphere.

The LiteSpeed Ingress Controller passes the rigorous Gateway Conformance tests and is thus consistent in behavior with the goals of the Gateway API. It includes all core support and quite a bit of extended support, including optional features, for GatewayClass, Gateway, HTTPRoute and ReferernceGrant.

In most cases you will only need to worry about defining definitions using Gateway and HTTPRoute. A GatewayClass is automatically created and has what you need in it. ReferenceGrant is only needed if you want to have cross-namespace references for Secrets and Services. These features are not available in Ingress, so you may be using them once you are familiar with the basic features of Gateway.

Since most of the features of Gateway are described in the API documentation you will be spending quite a lot of time looking at the Beta Gateway API documentation and the Alpha Gateway API documentation for those features not in beta yet.

Updating our simple Ingress example for Gateway¶

Our Ingress examples, olsup.sh and olsdown.sh are described here. We are going to create new samples named olsup-gw.sh and olsdown-gw.sh, which are available in the sample distribution described there. What is below is a full explanation of how they were created and why.

To demonstrate the similarity between Ingress and Gateway and show the migration path between the two, the same Deployment and Service will be used for both. Thus they can not be run at the same time. The key files, deployment and service are described in the samples section.

All of the samples require that you use the namespace sandbox. If this namespace does not already exist, you should create it:

$ kubectl create namespace sandbox

Running the test with olsup-gw.sh¶

This sample installs a preconfigured cloud based version of OpenLiteSpeed with limited TLS support into your cluster, and can uninstall it completely. This lets you see how each step is performed.

The olsup-gw.sh script looks like this:

#!/bin/bash
DIR=$( dirname "$0" )
. "$DIR/olsenv.sh"
echo "Create the OLS Gateway sample"
echo "Create the TLS secret.  In production you would use previously created certificate and key files."
openssl genrsa -out "${GATEWAY_HOSTNAME}.key" 2048
openssl req -new -key "${GATEWAY_HOSTNAME}.key" -out "${GATEWAY_HOSTNAME}.csr" -subj "/CN=${GATEWAY_HOSTNAME}"
openssl x509 -req -days 3650 -in "${GATEWAY_HOSTNAME}.csr" -signkey "${GATEWAY_HOSTNAME}.key" -out "${GATEWAY_HOSTNAME}.crt"
echo "Make Kubernetes aware of your certificate"
kubectl create secret tls $GATEWAY_HOSTNAME --cert "${GATEWAY_HOSTNAME}.crt" --key "${GATEWAY_HOSTNAME}.key" -n $NAMESPACE
echo "Bring up the OLS Gateway environment"
kubectl apply -f examples/ols-backend.yaml -n $NAMESPACE
kubectl apply -f examples/ols-backend-svc.yaml -n $NAMESPACE
kubectl apply -f examples/ols-backend-gateway.yaml -n $NAMESPACE
kubectl apply -f examples/ols-backend-httproute.yaml -n $NAMESPACE
echo "Get the IP address to access it"
kubectl get gateway ${GATEWAY_HOSTNAME} -n $NAMESPACE
#curl https://IPADDRESS/ -H 'Host: ols-gateway.com' -k

olsenv.sh¶

olsenv.sh is used as a source variables file so you can define NAMESPACE and GATEWAY_HOSTNAME in a common place, without having to change the script.

Your application will need to run in a specified name space. As for olsup.sh, the sandbox namespace is used.

Creating the certificate, key, deployment and service¶

Those are represented with the following lines:

openssl genrsa -out "${GATEWAY_HOSTNAME}.key" 2048
openssl req -new -key "${GATEWAY_HOSTNAME}.key" -out "${GATEWAY_HOSTNAME}.csr" -subj "/CN=${GATEWAY_HOSTNAME}"
openssl x509 -req -days 3650 -in "${GATEWAY_HOSTNAME}.csr" -signkey "${GATEWAY_HOSTNAME}.key" -out "${GATEWAY_HOSTNAME}.crt"
echo "Make Kubernetes aware of your certificate"
kubectl create secret tls $GATEWAY_HOSTNAME --cert "${GATEWAY_HOSTNAME}.crt" --key "${GATEWAY_HOSTNAME}.key" -n $NAMESPACE
echo "Bring up the OLS Gateway environment"
kubectl apply -f examples/ols-backend.yaml -n $NAMESPACE
kubectl apply -f examples/ols-backend-svc.yaml -n $NAMESPACE

Creating the Gateway¶

As described in the Kubernetes Gateway API Concepts Guide, the Gateway describes how traffic can be translated to Services within the cluster.

The Gateway is the object that is exposed to the LiteSpeed Ingress Controller. The examples/ols-backend-gateway.yaml file creates the gateway which is exposed to the LiteSpeed Ingress Controller for access to the wider internet:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: ols-gateway.com
spec:
  gatewayClassName: lslbd
  listeners:
  - name: ols-gateway.com
    port: 443
    protocol: HTTPS
    tls:
      certificateRefs:
      - name: ols-gateway.com

It's simpler than Ingress because the functionality is split between Gateway and HTTPRoute resources.

Some interesting parameters are:

• metadata.name: ols-gateway.com The name used by the LiteSpeed Ingress Controller or other Kubernetes resources to access the gateway. In particular it is used by the HTTPRoute as the spec.parentRefs.name.
• spec.gatewayClassName: lslbd Required for Gateway, the controller name. For LiteSpeed it is specified as the gateway-class parameter in the LiteSpeed Ingress Controller startup, defaulting to lslbd.
• spec.listeners.name: ols-gateway.com The name of the listener. Can be any name.
• spec.listeners.port: 443 The external port. The specification is the external HTTPS port.
• spec.listeners.protocol: HTTPS The protocol, HTTPS. With HTTPS you must also specify spec.listeners.tls.certificateRefs.
• spec.listeners.tls.certificateRefs.name: ols-gateway.com The name of the secret we created in this script earlier.

There are a number of optional parameters specified in the Gateway API specification.

Creating the HTTPRoute¶

As described in the Kubernetes Gateway API Concepts Guide, the HTTPRoute is for multiplexing HTTP or terminated HTTPS connections.

The examples/ols-backend-httproute.yaml file creates the HTTPRoute which is exposed to the LiteSpeed Ingress Controller for access to the Gateway:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: ols-gateway.com
spec:
  parentRefs:
  - name: ols-gateway.com
  hostnames:
  - ols-gateway.com
  rules:
  - matches:
    - path:
        value: /
        type: PathPrefix
    backendRefs:
    - name: ols-backend-svc
      port: 80

Some interesting parameters are:

• metadata.name: ols-gateway.com The name of the HTTPRoute, can be any name.
• spec.parentRefs.name: ols-gateway.com Must match the metadata.name of the Gateway.
• spec.hostnames: ols-gateway.com The hostname of this particular route.
• spec.rules.matches.path.value: / Specifies that the root can be used as the path of this route.
• spec.rules.matches.path.type: PathPrefix Indicates that any value from the path value downwards is acceptable. Can also be Exact in which only the specified path value would match.
• spec.rules.backendRefs.name: ols-backend-svc The name of the service that this route will be attached to. For this example, created in examples/ols-backend-svc.yaml.
• spec.rules.backendRefs.port: 80 The service port number and is required for backend service specs.

Testing it¶

The last two lines in the olsup-gw.sh are intended to help you test the load balancer's access to the backend.

kubectl get gateway ${GATEWAY_HOSTNAME} -n $NAMESPACE will resolve to kubectl get gateway ols-gateway -n sandbox in the example and will show the exported IP address generated by Kubernetes. This may take a few manual interations as Kubernetes may need several minutes to get an address assigned.

In our environment we ran:

$ kubectl get gateway ols-gateway -n sandbox
NAME          CLASS    HOSTS             ADDRESS          PORTS     AGE
ols-gateway   <none>   ols-gateway.com   143.244.212.14   80, 443   21m

• #curl https://IPADDRESS/ -H 'Host: ols-gateway.com' -k In our environment we ran curl https://143.244.212.14/ -H 'Host: ols-gateway.com' -k which displays the output of the default OpenLiteSpeed banner screen. You specify the -H 'Host: ols-gateway.com' to indicate that it should transmit the Host header entry as we are not able to immediately use the domain name. You specify -k to support a self-signed cert.

To run using the proper domain name, see your Cloud Provider's documentation as to the steps to add a DNS name using the address assigned by Kubernetes.

Taking it down with olsdown-gw.sh¶

The olsdown.sh script looks like this:

#!/bin/bash
set -x
DIR=$( dirname "$0" )
. "$DIR/olsenv.sh"
echo "Delete the OLS Gateway Sample"
kubectl delete -f examples/ols-backend-httproute.yaml -n $NAMESPACE
kubectl delete -f examples/ols-backend-gateway.yaml -n $NAMESPACE
kubectl delete -f examples/ols-backend-svc.yaml -n $NAMESPACE
kubectl delete -f examples/ols-backend.yaml -n $NAMESPACE
kubectl delete secret $GATEWAY_HOSTNAME -n $NAMESPACE
rm "${GATEWAY_HOSTNAME}.crt"
rm "${GATEWAY_HOSTNAME}.csr"
rm "${GATEWAY_HOSTNAME}.key"

The steps are the reverse of the creation of the gateway route:

• . "$DIR/olsenv.sh" Sources the olsenv.sh script to define NAMESPACE and GATEWAY_HOSTNAME.
• kubectl delete -f examples/ols-backend-httproute.yaml -n $NAMESPACE Deletes the HTTPRoute.
• kubectl delete -f examples/ols-backend-gateway.yaml -n $NAMESPACE Deletes the Gateway
• kubectl delete -f examples/ols-backend-svc.yaml -n $NAMESPACE Deletes the Service
• kubectl delete -f examples/ols-backend.yaml -n $NAMESPACE Deletes the Deployment
• kubectl delete secret $GATEWAY_HOSTNAME -n $NAMESPACE Deletes the secret
• rm "${GATEWAY_HOSTNAME}.crt", rm "${GATEWAY_HOSTNAME}.csr", and rm "${GATEWAY_HOSTNAME}.key" Deletes the certificate and key files.

Gateway with cert-manager¶

cert-manager has been updated for Gateway and LiteSpeed works with it. There are some preliminary steps necessary to using Gateway with cert-manager and they are discussed in the cert-manager Gateway documentation. You may need to verify that your infrastructure provider's DNS servers support Gateway as well as Ingress.

As for LiteSpeed itself, you will need to install cert-manager using the instructions.

Those instructions involve adding an annotation to your Ingress definition. Since this is Gateway, you'll be adding an annotation to your Gateway definition. If you used the recommended cluster-issuer letsencrypt-production, for example:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: echo-ingress
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-production

Otherwise cert-manager is used the same as for Ingress, including troubleshooting.

Using the more advanced features of Gateway¶

The Gateway documentation is quite complete so this section will just touch on some of the features that you can employ as you define your Gateway resources. Note that the example above is just about getting a Gateway definition working and doesn't show practically any of the features of Gateway.

• Since Gateway is intented to be role based, the goal is to have different groups responsible for different parts of the definition. For example, as the infrastructure provider you might be responsible for the Gateway resources and the consumers might be responsible for the HTTPRoutes. To keep these responsibilities separate, you may choose to have these implemented in different namespaces, which is available in Gateway, but was not in Ingress. To support that there is the more advanced resource of ReferenceGrant which allows you as the infrastructure provider to allow specific cross-namespace interactions.
• Ingress supports a wildcarded domain where only the highest level is allowed to match the wildcard: For example *.abc.com would match one.abc.com but not two.one.abc.com. Gateway will match everything with the suffix so both would match.
• As part of the architecture, you can match hostname, path, headers and query-strings and order is quite important. This gives you a tremendous amount of control in your matches and enables architected red-blue deployments.
• Canary deployments are well defined with control of traffic Weight
• Redirection control including the setting of the HTTP status code.
• You can add/set/delete header items which allow you fine-grained control of the identification of your traffic and cache optimization.

All of the regular features of the LiteSpeed WebADC controller are still available including Prometheus and Grafana statistics and graphs, a high level of security with a very high speed Web Application Firewall, cache and other WebADC feature control in the graphical configuration.

Extended Gateway features¶

There are a number of Gateway HttpRoute Extended Features, which are beyond the Core features and include:

• Matches: Path, Type: RegularExpression where PCRE compatible regular expressions are supported
• Filters:
  • RequestRedirect: New features include Scheme, Hostname, Path, Port and StatusCode. Path can specify a Type of ReplaceFullPath (an absolute path) or ReplacePrefixMatch (a path which will add subdirectories), either of which can include PCRE regular expression replacement strings.
  • URLRewrite: Hostname and Path. Hostname can include a port and Path can specify a Type of ReplaceFullPath (an absolute path) or ReplacePrefixMatch (a path which will add subdirectories).

Gateway Troubleshooting¶

As with all of Kubernetes kubectl describe can be your best tool. When a Gateway is created, it is up to the Gateway Controller, LiteSpeed in this case (named lslbd), to set the Conditions.

• For Status, Conditions it must have the Status of True and the Type of Ready if the LiteSpeed controller is available.
• For Listeners, Conditions again it must have the Status of True and the Type of Ready if the HTTPRoute is attached.

As in the following kubectl describe gateway ols-gateway -n sandbox in the Status section from the olsup-gw.sh example above:

Status:
  Addresses:
    Type:   IPAddress
    Value:  146.190.196.211
  Conditions:
    Last Transition Time:  2022-11-01T16:05:31Z
    Message:               
    Reason:                Ready
    Status:                True
    Type:                  Ready
  Listeners:
    Attached Routes:  0
    Conditions:
      Last Transition Time:  2022-11-01T16:05:31Z
      Message:               
      Reason:                Ready
      Status:                True
      Type:                  Ready
    Name:                    ols-gateway.com
    Supported Kinds:
      Group:  gateway.networking.k8s.io
      Kind:   HTTPRoute

Similarly, the Status from the example above shows for kubectl describe httproute ols-gateway -n sandbox:

• For Status, Parents, Conditions the Status of True and the Type of Accepted if the LiteSpeed controller has accepted the route.
• For Parent Ref it will display the Kind (Gateway) and the Name of the Gateway that the route is attached to.

Status:
  Parents:
    Conditions:
      Last Transition Time:  2022-11-01T18:14:12Z
      Message:               
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
    Controller Name:         litespeedtech.com/lslbd
    Parent Ref:
      Group:  gateway.networking.k8s.io
      Kind:   Gateway
      Name:   ols-gateway.com

Note that an HTTPRoute without a Status almost certainly has a mismatch between its spec.parentRefs.name and the name in the corresponding Gateway spec.listeners.name.