Container:Development/ServiceMesh

Service mesh

This page explains the process building and packaging of the following service mesh components:

• envoy-proxy
• istio-proxy - We can drop it for now until we want to package the full Istio! cilium-proxy used to require istio-proxy to build, but it does not anymore
• cilium-proxy

Package structure

envoy-proxy is a package in the devel:kubic project which contains sources and specs for:

• envoy-proxy
• istio-proxy - we can drop it now
• cilium-proxy

istio-proxy and cilium-proxy packages are links to the envoy-proxy package. This is done to build those three projects all together, because they depend on sources of each other.

Problems

Packaging of those projects has several problems which make it non-trivial.

Bazel

Bazel is the buildsystem which is designed for building software with bundled libraries. Usually the most of C and C++ projects using Bazel are downloading all libraries from Github or any other public provides which host tarballs based on git repositories.

That approach stands in contradiction with openSUSE packaging which uses shared libraries and *-devel packages for bulding C and C++ software.

To make Bazel more compatible with our approach, there is a project called bazel-workspaces which contains "fake" workspaces for Bazel, which tell to link libraries dynamically, instead of bundling. This is achieved by custom BUILD files like:

cc_library(
    name = "gperftools",
    hdrs = glob(["thirdparty_build/include/gperftools/**/*.h"]),
    copts = [],
    linkopts = ["-ltcmalloc", "-lprofiler"],
    visibility = ["//visibility:public"],
    linkstatic = False,
)

Which tells Bazel to:

• look for headers in /usr/include/gperftools, recursively in all possible subdirectories
• link libtcmalloc.so and libprofiler.so

There is an RPM package available for bazel-workspaces which installs those files in /usr/share/bazel-workspaces.

Let's assume that you want to build a project using Bazel, which has a dependency on gperftools with the name com_github_gperftools_gperftools (that's a commonly used one). To use the shared library from gperftools-devel package, use the --override_repository option like:

bazel build \
    --override_repository="com_github_gperftools_gperftools=/usr/share/bazel-workspaces" \
    //...

Using OpenSSL instead of BoringSSL

Upstream Envoy uses BoringSSL as the library for certification features. BoringSSL is Google's fork of OpenSSL. Its README says:

Although BoringSSL is an open source project, it is not intended for general use, as OpenSSL is. We don't recommend that third parties depend upon it. Doing so is likely to be frustrating because there are no guarantees of API or ABI stability.

That's why we don't want to package BoringSSL and we are going to patch all sources to use OpenSSL instead.

We achieve that by the following projects from Maistra - OpenShift service mesh:

• bssl_wrapper – a library which translates BoringSSL calls to OpenSSL calls
• OpenSSL-cbs – a library which provides Crypto ByteString (CBS) functionality for OpenSSL – BoringSSL’s feature
• jwt-verify-lib-openssl, envoy-openssl, istio-proxy-openssl – set of patches and modules which make jwt_verify_lib, Envoy and Istio build with OpenSSL, using the two projects above

Building with OpenSSL on local environment

It's the best if all the repositories mentioned above are cloned in one directory (let's assume ~/repos):

git clone git@github.com:google/jwt_verify_lib.git
git clone git@github.com:Maistra/jwt-verify-lib-openssl.git
git clone git@github.com:envoyproxy/envoy.git
git clone git@github.com:kubic-project/envoy-openssl.git
git clone git@github.com:istio/proxy.git istio-proxy
git clone git@github.com:Maistra/istio-proxy-openssl.git
git clone git@github.com:cilium/proxy.git cilium-proxy

Which after cloning all repos has the tree like i.e.:

- repos/
|- jwt_verify_lib/
|- jwt-verify-lib-openssl/
|- envoy/
|- envoy-openssl/
|- istio-proxy/
|- istio-proxy-openssl/
|- cilium-proxy/
[...]

Then each *-openssl repository has a file openssl.sh. The syntax of that script is:

./openssl.sh [path-to-repository-to-patch] SSL

So, for the each repo, you need to do

cd jwt-verify-lib-openssl
./openssl.sh ../jwt_verify_lib SSL

cd envoy-openssl
./openssl.sh ../envoy SSL

cd istio-proxy-openssl
./openssl.sh ../istio-proxy SSL

Then you can build each of repos:

cd jwt_verify_lib
bazel build //...

cd envoy
bazel build \
    --override_repository="com_github_google_jwt_verify=$HOME/repos/jwt_verify_lib" \
    --copt="-DENVOY_SSL_VERSION=\"OpenSSL_1_1_1\"" \
    //source/exe:envoy-static

cd istio-proxy
bazel build \
    --override_repository="com_github_google_jwt_verify=$HOME/repos/jwt_verify_lib" \
    --override_repository="envoy=$HOME/repos/envoy" \
    --copt="-DENVOY_SSL_VERSION=\"OpenSSL_1_1_1\"" \
    //...

The last one might still not build, but it's not a blocking issue for now.