Container:Development/ServiceMesh
Service mesh
This page explains the process building and packaging of the following service mesh components:
- envoy-proxy
- istio-proxy - We can drop it for now until we want to package the full Istio! cilium-proxy used to require istio-proxy to build, but it does not anymore
- cilium-proxy
Package structure
envoy-proxy is a package in the devel:kubic project which contains sources and specs for:
- envoy-proxy
- istio-proxy - we can drop it now
- cilium-proxy
istio-proxy and cilium-proxy packages are links to the envoy-proxy package. This is done to build those three projects all together, because they depend on sources of each other.
Problems
Packaging of those projects has several problems which make it non-trivial.
Bazel
Bazel is the buildsystem which is designed for building software with bundled libraries. Usually the most of C and C++ projects using Bazel are downloading all libraries from Github or any other public provides which host tarballs based on git repositories.
That approach stands in contradiction with openSUSE packaging which uses shared libraries and *-devel
packages for bulding C and C++ software.
To make Bazel more compatible with our approach, there is a project called bazel-workspaces which contains "fake" workspaces for Bazel, which tell to link libraries dynamically, instead of bundling. This is achieved by custom BUILD
files like:
cc_library(
name = "gperftools",
hdrs = glob(["thirdparty_build/include/gperftools/**/*.h"]),
copts = [],
linkopts = ["-ltcmalloc", "-lprofiler"],
visibility = ["//visibility:public"],
linkstatic = False,
)
Which tells Bazel to:
- look for headers in
/usr/include/gperftools
, recursively in all possible subdirectories - link
libtcmalloc.so
andlibprofiler.so
There is an RPM package available for bazel-workspaces which installs those files in /usr/share/bazel-workspaces
.
Let's assume that you want to build a project using Bazel, which has a dependency on gperftools
with the name com_github_gperftools_gperftools
(that's a commonly used one). To use the shared library from gperftools-devel
package, use the --override_repository
option like:
bazel build \
--override_repository="com_github_gperftools_gperftools=/usr/share/bazel-workspaces" \
//...
Using OpenSSL instead of BoringSSL
Upstream Envoy uses BoringSSL as the library for certification features. BoringSSL is Google's fork of OpenSSL. Its README says:
Although BoringSSL is an open source project, it is not intended for general use, as OpenSSL is. We don't recommend that third parties depend upon it. Doing so is likely to be frustrating because there are no guarantees of API or ABI stability.
That's why we don't want to package BoringSSL and we are going to patch all sources to use OpenSSL instead.
We achieve that by the following projects from Maistra - OpenShift service mesh:
- bssl_wrapper – a library which translates BoringSSL calls to OpenSSL calls
- OpenSSL-cbs – a library which provides Crypto ByteString (CBS) functionality for OpenSSL – BoringSSL’s feature
- jwt-verify-lib-openssl, envoy-openssl, istio-proxy-openssl – set of patches and modules which make jwt_verify_lib, Envoy and Istio build with OpenSSL, using the two projects above
Building with OpenSSL on local environment
It's the best if all the repositories mentioned above are cloned in one directory (let's assume ~/repos
):
git clone git@github.com:google/jwt_verify_lib.git
git clone git@github.com:Maistra/jwt-verify-lib-openssl.git
git clone git@github.com:envoyproxy/envoy.git
git clone git@github.com:kubic-project/envoy-openssl.git
git clone git@github.com:istio/proxy.git istio-proxy
git clone git@github.com:Maistra/istio-proxy-openssl.git
git clone git@github.com:cilium/proxy.git cilium-proxy
Which after cloning all repos has the tree like i.e.:
- repos/
|- jwt_verify_lib/
|- jwt-verify-lib-openssl/
|- envoy/
|- envoy-openssl/
|- istio-proxy/
|- istio-proxy-openssl/
|- cilium-proxy/
[...]
Then each *-openssl repository has a file openssl.sh
. The syntax of that script is:
./openssl.sh [path-to-repository-to-patch] SSL
So, for the each repo, you need to do
cd jwt-verify-lib-openssl
./openssl.sh ../jwt_verify_lib SSL
cd envoy-openssl
./openssl.sh ../envoy SSL
cd istio-proxy-openssl
./openssl.sh ../istio-proxy SSL
Then you can build each of repos:
cd jwt_verify_lib
bazel build //...
cd envoy
bazel build \
--override_repository="com_github_google_jwt_verify=$HOME/repos/jwt_verify_lib" \
--copt="-DENVOY_SSL_VERSION=\"OpenSSL_1_1_1\"" \
//source/exe:envoy-static
cd istio-proxy
bazel build \
--override_repository="com_github_google_jwt_verify=$HOME/repos/jwt_verify_lib" \
--override_repository="envoy=$HOME/repos/envoy" \
--copt="-DENVOY_SSL_VERSION=\"OpenSSL_1_1_1\"" \
//...
The last one might still not build, but it's not a blocking issue for now.