Enable sccache

Jump to: navigation, search

Enable Sccache in Your Project

Introduction

Sccache is similar to ccache which allows caching of build artifacts so that on a rebuild if sources have not changed, then they can be accessed from the cache. In some cases, this can lower rebuild times by more than 85% saving you time, allowing more jobs to proceed in OBS/IBS, and saving energy in our build farms.

Sccache supports caching of C, C++ and Rust, making it a "drop in" replacement to ccache. However, you must be careful to ensure that your environment variables and paths are identical when used, else sccache can not work correctly.

To enable this support in sccache, you need to edit your project config (not the package config).

CLI (osc)

You can do this with the command

osc meta prjconf <name of project> -e

Inside this, you need to specify the type of cache and the names of the packages to apply this to.

BuildFlags: ccachetype:sccache
BuildFlags: useccache:<package name>
# if the package uses multibuild you need to specify the flavors
BuildFlags: useccache:<package name>:<multibuild flavor>

You can specify useccache multiple times. For example, to apply this to the projects kanidm and nss_synth, the config would look like:

BuildFlags: ccachetype:sccache
BuildFlags: useccache:kanidm
BuildFlags: useccache:nss_synth

Webui

Navigate to your project page and select "Project Config". Then you can specify the buildflags the same as the cli.


Checking it works

You can see that this works by inspecting your buildlog. If working at the start of the build you should see:

[   27s] Unpacking sccache archive
[   28s] Compile requests                      0
[   28s] Compile requests executed             0
[   28s] Cache hits                            0
[   28s] Cache misses                          0
[   28s] Cache timeouts                        0
[   28s] Cache read errors                     0
[   28s] Forced recaches                       0
[   28s] Cache write errors                    0
[   28s] Compilation failures                  0
[   28s] Cache errors                          0
[   28s] Non-cacheable compilations            0
[   28s] Non-cacheable calls                   0
[   28s] Non-compilation calls                 0
[   28s] Unsupported compiler calls            0
[   28s] Average cache write               0.000 s
[   28s] Average cache read miss           0.000 s
[   28s] Average cache read hit            0.000 s
[   28s] Failed distributed compilations       0
[   28s] Cache location                  Local disk: "/.sccache"
[   28s] Cache size                          177 MiB
[   28s] Max cache size                        1 GiB

Near the end of the build you will see stats about how effective the cache was.

[  621s] Compile requests                    963
[  621s] Compile requests executed           680
[  621s] Cache hits                           13
[  621s] Cache hits (C/C++)                   13
[  621s] Cache misses                        660
[  621s] Cache misses (C/C++)                516
[  621s] Cache misses (Rust)                 144
[  621s] Cache timeouts                        0
[  621s] Cache read errors                     0
[  621s] Forced recaches                       0
[  621s] Cache write errors                    0
[  621s] Compilation failures                  5
[  621s] Cache errors                          2
[  621s] Cache errors (C/C++)                  2
[  621s] Non-cacheable compilations            0
[  621s] Non-cacheable calls                 114
[  621s] Non-compilation calls               169
[  621s] Unsupported compiler calls            0
[  621s] Average cache write               0.001 s
[  621s] Average cache read miss           1.157 s
[  621s] Average cache read hit            0.000 s
[  621s] Failed distributed compilations       0
[  621s] 
[  621s] Non-cacheable reasons:
[  621s] crate-type                           51
[  621s] -E                                   32
[  621s] -                                    21
[  621s] argument parse                       10
[  621s] 
[  621s] Cache location                  Local disk: "/.sccache"
[  621s] Cache size                          149 MiB
[  621s] Max cache size                        1 GiB

This was from an "initial" build so we can see the cache misses are "high". If we rebuild, the cache will take effect and we can see that in the statistics. If you are seeing cache misses frequently on rebuilds, this may be a sign your environment variables are changing. You can dump these with the "env" command in your percent build phase to investigate.

A hot build should look like the following:

[  249s] Compile requests                    963
[  249s] Compile requests executed           680
[  249s] Cache hits                          673
[  249s] Cache hits (C/C++)                  529
[  249s] Cache hits (Rust)                   144
[  249s] Cache misses                          0
[  249s] Cache timeouts                        0
[  249s] Cache read errors                     0
[  249s] Forced recaches                       0
[  249s] Cache write errors                    0
[  249s] Compilation failures                  5
[  249s] Cache errors                          2
[  249s] Cache errors (C/C++)                  2
[  249s] Non-cacheable compilations            0
[  249s] Non-cacheable calls                 114
[  249s] Non-compilation calls               169
[  249s] Unsupported compiler calls            0
[  249s] Average cache write               0.000 s
[  249s] Average cache read miss           0.000 s
[  249s] Average cache read hit            0.002 s
[  249s] Failed distributed compilations       0
[  249s] 
[  249s] Non-cacheable reasons:
[  249s] crate-type                           51
[  249s] -E                                   32
[  249s] -                                    21
[  249s] argument parse                       10
[  249s] 
[  249s] Cache location                  Local disk: "/.sccache"
[  249s] Cache size                          149 MiB
[  249s] Max cache size                        1 GiB

We can also see a difference in the execution time (621s cold vs 249s hot)