HTTPS SSH

Benchmark for gcc, musl and clang

This is a benchmark to measure the performance of GCC against musl and clang using various flags. This repo derived from some thoughts I've written in this post. Have in mind that not all the c files that can be compiled in gcc can also be compiled with musl or clang.

Kudos

The bzip2.c, oggenc.v and gcc.c files are single compilation unit C programs that I've found here

The musl-clang script is taken from here

Usage

To use the benchmark just run the script with the c file that you want to test. ./benchmark.sh file.c

You can also add more compilation flags like that: ./benchmark.sh oggenc.c -lm

You can either use one of the included test c files or any other file you want.

By default all the compiled executables are export in the output/ folder which is created from the script and at the end of the benchmark this folder is deleted. If you want to keep the resulted executables for testing the binaries then use the KEEP parameter like this: KEEP=true ./benchmark.sh bzip2.c

Results

These are the results for the included test files

test.c
./benchmark.sh test.c
Testing file: test.c
gcc            :  8976
gcc -Os        :  8984
gcc -O3        :  8984
gcc -flto      :  8968
gcc -flto -Os  :  8920
gcc -flto -O3  :  8920
musl           :  7792
musl -Os       :  7792
musl -O3       :  7792
musl -flto     :  7784
musl -flto -Os :  7728
musl -flto -O3 :  7728
clang          :  7664
clang -Os      :  7800
clang -O3      :  7832
clang+musl     :  4792
clang+musl -Os :  4896
clang+musl -O3 :  4928
bzip2.c
./benchmark.sh bzip2.c
Testing file: bzip2.c
gcc            :  138008
gcc -Os        :  79320
gcc -O3        :  115912
gcc -flto      :  130448
gcc -flto -Os  :  71456
gcc -flto -O3  :  107760
musl           :  136376
musl -Os       :  73512
musl -O3       :  114304
musl -flto     :  128840
musl -flto -Os :  69728
musl -flto -O3 :  106152
clang          :  129112
clang -Os      :  94568
clang -O3      :  113504
clang+musl     :  125840
clang+musl -Os :  90824
clang+musl -O3 :  109968
oggenc.c
./benchmark.sh oggenc.c -lm
Testing file: oggenc.c -lm
gcc            :  2147072
gcc -Os        :  2028656
gcc -O3        :  2179880
gcc -flto      :  2140944
gcc -flto -Os  :  1974736
gcc -flto -O3  :  2067600
musl           :  2141096
musl -Os       :  2022552
musl -O3       :  2173776
musl -flto     :  2134952
musl -flto -Os :  1972976
musl -flto -O3 :  2069824
clang          :  2112544
clang -Os      :  2020600
clang -O3      :  2116544
clang+musl     :  2107952
clang+musl -Os :  2016264
clang+musl -O3 :  2110528

##### bzip2.c

./benchmark.sh bzip2.c Testing file: bzip2.c gcc : 138008 gcc -Os : 79320 gcc -O3 : 115912 gcc -flto : 130448 gcc -flto -Os : 71456 gcc -flto -O3 : 107760 musl : 136376 musl -Os : 73512 musl -O3 : 114304 musl -flto : 128840 musl -flto -Os : 69728 musl -flto -O3 : 106152 clang : 129112 clang -Os : 94568 clang -O3 : 113504 clang+musl : 125840 clang+musl -Os : 90824 clang+musl -O3 : 109968

##### gcc.c

./benchmark.sh gcc.c Testing file: gcc.c gcc : 6879424 gcc -Os : 4263096 gcc -O3 : 6493624 gcc -flto : 6772760 gcc -flto -Os : 3886664 gcc -flto -O3 : 5889240 musl : failed musl -Os : failed musl -O3 : failed musl -flto : failed musl -flto -Os : failed musl -flto -O3 : failed clang : 7446328 clang -Os : 4785560 clang -O3 : 6537192 clang+musl : failed clang+musl -Os : failed clang+musl -O3 : failed ```