Wiki

Clone wiki

cef / AutomatedBuildSetup

This Wiki page describes the setup of an automated build system for CEF.

Note to Editors: Changes made to this Wiki page without prior approval via the CEF Forum or Issue Tracker may be lost or reverted.



Overview

CEF provides tools for automatically downloading, building and packaging Chromium and CEF source code. These tools are the recommended way of building CEF locally and can also be integrated with an automated build system. Examples of automated build systems for CEF include http://opensource.spotify.com/cefbuilds/index.html hosted by Spotify. This document outlines how to build a system similar to Spotify's. For general CEF build instructions see the BranchesAndBuilding Wiki page.

Services

An automated build system tracks changes to a CEF branch, automatically builds a new version of CEF when changes are detected, and uploads the resulting artifacts (CEF binary files) to an Internet or Intranet Website.

The following high-level services are necessary for an automated build system:

  1. Build execution service. Used to perform the CEF automated builds using agents running on different OS platforms. For example, a build system like TeamCity or Jenkins.
  2. Artifact storage service. Used to store artifacts referenced and generated by the build agents. For example, a shared file system or a storage system like Artifactory.
  3. Artifact publishing service. Used to publish artifacts for download on the Intranet or Internet. Generally provides some form of distributed caching. For example, a cloud provider like Amazon S3 + Cloudfront or Google Cloud Storage.

Setup of these specific services is outside the scope of this document. However, we will refer to the capabilities of these services in generic terms.

Execution Steps

The reference build system works by executing the following steps in order:

1. Change Trigger

Runs automatically at some interval (for example, once daily) to check if a new change has been committed to the CEF repository. The Bitbucket REST API exposes an endpoint that can be queried to identify the most recent commit in a branch. For example, this will return the most recent commit for 2785 branch:

https://api.bitbucket.org/2.0/repositories/chromiumembedded/cef/commit/2785

The resulting "hash" value can then be compared to the last successful CEF build version to identify if a new build is required (e.g. a hash value of "1e15b7626244fa6c6455b7a24552664e4cc14fd3" from branch 2785 will match a build artifact file named "cef_binary_3.2785.*.g1e15b76_*.tar.bz2").

If artifacts are stored on a shared file system then languages like Python can access the file list directly. If artifacts are stored in Artifactory then see the Artifactory REST API documentation for instructions on how to retrieve a directory listing.

2. Platform Builder

Initiated by the change trigger when a new commit is detected. A separate platform builder is configured for each supported build platform (linux32, linux64, macosx64, windows32, windows64). It executes a build on that platform and then uploads the resulting build artifact files to the storage service (shared file system, Artifactory, etc.).

See the "Platform Build Configurations" section below for recommended platform build configurations.

3. Artifact Publisher

Initiated by the platform builder upon a successful build. It retrieves the list of build artifact files from the storage service and uses the cefbuilds tools to compare against the index.json file generated by the previous run. When changes are detected it uploads the new changes to the publishing service (Amazon S3, GCS, etc.). The new index.json file is then stored for future reference.

In addition to generating index.json files the cefbuilds tools can also generate the index.html file used by the Spotify builder (provide your own custom branding to cef_html_builder.py via the "branding" parameter). See the cef_json_builder_example.py and cef_html_builder.py files from the cefbuilds directory for sample usage.

Platform Build Configurations

The following configurations are executed by the platform builder step. These are the recommended configurations for externally published artifacts but are not recommended for local builds. See the the BranchesAndBuilding Wiki page for general build instructions.

These instructions are intended for use with the currently supported release branches. This Wiki page will be updated as build requirements change.

Lower-case values between %% (%download_dir%, %cef_branch%, etc.) must be provided by you.

Linux Configuration

What's Required

  • Linux Build Requirements as listed on the BranchesAndBuilding Wiki page.
  • At least 6GB of RAM and 40GB of free disk space.

The following commands can also be executed in a chroot environment.

64-bit Build Commands

To build 64-bit CEF on a 64-bit Linux host system:

apt-get install aptitude
aptitude -y update
DEBIAN_FRONTEND=noninteractive aptitude -y install bison build-essential cdbs curl devscripts dpkg-dev elfutils fakeroot flex g++ git-core git-svn gperf libapache2-mod-php5 libasound2-dev libav-tools libbrlapi-dev libbz2-dev libcairo2-dev libcap-dev libcups2-dev libcurl4-gnutls-dev libdrm-dev libelf-dev libexif-dev libffi-dev libgconf2-dev libgconf-2-4 libgl1-mesa-dev libglib2.0-dev libglu1-mesa-dev libgnome-keyring-dev libgtk2.0-dev libkrb5-dev libnspr4-dev libnss3-dev libpam0g-dev libpci-dev libpulse-dev libsctp-dev libspeechd-dev libsqlite3-dev libssl-dev libudev-dev libwww-perl libxslt1-dev libxss-dev libxt-dev libxtst-dev mesa-common-dev openbox patch perl php5-cgi pkg-config python python-cherrypy3 python-crypto python-dev python-psutil python-numpy python-opencv python-openssl python-yaml python3 python3-cherrypy3 python3-crypto python3-dev python3-psutil python3-numpy python3-opencv python3-openssl python3-yaml rpm ruby subversion ttf-dejavu-core ttf-indic-fonts ttf-kochi-gothic ttf-kochi-mincho fonts-thai-tlwg wdiff wget zip
export GN_DEFINES="is_official_build=true use_sysroot=true use_allocator=none symbol_level=1 is_cfi=false use_thin_lto=false"
export CEF_ARCHIVE_FORMAT=tar.bz2
automate-git.py --download-dir=%download_dir% --branch=%cef_branch% --minimal-distrib --client-distrib --force-clean --x64-build --build-target=cefsimple

32-bit Build Commands

To build 32-bit CEF on a 64-bit Linux host system:

dpkg --add-architecture i386
apt-get install aptitude
aptitude -y update
DEBIAN_FRONTEND=noninteractive aptitude -y install bison build-essential cdbs curl devscripts dpkg-dev elfutils fakeroot flex g++ git-core git-svn gperf libapache2-mod-php5 libasound2-dev libav-tools libbrlapi-dev libbz2-dev libcairo2-dev libcap-dev libcups2-dev libcurl4-gnutls-dev libdrm-dev libelf-dev libexif-dev libffi-dev libgconf2-dev libgl1-mesa-dev libglib2.0-dev libglu1-mesa-dev libgnome-keyring-dev libgtk2.0-dev libkrb5-dev libnspr4-dev libnss3-dev libpam0g-dev libpci-dev libpulse-dev libsctp-dev libspeechd-dev libsqlite3-dev libssl-dev libudev-dev libwww-perl libxslt1-dev libxss-dev libxt-dev libxtst-dev mesa-common-dev openbox patch perl php5-cgi pkg-config python python-cherrypy3 python-crypto python-dev python-psutil python-numpy python-opencv python-openssl python-yaml python3 python3-cherrypy3 python3-crypto python3-dev python3-psutil python3-numpy python3-opencv python3-openssl python3-yaml rpm ruby subversion ttf-dejavu-core ttf-indic-fonts ttf-kochi-gothic ttf-kochi-mincho fonts-thai-tlwg wdiff wget zip lib32gcc1 lib32stdc++6 libc6-i386 linux-libc-dev:i386 libasound2:i386 libcap2:i386 libelf-dev:i386 libfontconfig1:i386 libgconf-2-4:i386 libglib2.0-0:i386 libgpm2:i386 libgtk2.0-0:i386 libgtk-3-0:i386 libncurses5:i386 libnss3:i386 libpango1.0-0:i386 libssl1.0.0:i386 libtinfo-dev:i386 libxcomposite1:i386 libxcursor1:i386 libxdamage1:i386 libxi6:i386 libxrandr2:i386 libxss1:i386 libxtst6:i386
export GN_DEFINES="is_official_build=true use_sysroot=true use_allocator=none symbol_level=1"
export CEF_ARCHIVE_FORMAT=tar.bz2
automate-git.py --download-dir=%download_dir% --branch=%cef_branch% --minimal-distrib --client-distrib --force-clean --build-target=cefsimple

ARM Build Commands

To build ARM CEF on a 64-bit Linux host system start with the 32-bit Build Commands and make the following changes:

1. Add the following packages to the install command:

binutils-arm-linux-gnueabihf g++-arm-linux-gnueabihf libc6-dev-armhf-cross linux-libc-dev-armhf-cross g++-5-multilib

Note: The gcc-multilib package conflicts with the arm cross compiler (at least in trusty) but g++-X.Y-multilib gives the necessary 32-bit support. The above example uses g++-5-multilib which may not exist on your system. Find out the appropriate value of X and Y by running this command:

apt-cache depends g++-multilib --important | grep -E --color=never --only-matching '\bg\+\+-[0-9.]+-multilib\b

2. Define the following environment variables before running automate-git.py:

export GYP_DEFINES=target_arch=arm
export CEF_INSTALL_SYSROOT=arm

3. Add the --arm-build flag to the automate-git.py command-line.

ARM64 Build Commands

To build ARM64 CEF on a 64-bit Linux host system start with the 32-bit Build Commands and make the following changes:

1. Add the following packages to the install command:

binutils-aarch64-linux-gnu g++-arm-linux-gnueabihf libc6-dev-armhf-cross linux-libc-dev-armhf-cross g++-5-multilib

See the above note about the gcc-multilib package.

2. Define the following environment variables before running automate-git.py:

export GYP_DEFINES=target_arch=arm64
export CEF_INSTALL_SYSROOT=arm64

3. Add the --arm64-build flag to the automate-git.py command-line.

Mac OS X Configuration

What's Required

  • macOS Build Requirements as listed on the BranchesAndBuilding Wiki page.
  • At least 8GB of RAM and 40GB of free disk space.

If Xcode is not installed to the default location then set the correct location using the xcodeselect tool.

64-bit Build Commands

To build 64-bit CEF on a 64-bit OS X host system:

export GN_DEFINES=is_official_build=true
export CEF_ARCHIVE_FORMAT=tar.bz2
automate-git.py --download-dir=%download_dir% --branch=%cef_branch% --minimal-distrib --client-distrib --force-clean --x64-build

Windows Configuration

What's Required

  • Visual Studio and Windows SDK versions as listed on the BranchesAndBuilding Wiki page.
  • At least 8GB of RAM and 40GB of free disk space.

Install the required Visual Studio sub-components by passing the following arguments to the Visual Studio installer:

$ PATH_TO_INSTALLER.EXE ^
--add Microsoft.VisualStudio.Workload.NativeDesktop ^
--add Microsoft.VisualStudio.Component.VC.ATLMFC ^
--includeRecommended

If building for ARM64 install the following additional sub-components:

$ PATH_TO_INSTALLER.EXE ^
--add Microsoft.VisualStudio.Component.VC.Tools.ARM64 ^
--add Microsoft.VisualStudio.Component.VC.MFC.ARM64 ^
--includeRecommended

If Visual Studio or Windows SDK is not installed to the default location then set environment variables like the following before executing automate-git.py:

set WIN_CUSTOM_TOOLCHAIN=1
set CEF_VCVARS=none
set GYP_MSVS_OVERRIDE_PATH=%vs_root%
set VS_CRT_ROOT=%vs_crt_root%
set SDK_ROOT=%sdk_root%
set PATH=%sdk_root%\bin\%sdk_version%\x64;%vs_root%\VC\Tools\MSVC\%vc_tools_version%\bin\HostX64\x64;%vs_root%\VC\Redist\MSVC\%vc_redist_version%\x64\%vc_redist_crt%;%vs_root%\SystemCRT;%PATH%
set LIB=%sdk_root%\Lib\%sdk_version\um\%arch%;%sdk_root%\Lib\%sdk_version\ucrt\%arch%;%vs_root%\VC\Tools\MSVC\%vc_tools_version%\lib\%arch%;%vs_root%\VC\Tools\MSVC\%vc_tools_version%\atlmfc\lib\%arch%;%LIB%
set INCLUDE=%sdk_root%\Include\%sdk_version%\um;%sdk_root%\Include\%sdk_version%\ucrt;%sdk_root%\Include\%sdk_version%\shared;%vs_root%\VC\Tools\MSVC\%vc_tools_version%\include;%vs_root%\VC\Tools\MSVC\%vc_tools_version%\atlmfc\include;%INCLUDE%

The correct PATH, LIB and INCLUDE values for your installation can be determined by running the vcvarsall.bat script and passing in the desired architecture and SDK version.

See comments in gclient_hook.py for Windows custom toolchain requirements. VS + SDK can be packaged for distribution to build agents using a script like Chromium's package_from_installed.py.

32-bit Build Commands

To build 32-bit CEF on a 64-bit Windows host system:

set GN_DEFINES=is_official_build=true
set GYP_MSVS_VERSION=2019
set CEF_ARCHIVE_FORMAT=tar.bz2
automate-git.py --download-dir=%download_dir% --branch=%cef_branch% --minimal-distrib --client-distrib --force-clean

64-bit Build Commands

To build 64-bit CEF on a 64-bit Windows host system:

set GN_DEFINES=is_official_build=true
set GYP_MSVS_VERSION=2019
set CEF_ARCHIVE_FORMAT=tar.bz2
automate-git.py --download-dir=%download_dir% --branch=%cef_branch% --minimal-distrib --client-distrib --force-clean --x64-build

ARM64 Build Commands

To build ARM64 CEF on a 64-bit Windows host system:

set CEF_ENABLE_ARM64=1
set GN_DEFINES=is_official_build=true target_cpu=arm64
set GYP_MSVS_VERSION=2019
set CEF_ARCHIVE_FORMAT=tar.bz2
automate-git.py --download-dir=%download_dir% --branch=%cef_branch% --minimal-distrib --client-distrib --force-clean --arm64-build

Updated