HTTPS SSH

Dullahan

What is it?

A headless browser SDK that uses the Chromium Embedded Framework (CEF). It is designed to make it easier to write applications that render modern web content directly to a memory buffer, inject synthesized mouse and keyboard events as well as interact with features like JavaScript or cookies.

Important note:

This is a very early release - more than alpha quality but less than beta. I have high hopes this will develop into something useful but there is still a lot of work to do first.

What could I use it for?

It might be useful in a number of situations - for example:

  • Anywhere you want to render web content in a graphics application - e.g. help pages in an OpenGL based game
  • VR based video chat and shared environments (I'm working on an app like this now)
  • HTML/JavaScript based user interfaces E.G. a tablet based paradigm for VR
  • Integration with JavaScript for automated site testing like PhantomJS
  • Web page contents analysis - E.G:
    • Capture images the most popular 500 web pages and count the top 5 colors in each.
    • Capture a Google Maps traffic map of where you live for 24 hours and see how really miserable your commute is :)
  • Web page capture to an image or a PDF file
  • Lots more I haven't thought of yet

There are some examples included in this repository and there is a description and a screen shot for each here.

What is CEF?

From the CEF Wiki page: The Chromium Embedded Framework (CEF) is an open source framework for embedding a web browser engine based on the Chromium core.

An essential site to visit when developing a CEF based application is the CEF forum started and run by (I believe) the original CEF author Marshall Greenblatt who is very generous with his time and expertise.

Building CEF locally is quite a dauting task and needs a toolset I do not have on my computer (E.G. Visual Studio 2015) so I used the builds that Spotify kindly hosts here.

Which version of CEF does Dullahan use?

Dullahan currently builds against the CEF 3.3202.1686.gd665578 / Chromium 62 build of CEF but with some small tweaks to take account of CEF interface changes etc. it should work with slightly older or more recent versions too.

What are the origins of this project?

Historically, the first incarnation was LLCEFLib - a Linden Lab internal library to render web content content inside Second Life. LLCEFLib had to support the outdated 32bit macOS version as well as many Second Life specific features so the code became very disjointed and messy. This effort is an attempt to unburden the API from those requirements and provide a more generic version which can be useful in other applications.

Which platforms are supported?

  • Windows 32/64 bit (primary development platform)
  • macOS 64 bit (works, is supported but needs a little work - any macOS devs out there want to help?)
  • Linux 64 bit (not available yet but hopefully one day - any Linux devs want to help?)

Which version of the C++ runtime library does it use on Windows?

For historical reasons, both Dullahan and the version of CEF it consumes are built with the C++ runtime library set to Multi-threaded DLL / Multi-threaded Debug DLL.

Is there a binary version I can download?

Not right now. I used to store a pre-built version on the Windows 64 bit version in my DropBox but that became untenable. Once I figure out a sensible place to store a large binary package, I'll bundle up the latest SDK and examples for all the platforms.

What are the system requirements to build CEF and Dullahan?

  • Windows
    • Microsoft Visual Studio 2013 update 4
    • CMake version 3.6.3 or later
    • 7-Zip 7-Zip or a similar tool that can uncompress .tar.bz2 files
  • macOS
    • Xcode version 8.2.1 or later
    • CMake version 3.6.3 or later

How do I build CEF?

Before you can build Dullahan, you will need to build a version of CEF. There are some scripts to help - a batch file for Windows and a shell script for macOS. The macOS version is somewhat automated but the lack of built-in tools in Windows to download and uncompress .tar.bz2 files means it needs some manual steps up front.

  • Windows
    • Grab a Windows 32 or 64 bit build of CEF from http://opensource.spotify.com/cefbuilds/index.html
    • Extract the archive to a local folder - I use the Windows Desktop
    • Edit the tools/make_dullahan_cef_pkg.bat batch file - specifically, the lines at the top of the file that set the SRC_DIR_32/64 (location of uncompressed CEF package you just downloaded) and DST_DIR_32/64 (location where you want to generate a build).
    • From a command prompt, run tools\make_dullahan_cef_pkg.bat with a parameter of either 32 or 64 to set the bit-width to build
    • If all goes well, you will end up with a set of libs, headers and binary files in the DST_DIR_32/64 folder that you can use to build Dullahan
  • macOS
    • Edit the tools/make_dullahan_cef_pkg.sh shell script and make sure the CEF_BUILD variable is set to the version you want and make sure the DST_DIR points to somewhere sensible. This is where the build will be generated.
    • From a terminal prompt, run tools/make_dullahan_cef_pkg.sh. This will download the right version of CEF, build it and create a set of libs, frameworks, headers etc. that you need to build Dullahan

Do I need to build Chromium?

The easy answer is no - CEF builds against Chromium but the Spotify builds already do that for you. However, you can build Chromium yourself and then build CEF against that in the same way as Spotify does - you might need to do this if you had to change the way part of Chromium works or more likely, you wanted a version of Chromium that has the proprietary media codec support turned on. The Dullahan project used to do this but the toolset required to build Chromium passed what I had available to me (Visual Studio 2015 is a dependency for recent builds) so we stopped doing it. The remnants of a batch file can be found in here - some pieces of that may still be useful. There is also a CEF wiki page that has lots of details too.

How do I build Dullahan?

  • Download or clone the Dullahan source code
  • Windows
    • Edit the tools\build_win.bat filer and change the CEF_32_DIR and CEF_64_DIR variables at the top to point to the appropriate build of CEF you just built.
    • From a command prompt, run tools/build_win.bat with a parameter of either 32 or 64 to set the bit-width to build
    • If Dullahan builds correctly, the example application in Release/examples/webcube will start
    • You can open the Visual Studio 2013 solution file in build or build64 as normal and edit files if you want to experiment
  • macOS
    • Edit the tools/build_mac.sh file and edit the cef_base_dir variable to point to the version of CEF you just built.
    • From a terminal prompt, run tools/build_mac.sh.
    • If it builds correctly, you can try to the run the included macOS example application (open build64/Release/osxgl.app)
    • You can open the Xcode 8 project file build64/dullahan.xcodeproj and edit files if you want to experiment

Which files do I need in my own project?

The reason to use Dullahan vs raw CEF in your application is that you merely include the dullahan.h header file in your application and link against the Dullahan, libcef and libcef_dll_wrapper static libraries. Then, copy over the various CEF/Chromium runtime files alongside your application - see the post-build step for the examples in Visual Studio or the build_mac.sh script for more details.

TODO: Create a batch file and shell script that generates a CMake file which creates an empty project with everything set up correctly.

How do I use it?

The short answer is look at the code in the examples folder and dullahan.h. Generally speaking, you create an instance of the library, call init() and regularly call the update() function in your own message loop. You can hook up a callback to be notified when the contents of the page changes and get access to the raw pixels. You can synthesize mouse and keyboard events and send them into the page. Finally, when you want to exit, make sure the onRequestExit callback is hooked up and call the requestExit() method. When the library and CEF have cleaned everything up, your callback will be triggered and you can call shutdown() before exiting normally.

Are there examples?

Why yes there is - take a look in the examples subdirectory of this repo. There are some screenshots and short descriptions in the README file too

What is left to do before the first real release?

  • Implement proper Dullahan <--> JavaScript interoperability
  • Improve the documentation
  • Make sure all the features are used in the examples to help understand usage
  • Unit tests
  • Add support for different pages on each side of the Web Cube example
  • Lots and lots more... please help if you can.

Alternatives

  • The most exciting alternative in this space is from Google themselves - headless Chromium appears to offer much of what Dullahan does albeit better because they have more than one person working on it :) It's very new and still developing but definitely worth a look and will likely supersede Dullahan one day.

Why the name?

Seemed appropriate