Wiki

Clone wiki

glLoadGen / New_Style_Step_By_Step

Making a style is a somewhat complex process. This will be a step-by-step guide to using the system to make a part of a style. The style we will create will be a simple C-style function loader.

Initial setup

All of the Lua scripts should go in the modules folder. For this demonstration, we will create two files: one for the style and one for the structure. The style will go into a file called StyleTest.lua, while the structure will go into StructTest.lua.

The basic boilerplate for the structure is very simple:

local struct = require "Structure"
local common = require "CommonStruct"

local my_struct = 
{
}

my_struct = struct.BuildStructure(my_struct)
return my_struct

The first two statements include the Structure module, which allows us to build our structure, and the CommonStruct module, which provides some commonly used structural elements.

my_struct is the base table for our structure. It's currently empty, but we'll work on that as we progress.

Notice that what is returned is the built structure, which is a table. Lua's module loading system will store this, so that the structure will only be built once.

The boilerplate for our style is fairly simple as well:

local util = require "util"
local struct = require "StructTest"
local common = require "CommonStyle"

local my_style = {}

local function Create()
    return util.DeepCopyTable(my_style), struct
end

return { Create = Create }

The util module contains some basic utility functions that we will use. The style is what decides which structure it uses, so we include our structure's module table. And the CommonStyle are functions commonly used by styles.

my_style is the base style table. It's empty; we will fill it in through this demonstration.

The return value is a table that exports a single function: Create. The processing system expects Create to return both a style and a structure. So we perform a table copy of our style and return the structure as presented.

There is one final step before we begin: we must hook our style into the glLoadGen system. To do this, we create a file called UserStyle.lua. This file should go in the modules directory with the rest. All it does is return a table containing any additional styles.

In our case, we will do this:

return
{
	test = require("StyleTest")
}

The key name "test" here is the command-line name for the style. So to execute our style, we use the -style=test parameter, with an appropriate set of other options.

Header

Our C-style loader will generate two files: a header and a source file. So our first major step is to generate the header file.

The header file and source files will have to do two very different things. The header file will contain function pointers, extension variables and #defines for enumerators that the user will use. But since this is C, variables in headers must be declared with extern. So the declaration in the headers has to be different from the definitions in the source files.

Therefore, we are going to split our main style (my_style) into two sub-styles: hdr and src. hdr contains the writing logic for header-specific issues, and the same goes for src for source-specific issues.

So let's start doing that. In our style file, we now have this:

local my_style = {}

local hdr = {}
my_style.hdr = hdr

local src = {}
my_style.src = src

Everything before and after this looks the same as before. my_style will contain functions that both the src and hdr sub-styles share.

Now that we have divided our style, we now must start building our structure. We need to generate a header file. So we use the file structure action:

local my_struct = 
{
  { type="file", style="hdr", name="GetFilename(basename, spec, options)",
  },
}

The style attribute in the file action says to use the substyle named hdr, which we have defined in our main style. The function GetFilename now must be placed into our style. We could put it in either my_style or hdr and the system would find it. But since it is header-specific, it should go into hdr; that's the whole point of splitting them up to begin with.

So, in our style, we add a function to get the appropriate filename:

local hdr = {}
my_style.hdr = hdr

function hdr.GetFilename(basename, spec, options)
  return basename .. ".h"
end

The variable basename contains all of the pathing information, as well as the specific name that the user requested. All we need to do is add an appropriate extension. We could decorate the name with anything as we see fit, but we only need to add an extension.

You should be able to run this and have an empty file be generated.

Every header in C or C++ should have include guards; this is standard practice and is very important. Include guards effectively have scope: the #ifndef/#define at the top, and the #endif at the bottom. So we will handle this in our structure using the block action:

local my_struct = 
{
  { type="file", style="hdr", name="GetFilename(basename, spec, options)",
    { type="block", name="IncludeGuard",
    },
  },
}

Here, we don't provide a parameter list, so the block's default parameter list is used. The default for block actions is (hFile, spec, options).

block actions will call two different functions. In our case, it will call WriteBlockBeginIncludeGuard before executing any child actions, and WriteBlockEndIncludeGuard after writing all child actions. There are no child actions as of yet, so we will just see include-guards guarding nothing.

To actually write our beginning and ending include guards, we need some style code. The first code we need is a function that will compute the include guard name.

local function GetIncludeGuard(spec, options)
  local temp = 
    options.prefix .. spec.GetIncludeGuardString() .. "_THIS_IS_A_TEST_H"
  return temp:upper()
end

Note that this function takes spec and options, and that it uses both of them. The reason for this goes back to some of the responsibilities that styles must fulfill. Remember that styles must allow the user to specify a prefix to allow two different invocations of the same style with the same specification to coexist in a program. So our include guard need to incorporate the user's specified prefix. Also it includes a string generated by the spec, which will differ between OpenGL, WGL, and GLX. That way, you can include the GL generated header in the same source file as the WGL generated one.

Now that we have a function that computes the include guard, we simply need to write it:

function hdr.GetFilename(basename, spec, options)
  return basename .. ".h"
end

function hdr.WriteBlockBeginIncludeGuard(hFile, spec, options)
  local guard = GetIncludeGuard(spec, options)
  hFile:fmt("#ifndef %s\n", guard)
  hFile:fmt("#define %s\n", guard)
end

function hdr.WriteBlockEndIncludeGuard(hFile, spec, options)
  hFile:fmt("#endif /*%s*/\n", GetIncludeGuard(spec, options))
end

Note the use of hFile:fmt, which works like a form of printf for Lua files. This is provided by the TabbedFile.lua module.

Header preamble

The next steps are very specific to writing OpenGL loading systems. If your intent is to generate something else with this system, then you won't need this preamble stuff.

First, we need to recognize the intent of our loading system. Like most OpenGL loading libraries, we expect our header to completely replace all uses of gl.h (though not WGL or GLX's headers). Because of that, we need to make sure that the user has not included gl.h or other similar files first. Also, if they try to include gl.h afterwards, we want it to not be included.

Similarly, there are wglext.h and glxext.h files (available from the OpenGL.org registry) that do the same job as our loaders. We want to stop users from including them too.

To do this, we need to put some #defines into our header. What we want to do is #define the include guards that these file use, so that they think they've already been included. And if we detect that they were already defined, we issue a #error to halt compilation.

The hard work has been done for us; the function spec.GetHeaderInit() will get the appropriate header #defines that perform these tricks. All we need to do is write what this function returns.

To do that, we add a write action to our structure:

local my_struct = 
{
  { type="file", style="hdr", name="GetFilename(basename, spec, options)",
    { type="block", name="IncludeGuard",
      { type="write", name="Guards(hFile, spec, options)", },
    },
  },
}

The function WriteGuards will be called. As a header-specific function, it goes into hdr. To write it, we simply do this in our style:

function hdr.WriteGuards(hFile, spec, options)
  hFile:rawwrite(spec.GetHeaderInit())
end

The rawwrite command ignores all indentation in the TabbedFile and simply directly writes the block of text to the output.

The next step is to write some necessary definitions. Our system is designed to replace gl.h and the platform-specific extension headers. However, these headers do more than just declare some functions. They also define typedefs, things like GLuint, GLenum, and so forth. These are crucial, and they must be defined before you can start declaring function pointers and such. If you're writing a C++ loader, it's probably best to not stick them in a namespace.

OpenGL versions have increased the number of typedefs over the years. And we will need to write these typedefs into our headers. If you were writing a non C/C++-based loader, you would have quite a time figuring out how to define these typedefs and such for your platform of interest, since the .spec files are geared towards C/C++. But you would work something out, based on the definitions in the OpenGL specification and your platform of interest.

Given that we're writing a C loader, we have an easier time; the hard work has been done for us. The typedefs exists as blocks of text waiting to be thrown into a header. The only thing we need is to simply regurgitate the information into the file of interest.

To do this, we need to add another write action to our style:

local my_struct = 
{
  { type="file", style="hdr", name="GetFilename(basename, spec, options)",
    { type="block", name="IncludeGuard",
      { type="write", name="Guards(hFile, spec, options)", },
      { type="blank" },
      { type="write", name="Typedefs(hFile, specData, spec, options)",},
    },
  },
}

Note that WriteTypedefs takes the specData parameter. That's because some of the data we need to write lives in the specification data and some of it lives in the spec.

Also, note that we use a blank to insert a blank line between them. This is just for clarity.

As for writing the data, here is how we do it in the style:

function hdr.WriteTypedefs(hFile, specData, spec, options)
  local defArray = common.GetStdTypedefs()

  hFile:write("#ifndef GL_LOAD_GEN_BASIC_OPENGL_TYPEDEFS\n")
  hFile:write("#define GL_LOAD_GEN_BASIC_OPENGL_TYPEDEFS\n")
  hFile:write("\n")
  hFile:inc()
	
  for _, def in ipairs(defArray) do
    hFile:write(def)
  end
	
  hFile:dec()
  hFile:write("\n")
  hFile:write("#endif /*GL_LOAD_GEN_BASIC_OPENGL_TYPEDEFS*/\n")
  hFile:write("\n")

  common.WritePassthruData(hFile, specData.funcData.passthru)
end

This is a bit complicated. common.GetStdTypedefs() is where the standard typedefs for the OpenGL 1.1 types live. We wrap them in their own set of include guards to ensure that they never get defined multiple times; all of these definitions will be used for our OpenGL, WGL, and GLX headers, which need to co-exist with one another.

Once we have that, we use the function common.WritePassthruData to write the spec-specific typedefs, passing it the passthru table loaded from the specification files.

You should be able to run the code generation process and get a header with a bunch of type definitions in it.

Extension variables

With that task out of the way, we can now proceed to write our extension variable declarations. However, if we want this file to co-exist with C and C++ (ie: you can include the header from C++ code, even though the source code for it is C), then we need to wrap all of our real C code in an extern "C" block.

Since this is a block in terms of C logic, we use the block for it in our structure:

local my_struct = 
{
  { type="file", style="hdr", name="GetFilename(basename, spec, options)",
    { type="block", name="IncludeGuard",
      { type="write", name="Guards(hFile, spec, options)", },
      { type="blank" },
      { type="write", name="Typedefs(hFile, specData, spec, options)",},
      { type="blank" },
      { type="block", name="Extern(hFile)",
      },
    },
  },
}

Our style needs two functions. Since both of these are header specific, they both go into the hdr table:

function hdr.WriteBlockBeginExtern(hFile)
  common.WriteExternCStart(hFile)
end

function hdr.WriteBlockEndExtern(hFile)
  common.WriteExternCEnd(hFile)
end

Because of how common this is, there are functions in the common table for doing exactly this task. Indeed, we can shorten this by directly copying those functions into our table:

hdr.WriteBlockBeginExtern = common.WriteExternCStart
hdr.WriteBlockEndExtern = common.WriteExternCEnd

Now that we have an extern block, we need to write an extension variable for every extension that we were told to export. Since this is plain C89, we don't have access to bool, so we'll use int for the type instead with the usual boolean semantics.

To iterate over extensions and write them, we use the ext-iter iterator in our structure:

{ type="block", name="Extern(hFile)",
  { type="ext-iter",
    { type="write", name="Extension(hFile, extName, spec, options)", },
  },
},

For each extension, we will call WriteExtension in the style, passing it the name of the extension via extName.

To write the extension variable declaration, we first need a function to compute the name of the extension variable. We'll make this a separate function because we will need to write the variable definitions in the source file, and it would be best not to copy-and-paste code. So we need a local function to generate this:

local function GetExtensionVarName(extName, spec, options)
	return options.prefix .. spec.DeclPrefix() .. "ext_" .. extName
end

Notice that we again decorate the name with the user-defined prefix as well as a specification-defined prefix. spec.DeclPrefix() is a common prefix used for user-created variables and functions. The spec also has functions for getting canonical prefixes (GL_, WGL_, etc), as we will see later.

Armed with this function, we can now add the writing function to hdr:

function hdr.WriteExtension(hFile, extName, spec, options)
  hFile:fmt("extern int %s;\n", GetExtensionVarName(extName, spec, options));
end

You should be able to run this and get a number of extern declarations in the header now.

Enumerators

Wasn't that easy? Well, it won't be quite so easy anymore.

Now it's time to generate our enumerators. Since this is a C-style loader, we will use the standard convention in C for enumerators: #defines. We could define an actual C enumeration, but we'll go with the common way it's done, for now.

Iterating over all of the extensions was simple. However, enumerators can come from one of three sources:

  • Extensions the user asked for.
  • OpenGL extensions that are core for the version the user asked for, but the user didn't specifically ask for that extension.
  • The version of OpenGL the user asked for, outside of any core extensions.

And we must iterate over each of these individually.

Compounding this is the fact that we also need to avoid writing the same enumerator #define twice (sometimes an enum will be in a version and an extension). This problem is easily solved via special structure actions.

So, the first step with our structure is to introduce an enum-seen action. Any enumeration that is iterated over by any child action will be captured. Later iterations that produce the same enumerator can detect it and choose not write the #define statement.

{ type="block", name="Extern(hFile)",
  { type="ext-iter",
    { type="write", name="Extension(hFile, extName, spec, options)", },
  },
  { type="enum-seen",
  },
},

In order to iterate over every enum in every extension the user requested, we must do exactly that: iterate over each extension, then iterate over every enumerator in that extension. For the former, we use the ext-iter as before; for the latter, we use enum-iter:

{ type="enum-seen",
  { type="ext-iter",
    {type="enum-iter",
      { type="write", name="Enumerator(hFile, enum, enumTable, spec, options, enumSeen)", },
      { type="blank", last=true },
    },
  },
},

This will call the WriteEnumerator function for every enumerator in every extension. The blank action here uses the attribute last, which means that it will only insert a blank line on the last iteration of the inner-most enumerator. So this puts a space between every extension's enumerator. But it doesn't insert a space if the extension had no enumerators at all. This is because enum-iter doesn't execute any of its children if there are no enumerators, much like a for-loop over an empty list.

So, how do we write an enumerator in our style? That's kind of complicated. First, we need a function to generate an enumerator name.

local function GetEnumName(enum, spec, options)
	return spec.EnumNamePrefix() .. enum.name
end

Note that the enumerator does not prefix the enum name with the user-specified prefix options.prefix. This is a concession to the purpose of this loader: it's trying to emulate the common OpenGL style as much as possible. Furthermore, multiply-defined enums are usually a warning, not an error, so it isn't too big of a problem.

To write the enumerator, we use this function:

function hdr.WriteEnumerator(hFile, enum, enumTable, spec, options, enumSeen)
  if(enumSeen[enum.name]) then return end

  hFile:fmt("#define %s %s\n",
    GetEnumName(enum, spec, options),
    common.ResolveEnumValue(enum, enumTable))
end

The if statement checks to see if the enum has been processed before. If not, we generate a #define. Note that it is not this function's responsibility to actually store a value in the enumSeen table. That is done by the enum-iter internally.

The value of the enumerator is stored oddly. One enum can reference another, so resolving the value requires recursively going from enum to enum until you find one with a real value. That is the job of common.ResolveEnumValue, which uses the enumTable.

You can run this code, and you will get a file that contains enumerator definitions for the requested extensions. But not for any core versions.

To get them, we need to augment our structure a bit. The user asked for a specific version and profile (assume we're talking about OpenGL, rather than WGL/GLX). This means that the user wants the enums for that particular version/profile. To do this, we have to iterate over all of the versions between the first version (1.1) and the one the user asked for, writing out the appropriate enumerators for each version.

We do that with a version-iter action:

{ type="enum-seen",
  { type="ext-iter",
    {type="enum-iter",
      { type="write", name="Enumerator(hFile, enum, enumTable, spec, options, enumSeen)", },
      { type="blank", last=true },
    },
  },
  { type="version-iter",
  },
},

Within that iterator, we need to iterate over all extensions that were not explicitly requested. There is a special extension iterator for this: core-ext-cull-iter. It only works in the presence of a version-iter. Once in place, we do our previous enumeration iteration code:

{ type="enum-seen",
  { type="ext-iter",
    {type="enum-iter",
      { type="write", name="Enumerator(hFile, enum, enumTable, spec, options, enumSeen)", },
      { type="blank", last=true },
    },
  },
  { type="version-iter",
    { type="core-ext-cull-iter",
      {type="enum-iter",
        { type="write", name="Enumerator(hFile, enum, enumTable, spec, options, enumSeen)", },
        { type="blank", last=true },
      },
    },
  },
},

This time, we don't even have to touch the style; this will now print the enumerators for any core extensions not asked for explicitly.

To add in the enumerators for a version that aren't in core extensions from that version, we make one more enum-iter pass, directly beneath the version-iter. enum-iter iterates over an extension if an extension iterator is in use, but if none is in use, it looks for a version iterator:

{ type="enum-seen",
  { type="ext-iter",
    {type="enum-iter",
      { type="write", name="Enumerator(hFile, enum, enumTable, spec, options, enumSeen)", },
      { type="blank", last=true },
    },
  },
  { type="version-iter",
    { type="core-ext-cull-iter",
      {type="enum-iter",
        { type="write", name="Enumerator(hFile, enum, enumTable, spec, options, enumSeen)", },
        { type="blank", last=true },
      },
    },
    {type="enum-iter",
      { type="write", name="Enumerator(hFile, enum, enumTable, spec, options, enumSeen)", },
      { type="blank", last=true },
    },
  },
},

Again, no style changes are necessary.

Now, you might be thinking that maybe this should all just be one big "iterate over all of the enumerators" action, rather than a bunch of little ones. While that is not available, there is the next best thing: the common structural elements in the common table we created in our StructTest.lua file. It has a number of useful structural elements, and this is one of them:

{ type="block", name="Extern(hFile)",
  { type="ext-iter",
    { type="write", name="Extension(hFile, extName, spec, options)", },
  },
  { type="blank" },
  common.Enumerators(),
},

This is almost exactly equivalent to the above block, right down to the name of the function it calls (Enumerator) and the parameters it uses. The only difference is that it adds some optional function calls to print headers (naming each extension and version number), but that's unimportant. And optional.

Functions

Functions work more or less like enumerators, in that you have to iterate over them like enumerators. In fact, they work so much like enumerators that the only difference between the final version of the structure for enums and for functions is the use of func-seen, func-iter, and writing with a call to Function(hFile, func, typemap, spec, options, funcSeen). So let's not waste time and skip right to the end:

{ type="func-seen",
  { type="ext-iter",
    {type="func-iter",
      { type="write", name="Function(hFile, func, typemap, spec, options, funcSeen)", },
      { type="blank", last=true },
    },
  },
  { type="version-iter",
    { type="core-ext-cull-iter",
      {type="func-iter",
        { type="write", name="Function(hFile, func, typemap, spec, options, funcSeen)", },
        { type="blank", last=true },
      },
    },
    {type="func-iter",
      { type="write", name="Function(hFile, func, typemap, spec, options, funcSeen)", },
      { type="blank", last=true },
    },
  },
},

As before, there is a common structural element to replace this: common.Functions(). And as before, it calls the same Function we defined with the same arguments.

The biggest issue here is the writing of the actual functions. First, we need a function to compute the name of the function pointer variable we want to declare:

local function GetFuncPtrName(func, spec, options)
  return options.prefix .. "_testc_".. spec.FuncNamePrefix() .. func.name
end

Here, we actually need to prefix the variable name; otherwise linker errors could occur. We also apply spec-based decoration, as well as a fixed string based on this particular generator.

The usual method of writing function pointers would involve creating typedefs for the function pointer types. You may have seen some "PFNGLVERTEXATTRIBPOINTERPROC"-kind of things. We won't be doing that.

We still need a function to build a string containing the full function pointer definition.

local function GetFuncPtrDef(hFile, func, typemap, spec, options)
  return string.format("%s (%s *%s)(%s)",
    common.GetFuncReturnType(func, typemap),
    spec.GetCodegenPtrType(),
    GetFuncPtrName(func, spec, options),
    common.GetFuncParamList(func, typemap))
end

This function calls a lot of things. Functions use types, for parameter types and return types. Resolving these into the actual GL types is a non-trivial process, so common.GetFuncReturnType and common.GetFuncParamList are used to get the return type and parameter list respectively. If you want the parameter names for the function parameter list, pass true for a third parameter to the function.

The spec.GetCodegenPtrType() part is very much not optional. It adds an important modifier to the function pointer, which is needed on some systems (Windows). Without it, bad things happen.

function hdr.WriteFunction(hFile, func, typemap, spec, options, funcSeen)
  if(funcSeen[func.name]) then return end

  hFile:fmt("extern %s;\n",
    GetFuncPtrDef(hFile, func, typemap, spec, options))

  hFile:fmt("#define %s %s\n",
    common.GetOpenGLFuncName(func, spec),
    GetFuncPtrName(func, spec, options))
end

After checking to see if the pointer has been written before, we get down to business. We write an extern declaration for our function pointer with that type. Then we write a #define statement, which effectively aliases the name. This is a common tactic when dealing with function pointers and OpenGL loaders: the pointer is given an innocuous name to avoid conflicting with user code, and a #define is used to map it to the actual OpenGL function name. That's what common.GetOpenGLFuncName returns.

Main loader

The last step for our header is to write the function prototype for our function that actually loads everything. It will just be a simple write statement:

{ type="block", name="Extern(hFile)",
  ...
  common.Functions(),
  { type="blank" },
  { type="write", name="MainLoaderFunc(hFile, specData, spec, options)",},
},

The hdr portion of our structure is now complete. In total, it looks like this (using the common structure elements):

local my_struct = 
{
  { type="file", style="hdr", name="GetFilename(basename, spec, options)",
    { type="block", name="IncludeGuard",
      { type="write", name="Guards(hFile, spec, options)", },
      { type="blank" },
      { type="write", name="Typedefs(hFile, specData, spec, options)",},
      { type="blank" },
      { type="block", name="Extern(hFile)",
        { type="ext-iter",
          { type="write", name="Extension(hFile, extName, spec, options)", },
        },
        { type="blank" },
        common.Enumerators(),
        { type="blank" },
        common.Functions(),
        { type="blank" },
        { type="write", name="MainLoaderFunc(hFile, spec, options)",},
      },
    },
  },
}

Reasonably compact and quite powerful.

The WriteMainLoaderFunc style function needs a function to get the name of the function to write.

local function GetMainLoaderFuncName(spec, options)
  return options.prefix .. spec.DeclPrefix() .. "LoadFunctions"
end

Note that our loader function name is prefixed with both the user-specified prefix and the spec-defined declaration prefix.

The actual function to write the prototype requires a small bit of explanation:

function hdr.WriteMainLoaderFunc(hFile, spec, options)
  hFile:fmt("int %s(%s);\n",
    GetMainLoaderFuncName(spec, options),
    spec.GetLoaderParams())
end

The function loader has parameters, but the specific parameters they take depends on the specification. The reason for this is that wglGetExtensionStringARB and glXQueryExtensionsString both take parameters. The former takes a HDC device context that has a pixel format on it, while the latter takes a Display and a screen number.

Our loader will load extensions based on what is advertised. As such, we need to call those functions to see what is advertised. So our loader function needs to take those parameters and pass them along.

Source

The header was the easy part. The source file is where things get tricky.

Since we're making a new file, we need to make a new file action in our structure:

local my_struct = 
{
  { type="file", style="hdr", name="GetFilename(basename, spec, options)",
    ...
  },
  { type="file", style="src", name="GetFilename(basename, spec, options)",
  },
}

We're switching to the src substyle table here. So we need a function to produce a filename for our source file:

local src = {}
my_style.src = src

function src.GetFilename(basename, spec, options)
  return basename .. ".c"
end

If you run this, you'll see a blank source file.

Source preamble

Next step: we need to #include the things our source file needs. Since we'll be doing some string comparisons, we need string.h, among others. Also, we need to include our own header, which we will assume is in the include path of the project.

First, we need to augment our structure with an include writer:

local my_struct = 
{
  { type="file", style="hdr", name="GetFilename(basename, spec, options)",
    ...
  },
  { type="file", style="src", name="GetFilename(basename, spec, options)",
    { type="write", name="Includes(hFile, basename, spec, options)", },
  },
}

In our structure, we add an appropriate WriteIncludes function:

function src.WriteIncludes(hFile, basename, spec, options)
  hFile:writeblock([[
#include <stdlib.h>
#include <string.h>
]])
  local base = util.ParsePath(hdr.GetFilename(basename, spec, options))
  hFile:fmt('#include "%s"\n', base)
end

The util.ParsePath function takes a path and breaks it up into a filename and a directory. We write the filename produced by the header as an include statement in our source file.

After this, we need to write the function that actually loads a function pointer. This is very platform specific; fortunately, the system has a simple way around all of that junk (well, for C/C++, at least).

In our structure, we add a write action for it:

{ type="file", style="src", name="GetFilename(basename, spec, options)",
  { type="write", name="Includes(hFile, basename, spec, options)", },
  { type="blank" },
  { type="write", name="LoaderFunc(hFile, spec, options)", },
},

And in our style, we add this:

function src.WriteLoaderFunc(hFile, spec, options)
  hFile:writeblock(spec.GetLoaderFunc())
end

All very easy. Everything is statically defined, so there will be no interference across different files.

Extension variables

We wrote extern declarations for our extension variables in the header; now we need to write proper definitions for them. So we need to iterate over the extensions again; we can use the exact same code as we did in the header:

{ type="file", style="src", name="GetFilename(basename, spec, options)",
  { type="write", name="Includes(hFile, basename, spec, options)", },
  { type="blank" },
  { type="write", name="LoaderFunc(hFile, spec, options)", },
  { type="blank" },
  { type="ext-iter",
    { type="write", name="Extension(hFile, extName, spec, options)", },
  },
},

The src table needs its own version of Extension which defines the variable and initializes it to 0:

function src.WriteExtension(hFile, extName, spec, options)
  hFile:fmt("int %s = 0;\n", GetExtensionVarName(extName, spec, options));
end

You can run that and get extension variable definitions in your source file. That was easy.

Functions

We now need to define the extension function pointers and initialize them to NULL.

We'll use our handy common structure element for iterating over all functions:

{ type="ext-iter",
  { type="write", name="Extension(hFile, extName, spec, options)", },
},
{ type="blank" },
common.Functions(),

This means we need a WriteFunction function in our src table:

function src.WriteFunction(hFile, func, typemap, spec, options, funcSeen)
  if(funcSeen[func.name]) then return end
  hFile:fmt("%s = NULL;\n", GetFuncPtrDef(hFile, func, typemap, spec, options));
end

You can run that and get function pointer definitions in your source file. That was easy too. Good thing we have GetFuncPtrDef lying around.

Function loaders

Before we can proceed further, we need to talk a bit about what it is we're actually trying to achieve here.

When the user calls our function to load the function pointers with actual functions, our code needs to do the following:

  • For each extension that the user asked us to export, if it is in the extension string:
    • Set its extension variable.
    • If it has functions, load its function pointers.
  • Load all of the functions associated with the version of OpenGL we were asked to export.

In order to do this, we need a few things. For every extension the user asked for, we need a function that will load its function pointers. And we need a function that will load the function pointers for the OpenGL version.

So let's do that.

The first step is one of those "for each extension" things, so obviously we need an extension iterator in our structure:

{ type="blank" },
common.Functions(),
{ type="ext-iter",
},

We need to write a function to load an extension's function pointers. These functions have a beginning, a series of load commands for each function, and an ending. That sounds like a block surrounding a function iterator:

{ type="blank" },
common.Functions(),
{ type="ext-iter",
  { type="block", name="ExtFuncLoader(hFile, extName, spec, options)",
    { type="func-iter",
    },
  }
},

We don't need a func-seen here, because we're not interested in duplicates. Or rather, we're not interested in doing things differently for duplicate functions. It's fine to load the same function twice (and we will certainly do that later).

There is one problem with this. We will generate loader functions for extensions that don't have functions. Now, this would just be a function that's empty, but it's a bit sloppy to deal with. We can do better.

We avoid this by using the cond attribute on our block:

{ type="blank" },
common.Functions(),
{ type="ext-iter",
  { type="block", name="ExtFuncLoader(hFile, extName, spec, options)", cond="func-iter",
    { type="func-iter",
    },
  }
},

The cond attribute means that the action (and its children) will only be executed if the condition is true. And the func-iter condition is only true if using a function iterator will iterate over at least one element.

Within the function iterator, we need to write some code for each function to do the actual loading:

{ type="ext-iter",
  { type="block", name="ExtFuncLoader(hFile, extName, spec, options)", cond="func-iter",
    { type="func-iter",
      { type="write", name="LoadFunction(hFile, func, typemap, spec, options)", },
    },
  },
  { type="blank", cond="func-iter",},
},

We also add a blank line after each function, just to make it nicer to read.

That's good enough for the structure for now. Time to do the style code. First, we need a function to get the name of the function we intend to write:

local function GetExtFuncLoaderName(extName, spec, options)
  return "Load_" .. extName;
end

Next comes the two functions for our block:

function src.WriteBlockBeginExtFuncLoader(hFile, extName, spec, options)
  hFile:fmt("static void %s()\n", GetExtFuncLoaderName(extName, spec, options))
  hFile:write("{\n")
  hFile:inc()
end

function src.WriteBlockEndExtFuncLoader(hFile, extName, spec, options)
  hFile:dec()
  hFile:write("}\n")
end

Since this is a static function, we don't have to worry about name conflicts (except for within this file, of course). The hFile:inc() and hFile:dec() calls are for incrementing and decrementing the tab count inserted by the various hFile writing commands. It helps us properly format our generated code.

For each function, we need to load a pointer into the appropriate function pointer. However, there's one problem: standard C does not allow implicit conversion between pointers-to-variables (like void*) and pointers-to-functions (like void(*)())). It also doesn't allow implicit conversions from different kinds of function pointers. So we need an explicit cast to our function pointer type.

To make this easier, we need a function to return the function pointer type:

local function GetFuncPtrType(hFile, func, typemap, spec, options)
  return string.format("%s (%s *)(%s)",
    common.GetFuncReturnType(func, typemap),
    spec.GetCodegenPtrType(),
    common.GetFuncParamList(func, typemap))
end

Now armed with that, we can write our function pointer loading code:

function src.WriteLoadFunction(hFile, func, typemap, spec, options)
  hFile:fmt('%s = (%s)%s("%s%s");\n',
    GetFuncPtrName(func, spec, options),
    GetFuncPtrType(hFile, func, typemap, spec, options),
    spec.GetPtrLoaderFuncName(),
    spec.FuncNamePrefix(),
    func.name)
end

The function spec.GetPtrLoaderFuncName() gets the name of the function pointer loading function written by spec.GetLoaderFunc(). spec.FuncNamePrefix() is the prefix used by OpenGL/WGL/GLX for its function names.

You can run that and get static functions that load any extensions that have added functions.

We still need one more thing: a function that loads all core function pointers. This will be a slightly more difficult to build. We first need a block that represents the function itself:

{ type="ext-iter",
  ...
},
{ type="block", name="CoreLoader(hFile, spec, options)", cond="core-funcs",
},

Here, we see the use of another cond, core-funcs. This one doesn't have an iterator analog. It is true if the specification has any core functions (which basically means if the specification is OpenGL and not WGL/GLX).

Inside, we must iterate over each version and print all of the functions. This requires the core extension trick we used before, but with one big difference:

{ type="block", name="CoreLoader(hFile, spec, options)", cond="core-funcs",
  { type="version-iter",
    { type="core-ext-iter",
      {type="func-iter",
        { type="write", name="LoadFunction(hFile, func, typemap, spec, options)", },
      },
    },
    {type="func-iter",
      { type="write", name="LoadFunction(hFile, func, typemap, spec, options)", },
    },
  },
},

Notice the use of core-ext-iter instead of core-ext-cull-iter. This iterates over all of the core extensions in that version, whether or not the user asked for them.

That means if the user asked for a core extension, we may load the function twice: once if it is in the extension list and once with all of the other core functions in this version. That's fine.

We already have a WriteLoadFunction, so all we need in the style is the block defining the function:

function src.WriteBlockBeginCoreLoader(hFile, spec, options)
  hFile:write("static void Load_Version()\n")
  hFile:write("{\n")
  hFile:inc()
end

function src.WriteBlockEndCoreLoader(hFile, version, spec, options)
  hFile:dec()
  hFile:write("}\n")
end

The name doesn't need to change, since there is only ever one of them.

Main loader

The most complex part is the generation of the main loading function. This is complicated because there are a lot of things you need to take into account. The easiest part of this is the structure. We simply need a write action to call into our src style:

local my_struct = 
{
  { type="file", style="hdr", name="GetFilename(basename, spec, options)",
    ...
  },
  { type="file", style="src", name="GetFilename(basename, spec, options)",
    ...
    { type="blank", },
    { type="write", name="MainLoaderFunc(hFile, specData, spec, options)",},
  },
}

That's the only part of this that is easy.

A detailed breakdown of our loading algorithm is as follows:

  1. Clear the extension variables. This will allow the user to call our loader multiple times. Resetting the function pointers is not necessary, since the user shouldn't be calling non-core function pointers if the extension variable is not set. And we'll be re-loading all the core functions anyway.
  2. Iterate over the extensions the context provides. For each extension, if it is among the extensions we care about:
    1. Set its extension variable to non-zero.
    2. If that extension has functions, call the function to load all of the function pointers.
  1. Load all of the functions for the OpenGL version, if any.

This tells us right away that we need some helper functions. So let's make some. Our WriteMainLoaderFunc will be broken up into two steps: the writing of helper functions/definitions and the writing of the main function:

local function WriteHelpers(hFile, specData, spec, options)
end

function src.WriteMainLoaderFunc(hFile, specData, spec, options)
  WriteHelpers(hFile, specData, spec, options)
  hFile:write("\n")

  hFile:fmt("int %s(%s)\n",
    GetMainLoaderFuncName(spec, options),
    spec.GetLoaderParams())
  hFile:write("{\n")
  hFile:inc()
  hFile:dec()
  hFile:write("}\n")
end

Running this will create an empty function definition.

Clear extension variables

We could have used the structure to help us build this (ie: used its ext-iter functionality), but it's simpler at this point to do this task ourselves.

local function WriteHelpers(hFile, specData, spec, options)
  common.WriteCClearExtensionVarsFunc(hFile, specData, spec, options,
    GetExtensionVarName, "0")
end

This will write a static function called ClearExtensionVars; it will walk through options.extensions, writing a line that sets the value to "0". To get the extension name, it calls the function we provided, which takes (extName, spec, options).

Load extensions

Well, that was the last easy part; now it all gets incredibly complicated. There are two big problems when dealing with trying to load the extensions.

  1. The function used to get the extensions string is a function which in many cases must be loaded.
  2. OpenGL (but not WGL/GLX) changed how you get the list of available extensions in GL 3.0.

That last problem is the most devious (we will solve #1 by just trying to load the function. If it's not there, we exit with a failure). Let's investigate further.

In OpenGL 2.1 and below, you use glGetString(GL_EXTENSIONS) to get a space-separated list of extensions. This was fairly simple, but people kept screwing it up. So in GL 3.0, they added glGetStringi, which gets a string by an enum name and an index. So you use glGetIntegerv(GL_NUM_EXTENSIONS) to get the number of extensions, then iterate through that number, calling glGetStringi(GL_EXTENSIONS, i) for each.

The problem is that in OpenGL 3.1, they removed the old space-separated list of extensions from core OpenGL. So if you create a core OpenGL 3.2 context, you have to use the new style. However, if you're dealing with a 2.1 or below context, glGetStringi doesn't exist, so we have to use the old way. If you create a 3.2 compatibility context, you can use either.

Incidentally, this is why GLEW has problems with core contexts; it only knows how to use the old way, and it doesn't bother to implement the machinery needed to actually test at runtime which to use.

If we wanted, we could write the code to process things both ways, then pick which one to use based on what the actual context is. However, for the sake of simplicity, we won't be doing that. We will instead do what pointer_c and pointer_cpp do: they expect you to actually be serious about the version number. If you create an OpenGL 3.3 header, then you expect to be using OpenGL 3.3 or greater, and the loader is allowed to fail if that's not available.

Therefore, if the user asks for OpenGL 3.0 or above, we will use the new-style functions; if the user asks for a lower version, we use the old style. Oh, and while we're doing this, remember that this code generator needs to work with WGL and GLX, which only use old-style. So we only use the new-style if we're writing OpenGL and version 3.0 or above is requested.

Is that complicated enough?

Extension Mapping Table

Well, let's get the simple part out of the way first. In both cases, we search the extension list, and if we find one of our extensions in the list, we set the extension variable and load those functions, if any. In order to do this, we need some way to map extension string names to extension variables and loading functions.

We do this with a mapping table. There is actually a nice common bit of code to do this all for us in common.WriteCMappingTable. But we'll do it by ourselves, just to show off what needs to be done.

First, we need a typedef for the function pointer type, because those are annoying to write without one. All of the loaders use the same prototype:

local function WriteHelpers(hFile, specData, spec, options)
  common.WriteCClearExtensionVarsFunc(hFile, specData, spec, options,
    GetExtensionVarName, "0")
  hFile("\n")
  hFile:write("typedef void (*PFN_LOADFUNCPOINTERS)();\n")
end

After this, we need to write a struct that serves as an entry in the mapping table. So we need a function to get the name of the struct:

local function GetMappingTableStructName(spec, options)
  return string.format("%s%sStringToExtMap",
    options.prefix, spec.DeclPrefix())
end

Each entry in the table has a string literal (the name of the extension), a pointer to the extension variable, and a function pointer to call to load the extensions (or NULL):

hFile:write("typedef void (*PFN_LOADFUNCPOINTERS)();\n")
hFile:fmt("typedef struct %s_s\n",
  GetMappingTableStructName(spec, options))
hFile:write("{\n")
hFile:inc()
hFile:writeblock [[
char *extensionName;
int *extensionVariable;
PFN_LOADFUNCPOINTERS LoadExtension;
]]
hFile:dec()
hFile:fmt("} %s;\n", GetMappingTableStructName(spec, options))
hFile:write "\n"

Because struct definitions can't be file-static, we have to prefix them with our specification and options prefixes.

We then need to declare a static global variable that represents our mapping table. So again, we need a function to compute that. However, since this will be file-static, there's no need to prefix it:

local function GetMappingTableVarName()
  return "g_stringToExtMap"
end

This will just be a global array of these structures, one for each extension.

hFile:write "\n" --From last line of previous code.
hFile:fmt("static %s %s[] = {\n",
  GetMappingTableStructName(spec, options),
  GetMappingTableVarName())
hFile:inc()
for _, extName in ipairs(options.extensions) do
  if(#specData.extdefs[extName].funcs > 0) then
    hFile:fmt('{"%s", &%s, %s},\n',
      spec.ExtNamePrefix() .. extName,
      GetExtensionVarName(extName, spec, options),
      GetExtFuncLoaderName(extName, spec, options))
  else
    hFile:fmt('{"%s", &%s, NULL},\n',
      spec.ExtNamePrefix() .. extName,
      GetExtensionVarName(extName, spec, options))
  end
end
hFile:dec()
hFile:write("};\n")

options.extensions contains the list of extensions the user asked for. So we iterate over each one. We also check to see if the named extension has actual functions; if not, we put NULL instead of a loader function.

Now it's time to think ahead. We're creating this table so that we can iterate through it and find an extension by name. In order to iterate through it, we need to know how big it is. While we could play some games with sizeof to compute it, there's no point, since we already know it: #options.extensions. So we write that as a static integer:

hFile:write("};\n") --From last line of previous code
hFile:write("\n")
hFile:fmt("static int g_extensionMapSize = %i;\n", #options.extensions);

Run it and see what you get.

Both the old-style algorithm and the new-style one are dependent on iterating through the table, looking for an extension by name. So we will create a function that does exactly that: it takes a string name of an extension and returns a pointer to an entry for that extension. Or NULL if none is available. Since that's mostly boiler-plate code, we'll do it by using a common method:

hFile:fmt("static int g_extensionMapSize = %i;\n", #options.extensions);
hFile:write("\n")
common.WriteCFindExtEntryFunc(hFile, specData, spec, options,
  GetMappingTableStructName(spec, options),
  GetMappingTableVarName())

This creates a function named FindExtEntry.

One last thing. Both algorithms will call this FindExtEntry with a string name. Both algorithms will set the extension variable to 1 if it is found. And both will call the loader function if it exists. Therefore, we should write a function to do that. There isn't quite one of those in the common system (there's something close), so we'll have to write the boiler-plate ourselves:

hFile:write("\n")

hFile:fmtblock([[
static void LoadExtByName(const char *extensionName)
{
	%s *entry = NULL;
	entry = FindExtEntry(extensionName);
	if(entry)
	{
		if(entry->LoadExtension)
		{
			int numFailed = entry->LoadExtension();
			if(numFailed == 0)
			{
				*(entry->extensionVariable) = 1;
			}
			else
			{
				*(entry->extensionVariable) = 1;
			}
		}
		else
		{
			*(entry->extensionVariable) = 1;
		}
	}
}
]], GetMappingTableStructName(spec, options))

Old style

This is where it gets complicated. Since WGL/GLX and many uses of OpenGL will use the old style, let's handle that one first.

To tell which is which, we could check the spec and the options against expected values. However, a certain bit of setup work has to be done to make the new style loader work out. The spec will provide part of it, and it will do the detection for us.

Thus, our next bit of code looks like this:

local indexed = spec.GetIndexedExtStringFunc(options);
if(not indexed) then
  --Old style
else
  --New style
end

We'll talk more about exactly what goes on in indexed later.

We're still writing helper functions. Remember that old-style extension processing deals with a single string containing space-separated extension names. So obviously, we need a way to walk this list, break it up into extensions, and run our LoadExtByName function. So let's write that function.

Actually, let's not; that's tedious. Let's just use the common code version:

local indexed = spec.GetIndexedExtStringFunc(options);
if(not indexed) then
  common.WriteProcessExtsFromStringFunc(hFile, "LoadExtByName(%s)")
else
  --New style
end

The second parameter to WriteProcessExtsFromStringFunc is the name of the function to call for each extension. That's the function we just wrote.

Now, we need to do one more thing: our helper writing function needs to return indexed, because we're about to shift to our main loader function:

hFile:write("\n")
local indexed = spec.GetIndexedExtStringFunc(options);
if(not indexed) then
  common.WriteProcessExtsFromStringFunc(hFile, "LoadExtByName(%s)")
else
  --New style
end

return indexed

Now, for our main loader, we have another if-statement:

function src.WriteMainLoaderFunc(hFile, specData, spec, options)
  local indexed = WriteHelpers(hFile, specData, spec, options)
  hFile:write("\n")

  hFile:fmt("int %s(%s)\n",
    GetMainLoaderFuncName(spec, options),
    spec.GetLoaderParams())
  hFile:write("{\n")
  hFile:inc()

  if(not indexed) then
  else
  end

  hFile:dec()
  hFile:write("}\n")
end

Load version

Updated