OpenGL Multi-threading, what it is and what it means

Disclaimer: this information is all easily available around, I’m trying to condense it a bit, and focus on a particular topic for discussion. I may gloss over, or simplify information, but the centrals ideas should apply.

There’s been a fair bit of talk recently about attempting to multi-thread using OpenGL so I thought I’d write a bit more about what “multi-threading” an OpenGL game is, what’s normally done, and how it compares to multi-threading in Vulkan.

Some History

For those not aware, OpenGL is, in computing terms, old. It was designed before multiple CPU cores were even available to the general consumer, and long before just about every part of a graphics pipeline was programmable.
The central concept of OpenGL is a state machine. This has done it very well for a long time, but a single OpenGL context (state machine) is based on sequential inputs from the application - the application calls the API, and the OpenGL implementation reacts. State is changed, rendering commands issued, resources loaded, and so on.


State of OpenGL

Being state based, and because any of the API calls has the potential to change state, this makes multi-threaded access to an OpenGL context very difficult - indeed. Let's say one thread is handling "x" and only meant for it, but another thread accesses it and changes "x" to "x+1", the original thread then tries to do something else and it doesn't know it has changed. It’s not permitted in most cases and can result in undefined behaviour like that. There is some exception to this: contexts are allowed to share certain data such as texture information and vertex buffers, but more on that later.
Furthermore about a state based design, is that OpenGL implementations must ensure that the state is always valid. It must ensure that data is correctly bound, in range, and that nothing will break the system. Up to this point, everything is CPU-side still. If everything is okay, the implementation may then generate commands that can be sent to the GPU itself.

Drivers can do some fancy things behind the scenes of course, but the end result, as presented to the application, is the same. Accept an API command, modify and verify state, send hardware commands to the GPU.

Recent versions of OpenGL have been a big help in cutting out a lot of the overhead. Checking state validity can be decreased, greatly reducing the time from API call to GPU command, but it’s still very much a case of having to do that in a single thread.


Threading

So how can developers “multi-thread” OpenGL? It is possible to have multiple contexts in multiple threads, and use them to load texture data, update vertex buffers, possibly compile new shaders, in different threads. The tricky part about this is that sharing this information between OpenGL contexts is dependent on the drivers behaving themselves, in addition to the application not trying (by accident or intent) to do anything strange, so it’s often unstable. It can be quite the adventure getting a game running with this approach, and the runtime improvements are often simply not even worth the effort - it can often run worse if drivers need to synchronise data between contexts often enough. For the curious, things like editors with multiple rendering windows do this, but that’s a different scenario - each window isn’t trying to interfere with every other window while rendering, so multi-threading doesn’t normally come into play.

This leads to the second approach to multi-threading OpenGL: developers don’t! If OpenGL works best by submitting commands sequentially on the thread where a context is active, then that’s simply the best thing to do. Nothing stops a game developer making their own queue of OpenGL API calls they want to perform though, and creating that can be done by multi-threading. To give an example, if a game has a big list of objects, there’s going to be a bit of processing to do when deciding whether to draw each object or not; for each object, the game decides if the object might be visible first, and only try to render if it will actually be seen. The check for each object takes time, but processing each object is independent. So the list can be split into multiple sub-lists, and each sub-list given to a separate thread to run a visibility check on. Each thread will have it’s own rendering list to which objects that should be rendered are added. When done, each rendering list can be iterated over in turn and objects submitted to OpenGL in a single thread. This is a very simple example, but there’s normally quite a fair amount of similar logic in deciding what to render. So it’s not multi-threading OpenGL, but rather multi-threading in deciding how to use OpenGL.


Vulkan

Before I mentioned that OpenGL verifies state information and then generates commands to the GPU.

Firstly, once a developer has finished making everything work, then all that verification is still done, but isn’t actually required. It’s useful during development, but later it’s (hopefully!) a waste of time. So even on a dedicated thread submitting commands to OpenGL, there’s quite the overhead for the final task of actually sending commands to the GPU itself. It would be nice if there was some way to pre-build a list of commands to send to the GPU that were known to be valid.
Secondly, as with the example above of a game splitting object visibility checks into multiple sub-lists, it would also be nice if multiple GPU command lists could be created on separate threads, and then submitted to the GPU in turn. They are separate after all, and don’t require GPU access to actually prepare.
This is essentially what Vulkan allows. There are some requirements: all the state must be known up-front, and prepared for well before it’s time to actually render something. The flip side is that there is much, much less driver overhead, and the API itself can be used multi-threaded. Actual submission of commands to the GPU is still done sequentially, in a single thread, however there’s very little overhead; all error checking has been done, and it’s just sending commands directly to the GPU (feeding the beast).
There are other areas of Vulkan that lend themselves nicely to application level multi-threading, but I won’t cover them here. Suffice to say that Vulkan does not contain a central state machine, and instead tries to keep everything as isolated and contained as possible, meaning things like building a shader don’t block loading a texture, making multi-threaded designs easier to achieve.


Not Always Applicable

On a final note: when porting games, the way a game handles its data is not always compatible with some of the multi-threading ideas mentioned above. It can’t be expected in every game. In addition, it might simply be easier in time and effort (not to mention with testing and stability) to run things in a single thread anyway. Not as efficient, but possibly less error prone and faster to bring a port. Article taken from GamingOnLinux.com.