Support us on Patreon
PayPal
You may remember recently I wrote about 'threaded GL dispatch' in Mesa possibly getting merged, after AMD's Marek suggested it and there was some backlash, well, it has continued.
For those that don't know, threaded GL dispatch will help to split up the application and OpenGL into different threads to hopefully improve performance. It can improve some games, while make things worse on others (some already do it in-engine).
Ian Romanick replied to the mailing list entry without pulling any punches:
However, Eric Anholt was keen to point out they often merge-in code that's not even close to being done:
There seems to be a lot of words being thrown around, which is perfectly normal for big changes to projects like this it's just that this is all on a public mailing list where anyone can subscribe to see it.
Also, it seems Civilization VI is another game that would benefit from it and Valve are apparently looking to use the code with a white-list on SteamOS, from Marek's post:
It will be very interesting to see what the real outcome is, hopefully whatever is better for the end-user and not what the developers feel should be the way because of ideological reasons or whatever. Article taken from GamingOnLinux.com.
For those that don't know, threaded GL dispatch will help to split up the application and OpenGL into different threads to hopefully improve performance. It can improve some games, while make things worse on others (some already do it in-engine).
Ian Romanick replied to the mailing list entry without pulling any punches:
QuoteNo, it absolutely is not fine to merge. We have never allowed such a thing, and I'll be damned if I'll allow this project to start. Things that land that are known to be broken never actually get fixed. Then we have to waste time fielding bug reports and Phoronix threads because
users turn on the performance features and everything breaks. It's just a terrible idea.
However, Eric Anholt was keen to point out they often merge-in code that's not even close to being done:
QuoteYeah, just like how we gated the GLSL compiler until it was completely done (we didn't) and NIR until it was completely done (we didn't) and Vulkan until it was completely done (we didn't) and...
Software that people care about gets fixed. I'm also concerned that nobody actually cares about getting glthread working completely, given Marek's attitude toward piglit conformance (and my also ignoring the branch for the last however many years). However, "we have never allowed merging broken software that's only turned on under env vars/configure" is totally false. We do that regularly for big things we care about.
There seems to be a lot of words being thrown around, which is perfectly normal for big changes to projects like this it's just that this is all on a public mailing list where anyone can subscribe to see it.
Also, it seems Civilization VI is another game that would benefit from it and Valve are apparently looking to use the code with a white-list on SteamOS, from Marek's post:
QuoteFYI. Civilization VI is another game that works with and benefits from glthread. The game was just released. Even Nvidia is CPU-bound on highest details and can't reach 45 fps with full hd. People wanting to play Civ VI with decent frame rate will want glthread. I don't think they care too much about our the community processes, so I expect there will be quite a few users using out-of-tree builds of Mesa.
Also, Pierre-Loup from Valve said on IRC yesterday that they are probably gonna ship glthread and make their own whitelist, regardless of the outcome of this discussion. It would be preferable to have that whitelist in master too, but that may be difficult if we can't merge it.
If distributions and vendors start shipping glthread, we might as well merge it, because at that point there is no advantage in keeping this out of tree if it forces users to use out of tree builds. We'll get bug reports regardless.
It will be very interesting to see what the real outcome is, hopefully whatever is better for the end-user and not what the developers feel should be the way because of ideological reasons or whatever. Article taken from GamingOnLinux.com.
Some you may have missed, popular articles from the last month:
The comments on this article are closed.
All posts need to follow our rules. For users logged in: please hit the Report Flag icon on any post that breaks the rules or contains illegal / harmful content. Guest readers can email us for any issues.
18 comments
GustyGhost Feb 10, 2017
If SteamOS does adopt a whitelist for this, it really could become a "best distro for gaming" out of the box.
2 Likes, Who?
Oh "whitelist"... I don't like where this is going...
This is exactly what proprietary NVIDIA and (ex-)AMD drivers do and it has a lot of repercussions on the maintainability and stabilization of the drivers. It is also a pain in the ass to debug since each and every games will have their own special tweaks specified in the drivers, which means more code paths to debug.
Oh this is a BAD idea..
This is exactly what proprietary NVIDIA and (ex-)AMD drivers do and it has a lot of repercussions on the maintainability and stabilization of the drivers. It is also a pain in the ass to debug since each and every games will have their own special tweaks specified in the drivers, which means more code paths to debug.
Oh this is a BAD idea..
1 Likes, Who?
Oh "whitelist"... I don't like where this is going...It shouldn't be as complicated as you think, it is after-all only checking a single white-list for one feature to see if they need to look at that code. That's until they make it smart enough to not need the white-list. Sure, it adds a bit of complexity, but everything they add does. Like Marek keeps saying, it will not be enabled by default so users shouldn't run into problems.
This is exactly what proprietary NVIDIA and (ex-)AMD drivers do and it has a lot of repercussions on the maintainability and stabilization of the drivers. It is also a pain in the ass to debug since each and every games will have their own special tweaks specified in the drivers, which means more code paths to debug.
Oh this is a BAD idea..
0 Likes
I think that GL threading is definitively not the silver bullet. I know... I keep posting that everywhere, but GL threading is almost the same as the threading capacities in Vulkan. And in the end, look at the benchmark differences between OpenGL and Vulkan: zero. Some games win a bit, but not that much, and some games loose a bit too.
For instance, look here: http://www.phoronix.com/scan.php?page=article&item=mesa-17-radeon&num=1
The improvements between 13.0 and 17.0 are very impressive on the RADV (kudos to the devs!), but if you compare the 17.0 results in Dota 2 for Vulkan and OpenGL, OpenGL wins all the time! And as for The Talos Principle, Vulkan is now just on par with OpenGL performances.
I'll repeat that again: multi-threading the renderer is, of course, a good thing, but a game engine is not just a 3D engine.
Edit:
About the whitelist, today it's "just" threaded GL, but if we open this door, a lot of other features will get in this whitelist as well.
Last edited by Creak on 10 February 2017 at 2:51 pm UTC
For instance, look here: http://www.phoronix.com/scan.php?page=article&item=mesa-17-radeon&num=1
The improvements between 13.0 and 17.0 are very impressive on the RADV (kudos to the devs!), but if you compare the 17.0 results in Dota 2 for Vulkan and OpenGL, OpenGL wins all the time! And as for The Talos Principle, Vulkan is now just on par with OpenGL performances.
I'll repeat that again: multi-threading the renderer is, of course, a good thing, but a game engine is not just a 3D engine.
Edit:
About the whitelist, today it's "just" threaded GL, but if we open this door, a lot of other features will get in this whitelist as well.
Last edited by Creak on 10 February 2017 at 2:51 pm UTC
1 Likes, Who?
About the whitelist, today it's "just" threaded GL, but if we open this door, a lot of other features will get in this whitelist as well.Well.. it was even faster than I expected... :(
http://phoronix.com/scan.php?page=news_item&px=Mesa-RFC-DRIRC-Compat-Change
0 Likes
We already have a list of programs that don't conform to strict OpenGL standards and thus driconf allows errors to be downgraded to allow games & programs to run. The only difference with GLThread is it's a performance boost rather than broken functionality workaround - and yes I know those work arounds are often disputed too
0 Likes
Mblackwell Feb 10, 2017
Nvidia's current driver doesn't have a whitelist for multithreading, instead it enables it on everything and performs some heuristics and disables it on the fly if performance is being affected.
0 Likes
Nvidia's current driver doesn't have a whitelist for multithreading, instead it enables it on everything and performs some heuristics and disables it on the fly if performance is being affected.I'm not sure it's "on the fly", I think it's more integrated profiles for specific games, as better explained here: https://www.quora.com/How-does-Nvidia-optimize-specific-games-via-driver-updates?share=1
0 Likes
@Creak
You do not see a lot of performance improvement by current vulkan implementations because the engines are FAR from finished porting to Vulkan.
Just quoting the article about Croteam before:
They need to get this done, and it's a lot of work. That's why I always say be careful, we'll probably not see the first real DX12/Vulkan games before next year. They're using the technology, but by far not optimized.
You do not see a lot of performance improvement by current vulkan implementations because the engines are FAR from finished porting to Vulkan.
Just quoting the article about Croteam before:
IHVs engineers know very well that I've been crying for years (since multi-core CPUs started to become standard) to allow us to record GPU command buffers on separate thread. But that didn't happen in DX10, nor DX11, nor any of OpenGL iterations. So our engine was adapted to work with what we had. Now that DX12 and Vulkan came, it will take quite some rewrite to get it all turned upside down. Until that's done, Vulkan will be at disadvantage. But the very fact that it is close to DX11 in speed while it's working practically in emulation shows how much potential there is. This discrepancy is what you see in Doom and TTP vs RorTR.
They need to get this done, and it's a lot of work. That's why I always say be careful, we'll probably not see the first real DX12/Vulkan games before next year. They're using the technology, but by far not optimized.
1 Likes, Who?
That's outdated, it's now enabled by default in the driver and NVIDIA stop if it detects that it's reducing performance...apparently.Nvidia's current driver doesn't have a whitelist for multithreading, instead it enables it on everything and performs some heuristics and disables it on the fly if performance is being affected.I'm not sure it's "on the fly", I think it's more integrated profiles for specific games, as better explained here: https://www.quora.com/How-does-Nvidia-optimize-specific-games-via-driver-updates?share=1
See this driver update: https://www.gamingonlinux.com/articles/nvidia-37809-beta-driver-released-adds-opengl-threaded-optimizations-by-default-and-more.8940
0 Likes
They need to get this done, and it's a lot of work. That's why I always say be careful, we'll probably not see the first real DX12/Vulkan games before next year. They're using the technology, but by far not optimized.I completely agree. I was referring to Vulkan because of the same feeling that "Vulkan will save us all" but for threaded GL.
To be perfectly clear, I never said threaded GL is a bad idea. It's a very good idea in fact. But it won't have the effect on performances that most of the forum users here seem to hope.
A game engine that only use 1 cpu has a lot more problems than Vulkan or threaded GL!
Last edited by Creak on 10 February 2017 at 3:32 pm UTC
0 Likes
That's outdated, it's now enabled by default in the driver and NVIDIA stop if it detects that it's reducing performance...apparently.Be careful here, they're just talking about the threaded GL optimization. My link is about all the other tweaks that are done in the NVIDIA drivers and are game specific.
See this driver update: https://www.gamingonlinux.com/articles/nvidia-37809-beta-driver-released-adds-opengl-threaded-optimizations-by-default-and-more.8940
0 Likes
Ah you're right, wires crossed :)That's outdated, it's now enabled by default in the driver and NVIDIA stop if it detects that it's reducing performance...apparently.Be careful here, they're just talking about the threaded GL optimization. My link is about all the other tweaks that are done in the NVIDIA drivers and are game specific.
See this driver update: https://www.gamingonlinux.com/articles/nvidia-37809-beta-driver-released-adds-opengl-threaded-optimizations-by-default-and-more.8940
0 Likes
I know that I won't change the mind of all the users in this forum, but if I can convince you @liamdawe that GL threaded is good, but not "that good", and that the game engines can be multi-threaded even without the GL threaded feature. Then maybe you would post new articles that won't get the hopes too high for your users.
As a developer in the game industry for 11 years now, I have some knowledge. I completely understand that you don't have to trust me on what I'm saying, but you have access to people that you trust more (Valve, Feral, Aspyr, ...). Ask them your questions like: "is it possible to do some multi-threading in a game engine even though there is no threaded GL?" or "What kind of algorithms can be multi-threaded in an engine apart from the 3D part?"
On my side, I won't say more about this, because I can feel that I'm getting angry at fighting against hopes and beliefs, and that can be very frustrating.
As a developer in the game industry for 11 years now, I have some knowledge. I completely understand that you don't have to trust me on what I'm saying, but you have access to people that you trust more (Valve, Feral, Aspyr, ...). Ask them your questions like: "is it possible to do some multi-threading in a game engine even though there is no threaded GL?" or "What kind of algorithms can be multi-threaded in an engine apart from the 3D part?"
On my side, I won't say more about this, because I can feel that I'm getting angry at fighting against hopes and beliefs, and that can be very frustrating.
4 Likes, Who?
Mblackwell Feb 10, 2017
I was talking about whitelists as it pertains to multithreading GL, which is what the discussion in Mesa is. Nvidia doesn't bother with a list, they just do it on the fly.
That doesn't mean they don't "fast path" certain calls for specific games.
Last edited by Mblackwell on 10 February 2017 at 4:30 pm UTC
That doesn't mean they don't "fast path" certain calls for specific games.
Last edited by Mblackwell on 10 February 2017 at 4:30 pm UTC
0 Likes
I perfectly understand why they don't want to merge bad/broken code. And not, that is not the same as 'unfinished code'. In case of unfinished code (like Vulkan when it was merged) most things that don't work are well understood (known to be unfinished) in this case we have an 'optimization' patch set, which fixes things in some cases, but causes crashes and other random behaviour in other.
The authors, instead of insisting on merging that as is right now, should make sure the code, when enabled, at least passes the Mesa standard test suite. Some tests may be skipped or ignored when they are known to be broken by some missing functionality, but ignoring failing tests 'because someone can fix that later' or 'it is ok, this is not a problem for the games it work with' is a very bad idea.
Unexplained test failure means some undefined behaviour with some unknown trigger. It may start affect games that work now when something changes anywhere in the build or runtime environment. Imagine: steam upgrade and game starts to crash, just because it was on the 'whitelist' and used the bad code, which just happened to work earlier. Steam developers, Mesa developers and game developers (provided they still care) won't be very happy with the bug reports coming.
The authors, instead of insisting on merging that as is right now, should make sure the code, when enabled, at least passes the Mesa standard test suite. Some tests may be skipped or ignored when they are known to be broken by some missing functionality, but ignoring failing tests 'because someone can fix that later' or 'it is ok, this is not a problem for the games it work with' is a very bad idea.
Unexplained test failure means some undefined behaviour with some unknown trigger. It may start affect games that work now when something changes anywhere in the build or runtime environment. Imagine: steam upgrade and game starts to crash, just because it was on the 'whitelist' and used the bad code, which just happened to work earlier. Steam developers, Mesa developers and game developers (provided they still care) won't be very happy with the bug reports coming.
1 Likes, Who?
I know that I won't change the mind of all the users in this forum, but if I can convince you @liamdawe that GL threaded is good, but not "that good", and that the game engines can be multi-threaded even without the GL threaded feature. Then maybe you would post new articles that won't get the hopes too high for your users.I'm well aware of the upsides and downsides to it all, as you say I have contacts in many studios and they all complain about OpenGL, especially the multi-threading. An engine being multi-threaded is different to OpenGL, that's obvious I don't think I ever said the are one and the same, so not really sure what you're getting at there.
As a developer in the game industry for 11 years now, I have some knowledge. I completely understand that you don't have to trust me on what I'm saying, but you have access to people that you trust more (Valve, Feral, Aspyr, ...). Ask them your questions like: "is it possible to do some multi-threading in a game engine even though there is no threaded GL?" or "What kind of algorithms can be multi-threaded in an engine apart from the 3D part?"
On my side, I won't say more about this, because I can feel that I'm getting angry at fighting against hopes and beliefs, and that can be very frustrating.
0 Likes
I know that I won't change the mind of all the users in this forum, but if I can convince you @liamdawe that GL threaded is good, but not "that good", and that the game engines can be multi-threaded even without the GL threaded feature. Then maybe you would post new articles that won't get the hopes too high for your users.
As a developer in the game industry for 11 years now, I have some knowledge. I completely understand that you don't have to trust me on what I'm saying, but you have access to people that you trust more (Valve, Feral, Aspyr, ...). Ask them your questions like: "is it possible to do some multi-threading in a game engine even though there is no threaded GL?" or "What kind of algorithms can be multi-threaded in an engine apart from the 3D part?"
On my side, I won't say more about this, because I can feel that I'm getting angry at fighting against hopes and beliefs, and that can be very frustrating.
Completely agree :)
The IT world in general is far too much about marketing BS, and the gaming world is sometimes part of that. The fact the Mantle/Vulkan/D3D12 exist is testament to the fact that there are issues to be resolved, but there are generally no silver bullets in coding. Threading APIs may be a convenience for developers, but the underlying implementation then has to do GENERIC work to deal with the threading in place of the developer doing SPECIFIC work for their threading needs; it will not always be a good idea.
0 Likes
See more from me