Note: I wrote a guide for the benchmarking, available here.
I decided to run some of my own tests in Dota 2 to see what kind of difference I am actually seeing, the results are surprising.
With many thanks to the person behind the recent video I covered, I was able to get a fair bit of help from him on twitter. I used the same demo file he used to run my tests. This demo file is heavier than the one used by Phoronix and others. Every demo file will give you a different result remember.
All tests run with "desktop friendly fullscreen" mode, not the borderless window mode which is a different option again.
Each test was run four times with the first removed to allow for all systems cache to be built for more accurate results. Tests done using the Dota 2 built-in benchmark tool using "timedemo benchmarkfilename", I simply ran Dota 2 with these launch options (console + API):
or
The benchmark does change graphical settings when you switch resolutions, so don't "do a Liam" and not notice. Make 100% sure the settings match after changing resolutions.
Note: As this is a benchmark, please remember this is probably different to what you will achieve. Unless you have the exact same setup your results will vary.
System
Linux Mint 17.3 64bit (Moving back over to Ubuntu soon, but that's another story)
Linux driver: 364.19 (latest available for Mint)
Intel i7 5960x 3.0ghz, 8 cores
Nvidia 980ti
Results
So in all the Linux tests, OpenGL outperformed Vulkan in Dota 2. Not by much at all in 1080p, but 4K is a different story.
I would love to run some Windows 10 tests in comparison, but Windows wanted to update which slowed everything down, Steam kept going to a 0b/s download for Dota 2 and it's just...so slow. I will hopefully do a Windows 10 comparison when Windows actually plays nicely. I was on it for half an hour and it only got to 40% done on updates, so it would probably take all night to do that + the tests.
Finally, just to remind you that Dota 2 isn't the best benchmark for Vulkan, Valve said it themselves that Dota 2 isn't doing all that much. Vulkan is early, Vulkan in Dota 2 is also very early. Article taken from GamingOnLinux.com.
I decided to run some of my own tests in Dota 2 to see what kind of difference I am actually seeing, the results are surprising.
With many thanks to the person behind the recent video I covered, I was able to get a fair bit of help from him on twitter. I used the same demo file he used to run my tests. This demo file is heavier than the one used by Phoronix and others. Every demo file will give you a different result remember.
All tests run with "desktop friendly fullscreen" mode, not the borderless window mode which is a different option again.
Each test was run four times with the first removed to allow for all systems cache to be built for more accurate results. Tests done using the Dota 2 built-in benchmark tool using "timedemo benchmarkfilename", I simply ran Dota 2 with these launch options (console + API):
+con_enable 1 -vulkanor
+con_enable 1 -glThe benchmark does change graphical settings when you switch resolutions, so don't "do a Liam" and not notice. Make 100% sure the settings match after changing resolutions.
Note: As this is a benchmark, please remember this is probably different to what you will achieve. Unless you have the exact same setup your results will vary.
System
Linux Mint 17.3 64bit (Moving back over to Ubuntu soon, but that's another story)
Linux driver: 364.19 (latest available for Mint)
Intel i7 5960x 3.0ghz, 8 cores
Nvidia 980ti
Results
So in all the Linux tests, OpenGL outperformed Vulkan in Dota 2. Not by much at all in 1080p, but 4K is a different story.
I would love to run some Windows 10 tests in comparison, but Windows wanted to update which slowed everything down, Steam kept going to a 0b/s download for Dota 2 and it's just...so slow. I will hopefully do a Windows 10 comparison when Windows actually plays nicely. I was on it for half an hour and it only got to 40% done on updates, so it would probably take all night to do that + the tests.
Finally, just to remind you that Dota 2 isn't the best benchmark for Vulkan, Valve said it themselves that Dota 2 isn't doing all that much. Vulkan is early, Vulkan in Dota 2 is also very early. Article taken from GamingOnLinux.com.
0 Likes
Some you may have missed, popular articles from the last month:
The comments on this article are closed.
All posts need to follow our rules. For users logged in: please hit the Report Flag icon on any post that breaks the rules or contains illegal / harmful content. Guest readers can email us for any issues.
Hmmm hd to 4k comparison got you weird results. 4 times more pixels to render and only like 2-5 fps less?
0 Likes
Hmmm hd to 4k comparison got you weird results. 4 times more pixels to render and only like 2-5 fps less?For those seeing this comment, the tests had to be re-done. There's a greyed-out slider for graphical quality on Dota 2 that you don't really notice unless you go to advanced options. It changes when you change resolution.
The tests are correct as of this comment btw.
Last edited by Liam Dawe on 15 Jun 2016 at 10:12 pm UTC
0 Likes
Very interesting. Since for me, well, I used the dem of the guy doing the video, Vulkan clearly outperformed OpenGL. Very interesting.
[edit] ok, on 1080 your test does too :D...
Last edited by STiAT on 15 Jun 2016 at 10:25 pm UTC
[edit] ok, on 1080 your test does too :D...
Last edited by STiAT on 15 Jun 2016 at 10:25 pm UTC
0 Likes
Very interesting. Since for me, well, I used the dem of the guy doing the video, Vulkan clearly outperformed OpenGL. Very interesting.I have a feeling Cinnamon is a bit of a hog for games.
Phoronix tests also showed OpenGL beating Vulkan at 4K: http://www.phoronix.com/scan.php?page=article&item=dota2-vulkan-redux&num=4
He had Vulkan doing better at 1080p than OpenGL. He was using a different demo file though and he is on a different distribution with a different CPU. Still, it shows my 4K results are within expectations right now for Vulkan vs OpenGL.
Last edited by Liam Dawe on 16 Jun 2016 at 6:29 am UTC
0 Likes
What kind of compositor does Cinnamon use?Muffin, a fork of Gnome's Mutter.
To be fair, I turned off the compositor for fullscreen applications (in general settings) and the 4K results for Vulkan were even worse.
I don't think Cinnamon is good for testing at all. I will reserve that final judgement for when I get Ubuntu back on though.
1 Likes, Who?
Comparing your test to the test of the guy on the video, your video card is more powerful than a GTX 1080 (same average and better minimum).
The resuls by dubigrasu already show that you can greatly change the results by using another demo :
https://www.gamingonlinux.com/articles/a-new-benchmark-video-shows-dota-2-with-vulkan-performing-better-on-windows-than-linux.7428/page=7#63762
I'm happy that Phoronix will do those Windows vs Linux Vulkan benchmarks too soon, not that I doubt your abilities but I always prefer someone who's got some experience to do them, at least I would be more incline to make some conclusions about Vulkan on Linux with his results.
The resuls by dubigrasu already show that you can greatly change the results by using another demo :
https://www.gamingonlinux.com/articles/a-new-benchmark-video-shows-dota-2-with-vulkan-performing-better-on-windows-than-linux.7428/page=7#63762
I'm happy that Phoronix will do those Windows vs Linux Vulkan benchmarks too soon, not that I doubt your abilities but I always prefer someone who's got some experience to do them, at least I would be more incline to make some conclusions about Vulkan on Linux with his results.
0 Likes
Comparing your test to the test of the guy on the video, your video card is more powerful than a GTX 1080 (same average and better minimum).A GTX 1080 is a more powerful card than my Nvidia 980ti.
The resuls by dubigrasu already show that you can greatly change the results by using another demo :
https://www.gamingonlinux.com/articles/a-new-benchmark-video-shows-dota-2-with-vulkan-performing-better-on-windows-than-linux.7428/page=7#63762
I'm happy that Phoronix will do those Windows vs Linux Vulkan benchmarks too soon, not that I doubt your abilities but I always prefer someone who's got some experience to do them, at least I would be more incline to make some conclusions about Vulkan on Linux with his results.
The results will change with every demo, which I did note in the article. I am also using a different distribution to others.
Phoronix knows plenty more than me that is true (his 4K results on Linux showed the same thing mine did, see previous comment). All I am doing is using the official benchmark tool with a demo, you can't do much else. The graphical settings were confirmed to be the same in every single run, I double checked.
0 Likes
The resuls by dubigrasu already show that you can greatly change the results by using another demoYeah, but which one is the good one ? :)
Personally I used what dang_valve told me to use (after all, he's the damn Source Vulkan dev) for relevant Vulkan benchmarks, but I ran other demos too before that and the results were very different.
So not sure what to believe.
Last edited by dubigrasu on 15 Jun 2016 at 10:55 pm UTC
0 Likes
Believe all of them. That's the point, each demo has a completely different set of actions going on.The resuls by dubigrasu already show that you can greatly change the results by using another demo :Yeah, but which one is the good one ? :)
https://www.gamingonlinux.com/articles/a-new-benchmark-video-shows-dota-2-with-vulkan-performing-better-on-windows-than-linux.7428/page=7#63762
Personally I used what dang_valve told me to use (after all, he's the damn Source Vulkan dev) for relevant Vulkan benchmarks, but I ran other demos too before that and the results were very different.
So not sure what to believe.
I wouldn't just use the ones the Valve dev pointed out, there aren't any that are the "best".
0 Likes
It's okay, different graphical patterns use different features on the GPU. The benchmarks we do can widely differ based on the demos we use, the state of implementation of drivers for a certain GPU architecture etc, driver versions we're using etc.
That all is perfectly okay. All we know now: It's not where we'd like to see Vulkan yet. There is room for improvement in drivers and engines. That's all we can say.
Last edited by STiAT on 15 Jun 2016 at 10:57 pm UTC
That all is perfectly okay. All we know now: It's not where we'd like to see Vulkan yet. There is room for improvement in drivers and engines. That's all we can say.
Last edited by STiAT on 15 Jun 2016 at 10:57 pm UTC
0 Likes
I wouldn't just use the ones the Valve dev pointed out, there aren't any that are the "best".No, not the "best" indeed, they are just meant to illustrate scenarios where Vulkan is efficient.
Ideally I would use more demos with specific scenarios or a single one but much longer.
For the moment though is too early (I think) to spend so much time for a benchmark that tomorrow might be obsolete.
0 Likes
On a GTX 980 over here, I got capped at 120, with the occasional drop to 119.
0 Likes
On a GTX 980 over here, I got capped at 120, with the occasional drop to 119.Are you using the "+fps_max 0" argument?
0 Likes
I have no solution for any of this, and I don't even play Dota. But my limited experience with Vulkan has shown me that it runs much more smoothly, without hitches like I tend to get with OpenGL. In this case, I'm referring to the Talos Principle. I'm guessing it takes stress off the CPU and relies more on the GPU. I have a GTX 970 but a pretty old i5 CPU. The less stress on my CPU, the better (at least until I can upgrade).
Maybe, even though you're seeing slightly lower FPS, it might still be smoother performance and better use of the GPU. I'm totally speculating, but just wondering.
Maybe, even though you're seeing slightly lower FPS, it might still be smoother performance and better use of the GPU. I'm totally speculating, but just wondering.
1 Likes, Who?
cRaZy-bisCuiT 16 Jun 2016
Hmmm hd to 4k comparison got you weird results. 4 times more pixels to render and only like 2-5 fps less?Due to CPU limitation! In 1080p the CPU may allready limit the scenario while the GPU gets bored. Therfor (with a powerful GPU) there's not much of a difference. With better vulkan optimisation of the game and driver the performance cap may be raised a bit. Therfor the gap might get bigger.
A GTX 1080 is a more powerful card than my Nvidia 980ti.True indeed, still the 1080 driver is not well-optimised since it's very new. The difference will grow. Except for the fact the game is CPU limited anyways.
1 Likes, Who?
Maybe, even though you're seeing slightly lower FPS, it might still be smoother performance and better use of the GPU. I'm totally speculating, but just wondering.
This is actually true. FPS tends to fluctuate a lot less on Vulkan. Also input lag is greatly reduced, which really makes a huge difference in competitive games and VR.
Also, I don't think it's because of less stress on CPU (unless you have a really bad cooling) but rather how Vulkan is multi threaded.
Last edited by Blodoffer on 16 Jun 2016 at 4:01 am UTC
0 Likes
There are new NVIDIA drivers out with Added support for the following GPUs:
GeForce GTX 1080
GeForce GTX 1070
and many bug fixes...
http://www.nvidia.co.uk/download/driverResults.aspx/104314/en-uk
Also can someone please post in details step by step how I can do this Dota 2 benchmarks?
I would like to do some benchmarks with my GTX980
Is weird that I cant find any info on this , on every benchmark site noone explains how they did their benchmarks step by step.
GeForce GTX 1080
GeForce GTX 1070
and many bug fixes...
http://www.nvidia.co.uk/download/driverResults.aspx/104314/en-uk
Also can someone please post in details step by step how I can do this Dota 2 benchmarks?
I would like to do some benchmarks with my GTX980
Is weird that I cant find any info on this , on every benchmark site noone explains how they did their benchmarks step by step.
0 Likes
Also can someone please post in details step by step how I can do this Dota 2 benchmarks?I am writing out a guide at the moment, I was also frustrated by the lack of a good guide to it all.
I would like to do some benchmarks with my GTX980
Is weird that I cant find any info on this , on every benchmark site noone explains how they did their benchmarks step by step.
Last edited by Liam Dawe on 16 Jun 2016 at 6:48 am UTC
1 Likes, Who?
Listening to a few Dota developer videos they state that Dota is not really draw call limited. It doesn't make enough calls to throttle DirectX or OpenGL, so you won't see much of a leap in performance with Vulkan.
From the benchmarks between OpenGL and Vulkan on Linux it would suggest the problem is driver (or render implementation) bound. As for the performance variance between Windows and Linux, a mixture of driver, compositor, X11, and a whole heap of other things. I suspect some developers will look into what is causing this variance.
Bare in mind this is from a game in-which the DirectX 9 implementation is still the fastest, even compared to DX10/11 renderers.
From the benchmarks between OpenGL and Vulkan on Linux it would suggest the problem is driver (or render implementation) bound. As for the performance variance between Windows and Linux, a mixture of driver, compositor, X11, and a whole heap of other things. I suspect some developers will look into what is causing this variance.
Bare in mind this is from a game in-which the DirectX 9 implementation is still the fastest, even compared to DX10/11 renderers.
0 Likes
Patreon
PayPal
An idiots guide to setting up Minecraft on Steam Deck / SteamOS with controller support
How to install extra software, apps and games on SteamOS and Steam Deck
See more from me