Note: I wrote a guide for the benchmarking, available here.
I decided to run some of my own tests in Dota 2 to see what kind of difference I am actually seeing, the results are surprising.
With many thanks to the person behind the recent video I covered, I was able to get a fair bit of help from him on twitter. I used the same demo file he used to run my tests. This demo file is heavier than the one used by Phoronix and others. Every demo file will give you a different result remember.
All tests run with "desktop friendly fullscreen" mode, not the borderless window mode which is a different option again.
Each test was run four times with the first removed to allow for all systems cache to be built for more accurate results. Tests done using the Dota 2 built-in benchmark tool using "timedemo benchmarkfilename", I simply ran Dota 2 with these launch options (console + API):
or
The benchmark does change graphical settings when you switch resolutions, so don't "do a Liam" and not notice. Make 100% sure the settings match after changing resolutions.
Note: As this is a benchmark, please remember this is probably different to what you will achieve. Unless you have the exact same setup your results will vary.
System
Linux Mint 17.3 64bit (Moving back over to Ubuntu soon, but that's another story)
Linux driver: 364.19 (latest available for Mint)
Intel i7 5960x 3.0ghz, 8 cores
Nvidia 980ti
Results
So in all the Linux tests, OpenGL outperformed Vulkan in Dota 2. Not by much at all in 1080p, but 4K is a different story.
I would love to run some Windows 10 tests in comparison, but Windows wanted to update which slowed everything down, Steam kept going to a 0b/s download for Dota 2 and it's just...so slow. I will hopefully do a Windows 10 comparison when Windows actually plays nicely. I was on it for half an hour and it only got to 40% done on updates, so it would probably take all night to do that + the tests.
Finally, just to remind you that Dota 2 isn't the best benchmark for Vulkan, Valve said it themselves that Dota 2 isn't doing all that much. Vulkan is early, Vulkan in Dota 2 is also very early. Article taken from GamingOnLinux.com.
I decided to run some of my own tests in Dota 2 to see what kind of difference I am actually seeing, the results are surprising.
With many thanks to the person behind the recent video I covered, I was able to get a fair bit of help from him on twitter. I used the same demo file he used to run my tests. This demo file is heavier than the one used by Phoronix and others. Every demo file will give you a different result remember.
All tests run with "desktop friendly fullscreen" mode, not the borderless window mode which is a different option again.
Each test was run four times with the first removed to allow for all systems cache to be built for more accurate results. Tests done using the Dota 2 built-in benchmark tool using "timedemo benchmarkfilename", I simply ran Dota 2 with these launch options (console + API):
+con_enable 1 -vulkanor
+con_enable 1 -glThe benchmark does change graphical settings when you switch resolutions, so don't "do a Liam" and not notice. Make 100% sure the settings match after changing resolutions.
Note: As this is a benchmark, please remember this is probably different to what you will achieve. Unless you have the exact same setup your results will vary.
System
Linux Mint 17.3 64bit (Moving back over to Ubuntu soon, but that's another story)
Linux driver: 364.19 (latest available for Mint)
Intel i7 5960x 3.0ghz, 8 cores
Nvidia 980ti
Results
So in all the Linux tests, OpenGL outperformed Vulkan in Dota 2. Not by much at all in 1080p, but 4K is a different story.
I would love to run some Windows 10 tests in comparison, but Windows wanted to update which slowed everything down, Steam kept going to a 0b/s download for Dota 2 and it's just...so slow. I will hopefully do a Windows 10 comparison when Windows actually plays nicely. I was on it for half an hour and it only got to 40% done on updates, so it would probably take all night to do that + the tests.
Finally, just to remind you that Dota 2 isn't the best benchmark for Vulkan, Valve said it themselves that Dota 2 isn't doing all that much. Vulkan is early, Vulkan in Dota 2 is also very early. Article taken from GamingOnLinux.com.
0 Likes
Some you may have missed, popular articles from the last month:
The comments on this article are closed.
Hmmm hd to 4k comparison got you weird results. 4 times more pixels to render and only like 2-5 fps less?
0 Likes
Quoting: wojtek88Hmmm hd to 4k comparison got you weird results. 4 times more pixels to render and only like 2-5 fps less?For those seeing this comment, the tests had to be re-done. There's a greyed-out slider for graphical quality on Dota 2 that you don't really notice unless you go to advanced options. It changes when you change resolution.
The tests are correct as of this comment btw.
Last edited by Liam Dawe on 15 June 2016 at 10:12 pm UTC
0 Likes
Very interesting. Since for me, well, I used the dem of the guy doing the video, Vulkan clearly outperformed OpenGL. Very interesting.
[edit] ok, on 1080 your test does too :D...
Last edited by STiAT on 15 June 2016 at 10:25 pm UTC
[edit] ok, on 1080 your test does too :D...
Last edited by STiAT on 15 June 2016 at 10:25 pm UTC
0 Likes
Quoting: STiATVery interesting. Since for me, well, I used the dem of the guy doing the video, Vulkan clearly outperformed OpenGL. Very interesting.I have a feeling Cinnamon is a bit of a hog for games.
Phoronix tests also showed OpenGL beating Vulkan at 4K: http://www.phoronix.com/scan.php?page=article&item=dota2-vulkan-redux&num=4
He had Vulkan doing better at 1080p than OpenGL. He was using a different demo file though and he is on a different distribution with a different CPU. Still, it shows my 4K results are within expectations right now for Vulkan vs OpenGL.
Last edited by Liam Dawe on 16 June 2016 at 6:29 am UTC
0 Likes
Quoting: STiATWhat kind of compositor does Cinnamon use?Muffin, a fork of Gnome's Mutter.
To be fair, I turned off the compositor for fullscreen applications (in general settings) and the 4K results for Vulkan were even worse.
I don't think Cinnamon is good for testing at all. I will reserve that final judgement for when I get Ubuntu back on though.
1 Likes, Who?
Comparing your test to the test of the guy on the video, your video card is more powerful than a GTX 1080 (same average and better minimum).
The resuls by dubigrasu already show that you can greatly change the results by using another demo :
https://www.gamingonlinux.com/articles/a-new-benchmark-video-shows-dota-2-with-vulkan-performing-better-on-windows-than-linux.7428/page=7#63762
I'm happy that Phoronix will do those Windows vs Linux Vulkan benchmarks too soon, not that I doubt your abilities but I always prefer someone who's got some experience to do them, at least I would be more incline to make some conclusions about Vulkan on Linux with his results.
The resuls by dubigrasu already show that you can greatly change the results by using another demo :
https://www.gamingonlinux.com/articles/a-new-benchmark-video-shows-dota-2-with-vulkan-performing-better-on-windows-than-linux.7428/page=7#63762
I'm happy that Phoronix will do those Windows vs Linux Vulkan benchmarks too soon, not that I doubt your abilities but I always prefer someone who's got some experience to do them, at least I would be more incline to make some conclusions about Vulkan on Linux with his results.
0 Likes
Quoting: NyamiouComparing your test to the test of the guy on the video, your video card is more powerful than a GTX 1080 (same average and better minimum).A GTX 1080 is a more powerful card than my Nvidia 980ti.
The resuls by dubigrasu already show that you can greatly change the results by using another demo :
https://www.gamingonlinux.com/articles/a-new-benchmark-video-shows-dota-2-with-vulkan-performing-better-on-windows-than-linux.7428/page=7#63762
I'm happy that Phoronix will do those Windows vs Linux Vulkan benchmarks too soon, not that I doubt your abilities but I always prefer someone who's got some experience to do them, at least I would be more incline to make some conclusions about Vulkan on Linux with his results.
The results will change with every demo, which I did note in the article. I am also using a different distribution to others.
Phoronix knows plenty more than me that is true (his 4K results on Linux showed the same thing mine did, see previous comment). All I am doing is using the official benchmark tool with a demo, you can't do much else. The graphical settings were confirmed to be the same in every single run, I double checked.
0 Likes
Quoting: NyamiouThe resuls by dubigrasu already show that you can greatly change the results by using another demoYeah, but which one is the good one ? :)
Personally I used what dang_valve told me to use (after all, he's the damn Source Vulkan dev) for relevant Vulkan benchmarks, but I ran other demos too before that and the results were very different.
So not sure what to believe.
Last edited by dubigrasu on 15 June 2016 at 10:55 pm UTC
0 Likes
Quoting: dubigrasuBelieve all of them. That's the point, each demo has a completely different set of actions going on.Quoting: NyamiouThe resuls by dubigrasu already show that you can greatly change the results by using another demo :Yeah, but which one is the good one ? :)
https://www.gamingonlinux.com/articles/a-new-benchmark-video-shows-dota-2-with-vulkan-performing-better-on-windows-than-linux.7428/page=7#63762
Personally I used what dang_valve told me to use (after all, he's the damn Source Vulkan dev) for relevant Vulkan benchmarks, but I ran other demos too before that and the results were very different.
So not sure what to believe.
I wouldn't just use the ones the Valve dev pointed out, there aren't any that are the "best".
0 Likes
Patreon
PayPal
See more from me