In an effort to make things as clear as possible, I decided to do some additional testing with Mad Max [Steam, Feral Store]. More specifically, in OpenGL for Mad Max at release vs the beta.
This is important, since our previous benchmarks (and every other website and person who ran benchmarks) will be affected by this.
Why was this not tested before? Well, we weren't made aware of any changes to the OpenGL renderer in Mad Max. There should have been no need to do tests like this since the whole idea behind the Beta was to test Vulkan specifically, but hopefully this will help clear things up a bit.
I ran some of my own tests using GLXOSD's benchmarking tool coupled with my speedy SSD on the Very High preset (with no extra game adjustments) and here's what I found. The results, sadly, speak for themselves. Done in two completely different areas to be sure it wasn't a fluke.
As always, benchmarks should always be taken with a grain of salt. It will depend heavily on your system setup. Not just hardware, but the exact distro and libs you have too all affect the numbers.
This was tricky to do, since the Beta version does have other issues with OpenGL, which is in the form of crash bugs. I've had a few crashes I've reported to Feral in the beta version when using OpenGL, so hopefully those can also get fixed. It's also tricky since the original release didn't feature a benchmark mode, so it had to all be done manually.
The good news is that Feral Interactive are aware of it, so they can look to fix it. The other good news, is that Vulkan is still performing better than both versions of the game when using OpenGL.
This regression might also be affecting the Vulkan renderer, so it's possible when fixed both the OpenGL and Vulkan versions could see a boost, although the OpenGL boost would likely only be putting it back to normal levels.
Finally, something to remember: This is exactly what a Beta test is for. Article taken from GamingOnLinux.com.
This is important, since our previous benchmarks (and every other website and person who ran benchmarks) will be affected by this.
Why was this not tested before? Well, we weren't made aware of any changes to the OpenGL renderer in Mad Max. There should have been no need to do tests like this since the whole idea behind the Beta was to test Vulkan specifically, but hopefully this will help clear things up a bit.
I ran some of my own tests using GLXOSD's benchmarking tool coupled with my speedy SSD on the Very High preset (with no extra game adjustments) and here's what I found. The results, sadly, speak for themselves. Done in two completely different areas to be sure it wasn't a fluke.
As always, benchmarks should always be taken with a grain of salt. It will depend heavily on your system setup. Not just hardware, but the exact distro and libs you have too all affect the numbers.
This was tricky to do, since the Beta version does have other issues with OpenGL, which is in the form of crash bugs. I've had a few crashes I've reported to Feral in the beta version when using OpenGL, so hopefully those can also get fixed. It's also tricky since the original release didn't feature a benchmark mode, so it had to all be done manually.
The good news is that Feral Interactive are aware of it, so they can look to fix it. The other good news, is that Vulkan is still performing better than both versions of the game when using OpenGL.
This regression might also be affecting the Vulkan renderer, so it's possible when fixed both the OpenGL and Vulkan versions could see a boost, although the OpenGL boost would likely only be putting it back to normal levels.
Finally, something to remember: This is exactly what a Beta test is for. Article taken from GamingOnLinux.com.
Some you may have missed, popular articles from the last month:
The comments on this article are closed.
I fear these are more realistic numbers. Would be happy to be wrong, of course. I think we can be happy when Mad Max runs without its frame rate drops and in weaker processors.
1 Likes, Who?
For me the performance increase OpenGL vs Vulkan was definitely a lot more significant. I'd still say it probably doubled or more. Maybe it is because I have quite low single-core performance with my AMD CPU.
5 Likes, Who?
rustybroomhandle Mar 31, 2017
I am running Manjaro and have noticed on kernel 4.9 this game stutters terribly, with frame rate wildly fluctuating. I booted up with kernel 4.5 and all of a sudden it's smoother than suppository dipped in Vaseline.
1 Likes, Who?
We also have to keep in mind that benchmarks are often run with fast CPUs. Vulkans brings more than just fps: It significantly decreases the load on the CPU, and therefore the system requirements on the CPU side. Feral games have high CPU requirements, in case of Mad Max lists 3.4 GHz as a minimum, which is really high. As the CPU load is now significantly lower with Vulkan, the game should run with a lower CPU.
And because the CPU in the average PC is a lot weaker than the CPU in the average benchmark, I expect the user benefit to be much more significant than the few fps difference that the benchmarks are showing us.
Last edited by dmantione on 31 March 2017 at 8:15 pm UTC
And because the CPU in the average PC is a lot weaker than the CPU in the average benchmark, I expect the user benefit to be much more significant than the few fps difference that the benchmarks are showing us.
Last edited by dmantione on 31 March 2017 at 8:15 pm UTC
10 Likes, Who?
natewardawg Mar 31, 2017
Which benchmarking expert was it on the BSD Now show that said something to the affect of, "The first thing to remember about benchmarks is that your benchmarks are incorrect" :)
I did a little bit of [informal] testing on this as well. My Bonobo Extreme laptop has the following specs for my testing.
1920x1080
Normal quality
i7-4810MQ Quad Core @ 2.8 GHz
GTX 860M 4GB
I concluded my results by staring at Steam's FPS counter, haha :D
Stable branch:
~20-22 FPS OpenGL
Beta branch:
~15-17 FPS OpenGL
~25-30 FPS Vulkan
So on this particular machine, regardless of this regression, it changed Mad Max from being a noticeably unplayable experience under OpenGL to being a noticeably playable one under Vulkan.
I will eventually (tonight?) test the same thing with my i7-6700K with a 960 GTX.
Last edited by natewardawg on 31 March 2017 at 8:19 pm UTC
I did a little bit of [informal] testing on this as well. My Bonobo Extreme laptop has the following specs for my testing.
1920x1080
Normal quality
i7-4810MQ Quad Core @ 2.8 GHz
GTX 860M 4GB
I concluded my results by staring at Steam's FPS counter, haha :D
Stable branch:
~20-22 FPS OpenGL
Beta branch:
~15-17 FPS OpenGL
~25-30 FPS Vulkan
So on this particular machine, regardless of this regression, it changed Mad Max from being a noticeably unplayable experience under OpenGL to being a noticeably playable one under Vulkan.
I will eventually (tonight?) test the same thing with my i7-6700K with a 960 GTX.
Last edited by natewardawg on 31 March 2017 at 8:19 pm UTC
2 Likes, Who?
I think it's worth mentioning that Mad Max's Vulkan backend is in beta. Feral is likely to keep tweaking and eking out as much performance as they can. Give it time.
3 Likes, Who?
Mountain Man Mar 31, 2017
As someone pointed out on reddit, are we sure there's not a bug that is affecting both OpenGL and Vulkan to the same degree?
But anyway, it's a beta build, so the kinks will undoubtedly be ironed out as development moves forward.
But anyway, it's a beta build, so the kinks will undoubtedly be ironed out as development moves forward.
1 Likes, Who?
I am generally experiencing some significant frame skips in Mad Max just recently, but it was ok earlier. It's worth to mention however, but the game performance seems far better when using Vulkan. Oh by the way, do you know any applications that can show FPS'es and stuff while running a game? I know of GLXOSD but it doesn't show any output when running the Vulkan beta. And also, it seems to stuck on 60fpses in any game.
0 Likes
Quoting: rustybroomhandleI am running Manjaro and have noticed on kernel 4.9 this game stutters terribly, with frame rate wildly fluctuating. I booted up with kernel 4.5 and all of a sudden it's smoother than suppository dipped in Vaseline.
Spoken like someone who knows what he's talking about!
1 Likes, Who?
Quoting: rustybroomhandleI am running Manjaro and have noticed on kernel 4.9 this game stutters terribly, with frame rate wildly fluctuating. I booted up with kernel 4.5 and all of a sudden it's smoother than suppository dipped in Vaseline.Is that with OpenGL or Vulkan, likely related to this if it's Vulkan.
0 Likes
Quoting: aFoxNamedMorrisI think it's worth mentioning that Mad Max's Vulkan backend is in beta.
Quoting: liamdawe, in the articleFinally, something to remember: This is exactly what a Beta test is for.
1 Likes, Who?
natewardawg Mar 31, 2017
Quoting: Adam_eMOh by the way, do you know any applications that can show FPS'es and stuff while running a game? I know of GLXOSD but it doesn't show any output when running the Vulkan beta. And also, it seems to stuck on 60fpses in any game.
Steam has a basic FPS counter you can turn on under Steam > Settings > In Game > In-game FPS counter
Obviously it's quite basic. I would also like to know if there's something available that overlays other thing too like CPU usage. :)
1 Likes, Who?
Quoting: natewardawg1920x1080
Normal quality
i7-4810MQ Quad Core @ 2.8 GHz
GTX 860M 4GB
I concluded my results by staring at Steam's FPS counter, haha :D
Stable branch:
~20-22 FPS OpenGL
Beta branch:
~15-17 FPS OpenGL
~25-30 FPS Vulkan
So on this particular machine, regardless of this regression, it changed Mad Max from being a noticeably unplayable experience under OpenGL to being a noticeably playable one under Vulkan.
I run a (very) old AMD FX 8120 Bulldozer with a gtx 970 and for me Vulkan gave a few fps more, but nothing tremendous.
It's still pretty choppy overall in the normal preset at 720P, with some regions with the frame rate boosting to the hundreds fps.
"benchmark 2" gave me an average of 32 fps in Vulkan and 28 fps in OGL.
I was a bit underwhelmed, I admit...
Probably time to jump on the Ryzen train.
1 Likes, Who?
These results are more realistic. 2.5x performance boost is just too much for just an API switch.
1 Likes, Who?
natewardawg Mar 31, 2017
Quoting: tripyI run a (very) old AMD FX 8120 Bulldozer with a gtx 970 and for me Vulkan gave a few fps more, but nothing tremendous.
It's still pretty choppy overall in the normal preset at 720P, with some regions with the frame rate boosting to the hundreds fps.
"benchmark 2" gave me an average of 32 fps in Vulkan and 28 fps in OGL.
I was a bit underwhelmed, I admit...
Probably time to jump on the Ryzen train.
I ran my benchmarks in 720p also and got 33.5 fps under Vulkan for benchmark #2, but on an 860M which is far inferior to your GTX 970. I do have a GTX 960 with an i7-6700K, it just hasn't downloaded Mad Max yet on that system for me to benchmark it :)
So, I agree, it's probably CPU bound, because I got better results and according to http://cpubenchmark.net/ your CPU scores in the 6000's and mine scores in the 8000's, which would explain why I got a better framerate on a significantly inferior GPU :)
If I remember, I will post my stats when I get it running on my other machine. I may do it at 1920x1080 though.
Last edited by natewardawg on 31 March 2017 at 10:16 pm UTC
1 Likes, Who?
Quoting: natewardawgIf I remember, I will post my stats when I get it running on my other machine. I may do it at 1920x1080 though.
Thanks. Much appreciated.
Edit: wups.. It was wishful thinking, but I've got a GTX 960, not 970.
Damn, time to get some Z I guess too.
Last edited by tripy on 31 March 2017 at 10:27 pm UTC
0 Likes
Quoting: Adam_eMI am generally experiencing some significant frame skips in Mad Max just recently, but it was ok earlier. It's worth to mention however, but the game performance seems far better when using Vulkan. Oh by the way, do you know any applications that can show FPS'es and stuff while running a game? I know of GLXOSD but it doesn't show any output when running the Vulkan beta. And also, it seems to stuck on 60fpses in any game.
It's definitely faster for me, but I've noticed the skipping/stuttering as well, even when it's above 60 fps. Hopefully, they can iron that out. I didn't try it with the framerate smoothing enabled. Would that help?
0 Likes
I've tested the old version against the new Vulkan in a couple (non benchmark) camps that gave me very low frames before. The bad one increased from 22 to 35, which is a big difference. Note that this is at very high settings and on 3440x1440. That changes the game from stuttery to perfectly playable. I just wish that NVidia would get G-SYNC working on Vulkan.
I have noticed a couple of graphical artefacts though. At one point the canteen flashed before filling it up. Another one was that I saw some flashing white lines that looked like they could have been ground mesh edges. Haven't been able to screenshot them, so I may have to try and screen capture a video.
I have noticed a couple of graphical artefacts though. At one point the canteen flashed before filling it up. Another one was that I saw some flashing white lines that looked like they could have been ground mesh edges. Haven't been able to screenshot them, so I may have to try and screen capture a video.
1 Likes, Who?
natewardawg Apr 1, 2017
Quoting: tripyQuoting: natewardawgIf I remember, I will post my stats when I get it running on my other machine. I may do it at 1920x1080 though.
Thanks. Much appreciated.
Edit: wups.. It was wishful thinking, but I've got a GTX 960, not 970.
Damn, time to get some Z I guess too.
Here it is... I actually did the benchmark at both 720p and 1080p with normal quality.
Specs:
i7-6700K
GTX 960
32 GB RAM
720p:
avg: 80.905754
min: 57.537403
max: 87.5733411080p:
avg: 81.353348
min: 64.316956
max: 86.662621Yep, almost exactly the same... so this scene is definitely CPU bound. So, I'm pretty sure upgrading your system with a Ryzen chip would cause this scene to outperform my system even with the same GPU. I'm going to do some more tests for the other two quality levels, at some point :)
Last edited by natewardawg on 1 April 2017 at 12:42 am UTC
1 Likes, Who?
natewardawg Apr 1, 2017
Gaming On Windows 10!!! Too funny Liam! :D
3 Likes, Who?
Patreon
PayPal
See more from me