Some early The Talos Principle Vulkan benchmarks

Updated article. Just having the Steam Overlay on won't actually affect Vulkan until something is rendered with it (fps counter, notification pop ups). Still best to just turn it off.

"From my observations Vulkan seems smoother, with little to no hitching/stutters, where as OpenGL did show some notable hitching/stutters for brief moments."

That's main selling point for me regarding Vulkan.

Quoting: JSalem

Quoting: tuubi

Quoting: JSalemIt would be really useful to have the actual frame data of the benchmark, instead of a short summary (average, extremes etc.). It would be possible to calculate other important metrics (median, for example) and have a more in-depth comparison of vulkan vs opengl vs dx

It's a bit too early for that sort of scrutiny. Their Vulkan backend is very much a work in progress, and you wouldn't get useful data out of the analysis.

I think it would still be useful; ideally you should be able to see exactly where, in their automated demo, there are significant differences between the APIs. This would potentially be insightful for the developers themselves.
More data is always good (provided that the data is annotated clearly)

I'm sure the developers already have this data. They can use dedicated tools for the analysis. What I meant was that providing better in-game benchmarking tools for us "outsiders" isn't a good use of their time at this point.

Quoting: KimyrielleNice to see it outperforming OpenGL (was there any doubt that it would btw.?) But the much more important question is...will it outperform DX12? Because if it doesn't devs will go "Why would we add a Vulkan render path for these 1% that use Linux?" To succeed on the market, being the second fastest API won't be good enough when the fastest one already controls 90% of the market.

Nope, it is a wrong question. Considering trade off - having your engine working on all Windows versions in business plus Linux plus Android plus potentially Nitendo/Sony platforms outweighs insignificant increases in framerates on just Windows 10+. Considering DX12 and Vulkan is so similar, and they both support same shader IR, most likely all serious game developers will support both anyway - if they will be interested in other market than Windows 10.

This "DX12 is so important" myth must end. It is not that important. Other aspects play bigger role.

Quoting: Kimyrielle

Quoting: Mountain Man

Quoting: KimyrielleBut the much more important question is...will it outperform DX12? Because if it doesn't devs will go "Why would we add a Vulkan render path for these 1% that use Linux?"

Performance isn't the only advantage of Vulkan or even the primary advantage. It's the fact that it can work on pretty much every platform that is capable of running games.

You're preaching to the converted in my case (obviously), but you know how these Windows devs operate: "Wait, what? People play games on platforms other than Windows? Are you kidding me?" If you want to get that crowd to use Vulkan, you need to give them something other than "It's running flawlessly on these platforms you don't even care about".

Wow, slow down. Baby steps first. You are NOT gonna convince Windows centric developers to use something else, period. You need to carter developers already *interested* in developing games for Linux. You need to help/serve them first. Make them successful. When they will be successful, *they* will be who will spread the world.

I just love how parts of Linux community still live in nineties. It is not how it works. VHS vs. Betamax everyone? Neither performance, not ease to use will bother Windows developers if they will be actually interested in porting games to SteamOS. They think in trends, in business terms, in branding. If SteamOS/Vulkan combo suddenly trends and becomes successful even with current dev base we have, it WILL make them notice. That will bring them over here.

Stop really think it is about DirectX12. It is NOT. Never have been. If it was, there wouldn't be Linux gaming at all.

Quoting: stssIs the steam overlay even necessary anymore?

Steam Controller seems not to run properly without it.
I felt like this person:
https://www.reddit.com/r/Steam/comments/3p2ms6/psa_you_need_steam_overlay_enabled_to_use_the/

Quoting: tuubi

Quoting: JSalemI think it would still be useful; ideally you should be able to see exactly where, in their automated demo, there are significant differences between the APIs. This would potentially be insightful for the developers themselves.
More data is always good (provided that the data is annotated clearly)

I'm sure the developers already have this data. They can use dedicated tools for the analysis. What I meant was that providing better in-game benchmarking tools for us "outsiders" isn't a good use of their time at this point.

I never mentioned asking other/better in-game benchmarking tools; on the contrary, it is possible already for us to look at the data

Quoting: JSalemI never mentioned asking other/better in-game benchmarking tools; on the contrary, it is possible already for us to look at the data

Ah, go ahead and analyse all you want then. :) I still think that while possibly interesting, any in-depth analysis and reporting on this preview-level Vulkan backend is not useful at this point to anyone but the developer, and based on their previous communication we can assume they already know what they need to work on.

Quoting: PeciskWow, slow down. Baby steps first. You are NOT gonna convince Windows centric developers to use something else, period. You need to carter developers already *interested* in developing games for Linux. You need to help/serve them first. Make them successful. When they will be successful, *they* will be who will spread the world.

I just love how parts of Linux community still live in nineties. It is not how it works. VHS vs. Betamax everyone? Neither performance, not ease to use will bother Windows developers if they will be actually interested in porting games to SteamOS. They think in trends, in business terms, in branding. If SteamOS/Vulkan combo suddenly trends and becomes successful even with current dev base we have, it WILL make them notice. That will bring them over here.

Stop really think it is about DirectX12. It is NOT. Never have been. If it was, there wouldn't be Linux gaming at all.

Most of the non-completely Windows centric studios support us already (seriously, what's left other than EA, Ubisoft, Blizzard, Rockstar and Bethesda?). They are successful, but obviously 95% of their sales are still Windows sales. SteamOS will become a major thing only if AAA games are available in large numbers AND if said games run at -least- as well as they would do in Windows. If the fastest available renderer is Windows/XBox-only, that's what gamers will buy. If you tell a mainstream gamer that he will get 80% of the possible performance out of his $500 graphics card he will laugh you off. These people invest 100s of dollars for a few more extra framerates.
This is not even remotely comparable to VHS vs BetaMax. This is waaaaay worse than that, as SteamOS is breaking into an existing market that's already dominated by a competing product. This is more like as if BetaMax tried to enter the market against VHS in the 1990s, when everybody had VHS already. If you attempt something like that, you have to be either massively better or massively cheaper than the established product. And in this case you can't be cheaper, as DX12 is free as in beer. So you have to be better.

Nothing suddenly "trends" without reason. Apple became trendy when they released innovative products nobody had before.
While platform-independence -does- matter from our point of view (it's why we want Vulkan to succeed, right?), from the perspective of one of the aforementioned Windows-only studios, it does not matter as long as they have no plans to release on alternative platforms anyway. Bethesda made over a billion dollars with Fallout 4, and they still couldn't be bothered investing a comparably laughable sum to have that game ported to Mac and Linux. Good luck with trying to convince these guys adding a second render path to their games. Even more good luck trying to convince the average gamer to run his games in Linux if they would perform -slower- there. Vulkan does simply not have a business case then. UNLESS Vulkan performs at least as well as DX12 and becomes an -alternative- to it. Then it suddenly has a business case. It would become a "Ok, this thing is as good as DX12 and we can increase our target market by 10% on top of it just by using it? Let's do it!" thing for these CEOs. And only THEN will SteamOS have a chance to become "trendy"
So if you think the DX12 comparison doesn't matter, you're a bit delusional, tbh. Mind you we're not talking about some mobile ports here, where Linux actually is market leader and Windows doesn't matter at all. We're talking about high end AAA games we desperately want to see coming to Linux, and these wouldn't run on 'droid anyway.

Quoting: GuestThe steam overlay is pretty bad for performance, and is really a hack job more than anything else. Plus, it has to work across the board - so it's using an ancient version of OpenGL, not even running in the same context as the application. So it's no surprise that it impacts performance so much.

I'm curious as to whether CPU usage evens out with Vulkan or not. It speaks a lot about where the improvements are really coming from.

Not sure im understanding this, are you saying if i disable the overlay in standard OGL games i can get a performance boost ? because i don't really use the overlay and work space switch instead


Last edited by on 14 April 2016 at 4:38 pm UTC

Quoting: Purple Library GuyWell, and correct me if I'm wrong but I seem to recall hearing that PS4 will run Vulkan . . .

If you go by the supporters list, Nintendo may support Vulkan with their upcoming NX console too.

You may want to re-test the game with steam overlay:

Steam Client Beta Update - April 14th
(...)
Steam Overlay
Improved rendering performance with Vulkan applications

I did other tests after updating Steam.

OS: Debian 8.4 (Jessie) - 3.16.0-4-amd64
CPU: i5 3570K
RAM: 8GB
GPU: Nvidia 660GTX - 364.15

Here are results I got yesterday:
Steam Overlay enabled: OpenGL 82fps - Vulkan 58fps
Steam Overlay disabled: OpenGL 83fps - Vulkan 107fps

Here are results I got today:
Steam Overlay enabled: OpenGL 82fps - Vulkan 107fps
Steam Overlay disabled: OpenGL 83fps - Vulkan 107fps

Steam Overlay has now no impact with Vulkan rendering on my system.

I also did the full benchmark test (184 sec) on Vulkan with Steam Overlay enabled to see if there is any downside, and I got 114 fps in average. ^_^

Spoiler, click me

### OpenGL with Steam Overlay

11:31:42 INF: - benchmark results -
11:31:42 INF:
11:31:42 INF: Gfx API: OpenGL
11:31:42 INF: Duration: 60.0 seconds (4922 frames)
11:31:42 INF: Average: 82.0 FPS (89.6 w/o extremes)
11:31:42 INF: Extremes: 152.3 max, 20.9 min
11:31:42 INF: Sections: AI=6%, physics=2%, sound=1%, scene=62%, shadows=23%, misc=7%
11:31:42 INF: Highs: 492 in 4.0 seconds (123.6 FPS)
11:31:42 INF: Lows: 970 in 17.4 seconds (55.7 FPS)
11:31:42 INF: 30-60 FPS: 14%
11:31:42 INF: > 60 FPS: 86%

### Vulkan with Steam Overlay

11:29:55 INF: - benchmark results -
11:29:55 INF:
11:29:55 INF: Gfx API: Vulkan
11:29:55 INF: Duration: 60.0 seconds (6434 frames)
11:29:55 INF: Average: 107.2 FPS (110.7 w/o extremes)
11:29:55 INF: Extremes: 255.1 max, 6.5 min
11:29:55 INF: Sections: AI=8%, physics=3%, sound=1%, scene=63%, shadows=14%, misc=11%
11:29:55 INF: Highs: 31 in 0.1 seconds (214.0 FPS)
11:29:55 INF: Lows: 158 in 3.5 seconds (45.7 FPS)
11:29:55 INF: 30-60 FPS: 1%
11:29:55 INF: > 60 FPS: 99%

### OpenGL without Steam Overlay

11:36:40 INF: - benchmark results -
11:36:40 INF:
11:36:40 INF: Gfx API: OpenGL
11:36:40 INF: Duration: 60.0 seconds (4984 frames)
11:36:40 INF: Average: 83.1 FPS (90.2 w/o extremes)
11:36:40 INF: Extremes: 150.5 max, 34.9 min
11:36:40 INF: Sections: AI=6%, physics=2%, sound=1%, scene=62%, shadows=23%, misc=6%
11:36:40 INF: Highs: 528 in 4.2 seconds (125.0 FPS)
11:36:40 INF: Lows: 962 in 17.0 seconds (56.5 FPS)
11:36:40 INF: 30-60 FPS: 13%
11:36:40 INF: > 60 FPS: 87%

### Vulkan without Steam Overlay

11:35:01 INF: - benchmark results -
11:35:01 INF:
11:35:01 INF: Gfx API: Vulkan
11:35:01 INF: Duration: 60.0 seconds (6469 frames)
11:35:01 INF: Average: 107.8 FPS (110.3 w/o extremes)
11:35:01 INF: Extremes: 242.0 max, 7.0 min
11:35:01 INF: Sections: AI=8%, physics=2%, sound=1%, scene=64%, shadows=14%, misc=11%
11:35:01 INF: Highs: 91 in 0.5 seconds (193.7 FPS)
11:35:01 INF: Lows: 177 in 3.3 seconds (53.3 FPS)
11:35:01 INF: 30-60 FPS: 1%
11:35:01 INF: > 60 FPS: 99%

### Vulkan with Steam Overlay - Full benchmark test

11:51:05 INF: - benchmark results -
11:51:05 INF:
11:51:05 INF: Gfx API: Vulkan
11:51:05 INF: Duration: 184.8 seconds (21230 frames)
11:51:05 INF: Average: 114.9 FPS (117.1 w/o extremes)
11:51:05 INF: Extremes: 302.7 max, 5.8 min
11:51:05 INF: Sections: AI=10%, physics=3%, sound=1%, scene=61%, shadows=15%, misc=11%
11:51:05 INF: Highs: 319 in 1.5 seconds (210.0 FPS)
11:51:05 INF: Lows: 559 in 9.5 seconds (58.6 FPS)
11:51:05 INF: > 60 FPS: 100%

Last edited by Nel on 15 April 2016 at 10:00 am UTC

Just now i finished the Talos Principle with Vulkan :D
I climped the tower. Nice ending.

Started it a few weeks ago with OpenGL and switch to Vulkan as soon as the nvidia driver got into Arch linux.

Quoting: GuestFor me too Talos is slower with Vulkan: 49 FPS instead of 65 FPS in OpenGL.
Nvidia 364.19, GTX 660, i5 4690K, Steam overlay disabled, 1920×1200, Xfce (no compositing).

You've got a 660 just like Nel above, and he gets much better performance using Vulkan, with a slightly older beta driver from the same series. Does the Vulkan implementation use more video memory (and you've got less than him), or maybe your settings are different somehow? Oh, or maybe you're running the game from a slower drive and their new Vulkan backend doesn't do asset/shader loading/caching/compiling as efficiently as the OpenGL backend? I also assume you might not be running Debian Jessie with an older kernel like him. The possibilites are endless. :)

Hey Stan, I just performed another benchmark with Nvidia 364.19

More informations about my system.

OS: Debian 8.4 (Jessie) - 3.16.0-4-amd64
CPU: i5 3570K
RAM: 8GB
GPU: Nvidia 660GTX - 364.19
Display: 1920x1080
xorg.conf: TripleBuffer True, RenderAccel True, DamageEvents True
DE: KDE 4.14.2, Desktop Effects enabled
Steam: Steam beta branch, Steam overlay enabled
Talos version: publicbeta - Public testing
Game launch options: %command% +gfx_strAPI "VLK"
Talos game: Play (64bits) on launch, graphics options autodetect

Results are pretty consistent:
OpenGL: 84fps
Vulkan: 106fps

Spoiler, click me

### Vulkan with Steam Overlay

12:56:37 INF: - benchmark results -
12:56:37 INF:
12:56:37 INF: Gfx API: Vulkan
12:56:37 INF: Duration: 60.0 seconds (6380 frames)
12:56:37 INF: Average: 106.3 FPS (110.8 w/o extremes)
12:56:37 INF: Extremes: 248.2 max, 9.4 min
12:56:37 INF: Sections: AI=8%, physics=3%, sound=1%, scene=62%, shadows=13%, misc=13%
12:56:37 INF: Highs: 92 in 0.4 seconds (204.8 FPS)
12:56:37 INF: Lows: 231 in 4.9 seconds (47.4 FPS)
12:56:37 INF: 30-60 FPS: 2%
12:56:37 INF: > 60 FPS: 98%

### OpenGL with Steam Overlay

12:58:20 INF: - benchmark results -
12:58:20 INF:
12:58:20 INF: Gfx API: OpenGL
12:58:20 INF: Duration: 60.0 seconds (5053 frames)
12:58:20 INF: Average: 84.2 FPS (91.8 w/o extremes)
12:58:20 INF: Extremes: 148.6 max, 12.2 min
12:58:20 INF: Sections: AI=6%, physics=2%, sound=1%, scene=63%, shadows=22%, misc=7%
12:58:20 INF: Highs: 461 in 3.7 seconds (126.0 FPS)
12:58:20 INF: Lows: 935 in 16.5 seconds (56.7 FPS)
12:58:20 INF: 30-60 FPS: 12%
12:58:20 INF: > 60 FPS: 88%