Ars Technica Benchmarks Show Windows 10 Beating SteamOS Performance

Arse Technica. There, I said it, for no special reason.

Quoting: mulletdeathI'll also add that I'm really disappointed in Valve's marketing, or complete lack thereof, of SteamOS. Over in /r/linux_gaming the meme going around is that "Valve is playing the long game, just like the launch of Steam itself, there's nothing to worry about guys!" The launch of Steam and SteamOS are two totally different situations, so I do wonder how well this strategy is going to work out. It really seems like Valve doesn't care if SteamOS totally flops because they're still rolling in dough either way.

It's in line with what Valve themselves stated as their plans for SteamOS and Steam hardware.
From an IGN interview:

Quoting: Valve on IGN"Usually, when a platform like this gets brought out, it’s a very different working method and proposition to customers,” he continued. “It looks more like a team that’s much larger than us has worked at perfecting something and finishing it, and then reducing risk as much possible and locking down that design, making it ready for a massive initial manufacturing push, spending billions of dollars on marketing. Very different from what we’re trying to do. We don’t have to be so risk-averse. We intentionally are operating this way because we think it will result in a much better product, in the short term and the long term, to be public about this, and to have it iterated with us and with partners and with users. But it lets us start small and grow over time.”

From a Guardian interview:

Quoting: Valve on The Guardian“We built Steam for the same reasons we did all this: it didn’t exist, a bunch of people wanted it to exist, no one was building it, so we built it. There was this post-rationalisation of Steam after it was launched; like ‘of course you’d build that, it’s obvious’ – but, at the time, when we were building it, no one else was,” says Valve’s business development specialist Erik Johnson.

“Who knows if it’ll be the same this time, but if you ask customers, ‘do you want to be able to play all your Steam games in your living room with a controller?’, they’re like: ‘yeah, that sounds cool.’ That’s good enough for us to move forward. We’re not obsessed with what market we’re creating or what users we’re cannibalising from other platforms. No one ever makes good decisions based off that kind of data.”

And as you say, they can afford it, they have the money for this kind of approach.

QuoteRight now most ports run worse on Linux, a lot of it is down to OpenGL...

This is false. Games that are designed to be cross platform from the beginning run just as well on Linux as they do on Windows. For example, the performance between the Linux, OSX, and Windows version of X-Plane 10 is identical across all platforms. Victor Vran is another great example that has no significant performance difference between platforms. Not to mention all the Source Engine games.

The real problem is that porting from DirectX to OpenGL introduces performance loss because of how each system handles certain things, and there's not a lot porters can do about it without rewriting a game's graphics engine from scratch, which really isn't a viable solution.

The sooner we can get more developers thinking cross platform from day one, the sooner the performance gap between Linux and Windows will shrink.

The other thing to keep in mind is that even with the reduced performance, performance in Linux is still more than acceptable and won't negatively affect anybody's gaming experience. That's the problem with looking at benchmark numbers, because you'll see Windows getting 200FPS and Linux getting 100FPS, and everybody says, "Oh my gosh! Linux has 50% of the performance of Windows!" but does it really matter when everybody locks their refresh rate to 60Hz?


Last edited by Mountain Man on 14 November 2015 at 3:35 pm UTC

Quoting: NyamiouSeriously WTF? Why do they benchmark with a GTX 660 and a dual core processor at a resolutions of 2560x1600 and 1792x1120? Doesn't that look like they tested all combinations and choose the one that was worst for SteamOS. They have a GTX 980 Ti benchmark PC (<a href="http://arstechnica.com/gaming/2015/09/building-the-ultimate-x99-gaming-and-benchmarking-pc/" target="_blank" rel="noreferrer" style="cursor:help;display:inline !important;">http://arstechnica.com/gaming/2015/09/building-the-ultimate-x99-gaming-and-benchmarking-pc/</a>) why didn't they use this one?

Also, the processor Intel Pentium G3220 have integrated graphics and SteamOS does not support multiple GPUs, if it's active that would explain the bad performances.

Also there isn't a 4.1.0-0 version of SteamOS, and the NVidia drivers 358.91 are not available right now on SteamOS, so this test could have been made ages ago because we have no information whatsoever on when it was made and on which version of SteamOS.

Also I would add that the guy who wrote this is not clean at all, he have been part of a scandal involving gaming journalists of big gaming sites colluding together on how to shape the opinion by choosing what to cover and how to cover it in a private mailing list : <a href="http://www.breitbart.com/london/2014/09/17/exposed-the-secret-mailing-list-of-the-gaming-journalism-elite/" target="_blank" rel="noreferrer" style="cursor:help;display:inline !important;">http://www.breitbart.com/london/2014/09/17/exposed-the-secret-mailing-list-of-the-gaming-journalism-elite/</a> I should add that the guy that made the Polygon article is also part of this.

All valid points.

Quoting: subEven worse, it feels like they want to prove that Steam Machines with SteamOS are crap. And they're doing a damn fine job at this.

Yes, there does seem to be a pattern in the broader gaming press of protecting Windows from the unsavory horde of competition, from gleeful reports about Steam Survey results to disparaging comments about the lack of games compared to Windows to the recent articles about performance disparity. We can't deny the facts, of course, but so many of these articles seem to be written with a condescending smirk and a wink to the Microsoft faithful, and that really bothers me. It's like they're trying to kill SteamOS before it even has a chance.

Quoting: Mountain ManThe other thing to keep in mind is that even with the reduced performance, performance in Linux is still more than acceptable and won't negatively affect anybody's gaming experience. That's the problem with looking at benchmark numbers, because you'll see Windows getting 200FPS and Linux getting 100FPS, and everybody says, "Oh my gosh! Linux has 50% of the performance of Windows!" but does it really matter when everybody locks their refresh rate to 60Hz?

This may be valid for Portal, but if you want to run Shadow of Mordor on the Alienware SteamMachine, you'll have to make some serious compromises in the graphics settings to run it at 60Hz. I don't have the data, but the point remains that if you got the Alienware Alpha with the same hardware but running Windows, you'd get better graphics. And while that might not matter to you, it does matter to a lot of people, just look at any discussion comparing PS4 and XB1 performance.
Also, not everybody locks their games at 60Hz, I have a 144 Hz monitor (running at 120 Hz) and I like to put it to good use. Of course, I also have a better PC than an Alienware SteamMachine, so I'm good :D

Two things:

It's interesting that no one has mentioned the elephant in the room: Virtual Programming. Despite being a "wrapper", their eON system is now getting excellent performance on DirectX games. Perhaps that may be the best route for developers who aren't going to begin with native OpenGL code when they start making a game.

As for those developers who do start with OpenGL, the performance seems to be great. When developers don't rely on third-party porters, their games run very well on Linux. I'm talking about games like Victor Vran and Magicka 2.


Last edited by melkemind on 14 November 2015 at 4:31 pm UTC

Quoting: Mountain Man

Quoting: subEven worse, it feels like they want to prove that Steam Machines with SteamOS are crap. And they're doing a damn fine job at this.

Yes, there does seem to be a pattern in the broader gaming press of protecting Windows from the unsavory horde of competition, from gleeful reports about Steam Survey results to disparaging comments about the lack of games compared to Windows to the recent articles about performance disparity. We can't deny the facts, of course, but so many of these articles seem to be written with a condescending smirk and a wink to the Microsoft faithful, and that really bothers me. It's like they're trying to kill SteamOS before it even has a chance.

Except that I don't really think it's about protecting Windows, or holding a particular grudge against SteamOS, but rather a closedmindedness, and maybe a preconception about Linux being less professional or something?
Now, I need only blink in the direction of scientific communities to debunk that preconception, but most people have a tendency to protect their preconceptions and grope at things that affirm them.

Quoting: reaVerEhm, I don't know how you can go more lower level than the most basic OpenGL calls. Don't forget that 90% of the OpenGL API is in fact redundant and if you look at the cleaner APIs such as OpenGL ES you will see that most of this stuff is in fact gone. I can imagine Vulkan making it easier, but not faster. I would again start with just the most basic problem, which is that they are using a distribution which is inherently slower than the top of the line such as gentoo and arch. And then drivers architecture, keep in mind that nvidia has used it's own architecture since its inception, so I'd guess it's probably optimized.

Keep in mind what I am going to say applies to Mantle (since I have more information about it), but also to Vulkan since it carries its legacy.
Actually, drawing a triangle with one of these APIs requires no less than 800 lines of code. For a triangle !
The developer is now responsible for low-level resource management/allocation, like memory. With previous APIs, you just kept uploading textures to the memory, until it was full, with little idea of what was remaining or how the driver managed the calls. Now, you can have a much finer control on resources, allocating what's needed, and minimizing copies, or keeping objects in memory for later.
The game developer is also responsible for draw calls, and any other form of communication with the device. Which means he can make those in batch, saving precious CPU time, and (maybe more importantly) in a multithreaded manner (provided he can manage the added complexity of multiple threads, that is.) Oxide games tested their game ashes of the singularity on a simulated infinitely fast GPU (I can't find the link to their article, unfortunately). They found that DX12 (which is close enough to Mantle/Vulkan) was capable of using all the CPU cores at 100%, whereas with the older API, only a core was always at full load. It has to do with the way OpenGL (and I think DX<12 too) uses their command buffer to send commands to the GPU, with the driver often running only on one core.
So, the area where the new APIs really shine are in CPU-bound scenarios, or when drawing multiple objects on screen (a lot of drawcalls). There are a few more points which are in favor of Vulkan, like built-in compute and multi-GPU capabilities, etc.
For more information, take a look at the DX12 demos. They should be quite representative of Vulkan, IMO.
There's also a handful Vulkan demos available (well, at least the results).
View video on youtube.com
View video on youtube.com
One added benefit of Vulkan is the simplicity of drivers. While that's not completely relevant with Gallium3D and such, you don't have to implement the whole OpenGL spec; since you let the developer take care of it.
Of course, the downside of all this is that it comes with added complexity for the developer. Moreover, Vulkan might not offer any substantial gain over OpenGL in some applications. On the plus side, there will very probably be some helper libs to help devs abstract away the complexity of Vulkan (thus, more or less recreating a complete OpenGL in the application :P); Moreover, from what I gathered, it seems possible to make Vulkan calls within an OpenGL context, at least on nVidia hardware (the opposite is probably possible too, but I didn't check).
So yeah, it looks somewhat like the golden apple of Linux gaming, and it's very promising for embedded systems and engine developers too.
I'm sure I forgot something, but there's really a lot to say about it :)

Quoting: lvlark

Quoting: Mountain Man

Quoting: subEven worse, it feels like they want to prove that Steam Machines with SteamOS are crap. And they're doing a damn fine job at this.

Yes, there does seem to be a pattern in the broader gaming press of protecting Windows from the unsavory horde of competition, from gleeful reports about Steam Survey results to disparaging comments about the lack of games compared to Windows to the recent articles about performance disparity. We can't deny the facts, of course, but so many of these articles seem to be written with a condescending smirk and a wink to the Microsoft faithful, and that really bothers me. It's like they're trying to kill SteamOS before it even has a chance.

Except that I don't really think it's about protecting Windows, or holding a particular grudge against SteamOS, but rather a closedmindedness, and maybe a preconception about Linux being less professional or something?

Now, I need only blink in the direction of scientific communities to debunk that preconception, but most people have a tendency to protect their preconceptions and grope at things that affirm them.

We're essentially saying the same thing.