Looks like Valve are adding a feedback system for Steam Deck Verified (update: it's live)

QuoteUpdated text on Download page to clarify when player is downloading a patch vs an initial download

Wasn't there supposed to be some designation (in the desktop client's download page) for shader downloads? I still see Steam downloading a bunch of (presumably shader) updates on most restarts, with no indication of what it is. The Tomb Raider games have a PATCH NOTES link, but they point at Windows updates from last year, so not very useful.

Quoting: ShabbyX

Quoting: Purple Library Guy

Quoting: Liam Dawe

Quoting: Purple Library GuySo just Verified? Might be good if they could figure something to ask about the other categories including Unsupported and Unknown.

The point isn't to crowdsource but to verify what's done.

Well, perhaps not Unknown then, although I think there's some grey area to "Crowdsourcing"--if they have a pool of "Unknowns" that people say are pretty playable, they wouldn't want to just trust that and designate, but it would be a good way to have an idea what games are good candidates to look at next, both because they have a good chance of going into at least "Playable" and because they know Deck users are playing them.

But both "Playable" and "Unsupported" are also the results of Valve doing an evaluation, and they could also be mistaken (or change), so would also be worth verifying.

They don't really need you to tell them the game is playable or verified. They know when and where and how long you play a game, they know when the game crashes etc:

- If you play a game for many hours without a crash on a steam deck: it's an indication of "verified"
- If you play a game for many hours with some crashes on a steam deck: it's an indication of "playable"
- If the gam crashes within minutes, or if rarely anyone plays it on deck for long: it's an indication of "unsupported"

Of course that's just heuristics, but it exactly serves the purpose you are talking about it.

Well, no it doesn't. The distinction between "verified", "playable", and "unsupported" doesn't actually have anything to do with crashes. Something can be "unsupported" even though it plays smoothly and fairly satisfyingly with never a crash--if there's a cutscene that doesn't play, it's "unsupported". And technically, a game can be "Verified" even if it crashes quite a bit--as long as every single bit of the game plays correctly, and it works well with the screen size and the Deck controls. So no, that heuristic wouldn't tell a thing about the categories as Valve draws them.

Quoting: Phlebiac

QuoteUpdated text on Download page to clarify when player is downloading a patch vs an initial download

Wasn't there supposed to be some designation (in the desktop client's download page) for shader downloads? I still see Steam downloading a bunch of (presumably shader) updates on most restarts, with no indication of what it is. The Tomb Raider games have a PATCH NOTES link, but they point at Windows updates from last year, so not very useful.

Same here, initially I thought that its just because Steam adds new precached shaders very frequently, but considering that it is after every PC restart and that 2 updates ago or so a bug for the Steam Client was fixed that made steam load every update twice, I now start to think that this also might be a similar bug.🤔

Quoting: Purple Library Guy

Quoting: ShabbyX

Quoting: Purple Library Guy

Quoting: Liam Dawe

Quoting: Purple Library GuySo just Verified? Might be good if they could figure something to ask about the other categories including Unsupported and Unknown.

The point isn't to crowdsource but to verify what's done.

Well, perhaps not Unknown then, although I think there's some grey area to "Crowdsourcing"--if they have a pool of "Unknowns" that people say are pretty playable, they wouldn't want to just trust that and designate, but it would be a good way to have an idea what games are good candidates to look at next, both because they have a good chance of going into at least "Playable" and because they know Deck users are playing them.

But both "Playable" and "Unsupported" are also the results of Valve doing an evaluation, and they could also be mistaken (or change), so would also be worth verifying.

They don't really need you to tell them the game is playable or verified. They know when and where and how long you play a game, they know when the game crashes etc:

- If you play a game for many hours without a crash on a steam deck: it's an indication of "verified"
- If you play a game for many hours with some crashes on a steam deck: it's an indication of "playable"
- If the gam crashes within minutes, or if rarely anyone plays it on deck for long: it's an indication of "unsupported"

Of course that's just heuristics, but it exactly serves the purpose you are talking about it.

Well, no it doesn't. The distinction between "verified", "playable", and "unsupported" doesn't actually have anything to do with crashes. Something can be "unsupported" even though it plays smoothly and fairly satisfyingly with never a crash--if there's a cutscene that doesn't play, it's "unsupported". And technically, a game can be "Verified" even if it crashes quite a bit--as long as every single bit of the game plays correctly, and it works well with the screen size and the Deck controls. So no, that heuristic wouldn't tell a thing about the categories as Valve draws them.

Yes, but! The crashes are part of the heuristic, but not all of if. If you paid attention, I also mentioned the amount of time users play. Obviously they can't exactly infer the class the game belongs to, but as an *indication* that valve should look into it, I think that's pretty sufficient (i.e. no need for the users to spell it out)


Last edited by ShabbyX on 26 March 2022 at 6:31 pm UTC

Quoting: ShabbyXYes, but! The crashes are part of the heuristic, but not all of if. If you paid attention, I also mentioned the amount of time users play. Obviously they can't exactly infer the class the game belongs to, but as an *indication* that valve should look into it, I think that's pretty sufficient (i.e. no need for the users to spell it out)

There are many cases where the time that the user spent or the amount of crashes wouldn't be enough:
- some graphical errors, like flickering, missing textures, or wrongly computed shaders
- the videos not playing correctly
- the sound being messy
- the game becoming slow after a while
- some functionalities of the game not working, like online stuff with anti-cheat
and basically anything else that doesn't cause the game to crash or to be completely unplayable.

Also, if it crashes, what does it mean? Does it crash randomly after a while? How long in average before it crashes? Or maybe it crashes systematically at the same moment in the game, preventing you to go further into the game?
There are many games that are released and played and that do crash after a while. How do you distinguish the crashes that are caused by the game itself or by the translation layer you are using?

If a game is classified as verified, but in the end has missing functionalities that don't prevent to play, I'm sure some people would still spend many hours on it, and yet the "verified" status would be false.

and I'm sure there are many more cases, actually so many that in principle the human testers should play the whole game before giving it a proper category, which is slow, unpractical and expensive.
Not only is this verified status not done on the whole game, it's also not necessarily up to date with the latest game version.

Using the players to confirm or deny a given category is the most practical choice, as they are human classifiers that come with no cost, some of them will cover parts of the game that were not covered by the testers, and they can report problems that are not easy to spot by a computer.

The ideal scenario would be somebody taking the time to open a bug ticket with the description of what happens and how to reproduce it, but that would be a minority of the players. On the other hand, just having a simple dialog that asks the player about confirmation requires very little involvement on his part, and is very valuable.

The time spent and the amount of crashes are sure, along with other information (such as the proton version, the system/game settings) also valuable. But I just don't think they are enough to confirm that the game ran in the expected conditions, and thus couldn't replace a human classification, only complete it.

Maybe some AI could be trained to do that, but I'm sure that wouldn't be easy.


Last edited by toor on 27 March 2022 at 1:09 am UTC

Quoting: ShabbyX

Quoting: Purple Library GuyWell, no it doesn't. The distinction between "verified", "playable", and "unsupported" doesn't actually have anything to do with crashes. Something can be "unsupported" even though it plays smoothly and fairly satisfyingly with never a crash--if there's a cutscene that doesn't play, it's "unsupported". And technically, a game can be "Verified" even if it crashes quite a bit--as long as every single bit of the game plays correctly, and it works well with the screen size and the Deck controls. So no, that heuristic wouldn't tell a thing about the categories as Valve draws them.

Yes, but! The crashes are part of the heuristic, but not all of if. If you paid attention, I also mentioned the amount of time users play. Obviously they can't exactly infer the class the game belongs to, but as an *indication* that valve should look into it, I think that's pretty sufficient (i.e. no need for the users to spell it out)

The point is that crashes are not actually part of the heuristic at all, as Valve have currently designed it; nowhere in the criteria for the categories does it actually say anything about how stable a game is. These are the current four criteria a game must pass in order to marked Verified, as listed here:

Quote

• Input: The title should have full controller support, use appropriate controller input icons, and automatically bring up the on-screen keyboard when needed.


• Display: The game should support the default resolution of Steam Deck (1280x800 or 1280x720), have good default settings, and text should be legible.


• Seamlessness: The title shouldn’t display any compatibility warnings, and if there’s a launcher it should be navigable with a controller.


• System Support: If running through Proton, the game and all its middleware should be supported by Proton. This includes anti-cheat support.

I'm not saying that's how it should be, just how it currently is. The problem is that they've designed this very objective, engineering-friendly system ("does game tick boxes? It's Verified!") without considering the emotional axis of playing games and losing an hour of progress to a crash and a save bug. I don't think you're wrong to assume that Verified should mean something like "plays basically flawlessly, or at least no more than the random crashes you'd get playing it on Windows", and I bet a lot of people are thinking along those lines as well, so I'm glad to see it looks like Valve are polling people's responses and possibly shifting to incorporate some more subjective criteria rather than purely objective ones. (Like, if the one movie that doesn't play in an otherwise flawless game isn't actually that important to the story, maybe it should be moved from Unsupported to at least Playable, that sort of thing.)

Quoting: Philadelphus

Quoting: ShabbyX

Quoting: Purple Library GuyWell, no it doesn't. The distinction between "verified", "playable", and "unsupported" doesn't actually have anything to do with crashes. Something can be "unsupported" even though it plays smoothly and fairly satisfyingly with never a crash--if there's a cutscene that doesn't play, it's "unsupported". And technically, a game can be "Verified" even if it crashes quite a bit--as long as every single bit of the game plays correctly, and it works well with the screen size and the Deck controls. So no, that heuristic wouldn't tell a thing about the categories as Valve draws them.

Yes, but! The crashes are part of the heuristic, but not all of if. If you paid attention, I also mentioned the amount of time users play. Obviously they can't exactly infer the class the game belongs to, but as an *indication* that valve should look into it, I think that's pretty sufficient (i.e. no need for the users to spell it out)

The point is that crashes are not actually part of the heuristic at all, as Valve have currently designed it; nowhere in the criteria for the categories does it actually say anything about how stable a game is. These are the current four criteria a game must pass in order to marked Verified, as listed here:

Quote

• Input: The title should have full controller support, use appropriate controller input icons, and automatically bring up the on-screen keyboard when needed.


• Display: The game should support the default resolution of Steam Deck (1280x800 or 1280x720), have good default settings, and text should be legible.


• Seamlessness: The title shouldn’t display any compatibility warnings, and if there’s a launcher it should be navigable with a controller.


• System Support: If running through Proton, the game and all its middleware should be supported by Proton. This includes anti-cheat support.

I'm not saying that's how it should be, just how it currently is. The problem is that they've designed this very objective, engineering-friendly system ("does game tick boxes? It's Verified!") without considering the emotional axis of playing games and losing an hour of progress to a crash and a save bug. I don't think you're wrong to assume that Verified should mean something like "plays basically flawlessly, or at least no more than the random crashes you'd get playing it on Windows", and I bet a lot of people are thinking along those lines as well, so I'm glad to see it looks like Valve are polling people's responses and possibly shifting to incorporate some more subjective criteria rather than purely objective ones. (Like, if the one movie that doesn't play in an otherwise flawless game isn't actually that important to the story, maybe it should be moved from Unsupported to at least Playable, that sort of thing.)

Let me clarify by saying there are two things being discussed here:

- A feedback system where valve double checks that their expectation matches the reality (what this article is about): yes that's fantastic, no arguments there.
- A proposed feedback system in an earlier comment where players *proactively* suggest a rating for a game that doesn't already have one. Follow up comment suggested the data could be used by Valve to know which game to verify next.

My reponses have all been about the latter suggestion. I maintain that Valve doesn't need you to guide them towards what game is working well or is popular etc so they would prioritize verifying it, because that information is easily inferable from the aggregate data they collect about how much time people spend on each game. For example:

- Distibution of time spent on Deck matches other platforms: game probably runs just fine
- Said distribution is skewed towards less time played: game probably has issues
- No one played more than 2 minuted on Deck: game is borked
- Distribution of time spent per session skewed towards less time + higher crash reports: drum roll ... the game has stability issues
- etc

Know what I'm saying?


Last edited by ShabbyX on 28 March 2022 at 5:43 am UTC

Quoting: ShabbyX- A proposed feedback system in an earlier comment where players *proactively* suggest a rating for a game that doesn't already have one.

Just to be clear, I did not in fact propose players "*proactively*" suggesting ratings. I merely suggested that the feedback system that is coming into existence, which asks players questions about their experience of Verified games they are playing, could be extended to asking similar questions about games in other categories, possibly including the "Unknown" category. Not that that necessarily has much bearing on all the back-and-forth, but as long as you're defining what's been going on, might as well define it accurately.

Quoting: ShabbyXKnow what I'm saying?

Yes, that clarifies things. Thanks.