UPDATE: shortly after, Valve officially announced this new system and it's live now. Valve confirmed they can already grab data on things like crashes but this new opt-in system is to gather more info on the "overall experience". They also said the "data collected by this system won't directly change the Deck compatibility category for a title". So they're not crowdsourcing the compatibility process but checking to see if their process is working well or not.
This is what else has rolled out today:
- Added alert to Steam letting reservation holders know they can purchase a Steam Deck, as well as how much time is remains in their order window
- Added support for advanced filters and dynamic collection collection to Library
- Added "View in Library" or "View in Store" to event items (like achievements earned) in the activity feed
- Updated layout and navigation for Community pages to fully support gamepad navigation
- Updated text on Download page to clarify when player is downloading a patch vs an initial download
- Updated localization
- Updated Controller Configurator navigation
- Fixed issue with Keyboard theme not updating properly when equipped directly from the Points Shop
- Fixed "Skip Shaders" button not working properly on game launch screen
- Fixed some focus issues using L1/R1 on tab rows
- Fixed some stalls on resume for users who own many, many games
- Fixed long loading times after logging in for users who own many, many games
Original article below:
An eagle-eyed reader in the GamingOnLinux Discord pointed out that it appears Valve are working to bring out a feedback system for Steam Deck Verified. We can see this thanks to all the work SteamDB does with their SteamTracking system, with a new update they found showing off some new language strings that will be used on Steam.
Some of the new strings are:
"SteamDeckVerified_Feedback_Question": "Does this match your experience playing the game?",
"SteamDeckVerified_Feedback_Agree": "Yes",
"SteamDeckVerified_Feedback_Disagree": "No",
"SteamDeckVerified_Feedback_Ignore": "Ignore",
"SteamDeckVerified_Feedback_Confirmation": "Thank you for the feedback.",
"SteamDeckVerified_Feedback_Confirmation_Ignore": "Gotcha, we won't ask you anymore questions about %1$s.",
"SteamDeckFeedback_UserPrefence_Question": "Can we occasionally ask you for feedback? Your input will help us improve Steam Deck.",
"SteamDeckFeedback_UserPrefence_Yes": "Yes",
"SteamDeckFeedback_UserPrefence_No": "No Thanks",
"SteamDeckFeedback_UserPrefence_LearnMore": "Learn More",
This makes it pretty clear that we're going to see a system of questions, to see if the experience matches up with the current Deck Verified rating for a game. Something that's definitely needed, as there's numerous issues with the current way games are going through this verifying system.
Seeing this makes me quite happy, and shows clearly how Valve want to make the best of it and keep improving.
Update: the new Steam Deck feedback system is live.
I wonder if it would allow you to give a feedback without a random process picking you up
So just Verified? Might be good if they could figure something to ask about the other categories including Unsupported and Unknown.The point isn't to crowdsource but to verify what's done.
Well, perhaps not Unknown then, although I think there's some grey area to "Crowdsourcing"--if they have a pool of "Unknowns" that people say are pretty playable, they wouldn't want to just trust that and designate, but it would be a good way to have an idea what games are good candidates to look at next, both because they have a good chance of going into at least "Playable" and because they know Deck users are playing them.So just Verified? Might be good if they could figure something to ask about the other categories including Unsupported and Unknown.The point isn't to crowdsource but to verify what's done.
But both "Playable" and "Unsupported" are also the results of Valve doing an evaluation, and they could also be mistaken (or change), so would also be worth verifying.
Last edited by Purple Library Guy on 25 March 2022 at 9:00 pm UTC
Updated text on Download page to clarify when player is downloading a patch vs an initial downloadSeems like a small thing, but I'm happy to read this. It's more about the mentality this signifies: Tell your user what's happening. Since October, I have to use Windows for work again, and keep bumping into utterly useless messages, like 'Not implemented' (yes that's the full extent of the error message).
Well, perhaps not Unknown then, although I think there's some grey area to "Crowdsourcing"--if they have a pool of "Unknowns" that people say are pretty playable, they wouldn't want to just trust that and designate, but it would be a good way to have an idea what games are good candidates to look at next, both because they have a good chance of going into at least "Playable" and because they know Deck users are playing them.So just Verified? Might be good if they could figure something to ask about the other categories including Unsupported and Unknown.The point isn't to crowdsource but to verify what's done.
But both "Playable" and "Unsupported" are also the results of Valve doing an evaluation, and they could also be mistaken (or change), so would also be worth verifying.
They don't really need you to tell them the game is playable or verified. They know when and where and how long you play a game, they know when the game crashes etc:
- If you play a game for many hours without a crash on a steam deck: it's an indication of "verified"
- If you play a game for many hours with some crashes on a steam deck: it's an indication of "playable"
- If the gam crashes within minutes, or if rarely anyone plays it on deck for long: it's an indication of "unsupported"
Of course that's just heuristics, but it exactly serves the purpose you are talking about it.
Updated text on Download page to clarify when player is downloading a patch vs an initial download
Wasn't there supposed to be some designation (in the desktop client's download page) for shader downloads? I still see Steam downloading a bunch of (presumably shader) updates on most restarts, with no indication of what it is. The Tomb Raider games have a PATCH NOTES link, but they point at Windows updates from last year, so not very useful.
Well, no it doesn't. The distinction between "verified", "playable", and "unsupported" doesn't actually have anything to do with crashes. Something can be "unsupported" even though it plays smoothly and fairly satisfyingly with never a crash--if there's a cutscene that doesn't play, it's "unsupported". And technically, a game can be "Verified" even if it crashes quite a bit--as long as every single bit of the game plays correctly, and it works well with the screen size and the Deck controls. So no, that heuristic wouldn't tell a thing about the categories as Valve draws them.Well, perhaps not Unknown then, although I think there's some grey area to "Crowdsourcing"--if they have a pool of "Unknowns" that people say are pretty playable, they wouldn't want to just trust that and designate, but it would be a good way to have an idea what games are good candidates to look at next, both because they have a good chance of going into at least "Playable" and because they know Deck users are playing them.So just Verified? Might be good if they could figure something to ask about the other categories including Unsupported and Unknown.The point isn't to crowdsource but to verify what's done.
But both "Playable" and "Unsupported" are also the results of Valve doing an evaluation, and they could also be mistaken (or change), so would also be worth verifying.
They don't really need you to tell them the game is playable or verified. They know when and where and how long you play a game, they know when the game crashes etc:
- If you play a game for many hours without a crash on a steam deck: it's an indication of "verified"
- If you play a game for many hours with some crashes on a steam deck: it's an indication of "playable"
- If the gam crashes within minutes, or if rarely anyone plays it on deck for long: it's an indication of "unsupported"
Of course that's just heuristics, but it exactly serves the purpose you are talking about it.
Same here, initially I thought that its just because Steam adds new precached shaders very frequently, but considering that it is after every PC restart and that 2 updates ago or so a bug for the Steam Client was fixed that made steam load every update twice, I now start to think that this also might be a similar bug.🤔Updated text on Download page to clarify when player is downloading a patch vs an initial download
Wasn't there supposed to be some designation (in the desktop client's download page) for shader downloads? I still see Steam downloading a bunch of (presumably shader) updates on most restarts, with no indication of what it is. The Tomb Raider games have a PATCH NOTES link, but they point at Windows updates from last year, so not very useful.
Well, no it doesn't. The distinction between "verified", "playable", and "unsupported" doesn't actually have anything to do with crashes. Something can be "unsupported" even though it plays smoothly and fairly satisfyingly with never a crash--if there's a cutscene that doesn't play, it's "unsupported". And technically, a game can be "Verified" even if it crashes quite a bit--as long as every single bit of the game plays correctly, and it works well with the screen size and the Deck controls. So no, that heuristic wouldn't tell a thing about the categories as Valve draws them.Well, perhaps not Unknown then, although I think there's some grey area to "Crowdsourcing"--if they have a pool of "Unknowns" that people say are pretty playable, they wouldn't want to just trust that and designate, but it would be a good way to have an idea what games are good candidates to look at next, both because they have a good chance of going into at least "Playable" and because they know Deck users are playing them.So just Verified? Might be good if they could figure something to ask about the other categories including Unsupported and Unknown.The point isn't to crowdsource but to verify what's done.
But both "Playable" and "Unsupported" are also the results of Valve doing an evaluation, and they could also be mistaken (or change), so would also be worth verifying.
They don't really need you to tell them the game is playable or verified. They know when and where and how long you play a game, they know when the game crashes etc:
- If you play a game for many hours without a crash on a steam deck: it's an indication of "verified"
- If you play a game for many hours with some crashes on a steam deck: it's an indication of "playable"
- If the gam crashes within minutes, or if rarely anyone plays it on deck for long: it's an indication of "unsupported"
Of course that's just heuristics, but it exactly serves the purpose you are talking about it.
Yes, but! The crashes are part of the heuristic, but not all of if. If you paid attention, I also mentioned the amount of time users play. Obviously they can't exactly infer the class the game belongs to, but as an *indication* that valve should look into it, I think that's pretty sufficient (i.e. no need for the users to spell it out)
Last edited by ShabbyX on 26 March 2022 at 6:31 pm UTC
Yes, but! The crashes are part of the heuristic, but not all of if. If you paid attention, I also mentioned the amount of time users play. Obviously they can't exactly infer the class the game belongs to, but as an *indication* that valve should look into it, I think that's pretty sufficient (i.e. no need for the users to spell it out)
There are many cases where the time that the user spent or the amount of crashes wouldn't be enough:
- some graphical errors, like flickering, missing textures, or wrongly computed shaders
- the videos not playing correctly
- the sound being messy
- the game becoming slow after a while
- some functionalities of the game not working, like online stuff with anti-cheat
and basically anything else that doesn't cause the game to crash or to be completely unplayable.
Also, if it crashes, what does it mean? Does it crash randomly after a while? How long in average before it crashes? Or maybe it crashes systematically at the same moment in the game, preventing you to go further into the game?
There are many games that are released and played and that do crash after a while. How do you distinguish the crashes that are caused by the game itself or by the translation layer you are using?
If a game is classified as verified, but in the end has missing functionalities that don't prevent to play, I'm sure some people would still spend many hours on it, and yet the "verified" status would be false.
and I'm sure there are many more cases, actually so many that in principle the human testers should play the whole game before giving it a proper category, which is slow, unpractical and expensive.
Not only is this verified status not done on the whole game, it's also not necessarily up to date with the latest game version.
Using the players to confirm or deny a given category is the most practical choice, as they are human classifiers that come with no cost, some of them will cover parts of the game that were not covered by the testers, and they can report problems that are not easy to spot by a computer.
The ideal scenario would be somebody taking the time to open a bug ticket with the description of what happens and how to reproduce it, but that would be a minority of the players. On the other hand, just having a simple dialog that asks the player about confirmation requires very little involvement on his part, and is very valuable.
The time spent and the amount of crashes are sure, along with other information (such as the proton version, the system/game settings) also valuable. But I just don't think they are enough to confirm that the game ran in the expected conditions, and thus couldn't replace a human classification, only complete it.
Maybe some AI could be trained to do that, but I'm sure that wouldn't be easy.
Last edited by toor on 27 March 2022 at 1:09 am UTC
The point is that crashes are not actually part of the heuristic at all, as Valve have currently designed it; nowhere in the criteria for the categories does it actually say anything about how stable a game is. These are the current four criteria a game must pass in order to marked Verified, as listed here:Well, no it doesn't. The distinction between "verified", "playable", and "unsupported" doesn't actually have anything to do with crashes. Something can be "unsupported" even though it plays smoothly and fairly satisfyingly with never a crash--if there's a cutscene that doesn't play, it's "unsupported". And technically, a game can be "Verified" even if it crashes quite a bit--as long as every single bit of the game plays correctly, and it works well with the screen size and the Deck controls. So no, that heuristic wouldn't tell a thing about the categories as Valve draws them.
Yes, but! The crashes are part of the heuristic, but not all of if. If you paid attention, I also mentioned the amount of time users play. Obviously they can't exactly infer the class the game belongs to, but as an *indication* that valve should look into it, I think that's pretty sufficient (i.e. no need for the users to spell it out)
I'm not saying that's how it should be, just how it currently is. The problem is that they've designed this very objective, engineering-friendly system ("does game tick boxes? It's Verified!") without considering the emotional axis of playing games and losing an hour of progress to a crash and a save bug. I don't think you're wrong to assume that Verified should mean something like "plays basically flawlessly, or at least no more than the random crashes you'd get playing it on Windows", and I bet a lot of people are thinking along those lines as well, so I'm glad to see it looks like Valve are polling people's responses and possibly shifting to incorporate some more subjective criteria rather than purely objective ones. (Like, if the one movie that doesn't play in an otherwise flawless game isn't actually that important to the story, maybe it should be moved from Unsupported to at least Playable, that sort of thing.)
- Input: The title should have full controller support, use appropriate controller input icons, and automatically bring up the on-screen keyboard when needed.
- Display: The game should support the default resolution of Steam Deck (1280x800 or 1280x720), have good default settings, and text should be legible.
- Seamlessness: The title shouldn’t display any compatibility warnings, and if there’s a launcher it should be navigable with a controller.
- System Support: If running through Proton, the game and all its middleware should be supported by Proton. This includes anti-cheat support.
The point is that crashes are not actually part of the heuristic at all, as Valve have currently designed it; nowhere in the criteria for the categories does it actually say anything about how stable a game is. These are the current four criteria a game must pass in order to marked Verified, as listed here:Well, no it doesn't. The distinction between "verified", "playable", and "unsupported" doesn't actually have anything to do with crashes. Something can be "unsupported" even though it plays smoothly and fairly satisfyingly with never a crash--if there's a cutscene that doesn't play, it's "unsupported". And technically, a game can be "Verified" even if it crashes quite a bit--as long as every single bit of the game plays correctly, and it works well with the screen size and the Deck controls. So no, that heuristic wouldn't tell a thing about the categories as Valve draws them.
Yes, but! The crashes are part of the heuristic, but not all of if. If you paid attention, I also mentioned the amount of time users play. Obviously they can't exactly infer the class the game belongs to, but as an *indication* that valve should look into it, I think that's pretty sufficient (i.e. no need for the users to spell it out)
I'm not saying that's how it should be, just how it currently is. The problem is that they've designed this very objective, engineering-friendly system ("does game tick boxes? It's Verified!") without considering the emotional axis of playing games and losing an hour of progress to a crash and a save bug. I don't think you're wrong to assume that Verified should mean something like "plays basically flawlessly, or at least no more than the random crashes you'd get playing it on Windows", and I bet a lot of people are thinking along those lines as well, so I'm glad to see it looks like Valve are polling people's responses and possibly shifting to incorporate some more subjective criteria rather than purely objective ones. (Like, if the one movie that doesn't play in an otherwise flawless game isn't actually that important to the story, maybe it should be moved from Unsupported to at least Playable, that sort of thing.)
- Input: The title should have full controller support, use appropriate controller input icons, and automatically bring up the on-screen keyboard when needed.
- Display: The game should support the default resolution of Steam Deck (1280x800 or 1280x720), have good default settings, and text should be legible.
- Seamlessness: The title shouldn’t display any compatibility warnings, and if there’s a launcher it should be navigable with a controller.
- System Support: If running through Proton, the game and all its middleware should be supported by Proton. This includes anti-cheat support.
Let me clarify by saying there are two things being discussed here:
- A feedback system where valve double checks that their expectation matches the reality (what this article is about): yes that's fantastic, no arguments there.
- A proposed feedback system in an earlier comment where players *proactively* suggest a rating for a game that doesn't already have one. Follow up comment suggested the data could be used by Valve to know which game to verify next.
My reponses have all been about the latter suggestion. I maintain that Valve doesn't need you to guide them towards what game is working well or is popular etc so they would prioritize verifying it, because that information is easily inferable from the aggregate data they collect about how much time people spend on each game. For example:
- Distibution of time spent on Deck matches other platforms: game probably runs just fine
- Said distribution is skewed towards less time played: game probably has issues
- No one played more than 2 minuted on Deck: game is borked
- Distribution of time spent per session skewed towards less time + higher crash reports: drum roll ... the game has stability issues
- etc
Know what I'm saying?
Last edited by ShabbyX on 28 March 2022 at 5:43 am UTC
- A proposed feedback system in an earlier comment where players *proactively* suggest a rating for a game that doesn't already have one.Just to be clear, I did not in fact propose players "*proactively*" suggesting ratings. I merely suggested that the feedback system that is coming into existence, which asks players questions about their experience of Verified games they are playing, could be extended to asking similar questions about games in other categories, possibly including the "Unknown" category. Not that that necessarily has much bearing on all the back-and-forth, but as long as you're defining what's been going on, might as well define it accurately.
Know what I'm saying?Yes, that clarifies things. Thanks.
See more from me