Steam Game Recording was recently released, which came as part of a big Stable Steam Client update that noted it had fixes for "miscellaneous common crashes" on Linux. Now we know a little more on what's happening behind the scenes.
Coming from a blog post from developer Timothée Besset, who does a lot of Valve work, Besset mentions how a change to how they approach the setenv and getenv functions that made the "most significant impact to Steam client stability on Linux".
For some context:
- setenv: change or add an environment variable.
- getenv: get an environment variable.
According to Besset, one of their colleagues claimed setenv as "the worst Linux API", even though it's such a common API available across all platforms "it was a little difficult to convince ourselves just how bad it is" — ouch. And they already knew, at least on Linux, that "setenv and getenv aren't safe to use in multithreaded environments".
Pictured - Steam Client
Part of their problem though is how on Linux you have so many distributions with different window managers, desktop environments, driver versions, extensive user customization and so on. So their own crash reports end up "very noisy". Another issue is that setenv will at times in a multithreaded program "crash outright", on top of that they found that "other threads would blow up though, usually with a SIGABRT, shortly after calling getenv", and their backtraces for the crashes were "all over the place and could not be easily tied to a single cause".
So what did they do? Here's what:
- We removed the majority of
setenvcalls. It was mostly used when spawning processes, and refactoring to useexevpeto pass down a prepared environment is an all around improvement.- We reduced how much we rely on
getenv, mostly by caching the calls. There is still an uncomfortable amount of it, but it's in OS libraries at this point (x11, xcb, dbus etc.) and we continue reducing it's usage.- For the few remaining
setenvuse cases that could not be easily refactored, we introduced an 'environment manager' that pre-allocates large enough value buffers at startup for fixed environment variable names, before any threading has started.
It's a fun little read worth a couple minutes of your time if you're interested on behind the scenes info, and how at times building for Linux can be a minefield.
According to a comment on the post, these issues are in fact being worked on in glibc, so it may not be such a problem in future for Linux developers.
The more stable it is, the better the experience will be for people switching to Linux, and of course for the Steam Deck too since SteamOS is Linux. Nice to see Valve developers constantly trying to make everything better.
Article taken from GamingOnLinux.com.
Some you may have missed, popular articles from the last month:
All posts need to follow our rules. For users logged in: please hit the Report Flag icon on any post that breaks the rules or contains illegal / harmful content. Guest readers can email us for any issues.
14 comments
They may be better off embracing the flatpak more closely, as it'll make it easier to port between different distributions.
In any case, more stability is always welcome! I did find it a little amusing at how steam recording on my steam deck kills the game and GUI, resetting it entirely (drop to CLI, then GUI restarts) when you stop recording or go to recording settings while recording in a non-steam game.
I was just testing it though, so it wasn't a big deal. No matter, I don't need it nor care.
In any case, more stability is always welcome! I did find it a little amusing at how steam recording on my steam deck kills the game and GUI, resetting it entirely (drop to CLI, then GUI restarts) when you stop recording or go to recording settings while recording in a non-steam game.
I was just testing it though, so it wasn't a big deal. No matter, I don't need it nor care.
3 Likes, Who?
pleasereadthemanual Nov 12
They may be better off embracing the flatpak more closely, as it'll make it easier to port between different distributions.I don't know what the Steam client's codebase looks like, but all I can see with the Steam Flatpak manifest is black magic: https://github.com/flathub/com.valvesoftware.Steam
What an insane amount of work for a Flatpak package.
6 Likes, Who?
I'd just like to see Valve move into the 2010's and make the Steam client 64-bit.
8 Likes, Who?
I do not understood. Why Valve allocates buffers, etc. As I read setenv is broken in multithread software. Why do not write overlap functions, which use mutexes? By writing overlap version, I mean wrapper.
Last edited by Lachu on 12 November 2024 at 4:42 pm UTC
Last edited by Lachu on 12 November 2024 at 4:42 pm UTC
0 Likes
Vortex_Acherontic Nov 12
They may be better off embracing the flatpak more closely, as it'll make it easier to port between different distributions.I don't know what the Steam client's codebase looks like, but all I can see with the Steam Flatpak manifest is black magic: https://github.com/flathub/com.valvesoftware.Steam
What an insane amount of work for a Flatpak package.
Yes it is. As a Steam flatpak user I am still surprised how well it runs. For me personally better than native packages ever did. But an official flatpak support by Valve would be much appreciated. But would probably also require Valve to rework a lot of their client to do it properly I guess.
I mean I don't have any insights on the actual source code of the client but dabbing through all it's files and directory, random libraries and shell scripts they ship it feel like they reinvented the wheel several times. I would not be surprised if the technical debs probably build up over time that there is no easy path in clean everything up except wiping everything and start from scratch and then aim for proper flatpak support.
Yet I am surprised it works in flatpak at all. Especially the Steam Runtimes which kinda are their own flatpaks inside flatpak.
Last edited by Vortex_Acherontic on 12 November 2024 at 1:54 pm UTC
4 Likes, Who?
it's kind for them to make steam stable on linoox. (pray to make it native on bsd)
What about the bugs for
- "filter by" in the wishlist who is buggy so you have to pass by the mobile app to make sure it shows the same results on pc ?
- the edit / delete buttons for the posts on steam discussions ? (quote works)
- Filter negative reviews and other filters on the main page for the game
Those bugs are waiting to be fixed
- also another bug, my pfp isn't shown when i'm logged
Those bugs appear on stable version and beta version
What about the bugs for
- "filter by" in the wishlist who is buggy so you have to pass by the mobile app to make sure it shows the same results on pc ?
- the edit / delete buttons for the posts on steam discussions ? (quote works)
- Filter negative reviews and other filters on the main page for the game
Those bugs are waiting to be fixed
- also another bug, my pfp isn't shown when i'm logged
Those bugs appear on stable version and beta version
0 Likes
it's kind for them to make steam stable on linoox. (pray to make it native on bsd)if by "what about this other thing" you meant "thanks! while at it, please also remember these!", then yeah, sure, these are pretty annoying (though at least not crashes!)
What about the bugs for
- "filter by" in the wishlist who is buggy so you have to pass by the mobile app to make sure it shows the same results on pc ?
- the edit / delete buttons for the posts on steam discussions ? (quote works)
- Filter negative reviews and other filters on the main page for the game
Those bugs are waiting to be fixed
- also another bug, my pfp isn't shown when i'm logged
Those bugs appear on stable version and beta version
if you meant it as "why waste time on linux instead of this other thing", then you should understand not all devs work on all issues
most of those you mentioned are probably not under the scope of Valve's linux devs... it's pretty much all web UI design bugs, which is developed once for all systems, probably by some main steam app devs team that actually focuses on windows (and/or the steam deck nowadays) more than anything else
2 Likes, Who?
ElectricPrism Nov 13
I'd just like to see Valve move into the 2010's and make the Steam client 64-bit.
I'll second that. I think Steam is probably the only thing keeping 32-bit multilib around for me. It would be nice to see it go for organization and cleanliness.
1 Likes, Who?
Yet I am surprised it works in flatpak at all. Especially the Steam Runtimes which kinda are their own flatpaks inside flatpak.It didn't work, specifically because of that issue with nested bubblewrap. They made changes to both the Steam client and flatpak so that the client can negotiate to run things with flatpak's bubblewrap. And now it works.
0 Likes
If the steam client moves entirely to 64-bit, they'll still need 32-bit libs to run old 32-bit games.I'd just like to see Valve move into the 2010's and make the Steam client 64-bit.
I'll second that. I think Steam is probably the only thing keeping 32-bit multilib around for me. It would be nice to see it go for organization and cleanliness.
Wine/Proton may be able to use 64-bit libs for 32-bit windows games (not yet but working on it afaik). And then there are native linux games, where it's a bit trickier, though most linux native games are 64-bit iirc.
0 Likes
My wishlist for things I would love if Steam changed on their linux client:
- Drag and drop support for steam chat when it comes to pictures and videos from your file manager like dolphin, nautilus, nemo, etc. please, oh my god that is straight up one of the only remaining areas where I think linux is an objective downgrade from windows. Discord clients on linux can do it and they can't even stream with audio; Why can't steam have drag and drop file support?
- Store/Community webpage black screen bug until you click library and go back is fixed.
- You can pick and choose which games get shader pre-caching and which don't instead of checking it for every game or no game. It is a straight up detriment to games like Warframe which have to download 10GB of shaders basically every single time I boot up my PC only to have new assets stutter anyway, and takes like 6 hours to actually compile the vulkan shaders for that game on steam. Other games though don't have this issue and benefit from fossilize.
- Media codec fixed versions of videos in proton are decoupled from fossilize's shader cache as a separate setting.
- Fix Big Picture mode running like ass on Nvidia Wayland with and without hardware acceleration
- Gamescope-session (Steam deck's "Gaming Mode") becomes as normalized on Nvidia as much as it is on AMD.
- 64 bit client
Last edited by WMan22 on 13 November 2024 at 9:39 am UTC
- Drag and drop support for steam chat when it comes to pictures and videos from your file manager like dolphin, nautilus, nemo, etc. please, oh my god that is straight up one of the only remaining areas where I think linux is an objective downgrade from windows. Discord clients on linux can do it and they can't even stream with audio; Why can't steam have drag and drop file support?
- Store/Community webpage black screen bug until you click library and go back is fixed.
- You can pick and choose which games get shader pre-caching and which don't instead of checking it for every game or no game. It is a straight up detriment to games like Warframe which have to download 10GB of shaders basically every single time I boot up my PC only to have new assets stutter anyway, and takes like 6 hours to actually compile the vulkan shaders for that game on steam. Other games though don't have this issue and benefit from fossilize.
- Media codec fixed versions of videos in proton are decoupled from fossilize's shader cache as a separate setting.
- Fix Big Picture mode running like ass on Nvidia Wayland with and without hardware acceleration
- Gamescope-session (Steam deck's "Gaming Mode") becomes as normalized on Nvidia as much as it is on AMD.
- 64 bit client
Last edited by WMan22 on 13 November 2024 at 9:39 am UTC
0 Likes
Vortex_Acherontic Nov 14
Yet I am surprised it works in flatpak at all. Especially the Steam Runtimes which kinda are their own flatpaks inside flatpak.It didn't work, specifically because of that issue with nested bubblewrap. They made changes to both the Steam client and flatpak so that the client can negotiate to run things with flatpak's bubblewrap. And now it works.
Yes I am pretty much aware of this. I've read the news too back then which is why I phrased it like that.
Still it works now and it is still surprising it does so well. However I still feel like the Steam Client needs a lot of refactor to properly integrate with the Linux desktop as it is today.
Still no 64bit, no native Wayland support, broadcasting is still bugged, Big Picture Mode still lags on nvidia hardware and I feel like for the Deck they need to entirely reinvent the store page because the current state, trying to force the webpage on deck and somehow try to navigate with a controller is super uncomfortable to use.
Sometimes also buggy in a way that the selection indicator is somewhere off screen or in the wrong layer and controls something in the background. Sometimes the toolbar which shows the buttons to press is flashing on the store page. There are still after so many years so many little annoying buggy things which definitely need a lot of rework to the underlying client if not an entire rewrite of the software. In the end this could benefit a proper flatpak integration too.
Last edited by Vortex_Acherontic on 14 November 2024 at 12:42 am UTC
0 Likes
If the steam client moves entirely to 64-bit, they'll still need 32-bit libs to run old 32-bit games.I'd just like to see Valve move into the 2010's and make the Steam client 64-bit.
I'll second that. I think Steam is probably the only thing keeping 32-bit multilib around for me. It would be nice to see it go for organization and cleanliness.
Wine/Proton may be able to use 64-bit libs for 32-bit windows games (not yet but working on it afaik). And then there are native linux games, where it's a bit trickier, though most linux native games are 64-bit iirc.
They could just use a light container and run those games with that. These are old software, so probably one container would suffice. There are tons of them, and they do not slow games down or dictate the user to enable a foreign distribution channel like flatpak does.
0 Likes
When is the linux PC steam app going to stop causing issues with playing games, like Final Fantasy 14, for more than 30 minute at a time before the graphics start bouncing around like a fiddler's elbow? It's like slowly going through an old flip book when moving, and I have to exit the game and relaunch. Sometimes I have to exit the steam app too.
0 Likes
Patreon
PayPal
See more from me