After a long bumpy road with many revisions, it appears that the futex2 work sponsored by Valve is finally heading into the upstream Linux Kernel. Initially much larger, the work was slimmed down to get the main needed parts done and enabled before the rest can make it in.
So what is it? As developer André Almeida previously described it: "The use case of this syscall is to allow low level locking libraries to wait for multiple locks at the same time. This is specially useful for emulating Windows' WaitForMultipleObjects. A futex_waitv()-based solution has been used for some time at Proton's Wine (a compatibility layer to run Windows games on Linux). Compared to a solution that uses eventfd(), futex was able to reduce CPU utilization for games, and even increase frames per second for some games. This happens because eventfd doesn't scale very well for a huge number of read, write and poll calls compared to futex. Native game engines will benefit of this as well, given that this wait pattern is common for games.".
Speaking on Twitter, Valve developer Pierre-Loup Griffais said "It's amazing news that futex_waitv() seems to be on its way to the upstream kernel! Many thanks to the continued efforts of our partners at Collabora, CodeWeavers, and to the upstream community.".
Ideally then this will help Windows games in Proton on Linux run better. But that's not all!
Also interesting is the follow-up post from Griffais that mentions "Beyond Wine/Proton, we are also excited to bring those multi-threaded efficiency gains to Linux-native game engines and applications through some variant of the following primitive, pending more discussion with the glibc community:" with a link to some glibc work.
Article taken from GamingOnLinux.com.
Quoting: jordicomaWhat? A functionality coming from windows world that its good?
Not really, WaitForMultipleObjects() is different/higher-level than futex2, however futex2 can be used to implement a faster/better version of WaitForMultipleObjects().
Quoting: jordicomaWhat? A functionality coming from windows world that its good?Well, saying that gaming has not been the scientific focus on Linux would not be a big surprise right ?
From there on, it figures that for at least some cases windows has some better mechanism for gaming related workloads. Also, this just takes some ideas from windows and extend a linux idea (futexes), not an implementation from the windows syscall itself.
Generally speaking, all OSes have their pro and cons, and looking at your neighbor for some inspiration is always a good idea. As long as you do not blindly copy :)
Last edited by ShabbyX on 10 October 2021 at 5:03 pm UTC
Quoting: jordicomaWhat? A functionality coming from windows world that its good?
It's not necessarily good. If windows has feature X, games will depend on it. Now it may have been better if games did Y instead, but that doesn't exist and they are stuck with X.
Just because Linux now needs a way to implement X doesn't mean that X was the best option, just that because of windows we are now stuck with it.
Quoting: ShabbyXI wonder why they didn't instead try and make eventfd more effiecient?Probably because they would have the burden of not breaking software depending on early versions, same reason of why they called it "futex2" instead of just modify the original futex.
With futex2 they have all the freedom they need without have to worry about legacy compatibility.
Quoting: nenoroI thought it was already in Zen Kernel and maybe Xanmod ?Yes but those are not the "official kernel linux", or in another words they are linux kernels with custom patches (like Wine and Proton or Proton and Proton GE for example)
What changed in this case is that futex2 will also be part of the official kernel, so any distro using the mainline kernel 5.16+ will have support to futex2 by default, without the necessity of custom patches (commonly called "out of tree") like Xanmod does for example.
Quoting: BielFPsI wonder why they didn't instead try and make eventfd more effiecient?
What changed in this case is that futex2 will also be part of the official kernel, so any distro using the mainline kernel 5.16+ will have support to futex2 by default, without the necessity of custom patches (commonly called "out of tree") like Xanmod does for example.
sounds good to me
Last edited by nenoro on 10 October 2021 at 4:25 pm UTC
Quoting: ShabbyXI wonder why they didn't instead try and make eventfd more efficient?
Short answer: they kinda already tried that at first, and judged it to be a dead end.
Long answer:
Modifying an existing (set of) syscall(s) is extremely limited. You can not break compatibility in any way, since that would break thousands of apps, with no way for users to fix it. Contrary to libraries, you can not just install another kernel, or use a lightweight container to fix a kernel ABI breakage. So all issues with that syscall set are pretty much set in stone.
More generally, it seems more natural and clean to use a tool that is actually made to fix your issue. File descriptors (which eventfd is based on) are made to deal with file-like stuff (that's a lot of stuff in Linux). Futexes are made to deal with synchronization issues. Futexes are also made to be used in large numbers, file descriptors... not that much (the overhead in memory and so on). Futexes are generally made to be accessed many times too - they were made to deal with heavily multi threaded workloads synchronization. So, futexes are in principle a better tool for the task at hand, but they actually miss a couple of syscalls around them to match the need. Well, the solution which has been chosen is to add a syscall to be able to wait on multiple futexes, which is the need here. Futex2 is kinda misnamed from what I saw of the patch, as it just adds syscalls around futexes, not a whole new concept/object - at least that's what I saw. The need of waiting on multiple futexes is also sound, and seems reasonable to me. So solving that need once and for all does seem like the thing to do. And it gives a cleaner/better solution to the problem at hand than eventd.
Last edited by 3zekiel on 10 October 2021 at 6:43 pm UTC
soulsource Oct 10, 2021
Quoting: ShabbyXQuoting: jordicomaWhat? A functionality coming from windows world that its good?
It's not necessarily good. If windows has feature X, games will depend on it. Now it may have been better if games did Y instead, but that doesn't exist and they are stuck with X.
Just because Linux now needs a way to implement X doesn't mean that X was the best option, just that because of windows we are now stuck with it.
This.
Usually the problem of having to wait on multiple events can be solved by building cleaner (more readable, better maintainable) software architecture. I'm writing "usually" on purpose here, as there definitely are problems where going for a cleaner solution is not worth the effort, or where indeed waiting on multiple events is the most readable implementation (to be honest, right now I can't think of a problem where the latter would be the case, but that's probably just lack of experience on my side).
However, synchronization by waiting on multiple events is nearly always the easiest (quickest, cheapest in the short-term) solution to implement. If there's a deadline coming up, it's almost certainly the solution that's going to be picked, even though in the long-term it might cost more due to it being inherently more difficult to read and debug.
Generally speaking, if an API offers certain functionality, it will be used sooner or later. If one wants to be compatible, that functionality has to be there, and has to be about as performant as the original implementation (at least have the same asymptotic scaling behaviour).
Patreon
PayPal
See more from me