A new Steam Beta is up with Vulkan pipeline dumping and collection along with Steam Play improvements

Quoting: HubroI'm just crossing all my fingers and toes that Sekiro is playable through Steam Play at- or shortly after release

I really hope so too, this is the game I've been the most excited about in a long time. It seems they're still using the same engine as Dark Souls 3 and system requirements look similar too, so it's at least pretty likely that it'll work.

Quoting: EikeI wonder how hard it can possibly be to fix this 0 byte update bug.

I imagine something along the lines...

int updateSize = steamServer.calcUpdateSize();
if (updateSize <= 0) return false;

(Obviously, if it would be that easy, Valve would have solved it in the first attempt.)

Doing a wild guess here but I assume that one major problem is that the checksum of the files on the steam server somehow differs from what the one on the local machine which triggers a download and then it somehow finds out via some other means that all of the files in the package are unchanged so the download size gets to be 0 but that it still needs to do that last step where it's queued up in order for it to change the checksum on the local files.

So doing the simple obvious check that you posted would make the checksum still be wrong on the local side which means that the client will perform needless checks on the steam servers which valve wants to avoid in order to lessen the load (imagine the load generated when several million people launch steam for the day). And this checksum is probably messed up due to proton for some reason.

As I said however just 100% guesswork here from how I would imagine a situation like this arising when building a distribution system like steam.

Quoting: Purple Library GuyI know there are knowledgeable people here. What the heck is "Vulkan pipeline dumping"? It sounds like either a good thing or something oil companies do when something has gone very wrong.

I suspect "Vulkan pipeline dumping" is a term invented by someone at Valve, as I don't remember it being in the Vulkan specification.

The Vulkan specification is designed such that many of the Vulkan data structures you create are visible as simple memory blocks in CPU-accessible memory. That makes it easy to save the data structure content to disk. If you will always need the same data structures when you run an application, it may be quicker and more computationally predictable to load their content from disk, rather than re-create them from scratch.

A Vulkan pipeline used by a game ( which may use thousands of pipelines ) would be unlikely to change between one run and the next unless you modify the game's quality settings, so it is a good candidate for saving to disk. So, I would assume the Steam SDK now provides some form of helper function(s) to make cacheing pipelines easier for developers. I may be wrong in that assumption, of course :)


Last edited by etonbears on 7 March 2019 at 9:56 pm UTC

Quoting: Purple Library GuyI know there are knowledgeable people here. What the heck is "Vulkan pipeline dumping"? It sounds like either a good thing or something oil companies do when something has gone very wrong.

Hello PLG. Out of curiosity, I looked into this some more; the answer requires some explanation of the Vulkan architecture, which may or, more likely, may not be of interest to you. Skip to the last 3 paragraphs if this sort of thing sends you to sleep :)

Vulkan, on Linux, is provided as a shared library for each supported architecture. Your package manager should report "libvulkan1:amd64" and/or "libvulkan1:i386" or similar if you take a look. This does not require that you actually have any valid Vulkan hardware installed. Although this shared library can be provided by anyone, we probably all use a version written by LunarG, sponsored by Valve, which I think was recently donated to Khronos and will likely become the standard desktop version ( Linux, MacOS, Windows ).

This Vulkan library is formally called the "Vulkan Loader", in that it does not provide much function in itself, but coordinates access to Vulkan services provided by "Vulkan ICDs" and "Vulkan Layers".

Vulkan ICDs ( installable client drivers ) provide access to Vulkan-capable hardware through implementing all of the function calls documented in the Vulkan specification, and can additionally advertise extended function calls, usually for new hardware features that are not in the Vulkan specification yet. Vulkan ICDs are VERY lean wrappers around the hardware features, intended for maximum performance at runtime.

Vulkan Layers, on the other hand, are intended to provide all of the additional features that are not normally required at runtime, such as debugging, function call tracing, error reporting, function call parameter checking and validating the Vulkan state ( i.e. checking what you are requesting is complete and sane ).

A Vulkan Layer can implement one or more of the function calls documented in the Vulkan Specification, but unlike ICDs, is not obliged to implement them all. As with ICDs, a layer can offer extended function calls as well.

When an application initializes Vulkan ( a function call in libvulkan1.so ), the loader constructs a call "chain" for each Vulkan function. If any activated layer implements a particular function, that implementation is included in the call chain for the function; the end of the call chain is the guaranteed implementation in the ICD for your hardware.

These call chains are not at all efficient, but are intended to allow all sorts of checking, recording and analysis of an application while it is in development, when the execution speed is of less interest. At runtime, all development layers are deactivated, so the call chain just goes straight to the ICD implementation and the application/game runs as fast as possible.

If you are still awake and reading, you might be asking what this has to do with "Vulkan pipeline dumping". The answer is that Valve have used the Vulkan Layer mechanism to insert their own code into Steam-Launched Vulkan games, without the need to the game developer to do anything at all.

Vulkan on Linux uses 5 possible configuration directories ( to cater for different distributions liking things in different places, I guess )

/usr/local/etc/vulkan/
/usr/local/share/vulkan/
/etc/vulkan/
/usr/share/vulkan/
$HOME/.local/share/vulkan/

The Vulkan loader checks those locations for subdirectories "icd.d", "explicit_layer.d" and "implicit_layer.d", and reads any JSON-format text files in those directories to find which libraries are installed that implement ICDs and Layers.

Note that there are 2 types of Layer. Layers defined inside any "explicit_layer.d" are loaded by an application directly requesting them from code, but Layers defined in any "implicit_layer.d" are loaded automatically depending on environment variables declared in their JSON file.

If you have Steam installed, check $HOME/.local/share/vulkan/ and you should find 4 JSON files declaring implicit Vulkan Layers for 32bit & 64bit versions for steamoverlay and steamfossilize.

While there is no source code available for the libraries declared in these layers, it is clear that they intercept a small number of Vulkan calls to provide the Steam Overlay and Shader/Pipeline caches. If you activate these features in Steam, the correct environment variables will be set to let the Vulkan Loader know it should load those implicit layers.

More interesting, perhaps, is that this provides a non-intrusive mechanism for enhancing any/all Vulkan game presentations. This would allow overlaying data ( as with Steam Overlay ) of your choice from elsewhere, or applying post-process effects to your liking, as with the interesting Windows ENBSeries utility, for example.

Does anyone know if "Vulkan pipeline dumping" is working with the Mesa Radv Driver right now ? I have really bad stuttering in most of my games that are utilizing Proton with DXVK since i have switched to an AMD Radeon RX 590. Games using wined3d and therefore the Mesa GL Driver don't have that problem at all.

Quoting: ripper81358Does anyone know if "Vulkan pipeline dumping" is working with the Mesa Radv Driver right now ? I have really bad stuttering in most of my games that are utilizing Proton with DXVK since i have switched to an AMD Radeon RX 590. Games using wined3d and therefore the Mesa GL Driver don't have that problem at all.

Obviously it depends on the game and how efficiently the virtualization provided by Wine and DXVK maps the game to your hardware.

One thing Feral often mention when you start their native games is the effect of the CPU cores not being set to constant performance mode. It takes time for cores to clock down and up again between bursty loads which can lead to stuttering.

You will probably find the instructions for checking and changing the CPU mode on their Web site support FAQ. See if that is affecting you ( I have pkexec scripts to switch into 'game mode' and back to 'stop heating my room' mode, and use them as needed).