Flathub in 2023, they have some big plans

I will admit I jumped on the flatpak bandwagon early. After reading this, I am beginning to think that those that were advocating for appimage might have been right the whole time.

Quoting: GuestI can't think of a single project that clashes with everything Unix more than Flatpak.

I'm a bit annoyed by this "this is not UNIX" argument by now that I'm taking some time to reply to this in a bit more verbose fashion. It's not against you as a person, but the argument is just not helpful at all.

First, what is UNIX? UNIX can be all sorts of things, a trademark, the Single UNIX Specification, the POSIX standard (which is equal to the SUS), a historical OS, a line of heritage of OSes or some wobbly idea of "what a OS should look like". In this case, it often refers to the last wobbly idea. There are a lot of slightly different concepts of what this idea precisely is, such as:

• Everything is a file


• Do one thing and do it right


• Keep it simple, stupid


• Perfect is the enemy of the good, or, good enough is perfect


• ...

Historically, UNIX was born from the ashes of MULTICS. Basically, MULTICS was "perfect", or strived to be so, with complex handling of errors and exceptions, which propagated through to the user, ideally being caught somewhere, processed, eventually something restarted, handled, etc. It died in its infancy.

UNIX was first not much more than a filesystem, also supporting some games. It's error handling was simple, bordering on non-existent:

QuoteWe went to lunch afterward, and I remarked to Dennis that easily half the code I was writing in Multics was error recovery code. He said, "We left all that stuff out. If there's an error, we have this routine called panic(), and when it is called, the machine crashes, and you holler down the hall, 'Hey, reboot it.'"

source

This also shone through in C when UNIX was re-written in it. C doesn't really have error-handling techniques, most core utils just bail when there's something "wrong" -- which is fine for small programs. Arguably, the "keep it simple" method in the infrastructure (OS, language) propagated into the user space and forced us to write either very primitive programs, or invent our own error handling complexity which almost always failed. I'll come back to this downside in a minute.

So UNIX tools where basically all "do one thing and do it right" due to KISS on the lower layer. Of course having small composable tools is better than huge multi-tools, looking at you GPG (oddly enough, most die-hard UNIX people also defend GPG and GPG-Mail which definitely is not UNIX anymore, neither is). However, this concept still had it's limitations, some things couldn't be just "do one thing" (or it is unfathomly hard to come up with such an architecture). Thus, there were always key components which were not do-one-thing in UNIX, such as the Kernel itself, the initd, the filesystem as a separate component, the network stack, and virtually anything relating to security. Whoops. (As a side note, this shows how much KISS and do-one-thing are not the same: Implementing a micro-kernel would be very much the latter but not at all the former.)

Everything is a file is also often named as one of the key UNIX ideas. Unfortunately, never was this true: A process famously hasn't been a file (descriptor) until Linux came around with that idea rather recently. And even then, the process itself is also a directory (yes, can be represented through an fd) containing multiple directory entries of all the resources. Basically, this "everything" is only "everything that is a single entity in its own right and no group of entities"---i.e., exactly what a process is (a process groups one or multiple threads with their resources plus a slice of virtual memory). In those cases where everything is a file is indeed implemented, it's often a bad abstraction, because it pretends that there exists such a uniform access method. This easily falls apart with things like /dev/stdin: A file that can't be seekable? So yes, writing a C program that can simultaneously read from stdin and files is "easy" until it suddenly, silently and possibly unexpectedly falls apart because you didn't query whether the "file" behind your fd is actually a file. What kind of lousy abstraction is that where you constantly need to call into the abstracted/generic object to query what it actually is?! This is worse than untyped programming, it's typeless programming, where the programmer has to keep local state for every opened fd to know what kind of file it represents. Note that this is much better solved by actually getting rid of this dumb abstraction and having a proper representation such as Rust's trait-system:

Spoiler, click me

A really simple PoC Rust program that implements a simple cat(1) without needing to wrestle with fds:
use std::env;
use std::io::*;
use std::fs::File;

const BUFSIZ: usize = 1024;

fn copy(mut dest: impl Write, mut src: impl Read) -> Result<()> {
    let mut buffer = [0; BUFSIZ];

    loop {
        let read = src.read(&mut buffer)?;
        if read == 0 {
            return Ok(());
        }

        let mut remaining = read;
        while remaining > 0 {
            let written = dest.write(&mut buffer[read-remaining..read])?;
            remaining -= written;
        }
    }
}

fn main() -> Result<()> {
    let args: Vec<_> = env::args().skip(1).collect();
    if args.is_empty() {
        copy(stdout(), stdin())?;
    }

    for arg in args {
        if arg == "-" {
            copy(stdout(), stdin())?;
        } else {
            let f = match File::open(&arg) {
                Err(e) => {
                    eprintln!("{}: {}", &arg, e);
                    continue;
                }
                Ok(f) => f,
            };
            copy(stdout(), f)?;
        }
    }

    return Ok(());
}

Do note that "everything is a file" proved such a terrible idea that it was amongst the first things changed when the original authors of UNIX went on to design Plan 9.

Regardless, as someone who has actually worked on an internal fork of an actual UNIX system---and I mean developed the UNIX itself, not only worked "on" the UNIX---all this "everything is a file" is bullshit that was soon scrapped by all reasonable UNIX systems soon enough. And all the over-interpretation of "do one thing and do it good" as well (cf., Sun SMF, Zones, ZFS, ldoms, and Apples launchd, etc.). There's merit to this concept, but wielded more carefully.

Finally, "good enough is perfect" methodology is nice for a "finished" product. Something you can develop, sell/distribute and don't care afterwards anymore. It's absolutely the worst if you have to iterate on it or god-forbid, have other people work ontop of your infrastructure (e.g., if you are a Kernel). Of course, not being perfect shouldn't hold you from releasing necessarily, but you should strive for better. A lot. Otherwise you have code rot and wind up with [insert XKCD here].

So what was I saying about KISS inducing more complexity? If you keep your underlying system (overly) simple, the tools built ontop of it will need to add additional complexity to actually serve the demands of the user. You cannot, if Google crashes, just call 'em and let 'em restart the server farm. But I bet you also don't want to reboot in the middle of your gaming session, just because some daemon hangs. The key here is "serve the demands" and the demands have changed a lot. Remember that UNIX got funding because the developers promised AT&T to build a text-processing system which they could use to better print their patents and bills. That, plus serving the programmers of UNIX itself was the "demand" UNIX had to fulfill. Back then, the tools you used on UNIX were called PWB: Programmer's Workbench. Incidentally, not all users nowadays are programmers, nor is all you do coding, typing texts and playing a little console game.

UNIX wasn't bad. Not at all. I damn love UNIX, I even love C! But it's vintage. It's served its purpose. It was actually perfect for what it should do, so it never was "KISS" but "do exactly what's required, but not more". Nowadays we require more of a system, that's the natural way things go, hence, more is required from a system. Nowaday's UNIX would look a lot different.

Requirements that we nowadays have are things like:

• Easy rollback of single applications


• Fearlessly extending the system with apps, them not interfering with each other


• Automatic deployment, updating, ...


• Restriction of app's read/write permissions to per-app boundaries

This list is absolutely incomplete but does contain some of our new requirements (yes, maybe not for you since you may use only small FOSS apps and reboot to Windows for the other things, but for some people that's no feasable setup). Now Flatpak isn't perfect (that's the enemy of the good) and doesn't address all these problems, but it's a good start. And it's actually not "one tool" but building ontop of a lot of small tools such as cgroupsv2, bwrap, xdg-portals, pipewire: It's more an umbrella for a collection of tools that are designed properly to work together.

Flatpaks are basically even the most small applications as they're defined in terms of almost completely declarative "code" for building, metadata and the code itself. Just a super small archive of this will allow you to reproducibly build the Flatpak! It's serving the current needs of our world but not more.

To wind this down: Software development, historically, is always an iteration of ground/infrastructure work which starts out really primitive. Then apps, building upon this, and there's this "idea of a moment" where everything fits perfectly: The user's requirements, the apps fulfilling these, and the infrastructure providing the necessary tooling fulfilling the apps' requirements. They all line up and Dennis Ritchie sheds a tear. But then the users' requirements change, the apps stretch the boundaries of what the infrastructure provides and, inevitably, the infrastructure needs to change as well. It adapts, people cry for the lack of simplicity (sometimes rightly so!), the tools can then shed a bit of old, musty code not needed anymore and all is good again. Waiting for the next cycle.

While those things are, in principle, all positive I still worry a bit that this move might end up going in a less than ideal direction.

Flathub is already pretty great now. And most app stores are complete garbage, filled with proprietary shovelware full of malicious functionality (spying, ads, lock-in...). Making flathub into an app store could easily mean a huge downgrade.

Similarly, Linux apps - in particular the simple apps that you'd get from an app store¹ - are already pretty damn good. PDF viewers, torrent clients, media players, image programs, system utilities, text editors, browsers, compression tools, it's all good; the only reason Linux isn't better than other systems is because open-source apps are generally multiplatform and make their way to Windows at least. It is definitely fair that those great apps could make their developers some cash, but I don't know if the "economic incentive" (or profit motive) would make for better apps. It would be an incentive for shovelware, for allowing bullshit into the repos/store, for showing ads "to help devs"... in my experience, the FOSS apps made with no expectation of payment are generally better than the commercial alternatives, despite the obvious advantages the latter enjoys. In the end, it depends on what posture Flathub will have - what it will allow, what it will promote, who it will hear, etc. But this "make everything into a market, it is the best thing always" mentality is suspicious.

¹ For big professional software suites like Adobe or Autodesk, they definitely aren't waiting for payment processing in an app store. For specific things like games, it's not like Flathub will replace the dedicated stores. And for many things like my city's bus app, local university or restaurant apps, and many others the barrier is not the distribution, just adoption of the system and people giving a fuck about it

It's nice to see that Flatpak - and Flathub as the default store - are continuing to focus on improving the desktop experience not just for the user, but for the developer as well.

Hopefully the system they build for doing the entitling of purchased applications is something that's going to be usable for more Flatpak repositories than just Flathub, since there's really not any good ways to distribute paid Linux applications right now, especially not in a way that lets them properly integrate with the system.


Last edited by Ananace on 7 March 2023 at 1:18 pm UTC

Quoting: GuestI can't think of a single project that clashes with everything Unix more than Flatpak.

How so? It's debatable if it even needs to adhere to a philosophy from the 70s, but I mean, the UNIX philosophy isn't some abstract feeling, it's four specific points. Which one does it clash with so badly? I mean the point about one program being an input to another maybe, but that applies to most gui apps not written in the 70s. If it's the part about one tool doing its thing very well, flatpak is made up of ostree, bubblewrap, etc. It isn't some huge monolith. I find "Unix" and "Unix philosophy" and the like gets misappropriated a lot to basically just mean "thing I don't like."

If you just mean that it's big, that you don't like repackaging a bunch of stuff in user space . . .

- statically linked binaries have been a debate for decades
- Linux itself (no one wants to die on that microkernel hill anymore?)
- BSD chroots anyone
- Docker
- Kubernetes
- a huge amount of apps where the client and server are distributed as one binary, eg everything Hashicorp

I would in fact argue that 45 years later, flatpak and most of these tools are *very* UNIX like, to paraphrase Brian Kernighan:

What makes UNIX so powerful is that it decouples how the program is written, what language, what style, etc from the environment around them. A bunch of tools combine to create a coherent environment. Is that not exactly what flatpak does in 2023?

Overall the direction sounds good.

I think the main focus for Flatpak/Flathub should be:

- Ensure any application on Flathub works as intended, with all functionality the user would expect to see working, without issues. If that isn't the case, use whatever 'temporary' solutions are necessary to make it so until better long term solutions are in place. No one is going to want to use Flathub to download applications if they experience too many occasions of applications not working due to mishandled sandboxing. Prioritising 'Everything must be sandboxed!' at the expense of functionality, is prioritising ideals over users, which will result in less/no growth over time.

- "Developers, developers, developers!"... laugh if you want, but Palmer was right when he said that. If you want to make a platform successful, make developers happy. It should not be assumed that developers will just 'figure it out', if you leave the pieces on the floor for them to pick up and assemble a solution. There should be a very straight forward 'hand holding' process for getting apps onto Flathub, and enough documentation that even a non-programmer can easily figure out how to get an app on Flathub.

- Finish the permissions system, and by that I mean, do the other 50% of the 'Android on Desktop' like permission system Flathub/Flatpak has started. Add APIs for apps to check if they have permissions they want, add APIs for apps to request permissions, list what permissions app are asking for before install, if permissions change with an app then list the changes, make it easy to grant/revoke permissions, notify the user when something is blocked by a lack of permissions (nothing is a greater source of user frustration than something failing silently without explanation), etc etc.

Quoting: eldakingWhile those things are, in principle, all positive I still worry a bit that this move might end up going in a less than ideal direction.

Flathub is already pretty great now. And most app stores are complete garbage, filled with proprietary shovelware full of malicious functionality (spying, ads, lock-in...). Making flathub into an app store could easily mean a huge downgrade.
…

Thanks for the thoughts! I'm obviously very concerned about this not happening, which is part of why we're moving so slowly and making sure our governance is solid so that when we move, we do so with community support and trust. But I think if we're cautious about the incentives we put together and how we set up our goals, we can bring convenient payment/sponsorship/etc options alongside convenient app availability without just setting up this pure-profit race to the bottom.

The unique thing about Flathub compared to other app stores is that at its heart we exist to promote and serve the interests of the FLOSS community, and we'll ensure that through the governance we set up through GNOME and KDE, and any future steps we make with advisory boards etc. We will prioritise FLOSS applications over proprietary applications - for instance the fees will be higher for proprietary apps than free ones. None of this means we need to change the licensing properties, trust, transparency, etc - we will indicate when things are not open source or have not been built in transparent/trusted places, we'll scan for up to date dependencies and known malware, and we're going to reject and call out bad behaviour.

My personal hypothesis is that sponsorship (paying for the work of the creator, on the next update/version) is a better match for the model of how FOSS is developed compared to paying up front for something, or getting it for free and then trying to "monetize" () later with ads/spyware/etc. It's up to each developer how they set their apps up, but I expect - sponsorship, donations, and supporter content / paid upgrades (note: this doesn't mean open core - you can charge to distribute something which is still open source) - to be the most common approaches. We want to be led by the developers which is why the focus groups at LAS and online surveys are the next step on this payments journey.

Obviously it's inevitable that "junk" will appear - but I don't think it will be very popular and we won't work to promote it. I think the majority of the proprietary apps on the platform will be the things we already have - ie free of charge as they are actually clients to existing services/networks where payments are handled elsewhere - so proprietary apps being purchased for a fee will be a minority of the volume on the platform. There is a small cluster of extremely niche software which is available for Linux - things like a certain audio workflow or software for fitting carpets or something - maybe we'll get some of them if we can help bring those developers and users together?

Quoting: ramcq

Quoting: eldakingWhile those things are, in principle, all positive I still worry a bit that this move might end up going in a less than ideal direction.

Flathub is already pretty great now. And most app stores are complete garbage, filled with proprietary shovelware full of malicious functionality (spying, ads, lock-in...). Making flathub into an app store could easily mean a huge downgrade.
…

Thanks for the thoughts! I'm obviously very concerned about this not happening, which is part of why we're moving so slowly and making sure our governance is solid so that when we move, we do so with community support and trust. But I think if we're cautious about the incentives we put together and how we set up our goals, we can bring convenient payment/sponsorship/etc options alongside convenient app availability without just setting up this pure-profit race to the bottom.

The unique thing about Flathub compared to other app stores is that at its heart we exist to promote and serve the interests of the FLOSS community, and we'll ensure that through the governance we set up through GNOME and KDE, and any future steps we make with advisory boards etc. We will prioritise FLOSS applications over proprietary applications - for instance the fees will be higher for proprietary apps than free ones. None of this means we need to change the licensing properties, trust, transparency, etc - we will indicate when things are not open source or have not been built in transparent/trusted places, we'll scan for up to date dependencies and known malware, and we're going to reject and call out bad behaviour.

My personal hypothesis is that sponsorship (paying for the work of the creator, on the next update/version) is a better match for the model of how FOSS is developed compared to paying up front for something, or getting it for free and then trying to "monetize" () later with ads/spyware/etc. It's up to each developer how they set their apps up, but I expect - sponsorship, donations, and supporter content / paid upgrades (note: this doesn't mean open core - you can charge to distribute something which is still open source) - to be the most common approaches. We want to be led by the developers which is why the focus groups at LAS and online surveys are the next step on this payments journey.

Obviously it's inevitable that "junk" will appear - but I don't think it will be very popular and we won't work to promote it. I think the majority of the proprietary apps on the platform will be the things we already have - ie free of charge as they are actually clients to existing services/networks where payments are handled elsewhere - so proprietary apps being purchased for a fee will be a minority of the volume on the platform. There is a small cluster of extremely niche software which is available for Linux - things like a certain audio workflow or software for fitting carpets or something - maybe we'll get some of them if we can help bring those developers and users together?

This clarification does make me feel somewhat less uneasy.

Personally, I like Flatpaks as a concept, I feel they're appropriate for a significant niche, but I've never actually used one. If I want an app (that isn't a game), I get it from my distro, which, being Mint, has a nice easy GUI for software installation. I guess my needs aren't all that specialized. And that way it gets automatically updated, in ways that don't create conflicts, and I generally don't have to worry about it, and if I ever want to uninstall one that is very easy to do.

So for regular open source applications, Flathub is solving a problem I don't have, with a solution that is slightly more labour intensive than my existing approach. If there were closed source applications that I needed, that weren't games, I can see getting them from Flathub. Or I guess if there were open source applications that I needed, that were not available from my distro but were available from Flathub, I'd grumble and use it. But for most stuff I would prefer to use traditional package management integrated with my distro, and I will be unhappy if there is a general trend away from existing package management and I end up having to install everything as Flatpaks.

Quoting: Purple Library GuyPersonally, I like Flatpaks as a concept, I feel they're appropriate for a significant niche, but I've never actually used one. If I want an app (that isn't a game), I get it from my distro, which, being Mint, has a nice easy GUI for software installation.

Do bear in mind that these days Mint's friendly software manager GUI (mintinstall) integrates Flathub and the update manager updates your flatpaks seamslessly. When both sources offer the same version of a piece of software, there's no reason to go for the flatpak, but there are several apps that are a lot more recent on Flathub. Godot is a good example. And then there's software like DBeaver (database manager) that are not in the default Ubuntu/Mint repositories at all, but can be installed from Flathub using your friendly and familiar GUI.