What about something like navi - github.com/denisidoro/navi. Basically an interactive cheat sheet that has commands pre-loaded (or that you can make yourself).
Better organization and backup / restore. For example if you want to restore config files but don’t want to move over the large “.local” folder, applications that write to $HOME will create diifculty.
For me personally I just hate that I do not know where to find configs, especially when using a dotfiles repo, it becomes harder than if they’re all available under a common path.
freedesktop.org produces specifications for interoperability, but we are not an official standards body. There is no requirement for projects to implement all of these specifications, nor certification.
Below are some of the specifications we have produced, many under the banner of ‘XDG’, which stands for the Cross-Desktop Group.
Its nit-picking, but this is a specification, i.e a preference, not an official standard. It would be great if everyone would agree on just one of these to use, but that isn’t a foregone conclusion. Even the actual standard, the FHS, isn’t followed by popular OS’s like NixOS.
I’ve only ever heard of FHS or XDG. Due to the free nature of linux distros, there is no central authority on how they are to be set up, and so there is no difference between those two options in terms of authority. Standards (which XDG is, colloquially) are followed based on popularity.
Yeah, I fully get that. The post and comments were very specific about how if you dont follow XDG, you’re fucking up, while only generally saying that “everything would be better if everyone followed the same standard.”
I pointed out that there are several standards and asked for a unique reason why XDG was the best to use.
I still haven’t heard one, which is fine, but it undermines the “If youre not using, XDG youre a idiot” tone of the post and comments.
So far, thats the most specific reason someone has given to use XDG, but I dont think it accurate.
FHS is the most used, as it’s been the primary linux filesystem standard for decades. Isn’t it better to stick with it if the only metric is popularity?
False on at least Fedora Atomic^[1]^, NixOS^[2]^ and openSUSE Aeon^[3]^…
Which ‘immutable’ distros are you referring to?
On Fedora Atomic, changing /etc is literally identical to how it goes any other distro; or at least 1-to-1 as on traditional Fedora. The bonus is that a pristine copy of the original /etc is kept inside a sub-directory of /usr. Furthermore, all changes compared to the pristine copy are kept track of.
On NixOS, changes have to be applied through configuration.nix. Though, regardless, it’s effectively possible to edit and populate /etc like it is on other distros.
It’s explicitly mentioned that /etc does not belong to the immutable base.
No sorry, Fedora Atomic has allowed changes to /etc since at least 2019. Regarding NixOS, the consensus is that it’s an immutable distro. The immutability of /nix/store/ suffices for this.
Your notion on Fedora Atomic was false. So, what other ‘immutable’ distro did you have in mind when making that comment?
Thank you for your honesty! I only intend for the truth to prevail and/or to reach mutual understanding. So please don’t feel attacked. If somehow I came off as such, my apologies; that has never been my intent.
I can only imagine someone asking this if they a) don’t use the terminal except if Stackexchange says they should and b) have yet to try and cleanup a system that’s acquired cruft over a few years. If you don’t care about it, then let me flip that around and ask why you care if people use XDG? The people who care about it are the people in the spaces that concern it.
Off the top of my head this matters because:
it’s less clutter, especially if you’re browsing your system from terminal
it’s a single, specified place for user specific configs, session cache, application assets, etc. Why wouldn’t such important foundational things required for running apps not be in a well defined specification? Why just dump it gracelessly in the user’s root folder outside of pure sloppy laziness?
it makes uninstalling apps easier
it makes maintenance easier
it makes installing on new machines easier
It’ll be in /home anyways and I heard BSD had some issues with something that could be XDG.
Someone asking a question doesnt merit the insult of saying they “would never ask if they used a terminal.” I have no particular dog in this fight, but not being a dick isn’t that hard.
It may actually be the best now, but so were the 14 others that came before it. Your stated reasons are the same reasons as everyone agreeing to use any other standard. Consistency, predictability, automation,ease of backup/restore, etc.
What sets this standard apart from all the rest? Based on their own description, they aren’t even an official standard, just one in “very active” use.
So why this, specifically? Just because its what you’re already doing?
Someone asking a question doesnt merit the insult of saying they “would never ask if they used a terminal.” I have no particular dog in this fight, but not being a dick isn’t that hard.
This is true, and something that I’m working on. For some reason my brain is uncharitable in these situations and I interpret it not as a simple question but a sarcastically hostile put down in the form of a question. In this case, “Why would you be dumb and not just put things in /home”. That really is a silly interpretation of the OP question, so I apologize.
As to using this standard, just because this is your preferred standard, doesnt mean its the only standard.
Sure, but the OP was essentially asking “Why isn’t dumping everything into a user’s /home the standard? Why are you advocating for something different?”
Based on their own description, they aren’t even an official standard, just one in “very active” use.
There are a LOT of “unofficial standards” that are very impactful. System D can be considered among those. The page you link to does talk about a lot of specifications, but it also says that a lot of them are already under the XDG specification or the reason for XDG is to bring such a scheme under a single specification, i.e. XDG.
So why this, specifically? Just because its what you’re already doing?
yes I do use it, so I am definitely biased in that regard
it bring a bunch of disparate mostly abandoned specification into a single, active one
it’s the active specification that has learned from past attempts
It’s weird to me that you think I think that. I do primarily browse files by terminal, but not always. Before I got into heavy terminal use I was a power user of Nemo. In any case, dumping everything in /home does not make for a better gui file browsing experience, either
The implication seemed to be “if you don’t care exactly where all your files are you must not use terminal”. Which I still don’t get. Just about anyone who would even be in a community like this uses terminal a lot anyway.
To give one example, what if someone wants to have more than one set of options for the same app? That’s something I’ve needed before, and it’s really hard to accomplish if the app always looks in one specific place for its options.
Oh so it makes it impossible to change config path? Yea that’s a bit inconvenient but you always can just make many files and replace the file in the right directory with the one you want.
Not if you want to use both at the same time. Due example, I’ve wanted to have a local Gnome session that I leave signed in, and another session with different settings that I remote into.
Coming back to this with thoughts. What you're describing sounds a lot like a menu tree.
"Press 1 to do this, 2 to do that, 3 to go to submenu A, 4 for B," etc. 1
"You have pressed 1. Do you want to turn on option ABC? [Y / n]" Y
"Do you want option QWERTY47? [Y/n]" N
"Are you sure you want to run notthebees --abc --no-qwerty47? [Y/n]" N
"Aborted."
It sounds like a standards problem waiting to happen because no two menus will be alike, but hey, things like this can and do exist, and setting one up isn't that hard, only time consuming.
This was my thought exactly. And I also had the same assessment that having various arguments be context aware would be challenging, since some have sub-arguments of their own (with further sub-arguments of their own, etc.) but can sometimes be strung together all on the same line. How do you determine if someone wants an ascendant argument or a descendant argument when you’re three layers deep into the tree?
You would have to make opinionated decisions, which was the whole reason to avoid scripts in the first place. Seems like it would be better to just make executable scripts (which is what Fedora Atomics basically do with the just command) or gamify learning how to work in the terminal.
Something like this can kind of be achieved programmatically by unraveling bash completion arguments and loosely parsing terminal help strings.
They aren’t all formatted uniformly though, so you’ll need to come up with a filtering mechanism to prevent returning garbage. You’ll also always be a little out of date…
I didn’t know about this (and thankfully, haven’t written anything public). I’ve been trying to fix an install script for an OSS project that doesn’t work on immutable distros, and using the XDG Base Directory specs might just be the panacea I was looking for!
Edit: adding some context. I am planning to setup a dev machine that I will connect to remotely and would like to babysit very little while having stable and fresh packages. In the Ubuntu world we would go to an LTS release but on the RPM/Dnf world is there any other distro apart from CentOS Stream? And also is CentOS Stream comparable to an LTS release at all considering that they do not have release number?
Wanting both stable and fresh packages is unfortunately somewhat difficult in my experience. I think the primary choice within the Fedora ecosystem is if you want to have fresh packages (Fedora) or if you prefer a slower update cycle and more stable packages (RHEL/Alma/Rocky). In the second case you can also choose if you wish to pay Red Hat for support (RHEL) or not (Alma or Rocky).
One thing that’s quite different in RHEL vs Ubuntu/Debian ist that it gets minor releases that include substantial new features. For example you’ll get new compilers, python versions, drivers, … CentOS Stream gets those slightly ahead of RHEL/Alma/Rocky (a cynical person might say that CentOS Stream is a rolling beta for RHEL). But, IMHO that’s not really a strong reason to use CentOS Stream.
If you’d go with an Ubuntu LTS release, then I’d look into RHEL/Alma/Rocky.
Well, when software supports this standard, you as a user have a way to not confirm to it by setting the env variables to whatever you want, even per app. So you have two choises, either use it as is or change it.
But if software doesn’t supportthe spec, there is no choise of using it. So ons choise less.
You can of course not give users a choice. And a lot of applications do their own thing, having their own variables like GOPATH or a cli option like –config or some way to do that in a config file like Idea IDEs. But implementing XDG from start is miles simpler for all parties, it’s good practice to have your paths and variables somewhat organized in code anyway.
Even if they use the same technique, they’re entirely different algorithms and h.264 also takes information from multiple different frames, which is why the video is 1.7gb but a folder with each frame saved as a png is over 300gb.
The formats with the best compression, where it might be fine, are jpeg xl and webp, as far as I know. They’re even slower tho because they’re so CPU intensive and only use one thread.
Setting the png compression to 0 doesn’t help because the bottleneck for png is the hard drives write speed. I already tried that.
This would just further complicate things for me. It assumes that 1) the system even has a windowing system/desktop environment or 2) all the installed software is XDG-aware. Most of the time I’m fiddling with headless environments.
Are you saying that you don’t want to write your software according to the XDG spec, or that you don’t want to set the XDG env vars on your system? If it’s the second that’s fine - apps using XDG work just fine if you ignore it. If it’s the first I’d suggest reconsidering because XDG can make things much easier for users of your software who have system setups or preferences that are different from yours; and using XDG doesn’t cause problems for users who ignore it.
OP’s recommendation is aimed mostly at software authors.
I meant the second. But as to the first: I generally write in-house software for headless server environments, and my peers are going to push back if I add irrelevant XDG foo to my PR.
So yes, “XDG” stands for “Cross-Desktop Group” - but I don’t agree that using the spec assumes a windowing system. The base directory spec involves checking for certain environment variables for guidance on where to put files, and falling back to certain defaults if those variables are not set. It works fine on headless systems, and on systems that are not XDG-aware (I suppose that means systems that don’t set the relevant env vars).
OTOH as another commenter pointed out the base directory spec can make software work when it otherwise wouldn’t on a system that doesn’t have a typical home directory layout or permissions.
The spec doesn’t make those assumptions at all, idk where that’s coming from.
I have headless machines with XDG vars configured and ones without them. XDG compliant software works in either case, but I’m less likely to use a piece of software that clutters my $HOME.
Honestly I don’t know, but it seems to me like extracting every single frame of a video as a lossless PNG is only really something that’s necessary if you’re trying to archive something or do frame by frame restoration. Either way, it is something that you hopefully aren’t doing every day, so why not just let it run overnight & move on?
Otherwise ask yourself if you can settle with just extracting a single clip/section, or what’s actually wrong with lossy jpeg with a low -qscale:v (high quality) - start around 5 and work down until you visually can’t see any difference
I’m doing this to upscale and interpolate the video and I want the best quality possible, since the source is using h.264 and I’m exporting to AV1. I was using jpeg with qscale:v 0 and 100% quality but you could still see compression artifacts, which is why I want to use a lossless format now. The upscaling and interpolation also takes quite a lot of time, so I’m also trying to minimize the time each step takes, if possible, since I’ll be doing this with multiple videos and I’ll probably use these scripts I made in the future a few more times.
Have you considered using av1an? it supports vaporsynth which has a large amount of upscale and frame interpolation tools, AI or not. If your upscaler supports vapoursynth, it could be a lot better option.
I use upscayl-ncnn (basically just the cli version of Upscayl) and it doesn’t support vapoursynth. I’ve heard of it before but I don’t really know what it is or how to use it.
Yeah, that’s the probably the case for those. I looked at CPU usage when using webp and one CPU core was always at 100%. Even tough it seems to not be able to use multiple cores, that’s still really slow, no? Or is that normal?
Also, my CPU is a Ryzen 5 3600, just to get an idea of what performance would be expected.
My first thought was similar - there might be some hardware acceleration happening for the jpgs that isn’t for the other formats, resulting in a CPU bottleneck. A modern harddrive over USB3.0 should be capable of hundreds of megabits to several gigabits per second. It seems unlikely that’s your bottleneck (though you can feel free to share stats and correct the assumption if this is incorrect - if your pngs are in the 40 megabyte range, your 3.5 per second would be pretty taxing).
If you are seeing only 1 CPU core at 100%, perhaps you could split the video clip, and process multiple clips in parallel?
At this point I’m very sure that the drive speed is actually the bottleneck. I’m not sure why it’s so slow tho. Splitting it is an interesting idea, maybe it’s also possible to tell ffmpeg to only extract every 6th frame and start at a different frame for each of the 6 cores.
Coming back to this, what you said at the end was really interesting. I could manually split up the file and run the frame extract script for each one at the same time but do you know if it’s possible to automate this? Or even better, run each instance of ffmpeg on the same video file and just extract every nth frame, like I said in my earlier reply?
If your drive is the bottleneck, this will make things worse. If you want to proceed:
You’re already using ffmpeg to get the sequence of frames, correct? You can add the https://superuser.com/a/377407 to give a start time and a duration. Generate a list of offsets by dividing the length of video by the number of processes you want, and feed them through gnu parallel to your ffmpeg command.
A) Export using a lower effort, with libjxl effort 2 or something will be fine.
B) Export to a faster image format like QOI or TIFF or PPM/PNM etc.
PNG, JXL, WEBP, all have fairly high encode times by default with ffmpeg. lower the effort or use a faster format
If you think that it really could be write speed limitations, encode to a ramdisk first then transfer if you have the spare ram, but using a different and faster format will probably help as PNG is still very slow to encode. (writing to /tmp is fine for this)
A) I actually didn’t know about this before, do you know what option I need to use in ffmpeg to set the effort?
B) I tried those but it’s the same issue as with png, that the hard drive’s write speed is too slow (or it’s the USB 3 connection but the result is the same)
Edit: Just found out how to set the effort. Setting it to 1 is quite a bit faster but still slow at only 3.8 fps.
what are your system specs? at a low effort you should be getting a lot more FPS, what cli command are you using? but I guess it would be best for you to export to /tmp given enough ram and then go from there
EDIT: for context, when encoding libjxl I would do -distance 0 -effort 2 for lossless output
PNG is a good format for graphics, lettering, logos... not photography so unless your video is some cartoons you're using png compression for something is not meant for.
I agree that you’re not really leveraging any features of PNG like you would using JPEG or RAW here, but saying it’s not meant for this use is an odd way to phrase it. There’s nothing inherently wrong with wanting lossless compression on an image…
linux
Oldest
This magazine is from a federated server and may be incomplete. Browse more on the original instance.