Yeah the documentation (if it even exists) of most projects is usually clearly written by people intimately familiar with the project and then never reviewed to make sure it makes sense for people unfamiliar with it. But writing good detailed documentation is also really hard, especially for a specialist because many nontrivial things are trivial to them and they believe what they’re writing is thorough and well explained even though it actually isn’t.
You have to assume some level of end user knowledge, otherwise every piece of documentation would start with “What a computer does” and “How to turn your computer on.”
I’ve found the best practice is to list your assumptions at the top of the article with links to more detailed instructions.
Another case is listing a huge number of steps to do some task, without acting describing what the end goal for each set of instructions is (common in “how to” guides, and especially ones that involve a GUI).
This means that less technical users don’t really understand what is going on and are just following steps in a rote way, and it wastes the time of technical users since they probably know how to achieve each goal already.
I agree with this. When I publish my code, it is documented for someone in my field with around my level of knowledge. I assume you know DNS, I assume you know what a vector is, I assume you know what a dht is, I assume you know what O(log n) is.
I’m not writing a CS50 course, I’m helping you use the code I wrote.
Might be different for software like libre office which is supposed to be used by anyone, but most software on earth is built with other developers in mind.
That’s why blog posts rock. Most popular projects will have a dozen blog posts for different configurations. For example, when looking to set up NextCloud, I found docs for almost all combinations of the following:
Apache and Nginx configuration
running through Docker or directly on the host
MariaDB and Postgres configs (and SQLite, with proper disclaimers)
Collabora and OnlyOffice config
It does take some knowledge of each of the above if you need one of the few configs that’s not available on a blog post, and some of the posts are outdated, but with a bit of searching almost everything is documented by someone on the internet.
This shouldn’t be necessary (official docs should be more comprehensive), but at least it’s available.
Okay, please point me to the blog posts that helped you with collabora/onlyoffice. Thanks have NEVER been able to get that to work with my nextcloud (currently using the Docker AIO).
I’m not at the same computer I used to look it up, so I don’t have my search history, but I think this one was pretty decent. I don’t use Traefik, but the rest describes the important bits w/ docker compose. I don’t know much about the AIO image though (I used separate images).
Same with me. I played around with the setup a few times on my local machines. It took quite a bit to get it set up, then I saw an error after a couple of days and gave up. Its easier to just pull down the file and run it locally than use collabora.
I’d rather have a great documentation than five different blog posts, where some of them might be outdated, wrong or insecure (and you only find out later).
But yes, they are helpful and easily available for popular software.
My CS major required me to take two upper division English classes and I think they helped me more in my career than my upper division CS classes. People forget that documentation is for ourselves too
I’m really thankful that I had a great English teacher in high school, and that my degree required a technical writing class. Being able to write a coherent email got me further in my career than the technical stuff I learned in college.
There is a case to be made that people should be a bit more well rounded in general, and not just find a specific niche.
So non-technical people should still have a decent familiarity with computers and maybe be able to do some very basic coding. And technical people should spend some time working on their written and verbal communication.
Because in both cases, it makes people more effective in their roles.
Totally agree. And I’d argue that we don’t even need technical writers. Even if all people do is correct grammar and spelling mistakes it would be helpful, let alone actually writing docs. It’s one of the easiest ways non-technical folks can get involved with open source projects.
I think this is why the “my code documents itself” attitude appeals, even though it’s almost never enough. Most developers just can’t write, nor do they want to.
The information that familiarity gives you is “WTF does this field means”, and it’s the only thing that’s actually there. How you get a value and how a value is formatted are things no amount of expertise will save you from having to tell the computer, and thus you can’t just forget about.
(And let me guess, the software recommended install is a docker image?)
I don’t think it’s (just) that. It’s also a different skill set to write documentation than code, and generally in these kind of open source projects, the people who write the code end up writing the documentation. Even in some commercial projects, the engineers end up writing the docs, because the higher ups don’t see that they’re different skill sets.
This is why I did a “walkthrough test” when I had to write documentation on this sort of thing. I’m a terrible technical writer, so this shit is necessary for me.
I grabbed my friend who knows enough about computers to attempt this, but not enough about infrastructure to automatically know what I meant when I was too vague.
Took two revisions, but the final document was way easier to follow at the end
Reminds me of the time I asked a question about a Magic: The Gathering card tomy local game store’s Facebook page. The card was Sublime Archangel. I asked what happened if it gave a creature Exalted that already had one. Someone sarcastically replied that it already says it on the card. I was a new player, how was I supposed to parse the phrase “If a creature has multiple instances of exalted, each triggers separately”? For all I knew that could mean that they didn’t stack because they would need to trigger together. I didn’t have the vocabulary to understand those things.
I try to remember this when explaining what I might believe are simple concepts to people because that person really upset me.
100% Agree, it feels like most documentation is written in a way that expects you to already know what it’s talking about… When it’s the documentation’s job to teach me about it.
I’m guessing this string can be whatever you want it to be.
But yeah, I agree in general, some of the docs can be pretty opaque. For example, I wanted to configure NextCloud w/ Collabora in Docker, and I kept getting errors when trying to do what a few sites recommended. I ended up figuring it out, but only through trial and error. I’m going to go through the same pain this weekend when I try out ownCloud Infinite Scale up and running to compare.
I had very similar experiences with OCIS. Got it all set up following the quick start guide, found extremely odd and unacceptable behaviour with storage space ballooning, start troubleshooting and find “oh you had to do this, this and this manually, it’s in the docs” It is in the docs, but never referenced by any other part of the docs. Because why would you mention the thing that the admin must manually set up in 100% of installs in your setup guide?
Anyway I’ve become that guy ranting on the internet that I don’t want to be. So just so you don’t suffer as much as I did; you have to create scheduled tasks via cron or your preference of scheduler to clean your uploads folder and data blobs. This also did not fix my specific issue and I ended up giving up on OCIS and sticking to Nextcloud.
I’m going to run both in parallel for a month or so before trying to get my SO to use it so I can better estimate the WAF. So far, NextCloud is good enough, but it’s kinda slow (and I have Redis configured) despite being on pretty beefy hardware (Ryzen 1700 w/ 16GB RAM). I really hate PHP, so I’d prefer a project I can contribute to if needed. I worked w/ Go for almost 10 years, so OCIS would be a natural fit, but I’d still contribute patches for PHP if that really was the best tool for the job. But I’m not going to get involved unless the project already does what I need (my contributions would be for smaller bug fixes).
But yeah, the OCIS docs look kinda mediocre from the little I’ve read of them. But at least I don’t need to mess w/ PHP config most likely and can hopefully just forward HTTP requests to it.
The move from php to go and the slowness of NC is what attracted me to the project. But I’m going to wait a bit longer until we’re flush with 3rd party setup guides cause I simply do not have the time to wade through their docs.
Yup, that’s why I started w/ NextCloud. It was painful enough getting Collabora working with NC, so hopefully OCIS is easier now that I know my Collabora setup at least works.
Matrix and its implementations like Synapse have a very intimidating architecture (I’d go as far as to call most of the implementations somewhat overengineered) and the documentation ranges from inconsistent to horrific. I ran into this particular situation myself, Fortunately this particular step you’re overthinking it. You can use any random string you want. It doesn’t even have to be random, just as long as what you put in the config file matches. It’s basically just a temporary admin password.
Matrix was by far the worst thing I’ve ever tried to self-host. It’s a hot mess. Good luck, I think you’re close to the finish line.
I still have to sort out having a different server name to the access name so I can use the domain as well. Do I just put a field into the config like the rest? Can it go anywhere?
Matrix seemed interesting right until I got to self hosting it. Then, getting to know it from up close, and the absolute trainwreck that the protocol is, made me love XMPP. Matrix has no excuse for being so messy and fragile at this point. You do you, but I decided that it isn’t worth my sysadmin time (especially when something like ejabberd is practically fire and forget).
funnily there’s an… ansible i think? project that makes selfhosting synapse easy as fuck, you basically just go “ansible deploy synapse” or whatever the syntax is and it does almost everything for you.
Alternatively, you can create new users from the command line.
This can be done as follows:
If synapse was installed via pip, activate the virtualenv as follows (if Synapse was installed via a prebuilt package, register_new_matrix_user should already be on the search path):
cd ~/synapse
source env/bin/activate
synctl start # if not already running
Run the following command:
register_new_matrix_user -c homeserver.yaml
This will prompt you to add details for the new user, and will then connect to the running Synapse to create the new user. For example:
New user localpart: erikj
Password:
Confirm password:
Make admin [no]:
Success!
This process uses a setting registration_shared_secret, which is shared between Synapse itself and the register_new_matrix_user script.
It doesn’t matter what it is (a random value is generated by --generate-config), but it should be kept secret, as anyone with knowledge of it can register users, including admin accounts, on your server even if enable_registration is false.
Honestly, as a newbie to Linux I think the ratio of well documented processes vs. “draw the rest of the fucking owl” is too damn high.
The rule seems to be that CLI familiarity is treated as though its self-evident. The exception is a ground-up documented process with no assumptions of end user knowledge.
If that could be resolved I think it would make the Linux desktop much more appealing to wider demographics.
That said, I’m proud to say that I’ve migrated my entire home studio over to linux and have not nuked my system yet. Yet… Fortunately I have backups set up.
Linux on the desktop almost never needs CLI interaction though. Maybe you’ll need to copy/paste a command from the internet to fix some sketchy hardware, but almost everything works OOTB these days.
However, self-hosting isn’t a desktop Linux thing, it’s a server Linux thing. You can host it on your desktop, but as soon as you do anything remotely server-related, CLI familiarity is pretty much essential.
That depends on your use case for desktop linux of course. For me, yabridge is the tool I needed to run VSTs on Linux. Its CLI only as far as I know.
Don’t get me wrong; I’m not afraid of the CLI. Its just some tools are assuming the end user is a server admin or someone with deeper than the upper crust knowledge of how Linux works.
Don’t forget the situations where you find a good blog post or article that you can actually follow along until halfway through you get an error that the documentation doesn’t address. So you do some research and find out that they updated the commands for one of the dependency apps, so you try to piece together the updated documents with the original post, until something else breaks and you just end up giving up out of frustration.
That sounds an awful lot like modifying an ESP32 script I’ve been trying to follow from a YouTube tutorial published a while back. Research hasn’t uncovered anything for me to troubleshoot the issue so it’s a really shit experience.
That shouldn’t be too bad if you understand systemd though, right? Or is there something weird i’m missing? Do you have an example guide that illustrates the problem?
CLI familiarity is fine. CD, Nano, mkdir, rm. I am proficient with that. But I am not necessarily proficient with Docker (went with it because it worked nicely for another thing which was well documented and very straight forward). It’s just I’m trying to self host stuff. Some things like Wordpress and Immich are straightforward. Some things aren’t like Matrix and Mastodon. Lemmy is also notoriously bad.
I think if you’re talking wider demographics your model OSs are (obviously) Windows and macOS. People buy into that because CLI familiarity isn’t required. Especially with Apple products everything revolves around simplicity.
I do dream of a day when Linux can (at least somewhat) rival that. I love Linux because I am (or consider myself) intricately familiar with it and I can (theoretically) change every aspect about it. But mutability and limitless possibilities are not what makes an OS lovable to the average user. I think the advent of immutable Linux distros is a step in the right direction for mass adoption. Stuff just needs to work. Googling for StackOverflow or AskUbuntu postings shouldn’t ever be necessary when people just want to do whatever they were doing on Windows with limited technical knowledge.
However on another note, if you’re talking a home studio migration, not sure what that entails, but it sounds rather technical. I don’t want to be the guy to tell you that CLI familiarity is simply par for the course. Maybe your work shouldn’t require terminal interaction. Maybe there is a certain gap between absolutely basic linux tutorials and the more advanced ones like you suggest. Yet what I do want to say is that if you want to do repairwork on your own car it’s not exactly like that is supposed to be an accessible skill to acquire. Even if there are videos explaining step by step what you need to do, eventually you still need to get your own practice in. Stuff will break. We make mistakes and we learn from them. That is the point I’m trying to get at. Not all knowledge can be bestowed from without. Some of it just needs to grow organically from within.
I haven’t done any programming in over 20 years, but I think I can make a contribution to projects by trying to improve documentation, once I start using some projects
I have a 2 page Google Doc that I wrote while installing Mateix (because I wanted to be able to recover from a complete system loss, and knew I'd forget what I did). Half of the doc is my HAProxy notes.
Are you still having issues? I could try cleaning up my notes for a wider audience (note: my professional background includes technical writing and corporate technical training, so I'd be super anal about, and it would take a few hours at least).
Is this meant to be single-user, or a larger host?
So you added the secret to the file and restarted the docker container, right?
Something that I think will help you with self-hosting in the future is to always read through the entire process for setting up whatever you want to set up first, beginning to end, so that you are familiar with what you need to do before attempting it the first time. It's helped me numerous times myself.
Which config file does it go in? Where does it go in that file? Do you literally just put “registration_shared_secret” or does it need a value? What is the syntax of setting the value? Does it accept spaces, special characters, etc.?
By the way, running synapse - docker or not - is a challenge. It can be very complex especially if you are interested in adding gateways to other services and such. Attempting to use https://github.com/spantaleev/matrix-docker-ansible-deploy might be a better choice as even though it is A LOT, it has a ton of good documentation and you can grow with it as it can help you install various different Matrix servers, gateways and clients as well.
Good luck, hope to hear more about how you get on with it.
So why didn’t they write that? It’s a bad documentation if someone doesn’t understand it. If you’re not going to explain something, at least share a source to where it’s explained.
I have to set literally everything up again on a new microSD for my Pi because the apt-get repositories no longer support the Raspbian version I’m on. I’m not mad; good for security to update, but I don’t have half a day free anytime soon for it.
A temporary one that you’re expected to remove as soon as you’ve created the admin user(s) you need, but yes. It should only be there during initial setup and ideally removed before the server is ever exposed to the internet.
Yes because having a user remember to do something is a great line of defense, better than encrypting it from the get go. It should just be encrypted in the file.
I think that’s the way both Splunk and JFrog work – you generate or enter a password into the key field in a YAML file somewhere, start the service, and next time you come back the field’s been encrypted.
While security has nothing to do with my disgust for docker and people advocating its use, docker adds a layer of complexity, which means it is not necessarily more secure.
What is extremely bad about docker:
it enables extremely shitty configuration control on the side of a developer. There are way too many developers who have a chaotic approach to configurations, and instead of being forced to write a proper installation and configuration guide from scratch, and thereby making themselves(!) aware of active configuration changes they made to make their system work, they just roll out the docker container they develop in, without remembering most of the configurations they made. Which, naturally, means that they are unable to assist in troubleshooting problems or reproduce issues that users might have.
In general, if you can’t write a good user manual, or at least clearly identify needed dependencies and configurations, you should not be developing software for other people.
it combines the disadvantages of a VM (shitty performance) and running directly on the host OS (sandboxing is not nearly as good as on a VM)
it creates insane bloat, by completely bypassing the concept of shared libraries and making people download copies of software they already have on their system
it adds a lot of security risks because the user would have to not only review the source code they are compiling and installing, but also would have to scan all the dependencies and what-not, and would basically have to trust the developer and/or anyone distributing an image that they did not add any malware.
It took a little time to get the hang of it, but stick with it and it will get so much easier and it'll make self-hosting anything you want less of a pain in the future.