Hello, I set up my own lemmy instance and it can’t comment on other instances. on the apps it gives me a language not allowed, on desktop the post button just spins eternally. And idea how I can fix this?...
I’ve recently played with the idea of self hosting a LLM. I am aware that it will not reach GPT4 levels, but beeing free from restraining prompts with confidential data is very nice tool for me to have....
You can absolutely self host LLMs. HELM team has done an excellent job benchmarking the efficiency of different models for specific tasks so that would be a good place to start. You can balance model performance for your specific task with the model’s efficiency - in most situations, larger models are better performing but use more GPUs or are only available via APIs.
There are currently 3 different approaches to use AI for a custom task and application -
Train a base LLM from scratch - this is like creating your own GPT-by_autopilot model. This would be the maximum level of control, however the amount of compute, time, and data required for training does not make this an ideal approach for the end user. There are many open source base LLMs already published on HuggingFace that can be used instead.
Fine-tune a base LLM - starting with a base LLM, it can be fine tuned for a certain set of tasks. For example, you can fine tune a model to follow instructions or use as a chatbot. InstructGPT and GPT3.5+ are examples of fine tuned models. This approach allows you to create a model that can understand a specific domain or a set of instructions particularly well as compared to the base LLM. However, any time that training a large model is needed, it will be an expensive approach. If you are starting out, I’ll suggest exploring this as a v2 step for improving your model.
Prompt engineering or indexing using an existing LLM - starting with an existing model, create prompts to achieve your objective. This approach gives you the least control over the model itself, but is the most efficient. I would suggest this as the first approach to try. Langchain is the most widely used tool for prompt engineering and supports using self hosted base- or instruct-LLM. If your task is search and retrieval, an embeddings model is used. In this scenario, you generate embeddings for all your content and store the embeddings as vectors. For a user query, you then convert it to an embedding using the same model, and finally retrieve the most similar content based on vector similarity. Langchain provides this capability, but IMO, sentence-transformers may be a better starting point for a self hosted retrieval application. Without any intention to hijack this post, you can check out my project - synology-photos-nlp-search - as an example of a self hosted retrieval application.
To learn more, I have found the recent deeplearning.ai short courses to be quite good - they are short, comprehensive, and free.
I like and trust Proton Mail, and they support setting up custom domains while hosting your email data (for subscriber users).
You can then access it via their web mail box, via their Android and iOS apps, or via a desktop email client if you install their "bridge" application. The bridge application basically maintains the secure encryption ethos of their email system by ensuring all email traffic between your desktop and their servers remains encrypted, but can still be accessed via your preferred email clients such as Thunderbird or Outlook. The bridge is available for Windows, iOS and Linux.
I personally recommend Protonmail as it's primary focus is security and encryption, yet it does this in a very well developed and slick interface, so you get the best of both worlds. I'm a subscriber and moved from Gmail about 2 years ago as I wanted better privacy and security (they even have great tools for importing your old emails from major web providers). I don't have a custom domain but from my experiences of everything else they provide, I'd be confident it works as intended.
EDIT: In terms of cost, its €4 a month for the first tier which includes support for 1 custom domain, 10 email addresses, and 15GB of storage, or €10 for 500GB, 3 domains, 15 emails. They also include VPN, calendar, drive storage and a password manager in both.
I can highly recommend purelymail.com. They allow multiple domains, users and catch all accounts. They are great value, with a flat rate $10 per year ‘simple’ price, or you can pay per resource which for most people works out cheaper.
Been with them for over a year and been really good. Had a slight issue setting up one domain and their support were friendly, emailed back and sorted it out straight away for me.
I have a couple domains that are very low volume for outgoing mail. I use Migadu. I’m happy with their cheapest tier ($19/year for both domains). They have catch-alls and many other nice features.
Edit: They have no hard limits on the number of addresses, users, or domains and such. They just want you to be reasonable. You choose a tier based on your average quantity of outgoing mails per day. Again, there are no hard limits; they won’t cut you off unless you abuse the system.
July 8 (Reuters) - Jack Sweeney, the creator of an account tracking Elon Musk’s private jet in real time, has moved to Mark Zuckerberg’s Twitter-rival Threads after being suspended from Twitter last year.>...
With matrix, which is also federated, this cannot be done. In any way. At all.
You could attempt editing the database directly, but this is a bad, superbad, idea.
This is because the domain of the original server was part of the usernames and chat room names, and all of that would be in the database, for the server, and other servers, to know what goes together with what.
If you change your domain, your server wont know who the users logging in with the new domain are. Other servers wont know who they are, or what chat rooms they should be in.
I imagine it’s much the same with the fediverse. Changing your domain would break everything, where the databases are concerned.
And even if you could edit your database to swap out all mentions of the old domain, you could never do so for all other servers out there with federation database entries for your old domain.
Afaik, whenever an Activitypub instance has defederated from another it has always had to do with some combination of bad user behavior, poor moderation, and/or spam. Are the various instance admins who have decided to preemptively block threads.net simply convinced that these traits will be inevitable with it? Is it more of a...
Just a reminder that on Mastodon, you can choose "Block domain..." on a post. So an instance federating with meta can be blocked by the user. It would be nice to have that here on Kbin and/or all the other Fedi platforms.
A link to an instance is always going to be absolute so you just type the name of the instance. Most of the time the editor links it, but it still doesn’t understand some of the domain names, for example lemm.ee links but lemmy.world does not. If the editor fails to link you can prefix the name with https://lemmy.worldle lemmy.world
The link fixer bot shows how to link a community for Lemmy, but it’s different for kbin. The problem is if you use an absolute link to a community, you redirect the user away from their instance. The [email protected] format makes links relative so you stay on your own instance. Now we have the bot and it will post a fix, but better not to call the bot. Just do it right from the start.
There’s still the problem of how to do a relative link for a post. If someone knows I’d like to see that. At this point all post links are absolute and direct you away from your instance.
I looked up a list of Lemmy World instances and signed up for a bunch, but how do I see everything at once in one app instead of downloading one for each instance? I am new to what the kids call “federation.” Thank you.
I’ll try to simplify some concepts about the Fediverse and Lemmy using some more common IT knowledge.
I assume you are familiar with emails: gmail, protonmail, hotmail, aol, each of them runs a Server which will give you access, when registered, to a mailbox which will have a name of your choice, followed by an AT(@), and then a Domain specific to the vendor you have chosen.
ID
Lemmy works in a similar way: When you choose an instance (for example lemmy.world, or lemmy.ml, or feddit.de, you can find a list here: lemmyverse.net ) you are choosing where your account will live. You’ll get a full ID composed by your name of choice followed by an @ and then the domain of your chosen instance.
In your case your full ID is [email protected] . This is unique to you in the whole Fediverse.
Communities
Contrary to emails we have another abstraction: Communities. They are collections of posts and comments, moderated by someone and about a specific subject. These communities will reside on a specific server, like your account, and you can, for example, go to !technology or !mildlyinfuriating . As you can see their format is similar to that of your account: they have both a community name, the name before the @, and a domain that specifies where the main copy of it physically resides.
You can find a full, searchable, list of communities here: lemmyverse.net/communitiesFor some more complex reasons just disregards any of them that reside on beehaw.org (they have defederated with lemmy.world, but you don’t need to understand this concept yet).
Feeds
When you start to look around the lemmy.world site, and you are logged in with your lemmy.world account, you’ll have the ability to follow three different types of feed: Subscribed, Local, All.
Local: This feed shows only communities which main copy actually resides on the instance that you are currently on.
All: All the posts from all the communities of all the instances will appear here.
Subscribed: All the posts from the communities that you have subscribed to will show here.
Federation
Until now the concepts were pretty straightforward and not so dissimilar from other services, but now starts the more complex subject: Federation.
For all this different instances (computers running the Lemmy server) to have the information it needs to spread and they need to duplicate it.
This will allow you to see the same exact content whichever instance your account is on: this means that if you have chosen to create an account on lemmy.world or lemmy.ml it, generally speaking, doesn’t matter. You will see and interact with the same content and people.
Your instance is just the gate towards the content, which you have to always pass through (so no, you shouldn’t make an account in each instance, you just need one, and you should choose an instance which rules and values are acceptable for you and, better yet, similar to yours).
Going back to the previous comparison with email services you can think of this as: Each time you take an action (writing a post, a comment, sending an upvote) you are really just sending an email to all the instances, so they get notified and update their copy of the content.
An email saying something along the lines of “[email protected] has just downvoted the post number 69420 in the community 42 of the instance 123” will propagate until every single server has received it.
Defederation
Defederation is the strength and the weakness of the whole fediverse: it helps to moderate the content but it also breaks the web connecting all the servers.
Each instance automatically connects itself with all the others present in the net, but an Admin can actively decide that some other instance - maybe because it is full of Bots, or Scammers, or contains communities clearly voted to illegal or immoral things, or just they don’t like it; the reasons are subjective - really rustled their jimmies and they don’t want to see it anymore. In that case the Admin could decide to Defederate from them: this means that each time a new email from that instance comes it just get dropped and ignored.
So, if Instance A defederates from instance B, each time B sends an update to the content produced by the communities or people from B it will just be ignored by A. All the users registered on A will just not see anything or anyone from B.
Basically an instance wide shadow-ban.
Conclusion
This was just a quick and rough explanation of the concepts, but I hope it was useful: if you have any question I’ll try to answer them at the best of my abilities. But keep in mind that they are not really high. And I’m kinda tipsy.
The main takeaways are this:
You only need an account in a single instance.
You can subscribe to any community that you want, wherever they are.
You can search instances and communities here: lemmyverse.net
You should choose an instance that has similar values as you.
Sometimes the federation (the mails sent around between the servers) can get slow, but all content will be eventually consistent. Kinda.
Communities are sprouting around at high speed, there are a lot of duplicates and everything is in flux, try to be patience.
I said… imagine back to when there were the two main fourms for our favorite author? Now imagine if you could sign up as a user for one… but the forums can talk to one another so you can post on the other too, but your username will reflect the domain of the one you made your username on.
And defederation is when the two factions of fandom get into some fandom drama and decide not to let members of the other board talk to them anymore, lol.
It’s like one part forum, one part irc with distinct chat rooms around a topic, and one part signing up for a new email address, where the place you do it becomes part of your email address.
@masterspace@mango_master@Kichae
…
If that’s given, every entity (@‘person, @‘community, …) on each instance is unique.
Therefore, there can never be a duplicate identity = <entity>@<instance.domain>
Which allows the general assumption (all implementations adhere to the standard) each instance (homing instance, where the user is based) can verify the every identity within it’s domain.
…
Is it possible to change lemmy’s domain after I have already started it once and produced some content? I am thinking of moving to a subdomain but I’m not sure if it will go smoothly
It can cause some wackiness… basically you will need to maintain that old domain forever and everything will still refer to that old domain.
For example, your post looks like this from an ActivityPub/federation perspective:
{
[...]
"id": "https://atosoul.zapto.org/post/24325",
"attributedTo": "https://atosoul.zapto.org/u/Soullioness",
[...]
"content": "<p>I'm curious if I can migrate my instance (a single user) to a different domain? Right now I'm on a free DNS from no-ip but I might get a prettier paid domain name sometime.</p>n",
}
The post itself has an ID that references your domain, and the the attributedTo points to your user which also references your domain. AFAIK there is no reasonable way to update/change this. IDs are forever.
It would also break all of the subscriptions for an existing instance, as the subscriptions are all set to deliver to that old domain.
IMO your best bet would be to start a new instance on the new domain, update your profile on the old one saying that your user is now @Soullioness and maintain that old server in a read-only manner for as long as you can bear.
Apart from actual system administration or kernel developing, there’s no real “learn Linux” .
Video/Photo/Vector editing on Linux is not “learning Linux”, it’s learning to use a tool which runs on Linux. You can learn to use Blender, Gimp or Inkscape on Windows. You don’t edit videos/photos/vectors with the Linux kernel. You can even “learn the linux terminal” installing bash on Windows.
You can also install Visual Code or IDLE on Windows and on Linux. Learning to code on Visual Code or IDLE is not really “learning Linux”.
Also going on distro hopping looking for the “perfect distro” many times means the hopper simply doesn’t stick to one long enough to learn how to customize the environment to their liking (which usually means the window manager).
Most of the things you can do on the GUI, even the administration ones are just layers and layers of tools to make things “easier” - and they’ll be different on each distro and release. Command line administration will change much less, or at least less frequently.
Things I consider “learning Linux” are for example:
installing Linux (specially a headless server)
understanding how to use the package managers - again, on the command line
understand how systemd works
(hard core) dive into the kernel workings
understand how grub works
learn the general filesystem structure
learn how to analyze logs
learn user administration and how the permissions (and extended permissions) work
learn how to integrate Linux to a Windows environment (join a workgroup or domain, share storage, authenticate users)
learn how to check resources usage and how to troubleshoot it
undertand the nuances and of partitioning and when they are needed, as well as the different filesystems
etc (and /etc)
And yes, many of those are not strictly “Linux”, but are specific to a Linux system, unlike photo editing.
I don’t think this is EEE, I think this is a chance for meta to dominate the narrative by drowning us out with algorithmically curated censorship, distractions, hatred, outrage etc. I would join threads if I want threads, I would be on Reddit if I want corporate influence....
You don't need to stick with Lemmy. Kbin is just as good. I actually prefer it because they already solved the problem of allowing users to block domains.
Red Hat is going full evil mode and Fedora, which is largely controlled by Red Hat, is also pushing forward with questionable decisions. At this time, as some Fedora users look for a new $HOME there are many recommending OpenSUSE but before doing this, please read the post below....
I started using Fedora after RH killed CentOS, mainly for this reason. However now I feel bit differently about all of this.
At the end of the day, it’s clear RH is not doing this out of good of their heart. They are looking for mutually beneficial relationship, yes. But importantly they are also steering the Linux ecosystem towards that mutually beneficial direction.
And I no longer feel like I can support that. I don’t trust Red Hat as a company to keep innovating and improving the ecosystem in such way it is truly mutually beneficial in long term. I expect that they are mainly interested in directions that benefit RHEL, and allow RHEL to maintain commercially viable, private codebase.
I think that without pushback, they will make desktop linux like so many other Open Source projects: in practice the commercial product is the only really working and well-rounded implementation, because developing alternatives is very complex and requires so much developer time.
So I’d much prefer sending my bug reports to some other community with some other domain. And I’d like to contribute towards pushing the mutually beneficial relationship to a direction where RHEL is just another distribution, and Gnome just another DE. I don’t want a future where it makes sense to say a user is missing Gnome-functionality or RHEL-features, when discussing software that has no reason to be exclusive to either.
If RH is the primary developer of Fedora, and Fedora is the exclusive testbed for desktop-linux, I feel like that’s likely to happen.
I already owned the domain and have access to a server with more than enough resources, so it didn’t have a downside to me.
Upside, I don’t really have to worry about anyone else’s federation choices. Undesirable content like loli/shouta stuff doesn’t appear at all, because I’m basically the only user and don’t subscribe to anywhere that exists so it doesn’t federate to me anyway. My instance never lags because nobody but me uses it. Sometimes it misses comments through federation from overloaded instances, but it seems like the newer version of Lemmy has helped that greatly.
I will soon start a new job where I expect to receive significantly more emails than I do currently. So far, I haven’t had a system in place, except for marking emails as unread until I respond and occasionally using flags....
Yeah basically the rules where “if from domain A go to folder A.”
The organized folders basically served as a way to filter through stuff that I didn’t need to respond to, break things down into tasks I actually needed to respond to, and to make it easier to search through later.
So if I got an email from user@xdomain, it would go to my xdomain folder and be listed as unread and I would respond from there. Then that email chain stayed in its appropriate folder.
Honestly, it depends on what you are doing, but there are definitely ways to keep things organized when receiving a lot of emails.
Just to give an example of my own job (I receive 50-100 or so emails per day), I use Outlook and make use of rules. Within my inbox, I have specific folders for each of the teams I work with, I have a folder for company-wide emails, I have folders for specific tasks I perform, and I have folders for projects that I work on.
The way that I set up rules is basically just to keep a list of important keywords and keep an up-to-date contact list. For the different teams, we have a robust active directory where everyone is discoverable, so I just grab names from a given department and add them to distribution lists labeled essentially “Team A”, “Team B”, etc. When I get emails from people who belong to those teams, I have rules that look for anyone belonging to those groups and automatically drop them in those folders.
For company-wide emails, those typically originate from a limited number of senders who handle communications, so I do the same thing and just drop them into a news/announcements folder.
For vendors, they’ll send emails from specific domains, so I add a rule to filter emails from those domains, regardless from who, and drop them into labeled subfolders within a larger “Vendors” folder.
For tasks that I work with, there are always keywords in the subject field that specify the nature of the request. If it’s a ticket, for example, I’ll create a rule looking for the keyword “Ticket” and drop them in the Tickets folder. If it’s for user enrollment, I’ll look for “Enrollment” and drop it in the New Users folder (and these always come from just a couple email addresses so I put those in the requirements, too).
For projects, we do a bit of intersecting. I’ve got people from different teams and different vendors who are working on a specific thing that I want to keep together. So here I do my best to identify the specific people involved with this project, and also identify common keywords I expect to see in the subjects of emails. So for example, if this is a project to install widgets in the conference room for improved synergy, I’d set a rule looking for the specific names on the project and keywords like “widgets”, “conference room”, “synergy” in the subject line, prioritize these over the other rules, and it catches about 90% of the emails I want it to, leaving me just a few that land in other places that may need to be manually sorted due to ambiguously named subjects (For the love of god, if you are emailing about a project, put some indicator about the project instead of just putting “Hey” or “Status Update” in the subject line).
I have Mastodon running on a VPS running Debian 11. Now I would like to add a Lemmy instance on the same server. I tried using the from scratch method from Lemmy documentation, but ran into errors that likely stemmed from minor version incompatibilities of the dependencies. I tried using the Lemmy easy deploy script but it wants...
Nope don't feel bad - it's totally understandable. Kbin is new, so the documentation is lacking.
Almost all content on Kbin, including users, has a follow / subscribe button in the sidebar beneath the magazine or user description. There's also a block button. These buttons can be used on almost all Kbin content, so it's very powerful - after spending about a week adding subs, my Kbin feed is far more active than Reddit was, even at its height.
Kbin breaks down your content into Magazines, Microblogs, and Threads. Magazines are synonymous with Communities on Lemmy, or subreddits on reddit. Microblogs are where the Mastodon Toots go, and how you interact with instances based on that architecture. Threads are just like posts on reddit, and can be text only, a link, a pic, or a video (although it seems video is still under development).
The best place to start getting subscriptions are in the magazines section:
...which lists all of the currently federated communities. Putting a domain search into the search bar will bring up all magazines in the instance on that domain:
This domain searching is extremely powerful, especially when you use the domain section (which can be hard to find) - you can get a breakdown of any domain currently federated on kbin by using the following link:
...where you'd put the domain you're interested in in place of 'lemmy.world'. You can then subscribe to the entire instance through that feed, or block it f you'd like. This also works for standard domains as well:
Since usernames are only unique to the instance it’s created on, what’s to stop someone from creating a copycat username in order to impersonate another user?
Mastodon has a sort of lightweight verification which just signifies that you are to some degree in control of the URLs linked to in your profile. So for example, if you have your own domain or something that people associate with you, then you can use that in your profile to show that it’s you. Of course, that depends on that domain meaning something to the end user, and the end user being savvy enough to, for example, know that someone could get the .com version of your .net domain, etc. etc.
Created a lemmy instance but can't post comments on other instances
Hello, I set up my own lemmy instance and it can’t comment on other instances. on the apps it gives me a language not allowed, on desktop the post button just spins eternally. And idea how I can fix this?...
My personal Newbie Sunday: How to install Teddit with docker. (lemmy.ml)
Hello all !...
Selfhosted LLM (ChatGPT)
I’ve recently played with the idea of self hosting a LLM. I am aware that it will not reach GPT4 levels, but beeing free from restraining prompts with confidential data is very nice tool for me to have....
Best alternative to selfhosting email? E.g. email hosting provider for a custom domain
Hey guys,...
Suspended Twitter account tracking Elon Musk's jet moves to Threads (www.reuters.com)
July 8 (Reuters) - Jack Sweeney, the creator of an account tracking Elon Musk’s private jet in real time, has moved to Mark Zuckerberg’s Twitter-rival Threads after being suspended from Twitter last year.>...
When someone loses the domain name, can they still set up the same lemmy server under a different name?
Is it possible or does it present difficulties with federation?...
What happened to vlemmy.net?
I had my original account there and now I can’t access it at all or load posts.
What does defederating from Meta's Threads.net actually accomplish?
Afaik, whenever an Activitypub instance has defederated from another it has always had to do with some combination of bad user behavior, poor moderation, and/or spam. Are the various instance admins who have decided to preemptively block threads.net simply convinced that these traits will be inevitable with it? Is it more of a...
how do I link a lemmy instance or community in a comment?
As title....
I don't understand.
I looked up a list of Lemmy World instances and signed up for a bunch, but how do I see everything at once in one app instead of downloading one for each instance? I am new to what the kids call “federation.” Thank you.
Does it feel like the fediverse is exclusively used by older tech nerds?
The mastodon and lemmy content I’m seeing feels like 90% of it comes from people who are:...
why can't we have federated identity ?
Why can’t we have federated identity to login into fediverse instead of creating login for each instance?
Is it possible to change lemmy's domain?
Is it possible to change lemmy’s domain after I have already started it once and produced some content? I am thinking of moving to a subdomain but I’m not sure if it will go smoothly
Just finished cs50 need some guidance.
My bachelor’s in software engineering starts in quite a few months...
lemmy.world has bent the knee to corporations. Consolidated comments into body.
I don’t think this is EEE, I think this is a chance for meta to dominate the narrative by drowning us out with algorithmically curated censorship, distractions, hatred, outrage etc. I would join threads if I want threads, I would be on Reddit if I want corporate influence....
Read this post by /u/[email protected] before recommending OpenSUSE during these trying times.
Red Hat is going full evil mode and Fedora, which is largely controlled by Red Hat, is also pushing forward with questionable decisions. At this time, as some Fedora users look for a new $HOME there are many recommending OpenSUSE but before doing this, please read the post below....
Second largest Lemmy instance preemptively un-friends Facebook (lemmy.ml)
Lemmy.ml has now blocked Threads.net
How do I manage my inbox?
I will soon start a new job where I expect to receive significantly more emails than I do currently. So far, I haven’t had a system in place, except for marking emails as unread until I respond and occasionally using flags....
YSK these e-mail tips (i.postimg.cc)
Why YSK: These email tips are helpful for people who struggle with boundaries and want to communicate more assertively.
Anyone hosting Lemmy and Mastodon on the same server?
I have Mastodon running on a VPS running Debian 11. Now I would like to add a Lemmy instance on the same server. I tried using the from scratch method from Lemmy documentation, but ran into errors that likely stemmed from minor version incompatibilities of the dependencies. I tried using the Lemmy easy deploy script but it wants...
What's to prevent someone from hijacking my username?
Since usernames are only unique to the instance it’s created on, what’s to stop someone from creating a copycat username in order to impersonate another user?