There have been multiple accounts created with the sole purpose of posting advertisement posts or replies containing unsolicited advertising.

Accounts which solely post advertisements, or persistently post them may be terminated.

Bots are running rampant. How do we stop them from ruining Lemmy?

Social media platforms like Twitter and Reddit are increasingly infested with bots and fake accounts, leading to significant manipulation of public discourse. These bots don’t just annoy users—they skew visibility through vote manipulation. Fake accounts and automated scripts systematically downvote posts opposing certain viewpoints, distorting the content that surfaces and amplifying specific agendas.

Before coming to Lemmy, I was systematically downvoted by bots on Reddit for completely normal comments that were relatively neutral and not controversial​ at all. Seemed to be no pattern in it… One time I commented that my favorite game was WoW, down voted -15 for no apparent reason.

For example, a bot on Twitter using an API call to GPT-4o ran out of funding and started posting their prompts and system information publicly.

dailydot.com/…/chatgpt-bot-x-russian-campaign-mem…

Example shown here

Bots like these are probably in the tens or hundreds of thousands. They did a huge ban wave of bots on Reddit, and some major top level subreddits were quiet for days because of it. Unbelievable…

How do we even fix this issue or prevent it from affecting Lemmy??

anzo ,

On an instance level, you can close registration after a threshold level of users that you are comfortable with. Then, you can defederate the instances that are driven by capitalistic ideals like eternal growth (e.g. Threads from meta)

anzo ,

Oh. And an invite-only could also work for new accounts.

NaoPb ,

I am glad clever people like yourselves are looking into this. Best of luck.

MediaSensationalism , (edited )
@MediaSensationalism@lemmy.world avatar

Signup safeguards will never be enough because the people who create these accounts have demonstrated that they are more than willing to do that dirty work themselves.

Let’s look at the anatomy of the average Reddit bot account:

  1. Rapid points acquisition. These are usually new accounts, but it doesn’t have to be. These posts and comments are often done manually by the seller if the account is being sold at a significant premium.
  2. A sudden shift in contribution style, usually preceded by a gap in activity. The account has now been fully matured to the desired amount of points, and is pending sale or set aside to be “aged”. If the seller hasn’t loaded on any points, the account is much cheaper but the activity gap still exists.
  • When the end buyer receives the account, they probably won’t be posting anything related to what the seller was originally involved in as they set about their own mission unless they’re extremely invested in the account. It becomes much easier to stay active in old forums if the account is now AI-controlled, but the account suddenly ceases making image contributions and mostly sticks to comments instead. Either way, the new account owner is probably accumulating much less points than the account was before.
  • A buyer may attempt to hide this obvious shift in contribution style by deleting all the activity before the account came into their possession, but now they have months of inactivity leading up to the beginning of the accounts contributions and thousands of points unaccounted for.
  1. Limited forum diversity. Fortunately, platforms like this have a major advantage over platforms like Facebook and Twitter because propaganda bots there can post on their own pages and gain exposure with hashtags without having to interact with other users or separate forums. On Lemmy, programming an effective bot means that it has to interact with a separate forum to achieve meaningful outreach, and these forums probably have to be manually programmed in. When a bot has one sole objective with a specific topic in mind, it makes great and telling use of a very narrow swath of forums. This makes Platforms like Reddit and Lemmy less preferred for automated propaganda bot activity, and more preferred for OnlyFans sellers, undercover small business advertisers, and scammers who do most of the legwork of posting and commenting themselves.

My solution? Implement a weighted visual timeline for a user’s points and posts to make it easier for admins to single out accounts that have already been found to be acting suspiciously. There are other types of malicious accounts that can be troublesome such as self-run engagement farms which express consistent front page contributions featuring their own political or whatever lean, but the type first described is a major player in Reddit’s current shitshow and is much easier to identify.

Most important is moderator and admin willingness to act. Many subreddit moderators on Reddit already know their subreddit has a bot problem but choose to do nothing because it drives traffic. Others are just burnt out and rarely even lift a finger to answer modmail, doing the bare minimum to keep their subreddit from being banned.

profdc9 ,

If they don’t blink and you hear the servos whirring, that’s a pretty good sign.

NaoPb ,

Ah yes, the 'bots.

drunkpostdisaster ,

Give up. There is no hope we already lost. Fuck us fuck our lives fuck everything we should just die.

hark ,
@hark@lemmy.world avatar

Is this a problem here? One thing we should also avoid is letting paranoia divide the community. It’s very easy to take something like this and then assume everyone you disagree with must be some kind of bot, which itself is damaging.

SpaceCadet ,
@SpaceCadet@feddit.nl avatar

Is this a problem here?

Not yet, but it most certainly will be once Lemmy grows big enough.

KairuByte ,
@KairuByte@lemmy.dbzer0.com avatar

Yeah, it’s a problem. You just don’t see it as often yet. A while back there were a large number of communities being blasted by bots, and they would make it into the hot category because nothing else was going on at the time.

Resol ,
@Resol@lemmy.world avatar

Create a bot that reports bot activity to the Lemmy developers.

You’re basically using bots to fight bots.

wuphysics87 ,

While a good solution in principle, it could (and likely will) false flag accounts. Such a system should be a first line with a review as a second.

ByteOnBikes ,

It’s reporting activity, not banning people (or bots)

UrPartnerInCrime ,

Are you willing to sift through all the reports?

Cause that’s gunna be A LOT of work

Melatonin ,

Let AI do it! See? Easy!

Resol ,
@Resol@lemmy.world avatar

Whenever I propose a solution, someone [justifiably] finds a problem within it.

I got nothing else. Sorry, OP.

Vandals_handle ,
@Vandals_handle@lemmy.world avatar

https://lemmy.world/pictrs/image/74986447-3d0c-4ba2-a62d-ddf9b62b4f42.jpeg

Resol ,
@Resol@lemmy.world avatar

Love that name too. Rock 'Em Sock 'Em Robots.

LarmyOfLone , (edited )

Fundamentally the problem only has temporary solutions unless you have some kind of system that makes using bots expensive.

One solution might be to use something like FIDO2 usb security tokens. Assuming those tokens cost like 5€. Instead of using an email you can create an account that is anonymous (assuming the tokens are sold anonymously) and requires a small cost investment. If you get banned you need to buy a new fido2 token.

PS: Fido tokens still cost too much but also you can make your own with a raspberry pico 2 and just overwrite and make a new key. So this is no solution either without some trust network.

sumguyonline ,

Make your own bot account that randomly(or not randomly) posts something bots will reply to, a system based response preferably. Last I was looking at bots they were simply programs, and have dev commands that can return information on things like system resources, or OS version. Your bot posts commands built in from the bot apps Dev, the bots reply like bots do with their version, system resources, or whatever they have built in. Boom - Banned instantly.

rglullis ,
@rglullis@communick.news avatar

The indieweb already has an answer for this: Web of Trust. Part of everyone social graph should include a list of accounts that they trust and that they do not trust. With this you can easily create some form of ranking system where bots get silenced or ignored.

A_Random_Idiot ,

A system like that sounds like it could be easily abused/manipulated into creating echo chambers of nothing but agreed-to right-think.

rglullis ,
@rglullis@communick.news avatar

That would be only true if people only marked that they trust people that conform with their worldview.

A_Random_Idiot ,

which already happens with the stupid up/downvote system.

Where popular things, not right things, frequently get uplifted.

rglullis ,
@rglullis@communick.news avatar

Well, I am on record saying that we should get rid of one-dimensional voting systems so I see your point.

But if anything, there is nothing stopping us from using both metrics (and potentially more) to build our feed.

A_Random_Idiot ,

Yeah, the up/down system is what prompted lots of bots to get created in the first place. because it leads to super easy post manipulation.

Get rid of it and go back to how web forums used to be. No upvotes, No downvotes, no stickers, no coins, no awards. Just the content of your post and nothing more. So people have to actually think and reply, rather than joining the mindless mob and feeling like they did something.

grrgyle ,

As a forum user I agree, but would like to add that many forums do have a kind of “demerit point” system for incivility. Where racking up enough points gets you temporarily muted or banned.

grepe ,

I was thinking about something like this but I think it’s ultimately not enough. You have essentially just two possible ends stages for this:

  1. you only trust people that you personally meet and you verified their private key directly and then you will see only posts/interactions from like 15 people. the social media looses its meaning and you can just have a chat group on signal.
  2. you allow some length of chains (you trust people [that are trusted by the people]^n that you know) but if you include enough people for social media to make sense then you will eventually end up with someone poisoning your network by trusting a bot (which can trust other bots…) so that wouldn’t work unless you keep doing moderation similar as now.

i would be willing to buy a wearable physical device (like a yubikey) that could be connected to my computer via a bluetooth interface and act as a fido2 second factor needed for every post but instead of having just a button (like on the yubikey) it would only work if monitoring of my heat rate or brainwaves would check out.

rglullis ,
@rglullis@communick.news avatar

Why does have it to be one or the other?

Why not use all these different metrics to build a recommendation system?

grepe ,

you are right - it doesn’t have to be one or the other… I just assume that for social media to work as I expect I don’t know most of the people on the platform. given that assumption and the lowering price of creating bots and ability to onboard them I expect that eventually most of the actors on the platform will end up being bots. people that write them are often insanely motivated (politically or financially) and creating barriers for them is not easy.

jjjalljs ,

The way I imagine it working is if I notice a bot in my web, I flag it, and then everyone involved in approving the bot loses some credibility. So a bad actor will get flushed out. And so will your idiot friend that keeps trusting bots, so their recommendations are then mostly ignored.

grepe ,

that is an interesting idea. still… you can create an account (or have a troll farm of such accounts) that will mainly be used to trust bots and when their reputation goes down you throw them away and create new ones. same as you would do with traditional troll accounts… you made it one step more complicated but since the cost of creating bot accounts is essentially zero it doesn’t help much.

rglullis ,
@rglullis@communick.news avatar

Just add “account age” to the list of metrics when evaluating their trust rank. Any account that is less than a week old has a default score of zero.

MediaSensationalism ,
@MediaSensationalism@lemmy.world avatar

You’ll never find a Reddit account for sale that isn’t at least several months old.

rglullis ,
@rglullis@communick.news avatar

Ok, which part of “multiple metrics” is not clear here?

Every risk analysis will have multiple factors. The idea is not to always have an absolute perfect ranking system, but to build a classifier that is accurate enough to filter most of the crap.

Email spam filters are not perfect, but no one inbox is drowning in useless crap like we used to have 20 years ago. Social media bots are presenting the same type of challenge, why can’t we solve it in the same way?

MediaSensationalism ,
@MediaSensationalism@lemmy.world avatar

I didn’t read very far up into the thread. Sorry.

Automated filters will just drive determined botters to play the system and perfect their craft until they can no longer be automatically identified, in my opinion. I’m more of the stance that accounts should be reviewed manually so that a leap into convincing bot accounts will need to be much more dramatic, and therefore difficult. If it’s done the hard way from the start with staff who know how to identify these accounts, it may keep it from growing into an issue to begin with.

Any threshold to be automatically flagged for review should be relatively low, but the process should also be quick and efficient. Adding more metrics to the flagging process only means botters will have a narrower gaze to avoid. Once they start crunching the numbers and streamline mimicking real user accounts it’s game over.

Fedizen ,

if the bots get so effective at mimicking users that they start to generate useful information that is also a win.

jjjalljs ,

But those bots don’t have any intersection with my network, so their trust score is low.

If they do connect via one of my idiot friends, that friend loses credit, too, and the system can trust his connections less.

The trust level is from my perspective, not global.

ByteOnBikes ,

Every time I see this implemented, it always seems like screwing over the end user who is trying to join for the first time. Platforms like reddit and Tumblr benefit from a friction-free sign up system.

Imagine how challenging it is for someone joining Lemmy for the first time and suddenly having to provide trust elements like answering a few questions, or getting someone to vouch for them.

They’ll run away and call Lemmy a walled garden.

rglullis ,
@rglullis@communick.news avatar

Platforms like Reddit and Tumblr need to optimize for growth. We need to have growth, but it is does not be optimized for it.

Yeah, things will work like a little elitist club, but all newcomers need to do is find someone who is willing to vouch for them.

Angry_Autist ,

You can’t just say ‘growth needs to be optimized for’ without sharing some optimizations…

SpaceCadet ,
@SpaceCadet@feddit.nl avatar

Platforms like reddit and Tumblr benefit from a friction-free sign up system.

Even on Reddit new accounts are often barred from participating in discussion, or even shadowbanned in some subs, until they’ve grinded enough karma elsewhere (and consequently, that’s why you have karmafarming bots).

Angry_Autist ,

lol reddit isnt friction free anymore, most subs want you to wait weeks or months before you post.

Same story, no experience, need work for experience, can’t get work without experience.

echodot ,

When I moderated a sub on Reddit I think I implemented a requirement that a poster must have at least positive three karma.

Was amazing how many scammers couldn’t even be bothered to do that little effort. Seriously they could have just upvoted each other but they couldn’t even do that.

All you have to do is introduce the smallest barrier to entry and it cuts bots admissions by about 95% as most of them out there are only looking for the lowest common denominator. They are unwilling to put in any effort at all.

grrgyle ,

My instance requires that users say a little about why they want to join. Works just fine.

If someone isn’t willing to introduce themselves, why would they even want to register? If they just want to lurk, they can do so anonymously.

EDIT I just noticed we’re from the same instance lol, so you definitely know what I’m talking about 😆

MangoPenguin ,
@MangoPenguin@lemmy.blahaj.zone avatar

How would I join a community without knowing anyone with that setup?

grrgyle ,

I think you’d work your way in naturally, same as any community throughout all of history.

I suppose an outsider might not be able to tell a web of trust that’s only bots trusting eachother, so you still have to think critically about what you read

ILikeBoobies ,

Keep the user base small and fragmented

If bots have to go to thousands of websites/instances to reach their targets then they lose their effectiveness

csm10495 ,
@csm10495@sh.itjust.works avatar

Thankfully we can federate both posts to make that easier :P

TheObviousSolution ,
@TheObviousSolution@lemm.ee avatar

This is another reason why a lack of transparency with user votes is bad.

As to why it is seemingly done randomly in reddit, it is to decrease your global karma score to make you less influential and to discourage you from making new comments. You probably pissed off someone’s troll farm in what they considered an influential subreddit. It might also interest you that reddit was explicitly named as part of a Russian influence effort here: www.justice.gov/opa/media/1366201/dl - maybe some day we will see something similar for other obvious troll farms operating in Reddit.

chiliedogg ,

To help fight bot disinformation, I think there needs to be an international treaty that requires all AI models/bots to disclose themselves as AI when prompted using a set keyphrase in every language, and that API access to the model be contingent on paying regain tests of the phrase (to keep bad actors from simply filtering out that phrase in their requests to the API).

It wouldn’t stop the nation-state level bad actors, but it would help prevent people without access to their own private LLMs from being able to use them as effectively for disinformation.

piccolo ,

Considering you can run LLMs on off the shlwf hardware, thats going to be as enforcable as piracy is…

TheGrandNagus , (edited )

I can download a decent size LLM such as Llama 3.1 in under 20 seconds then immediately start using it. No terminal, no complicated git commands, just pressing download in a slick-looking, user-friendly GUI.

They’re trivial to run yourself. And most are open source.

I don’t think this would be enforceable at all.

Hackworth ,

https://lemmy.world/pictrs/image/396bbb55-d995-460c-a690-007199d9049b.jpeg

fermuch ,
@fermuch@lemmy.ml avatar

That’s flux, isn’t it?

Hackworth ,

Aye, flux [pro] via glif.app, though it’s funny, sometimes I get better results from the smaller [schnell] model, depending on the use case.

brucethemoose ,

Trap them?

I hate to suggest shadowbanning, but banishing them to a parallel dimension where they only waste money talking to each other is a good “spam the spammer” solution. Bonus points if another bot tries to engage with them, lol.

Do these bots check themselves for shadowbanning? I wonder if there’s a way around that…

Crashumbc ,

I suspect they do, especially since Reddit’s been using shadow bans for many years. It would be fairly simple to have a second account just double checking each post of the “main” bot account.

brucethemoose ,

Hmm, what if the shadowbanning is ‘soft’? Like if bot comments are locked at a low negative number and hidden by default, that would take away most exposure but let them keep rambling away.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • [email protected]
  • random
  • lifeLocal
  • goranko
  • All magazines
    •