There have been multiple accounts created with the sole purpose of posting advertisement posts or replies containing unsolicited advertising.

Accounts which solely post advertisements, or persistently post them may be terminated.

Soliciting Feedback for Improvements to the Media Bias Fact Checker Bot

Hi all!

As many of you have noticed, many Lemmy.World communities introduced a bot: @MediaBiasFactChecker. This bot was introduced because modding can be pretty tough work at times and we are all just volunteers with regular lives. It has been helpful and we would like to keep it around in one form or another.

The !news mods want to give the community a chance to voice their thoughts on some potential changes to the MBFC bot. We have heard concerns that tend to fall into a few buckets. The most common concern we’ve heard is that the bot’s comment is too long. To address this, we’ve implemented a spoiler tag so that users need to click to see more information. We’ve also cut wording about donations that people argued made the bot feel like an ad.

Another common concern people have is with MBFC’s definition of “left” and “right,” which tend to be influenced by the American Overton window. Similarly, some have expressed that they feel MBFC’s process of rating reliability and credibility is opaque and/or subjective. To address this, we have discussed creating our own open source system of scoring news sources. We would essentially start with third-party ratings, including MBFC, and create an aggregate rating. We could also open a path for users to vote, so that any rating would reflect our instance’s opinions of a source. We would love to hear your thoughts on this, as well as suggestions for sources that rate news outlets’ bias, reliability, and/or credibility. Feel free to use this thread to share other constructive criticism about the bot too.

plz1 ,

I hate that I have to expand the section to see the rating. If that could be fixed, it’d be better.

solrize ,

Just kill it with fire.

AlexWIWA ,

Tell the bot to never be the first comment. I find it very frustrating when I see “a comment on this post” and it’s just the bot. I’m here to read what people have to say so it is very annoying when I think someone said something and it’s just the bot.

There was even a front page meme about this last year, but with another noisy bot. Lemmy doesn’t bury new comments like Reddit does, so there’s no real penalty to making the bot wait.

Carrolade ,

Personally I’m in favor of the bot. One complaint I’ve seen that I agree with is that it doesn’t need to float high up in the comments. If it was simply made to not upvote itself, it would stay nearer to the bottom naturally, which I think would be preferable.

cybervseas ,

I wish the comment count on a post didn’t include MBFC (or maybe bots in general).

jeffw OP ,
@jeffw@lemmy.world avatar

I’ll be honest, that’s probably outside of the scope of what we can do for now. It’s definitely valuable feedback in general and I wish I could offer some kind of solution but that’s probably even outside the control of the instance admins.

Someone can feel free to correct me if I’m wrong!

mozz , (edited )
@mozz@mbin.grits.dev avatar

Not directly related to MBFC bot, but what's your opinion on other moderation ideas to improve the nature of the discussion? Something Awful forums have strawmanning as a bannable offense. If someone says X, and you say they said Y which is clearly different from X, you can get a temp ban. It works well enough that they charge a not-tiny amount of money to participate and they've had a thriving community for longer than more existing social media has been alive. They're absolutely ruthless about someone who's being tricksy or pointlessly hostile with their argumentation style simply isn't allowed to participate.

I'm not trying to make more work for the moderators. I recognize that side of it... the whole:

This bot was introduced because modding can be pretty tough work at times and we are all just volunteers with regular lives. It has been helpful and we would like to keep it around in one form or another.

... makes perfect sense to me. I get the idea of mass-banning sources to get rid of a certain type of bad faith post, and doing it with automation so that it doesn't create more work for the moderators. But to me, things like:

  • Blatant strawmanning
  • Saying something very specific and factual (e.g. food inflation is 200%) and then making no effort to back it up, just, that's some shit that came into my head and so I felt like saying it and now that I've cluttered up the discussion with it byeeeeee

... create a lot more unpleasantness than just simple rudeness, or posting something from rt.com or whatever so-blatant-that-MBFC-is-useful type propaganda.

jeffw OP ,
@jeffw@lemmy.world avatar

It’s tricky because we could probably make 100 rules if we wanted to define every specific type of violation. But a lot of what you’re talking about could fall under Rules 1 and 8, which deal with civility and misinformation. If people are engaging in bad faith, feel free to report them and we’ll investigate.

DBT ,

I appreciate the bot. I like to play a game of “guess what the bot will say” before checking. I usually win, but it’s cool to have.

Aatube , (edited )
  1. Please, move the bias and reliability outside of the first accordion/spoiler. This is the sole purpose the bot was meant to provide. If we can't see that at a glance, it's bad. I don't see how these few words are "too long" either. I feel like a lot of the space could be cleared by turning the "Search Ground News" accordion into another link in the footer.
  2. While I personally don't see the point of the controversy, it wouldn't be too hard to manually enter Wikipedia's Perennial Sources list into the database that the bot references, especially with MediaWiki's watchlist RSS feed. This would almost certainly satisfy the community.
  3. Open source the database and the bot. Combined with #2, this could also offer an API to query Wikipedia's RSP for everyone to use in the spirit of fedi and decentralization.
mozz ,
@mozz@mbin.grits.dev avatar
  1. Open source the database and the bot.

Yes. A certain amount of my complaint about MBFC bot is not that it's a bad idea per se, it's just that the database and categorizations are laughably bad. It puts Al Jazeera in the same factual classification as TASS. It lists MSNBC as factually questionable and then when you look at the actual list, a lot of them are MSNBC getting it right and MBFC getting it wrong. It might as well be retitled "The New York Times's Awful Neoliberal Idea of Reality Check Bot". (And not talking about the biases ranking -- if that one is skewed it is fine, but they claim things are not factual if they don't match the appropriate bias, and the bias is unapologetic center-right.)

You can't set yourself up to sit in judgement of sources that write dozens of articles every single day about unfolding world events where the "objectively right" perspective isn't always even obvious in hindsight, and then totally half-ass the job of getting your basic facts straight about the sources you're ranking, and expect people to take you seriously. I feel like mostly the Lemmy hivemind is leaps and bounds ahead of MBFC bot at determining which sources are worth listening to.

  1. it wouldn't be too hard to manually enter Wikipedia's Perennial Sources list into the database that the bot references

FUCK FUCK FUCK YES

This is an actual up-to-date and very extensive list that people who care bother to keep up to date in detail (even making distinctions like "hey this source is ok for most topics but they are biased when talking about X, Y, Z"). This would immediately do away with like 50% of my complaint about MBFC bot.

Aatube , (edited )

For example, if we retain MBFC, the layout could look something like this:

Rolling Stone Bias: Left, Credibility: High, Factual Reporting: High - United States of America

MBFC report | bot support | Search topics on Ground.News

in which "Rolling Stone" is linked to the Wikipedia article.

With RSP, it could look something like this:

Rolling Stone is generally reliable on cultureThere is consensus that Rolling Stone has generally reliable coverage on culture matters (i.e., films, music, entertainment, etc.). Rolling Stone's opinion pieces and reviews, as well as any contentious statements regarding living persons, should only be used with attribution. The publication's capsule reviews deserve less weight than their full-length reviews, as they are subject to a lower standard of fact-checking. See also Rolling Stone (politics and society), 2011–present, Rolling Stone (Culture Council).

Rolling stone is generally unreliable on politics and society, 2011–presentAccording to a 2021 RfC discussion, there is unanimous consensus among editors that Rolling Stone is generally unreliable for politically and societally sensitive issues reported since 2011 (inclusive), though it must be borne in mind that this date is an estimate and not a definitive cutoff, as the deterioration of journalistic practices happened gradually. Some editors have said that low-quality reporting also appeared in some preceding years, but a specific date after which the articles are considered generally unreliable has not been proposed. Previous consensus was that Rolling Stone was generally reliable for political and societal topics before 2011. Most editors say that Rolling Stone is a partisan source in the field of politics, and that their statements in this field should be attributed. Moreover, medical or scientific claims should not be sourced to the publication.

RSP listing | bot support | Search topics on Ground.News

Both examples with everything necessary linked, of course

Zonetrooper ,
@Zonetrooper@lemmy.world avatar

I’m frankly rather concerned about the idea of crowdsourcing or voting on “reliability”, because - let’s be honest here - Lemmy’s population can have highly skewed perspectives on what constitutes “accurate”, “unbiased”, or “reliable” reporting of events. I’m concerned that opening this to influence by users’ preconceived notions would result in a reinforced echo chamber, where only sources which already agree with their perspectives are listed as “accurate”. It’d effectively turning this into a bias bot rather than a bias fact checking bot.

Aggregating from a number of rigorous, widely-accepted, and outside sources would seem to be a more suitable solution, although I can’t comment on how much programming it would take to produce an aggregate result. Perhaps just briefly listing results from a number of fact checkers?

jeffw OP ,
@jeffw@lemmy.world avatar

That’s fair. One idea could be a separate “community rating” and one that is more professional. Think Metacritic, RottenTomatoes, etc

Carrolade ,

I second this. This community is better than most social media, but it’s still that, and social media popularity is pretty bottom of the barrel as a means of determining accuracy. Additionally, that’d just open it up to abuse from people trying to weight the votes with fake accounts, scripts, whatever.

cybervseas ,

I’m not sure what to do here. On my mobile device the compacted media bias fact check post still takes up 50% of my phone screen.

How a post tag if we have a tagging system in Lemmy, instead of a whole long comment?

Maybe the bot could just post a one line summary with a link to more information?

jeffw OP ,
@jeffw@lemmy.world avatar

Thanks for the feedback. Can you elaborate a bit about the 50% of your screen thing? Is it the text itself, or is the issue that the app provides links at the bottom of the comment? I’m thinking of my experience on Voyager, where the links are summarized at the bottom of each comment, which does lead to a decent amount of screen being taken up. Would it be better if there weren’t any links?

cybervseas ,

yep I’m using Voyager on my iPhone. Maybe a super short summary without links. People could open the bot’s profile and look at the bot’s posts (not comments) if they want to dig deeper to understand a source.

jeffw OP ,
@jeffw@lemmy.world avatar

Interesting, so you think the bot should make posts too? Like, a post for each source with a summary of relevant info? Just making sure I understand what you mean

cybervseas ,

Yeah. It’s an idea for a way to create a user repository within Lemmy that could be edited by the bot as needed. I’m sure there are better ways.

catloaf , (edited )

It has been helpful and we would like to keep it around in one form or another.

Bull fucking shit. The majority of feedback has been negative. I can’t recall a single person arguing in its favor, but I can think of many, myself included, arguing against it. I hope you can find my report of one particular egregious example, because Lemmy doesn’t let me see a history of things I reported. I recall that MBFC rated a particular source poorly because they dared to use the word “genocide” to describe what’s going on in Gaza. Trusting one person, who clearly starts from an American point of view, and has a clearly biased view of world events, to be the arbiter of what is liberal or conservative, or factual or fictional, is actively harmful.

No community, neither reddit nor Lemmy nor any other, has suffered for lack of such a bot. I strongly recommend removing it. Non-credible sources, misinformation, and propaganda are already prohibited under rule 8. If a particular source is so objectionable, it should be blacklisted entirely. And what is and is not acceptable should be determined in concert with the community, not unilaterally.

Edit: And another thing! It’s obnoxious for bot comments to count toward the number of comments as shown in the post list. Nobody likes seeing it and thinking “I wonder what people are saying about this” and it’s just the damn bot again. But that’s really a shortcoming in Lemmy.

jeffw OP ,
@jeffw@lemmy.world avatar

I will start by saying that I feel like we are trying to address the criticism in your first paragraph with these changes. That being said, thanks for your feedback. I particularly like the comment you shared under the “edit,” because I hadn’t seen that sentiment shared before (not saying nobody else had that issue, just appreciating you for contributing that and challenging me to think more about how we execute things).

praise_idleness ,

This bot was introduced because modding can be pretty tough work at times and we are all just volunteers with regular lives.

Then maybe it can be an internal thing only. Let people do their own critical thinking. I believe that if you’re on Lemmy, you can make informed decision.

geekwithsoul ,

One problem I’ve noticed is that the bot doesn’t differentiate between news articles and opinion pieces. One of the most egregious examples is the NYT. Opinion pieces aren’t held to the same journalistic standards as news articles and shouldn’t be judged for bias and accuracy in the same way as news content.

I believe most major news organizations include the word “Opinion” in titles and URLs, so perhaps that could be something keyed off of to have the bot label these appropriately. I don’t expect you to judge the bias and accuracy of each opinion writer, but simply labeling them as “Opinion pieces are not required to meet accepted journalistic standards and bias is expected.” would go a long way.

jeffw OP ,
@jeffw@lemmy.world avatar

Thanks for this. As a mod of /c/news, I hadn’t really thought about that. We don’t allow opinion pieces, but this is very relevant if we roll out a new bot for all the communities that currently use the MBFC bot.

geekwithsoul ,

No problem. Specifically came to my attention about a week ago on this post where the bot reported on an opinion piece as if it was straight news.

BTW, I actually do appreciate the bot and think it’s doing about as well as it can given the technical limitations of the platform.

AlexWIWA ,

This contributes significantly to the noise issue most people complain about

MediaBiasFactChecker Bot ,

The news source of this post could not be identified. Please check the source yourself. Media Bias Fact Check | bot support

jeffw OP ,
@jeffw@lemmy.world avatar

Okay, so maybe we don’t need a comment if it’s a meta post or a mod announcement. Thanks for your inadvertent feedback, bot!

mozz ,
@mozz@mbin.grits.dev avatar

it also does this with a bunch of weird little local newspapers or etc which I've never heard of, which is like the one time I actually want it to be providing me with some kind of frame of reference for the source. MSNBC and the NYT, I feel like I already know what I think about them.

jeffw OP ,
@jeffw@lemmy.world avatar

Yeah, it’s tricky because who reviews those small guys? Granted, most of them are probably owned by a giant like Gannett, but that doesn’t mean we can just apply a rating from 1 small Gannett-owned paper to another. We’d like there to be some way for users to share their feedback/ratings on those small guys. But then it’s also true that some people will create a news site and try to share links on here to promote their new website and that’s typically just spam bots.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • [email protected]
  • random
  • lifeLocal
  • goranko
  • All magazines
    •