There have been multiple accounts created with the sole purpose of posting advertisement posts or replies containing unsolicited advertising.

Accounts which solely post advertisements, or persistently post them may be terminated.

Soliciting Feedback for Improvements to the Media Bias Fact Checker Bot

Hi all!

As many of you have noticed, many Lemmy.World communities introduced a bot: @MediaBiasFactChecker. This bot was introduced because modding can be pretty tough work at times and we are all just volunteers with regular lives. It has been helpful and we would like to keep it around in one form or another.

The !news mods want to give the community a chance to voice their thoughts on some potential changes to the MBFC bot. We have heard concerns that tend to fall into a few buckets. The most common concern we’ve heard is that the bot’s comment is too long. To address this, we’ve implemented a spoiler tag so that users need to click to see more information. We’ve also cut wording about donations that people argued made the bot feel like an ad.

Another common concern people have is with MBFC’s definition of “left” and “right,” which tend to be influenced by the American Overton window. Similarly, some have expressed that they feel MBFC’s process of rating reliability and credibility is opaque and/or subjective. To address this, we have discussed creating our own open source system of scoring news sources. We would essentially start with third-party ratings, including MBFC, and create an aggregate rating. We could also open a path for users to vote, so that any rating would reflect our instance’s opinions of a source. We would love to hear your thoughts on this, as well as suggestions for sources that rate news outlets’ bias, reliability, and/or credibility. Feel free to use this thread to share other constructive criticism about the bot too.

geekwithsoul ,

One problem I’ve noticed is that the bot doesn’t differentiate between news articles and opinion pieces. One of the most egregious examples is the NYT. Opinion pieces aren’t held to the same journalistic standards as news articles and shouldn’t be judged for bias and accuracy in the same way as news content.

I believe most major news organizations include the word “Opinion” in titles and URLs, so perhaps that could be something keyed off of to have the bot label these appropriately. I don’t expect you to judge the bias and accuracy of each opinion writer, but simply labeling them as “Opinion pieces are not required to meet accepted journalistic standards and bias is expected.” would go a long way.

jeffw OP ,
@jeffw@lemmy.world avatar

Thanks for this. As a mod of /c/news, I hadn’t really thought about that. We don’t allow opinion pieces, but this is very relevant if we roll out a new bot for all the communities that currently use the MBFC bot.

geekwithsoul ,

No problem. Specifically came to my attention about a week ago on this post where the bot reported on an opinion piece as if it was straight news.

BTW, I actually do appreciate the bot and think it’s doing about as well as it can given the technical limitations of the platform.

AlexWIWA ,

This contributes significantly to the noise issue most people complain about

SomeGuy69 ,
@SomeGuy69@lemmy.world avatar

Interesting that people say that opinion pieces should not be held to the same standard. I personally see such pieces contribute to fake news going around. Shouldn’t a platform with reach, held accountable for wrong information, they hide behind an opinion piece?

catloaf , (edited )

It has been helpful and we would like to keep it around in one form or another.

Bull fucking shit. The majority of feedback has been negative. I can’t recall a single person arguing in its favor, but I can think of many, myself included, arguing against it. I hope you can find my report of one particular egregious example, because Lemmy doesn’t let me see a history of things I reported. I recall that MBFC rated a particular source poorly because they dared to use the word “genocide” to describe what’s going on in Gaza. Trusting one person, who clearly starts from an American point of view, and has a clearly biased view of world events, to be the arbiter of what is liberal or conservative, or factual or fictional, is actively harmful.

No community, neither reddit nor Lemmy nor any other, has suffered for lack of such a bot. I strongly recommend removing it. Non-credible sources, misinformation, and propaganda are already prohibited under rule 8. If a particular source is so objectionable, it should be blacklisted entirely. And what is and is not acceptable should be determined in concert with the community, not unilaterally.

Edit: And another thing! It’s obnoxious for bot comments to count toward the number of comments as shown in the post list. Nobody likes seeing it and thinking “I wonder what people are saying about this” and it’s just the damn bot again. But that’s really a shortcoming in Lemmy.

jeffw OP ,
@jeffw@lemmy.world avatar

I will start by saying that I feel like we are trying to address the criticism in your first paragraph with these changes. That being said, thanks for your feedback. I particularly like the comment you shared under the “edit,” because I hadn’t seen that sentiment shared before (not saying nobody else had that issue, just appreciating you for contributing that and challenging me to think more about how we execute things).

KnightontheSun ,

I also would like it not add to the comment count. I am now getting inured to comment counts of “1”.

I generally like the bot and its intentions, but feel it inaccurate with my perception too often.

Aatube , (edited )
  1. Please, move the bias and reliability outside of the first accordion/spoiler. This is the sole purpose the bot was meant to provide. If we can't see that at a glance, it's bad. I don't see how these few words are "too long" either. I feel like a lot of the space could be cleared by turning the "Search Ground News" accordion into another link in the footer.
  2. While I personally don't see the point of the controversy, it wouldn't be too hard to manually enter Wikipedia's Perennial Sources list into the database that the bot references, especially with MediaWiki's watchlist RSS feed. This would almost certainly satisfy the community.
  3. Open source the database and the bot. Combined with #2, this could also offer an API to query Wikipedia's RSP for everyone to use in the spirit of fedi and decentralization.
mozz ,
@mozz@mbin.grits.dev avatar
  1. Open source the database and the bot.

Yes. A certain amount of my complaint about MBFC bot is not that it's a bad idea per se, it's just that the database and categorizations are laughably bad. It puts Al Jazeera in the same factual classification as TASS. It lists MSNBC as factually questionable and then when you look at the actual list, a lot of them are MSNBC getting it right and MBFC getting it wrong. It might as well be retitled "The New York Times's Awful Neoliberal Idea of Reality Check Bot". (And not talking about the biases ranking -- if that one is skewed it is fine, but they claim things are not factual if they don't match the appropriate bias, and the bias is unapologetic center-right.)

You can't set yourself up to sit in judgement of sources that write dozens of articles every single day about unfolding world events where the "objectively right" perspective isn't always even obvious in hindsight, and then totally half-ass the job of getting your basic facts straight about the sources you're ranking, and expect people to take you seriously. I feel like mostly the Lemmy hivemind is leaps and bounds ahead of MBFC bot at determining which sources are worth listening to.

  1. it wouldn't be too hard to manually enter Wikipedia's Perennial Sources list into the database that the bot references

FUCK FUCK FUCK YES

This is an actual up-to-date and very extensive list that people who care bother to keep up to date in detail (even making distinctions like "hey this source is ok for most topics but they are biased when talking about X, Y, Z"). This would immediately do away with like 50% of my complaint about MBFC bot.

Aatube , (edited )

For example, if we retain MBFC, the layout could look something like this:

Rolling Stone Bias: Left, Credibility: High, Factual Reporting: High - United States of America

MBFC report | bot support | Search topics on Ground.News

in which "Rolling Stone" is linked to the Wikipedia article.

With RSP, it could look something like this:

Rolling Stone is generally reliable on cultureThere is consensus that Rolling Stone has generally reliable coverage on culture matters (i.e., films, music, entertainment, etc.). Rolling Stone's opinion pieces and reviews, as well as any contentious statements regarding living persons, should only be used with attribution. The publication's capsule reviews deserve less weight than their full-length reviews, as they are subject to a lower standard of fact-checking. See also Rolling Stone (politics and society), 2011–present, Rolling Stone (Culture Council).

Rolling stone is generally unreliable on politics and society, 2011–presentAccording to a 2021 RfC discussion, there is unanimous consensus among editors that Rolling Stone is generally unreliable for politically and societally sensitive issues reported since 2011 (inclusive), though it must be borne in mind that this date is an estimate and not a definitive cutoff, as the deterioration of journalistic practices happened gradually. Some editors have said that low-quality reporting also appeared in some preceding years, but a specific date after which the articles are considered generally unreliable has not been proposed. Previous consensus was that Rolling Stone was generally reliable for political and societal topics before 2011. Most editors say that Rolling Stone is a partisan source in the field of politics, and that their statements in this field should be attributed. Moreover, medical or scientific claims should not be sourced to the publication.

RSP listing | bot support | Search topics on Ground.News

Both examples with everything necessary linked, of course

whostosay ,

Seconded. #1 is why I currently have it blocked.

Zonetrooper ,
@Zonetrooper@lemmy.world avatar

I’m frankly rather concerned about the idea of crowdsourcing or voting on “reliability”, because - let’s be honest here - Lemmy’s population can have highly skewed perspectives on what constitutes “accurate”, “unbiased”, or “reliable” reporting of events. I’m concerned that opening this to influence by users’ preconceived notions would result in a reinforced echo chamber, where only sources which already agree with their perspectives are listed as “accurate”. It’d effectively turning this into a bias bot rather than a bias fact checking bot.

Aggregating from a number of rigorous, widely-accepted, and outside sources would seem to be a more suitable solution, although I can’t comment on how much programming it would take to produce an aggregate result. Perhaps just briefly listing results from a number of fact checkers?

jeffw OP ,
@jeffw@lemmy.world avatar

That’s fair. One idea could be a separate “community rating” and one that is more professional. Think Metacritic, RottenTomatoes, etc

Carrolade ,

I second this. This community is better than most social media, but it’s still that, and social media popularity is pretty bottom of the barrel as a means of determining accuracy. Additionally, that’d just open it up to abuse from people trying to weight the votes with fake accounts, scripts, whatever.

Aurenkin , (edited )

My personal view is to remove the bot. I don’t think we should be promoting one organisations particular views as an authority. My suggestion would be to replace it with a pinned post linking to useful resources for critical thinking and analysing news. Teaching to fish vs giving a fish kind of thing.

If we are determined to have a bot like this as a community then I would strongly suggest at the very least removing the bias rating. The factuality is based on an objective measure of failed fact checks which you can click through to see. Although this still has problems, sometimes corrections or retractions by the publisher are taken note of and sometimes not, leaving the reader with potentially a false impression of the reliability of the source.

For the bias rating, however, it is completely subjective and sometimes the claimed reasons for the rating actually contradict themselves or other 3rd party analysis. I made a thread on this in the support community but TLDR, see if you can tell the specific reason for the BBC’s bias rating of left-centre. I personally can’t. Is it because they posted a negative sounding headline about Trump once or is it biased story selection? What does biased story selection mean and how is it measured? This is troubling because in my view it casts doubt on the reliability of the whole system.

I can’t see how this can help advance the goal (and it is a good goal) of being aware of source bias when in effect, we are simply adding another bias to contend with. I suspect it’s actually an intractable problem which is why I suggest linking to educational resources instead. In my home country critical analysis of news is a required course but it’s probably not the case everywhere and honestly I could probably use a refresher myself if some good sources exist for that.

Thanks for those involved in the bot though for their work and for being open to feedback. I think the goal is a good one, I just don’t think this solution really helps but I’m sure others have different views.

avidamoeba ,
@avidamoeba@lemmy.ca avatar

Removing the bias rating might be enough indeed.

Deceptichum ,
@Deceptichum@quokk.au avatar

Nah even credibility is subjective to MBFC.

solrize ,

Just kill it with fire.

cybervseas ,

I wish the comment count on a post didn’t include MBFC (or maybe bots in general).

jeffw OP ,
@jeffw@lemmy.world avatar

I’ll be honest, that’s probably outside of the scope of what we can do for now. It’s definitely valuable feedback in general and I wish I could offer some kind of solution but that’s probably even outside the control of the instance admins.

Someone can feel free to correct me if I’m wrong!

xmunk ,

On this topic the information would probably be ideally delivered by flairs/post tags which lemmy doesn’t support yet (AFAICT).

Simply having (bias:left) (factuality: high) would be much better than a whole comment.

cybervseas ,

I’m not sure what to do here. On my mobile device the compacted media bias fact check post still takes up 50% of my phone screen.

How a post tag if we have a tagging system in Lemmy, instead of a whole long comment?

Maybe the bot could just post a one line summary with a link to more information?

jeffw OP ,
@jeffw@lemmy.world avatar

Thanks for the feedback. Can you elaborate a bit about the 50% of your screen thing? Is it the text itself, or is the issue that the app provides links at the bottom of the comment? I’m thinking of my experience on Voyager, where the links are summarized at the bottom of each comment, which does lead to a decent amount of screen being taken up. Would it be better if there weren’t any links?

cybervseas ,

yep I’m using Voyager on my iPhone. Maybe a super short summary without links. People could open the bot’s profile and look at the bot’s posts (not comments) if they want to dig deeper to understand a source.

jeffw OP ,
@jeffw@lemmy.world avatar

Interesting, so you think the bot should make posts too? Like, a post for each source with a summary of relevant info? Just making sure I understand what you mean

cybervseas ,

Yeah. It’s an idea for a way to create a user repository within Lemmy that could be edited by the bot as needed. I’m sure there are better ways.

praise_idleness ,

This bot was introduced because modding can be pretty tough work at times and we are all just volunteers with regular lives.

Then maybe it can be an internal thing only. Let people do their own critical thinking. I believe that if you’re on Lemmy, you can make informed decision.

workerONE ,

I think it should be removed

Spuddlesv2 ,

Yes MBFC is Extremely American in its definitions of left and right. A less US-centric rating would be much preferred.

I generally like the idea otherwise.

Carrolade ,

Personally I’m in favor of the bot. One complaint I’ve seen that I agree with is that it doesn’t need to float high up in the comments. If it was simply made to not upvote itself, it would stay nearer to the bottom naturally, which I think would be preferable.

MediaBiasFactChecker Bot ,

The news source of this post could not be identified. Please check the source yourself. Media Bias Fact Check | bot support

jeffw OP ,
@jeffw@lemmy.world avatar

Okay, so maybe we don’t need a comment if it’s a meta post or a mod announcement. Thanks for your inadvertent feedback, bot!

mozz ,
@mozz@mbin.grits.dev avatar

it also does this with a bunch of weird little local newspapers or etc which I've never heard of, which is like the one time I actually want it to be providing me with some kind of frame of reference for the source. MSNBC and the NYT, I feel like I already know what I think about them.

jeffw OP ,
@jeffw@lemmy.world avatar

Yeah, it’s tricky because who reviews those small guys? Granted, most of them are probably owned by a giant like Gannett, but that doesn’t mean we can just apply a rating from 1 small Gannett-owned paper to another. We’d like there to be some way for users to share their feedback/ratings on those small guys. But then it’s also true that some people will create a news site and try to share links on here to promote their new website and that’s typically just spam bots.

Naich ,
@Naich@lemmings.world avatar

You don’t need every post to have a comment basically saying “this source is ok”. Just post that the source is unreliable on posts with unreliable sources. The definition of what is left or right is so subjective these days, that it’s pretty useless. Just don’t bother.

fine_sandy_bottom ,

My personal view is that the bot provides a net negative, and should be removed.

Firstly, I would argue that there are few, if any, users whom the bot has helped avoid misinformation or a skewed perspective. If you know what bias is and how it influences an article then you don’t need the bot to tell you. If you don’t know or care what bias is then it won’t help you.

Secondly, the existence of the bot implies that sources can be reduced to true or false or left or right. Lemmy users tend to deal in absolutes of right or wrong. The world exists in the nuance, in the conflict between differing perspectives. The only way to mitigate misinformation is for people to develop their own skeptical curiosity, and I think the bot is more of a hindrance than a help in this regard.

Thirdly, if it’s only misleading 1% of the time then it’s doing harm. IDK how sources can be rated when they often vary between articles. It’s so reductive that it’s misleading.

As regards an open database of bias, it doesn’t solve any of the issues listed above.

In summary, we should be trying to promote a healthy sceptical curiosity among users, not trying to tell them how to think.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • [email protected]
  • random
  • lifeLocal
  • goranko
  • All magazines
    •