There have been multiple accounts created with the sole purpose of posting advertisement posts or replies containing unsolicited advertising.

Accounts which solely post advertisements, or persistently post them may be terminated.

BurnSquirrel ,

Still couldn’t get me to use it, I use DDG which can switch between search engines and search sites very quickly with it’s ! syntax (Everyone goes on about privacy, but this is pretty much it’s best feature). Google results are consistently the worst for me if I’m hitting multiple search engines

Death_Equity ,

I’m not saying we should have gulags, but if we did, I have some suggestions for permanent residency.

TypicalHog ,

thats fucked

dabster291 ,
@dabster291@lemmy.zip avatar

paywalled link :(

hotpot8toe OP ,

I got you archive.ph/GS2I0

dabster291 ,
@dabster291@lemmy.zip avatar

yar har

reddig33 ,

I’m not understanding what stops a search engine from scraping a publicly accessible website. ?

Eril ,

robots.txt, I guess? Yes, you can just ignore it, but you shouldn’t, if you develop a responsible web scraper.

reddig33 ,

Doesn’t seem legal that a robots.txt could pick and choose who scrapes. Seems like legally it would have to be all or nothing. Here’s hoping one of the search engines ignores it and makes it a legal case.

Eril ,

Actually currently it contains this:


<span style="color:#323232;">User-agent: *
</span><span style="color:#323232;">Disallow: /
</span>

Well, that actually is a blanket ban for everyone, so something else must be at play here.

starman ,
@starman@programming.dev avatar

merj.com/…/investigating-reddits-robots-txt-cloak…

Reddit is serving different file to google

russjr08 ,

We believe in the open internet, but we do not believe in the misuse of public content.

That’s real rich, coming from Reddit.

capital ,

You’d probably feel differently if it were your service. Should you be able to control who scrapes your sites or should that be all or nothing?

For the record, I fucking hate what the internet is becoming. I naively believed that even if shit got cordoned off into the walled gardens that are mobile phone apps, the web would remain as open as it was. This is a terrible sign of things to come.

reddig33 ,

No, I wouldn’t feel differently. In fact letting search engines scrape and point to your content is what leads people to your site. It’s free advertising. If you’re going to let one search engine in, you should let them all in. If you want to be public, be public. Otherwise put up a login firewall and go private.

capital ,

It’s not just search engines. Lots of people on Mastodon were using robots.txt to block ChatGPT (and any other LLM company they knew of) from scraping their sites/blogs.

I disagree, to a point. I want to be able to control my services to the greatest extent possible, including picking who scrapes me.

On the other hand, orgs as large as Google doing this poses a real threat to how the internet works right now which I hate.

hotpot8toe OP ,

Also, rate limiting. A publicly accessible website doesn’t mean that it will allow scrapers to read millions of pages each week. They can easily identify and block scrapers because of the pattern of their activity. I don’t know if Reddit has rate-limiting, but I wouldn’t be surprised if they implement one.

sag ,

Fuck You Reddit

Imgonnatrythis ,

Meh, fuck em. The tighter they make their circle the less useful it is.

Reminder that Kagi searches Lemmy which is great.

UnderpantsWeevil ,
@UnderpantsWeevil@lemmy.world avatar

Kagi

Ah, yes. The “Fuck you, Pay me” search engine.

capital ,

A service that gasp costs money???

It surfaces better results for me than others so I’m paying for a service that I gain benefit from. Feel free not to…

Wolfram ,

It’s a paid service that in return respects privacy, and has results as good as Google’s.

Google doesn’t say “Fuck you, Pay me” because you are the product.

JackbyDev ,

If you use Bing, DuckDuckGo, Mojeek, Qwant or any other alternative search engine that doesn’t rely on Google’s indexing and search Reddit by using “site:reddit.com,” you will not see any results from the last week.

That’s absolutely insane… Reddit truly is making things awful. The “just add reddit” or “just add site:reddit.com” has been trash for a while because they bombard you with the “pwease use the app” and not showing more than like three comments at a time. It’s useless.

UnderpantsWeevil ,
@UnderpantsWeevil@lemmy.world avatar

Reddit truly is making things awful.

They’re no longer interested in driving traffic to the site, is my guess. They’re far more interested in devising new ways to extract rents from the existing participant base. So rather than pay Google to prioritize their site, or incentivize Google to link to their site with internal content hygenie techniques, now they’re getting paid by Google to exclusively serve up content.

It’s useless.

The sheer volume of junk content, the amount of content that just shows up as deleted or archived, and the rate at which I’m served “Reddit” as a source of data when there’s no conceivable reason why it should be near the top of my search list is very frustrating.

5oap10116 ,

https://lemmy.world/pictrs/image/941a7b15-cf66-407a-b8a6-d0c0f6b98f94.png

dullbananas ,
@dullbananas@lemmy.ca avatar

Poople

daniskarma ,

To be fair, Reddit is no longer that good of a source for answers in the later years.

Quality drop in comments is insane. Sometimes it looks like Quora.

Kecessa ,

I was looking for Bluetooth speakers recommendations and it’s the first time I really noticed “generic bot replies” like “I’ve got this great product to recommend, not only is it good but it offers great sound quality as well! The product is [link to Amazon page]”

Gotta start searching using “before:” to get quality results…

5oap10116 ,

Am I bbbrrrregnant?

BeigeAgenda ,
@BeigeAgenda@lemmy.ca avatar

If we consider all possible outcomes on a galaxy scale, then No.

villainy ,

Also my collection of hobbies seems to match up well with the people who nuked their post history after the API-ocalypse. Even when I get good search results I click through and… so many deleted comments…

werefreeatlast ,

Lemmy Will be king soon…is there a Lemmy search engine?

Fedizen ,

Reddit really fucked themselves. Not as much as Elon fucked twitter but super close.

Also pretty sure DDG uses Bing

vxx ,

Another nail.

AFC1886VCC ,

Fuck Reddit

cmrn ,

Every time I click a Reddit link now it’s just “download the app to verify your age” regardless of what it is

Wolf314159 ,

I feel your pain.

I edit the URL to remove the first part of the URL and replace it with “old.reddit.com”. That still seems to work, last I checked, but I fully expect it to be killed any day now.

PraiseTheSoup ,

There’s a firefox extension “old reddit redirect” that’ll do this for you. Been using it for years. But yeah any day now I expect old reddit to be offline.

Kowowow ,

They should include reddit in the list of search engines that don’t work well with reddit

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • [email protected]
  • random
  • lifeLocal
  • goranko
  • All magazines
    •