this post was submitted on 25 Jul 2024

373 points (98.7% liked)

Technology

34778 readers

94 users here now

This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.

Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.

Rules:

1: All Lemmy rules apply

2: Do not post low effort posts

3: NEVER post naziped*gore stuff

4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.

5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)

6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist

7: crypto related posts, unless essential, are disallowed

founded 5 years ago

MODERATORS

MinutePhrase@lemmy.ml

373

Reddit is now blocking major search engines and AI bots — except the ones that pay (www.theverge.com)

submitted 3 months ago by moe90@feddit.nl to c/technology@lemmy.ml

71 comments fedilink hide all child comments

top 50 comments

sorted by: hot top controversial new old

[–] mozz@mbin.grits.dev 138 points 3 months ago (4 children)

The cycle continues:

Hey you guys can have everything for free
WTF this is expensive to provide, I think I’m gonna start taking advantage of you guys which someone will pay me to do
WTF where’s everyone going
WTF I’m still losing money and always have been
Screw you guys, screw everybody, I didn’t want y’all anyway
(fades into irrelevance, gets bought by someone and stripped for parts)

Idk it’s not as pithy as Cory Doctorow’s version I guess

Anyway we’re at step 5 at this point

[–] friend_of_satan@lemmy.world 89 points 3 months ago (2 children)

Yeah, Reddit is Digging its own grave.

[–] zcd@lemmy.ca 45 points 3 months ago* (last edited 3 months ago)

[–] sylver_dragon@lemmy.world 20 points 3 months ago

It's getting Fark'd

[–] acosmichippo@lemmy.world 26 points 3 months ago* (last edited 3 months ago) (1 children)

honestly I'm not convinced step 6 is inevitable. I think enough people are okay with whatever reddit does.

[–] Reddfugee42@lemmy.world 12 points 3 months ago (3 children)

Enough consumers are okay with it, but the core geeks and nerds that created, curated, and moderated the content have jumped ship.

The cruise line is still sailing and there are still drinks and snacks so nobody has noticed the staff have jumped ship. There's management, low level volunteers, and thousands of kids, moms, and dads.

But sooner or later people are gonna get tired of snacks and flip their shit when management tells them the people who know how to make the steaks have just all fuckin ✨ inexplicably disappeared✨

[–] cRazi_man@lemm.ee 16 points 3 months ago* (last edited 3 months ago) (1 children)

That's a fantasy we all hold here because we don't like Reddit. Reddit doesn't care that nerds have gone and normies are left behind. People keep using the site and throwing money at "super upvotes". They've floated on the stock market and are doing well. The site is nowhere near dying like Digg. Deep, cerebral, meaningful content might have suffered; but hardly anyone cares as long as they get to see recycled memes, making judgemental comments on other people's relationships, porn and politics. Their main content is lowest common denominator shit, and it always has been. Facebook is far more shitty and is still going strong. I'm sure Reddit will be fine without us and with their ongoing enshitification, no matter how much we fantasise about their demise.

https://imgur.com/a/aLhmJSE.jpg

[–] pop@lemmy.ml 11 points 3 months ago (1 children)

People don't realize how much of reddit content is made by bot farms and advertising agencies, propaganda outlets with bots to spare. Which is what's keeping the normies entertained, not the nerds, not the niche community of a few thousand people.

People like the one you're replying to always are so sure their echo chamber is right when reality is like complete opposite. Most people on reddit are lurkers and not terminally online people. They just want to scroll and fucking waste time. Community, subs and their mods or rules be damned.

They don't care if the cat videos are on a banana sub. they'll happily upvote and scroll away while the terminally online will start complaining why the post is not fit, a repost, or against the rules for 100s of time. And as always, once they leave, they think it is dead.

load more comments (1 replies)

[–] Ferk@lemmy.ml 3 points 3 months ago* (last edited 3 months ago)

Content curated by "the core geeks and nerds" might appeal to "geeks and nerds", not to those consumers.

They want "consumer" content. And if one day they get tired of it then I doubt any amount of "steak" would have stopped them leaving anyway, since that was never what they were looking for. It's not like reddit has to be the only place they visit in the internet, nor is the internet their only source of consumption. Just because you go to a snack bar does not mean that's the only place you go for meals.

load more comments (1 replies)

[–] xavier666@lemm.ee 17 points 3 months ago

The key capitalistic trick is to time your step 2 just when you have a critical mass on your platform. Upper management has understood that our shitty paywall will remove x% of our users from our platform. But if (100-x)% of our users can pay $y annually, we can sustain our business model and make $z of profit each year. PR will take care of all the backlash but it's all calculated.

load more comments (1 replies)

[–] Gullible@sh.itjust.works 107 points 3 months ago (2 children)

Tangentially related- I fucking hate discord

[–] PythagreousTitties@lemm.ee 71 points 3 months ago (5 children)

Discord is fine for chatting, voice, and iterating quickly on projects. I have no idea why people want to think it's a forum. That's ridiculous.

[–] delirious_owl@discuss.online 21 points 3 months ago* (last edited 3 months ago) (5 children)

Its pretty awful for all those things if you care about privacy or can't signup for an account

[–] umbrella@lemmy.ml 8 points 3 months ago

also not searchable at all. its an information blackhole.

load more comments (4 replies)

[–] Lennnny@lemmy.world 10 points 3 months ago (1 children)

We use it for our friend group, as we have pub nights, group meals, vacations etc. we also all do each other's cat care when we're out of town, so we have a channel devoted to pet photos etc. works well enough for us.

[–] PythagreousTitties@lemm.ee 3 points 3 months ago

Exactly. That's a great use for it.

load more comments (3 replies)

[–] MalReynolds 18 points 3 months ago

I fucking hate discord

It's Cancer, have an upvote.

[–] ohwhatfollyisman@lemmy.world 58 points 3 months ago

thanks to them for making my deredditification that much easier!

[–] scrubbles@poptalk.scrubbles.tech 51 points 3 months ago (1 children)

The only way they get my clicks now are when I Google something and they come up.

They really keep making sure that I don't end up there.

[–] pupbiru@aussie.zone 18 points 3 months ago

libredirect helps with that on desktop

(browser extension that turns links to sites like reddit, youtube, etc into links to redlib, invidious)

[–] blackris@discuss.tchncs.de 26 points 3 months ago (2 children)

Too bad. Hey, crazy idea: let's create an open alternative for reddit with good content! Maybe something in the fediverse or so.

[–] _g_be@lemmy.world 11 points 3 months ago

I think you're onto something

[–] Swedneck@discuss.tchncs.de 5 points 3 months ago

that would never work

[–] GnuLinuxDude@lemmy.ml 24 points 3 months ago (1 children)

There are numerous occasions where someone has a lingering question on Reddit that I see and know the answer to. It’s too bad it’s on Reddit because I no longer contribute to that website, and refuse to.

[–] Ragnarok314159@sopuli.xyz 6 points 3 months ago

All the decent answers I find are from 5+ years ago. I check the user’s activity and they normally quit the place. Warms the heart.

[–] moe90@feddit.nl 19 points 3 months ago (3 children)

just begin with site:reddit.com test for ddg and it still works

[–] KevonLooney@lemm.ee 24 points 3 months ago (5 children)

Are they new posts or old ones? They are blocking new ones, not old ones.

[–] pupbiru@aussie.zone 18 points 3 months ago

new posts do not work

this post in /r/selfhosted is from 8hr ago: SWEKIT v0.1 - an open source library to build software engineering agents (DEVIN) in a agentic framework agnostic manner!

reddit/redlib: https://redlib.kylrth.com/r/selfhosted/comments/1eb86lf/swekit_v01_an_open_source_library_to_build/

doesn’t appear in DDG results: https://duckduckgo.com/?q=site%3Areddit.com+SWEKIT+v0.1+-+an+open+source+library+to+build+software+engineering+agents+%28DEVIN%29+in+a+agentic+framework+agnostic+manner%21&t=ffit

load more comments (4 replies)

[–] acosmichippo@lemmy.world 7 points 3 months ago

Based on my testing if you filter results by the last week or last day you get nothing. Past month works.

[–] Mereo@lemmy.ca 3 points 3 months ago (1 children)

For old posts. I can't find new posts on DDG. I find them on Google but not on DDG.

load more comments (1 replies)

[–] fsxylo@sh.itjust.works 17 points 3 months ago

LMAO searching "____ reddit" is the only time I visit their site.

They just really have no clue.

[–] brucethemoose@lemmy.world 13 points 3 months ago* (last edited 3 months ago) (3 children)

Would lemmy instances do this?

I know they can't afford to now, but hypothetically? A lot of people here don't seem to like data scraping for AI.

[–] mozz@mbin.grits.dev 38 points 3 months ago* (last edited 3 months ago) (2 children)

Your Lemmy posts are already being scraped for AI

The level of effort it would take to prevent would be infeasible to ask of even a non volunteer admin let alone a volunteer let alone literally all of them

[–] pennomi@lemmy.world 6 points 3 months ago (2 children)

Your Lemmy posts are already being scraped for AI

Good, hopefully it’ll make AI that is slightly less toxic than the rest of the internet.

It always baffles me that people don’t want their content represented in an AI - every word you write that gets indexed is a vote for how future AI will behave.

load more comments (2 replies)

[–] brucethemoose@lemmy.world 4 points 3 months ago (2 children)

That's what I figured, but I am envisioning a future where lemmy is huge and the network of admins is quite sizable.

I guess that doesn't change much?

[–] epyon22@programming.dev 17 points 3 months ago

Run Lemmy instance
Gain userbase
Intercept data users are reading and posting from your instance and others
Feed to AI
Profit?

Lemmy is way less privacy oriented than reddit and that's by design.

[–] theneverfox@pawb.social 4 points 3 months ago

It's structural - you can be open or locked down, and it's hard to decentralize if you're not open

You can make it easier or harder to work with that data, but ultimately it's obsfucation - you could make it hard to parse and obscure details, but ultimately if you want decentralized federation you can't hide too much

[–] Dave@lemmy.nz 19 points 3 months ago* (last edited 3 months ago)

You don't need to scrape. If you want to get all the content on Lemmy, just set up an instance and subscribe to all the top communities, and the instances will just send you all the content.

So there isn't really a way to monetise or block it. I guess you could only federate to a whitelist, but the biggest instances will federate by default with any new instances until they are given a reason to defederate.

[–] cmnybo@discuss.tchncs.de 9 points 3 months ago (1 children)

Some Lemmy instances disallow indexing in robots.txt, however indexers can choose to ignore that and actually blocking them takes a lot more effort.

[–] brucethemoose@lemmy.world 8 points 3 months ago

Some places on a "budget" like Ao3 just rate limit hard.

I don't like that solution at all though.

[–] tabular@lemmy.world 11 points 3 months ago

The users who wrote the content are going to get a share of the money, right Reddit? Riiight? /s

[–] Unchanged3656@infosec.pub 7 points 3 months ago (3 children)

Brave search got an option for that.

[–] moe90@feddit.nl 3 points 3 months ago

begin with site:reddit.com test is much more accurate to get reddit search on brave search tbh

load more comments (2 replies)

[–] Happywop@lemmy.world 7 points 3 months ago (1 children)

So glad I found this alternative. reddit, mods are psychos and the average user not much better

load more comments (1 replies)

load more comments