this post was submitted on 26 Jan 2023
12 points (100.0% liked)
Fediverse
17741 readers
56 users here now
A community dedicated to fediverse news and discussion.
Fediverse is a portmanteau of "federation" and "universe".
Getting started on Fediverse;
- What is the fediverse?
- Fediverse Platforms
- How to run your own community
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
@anders There would need to be a way for that search engine to collect data that is both possible to contribute to as an individual, and doesn't unintentionally DDoS sites it indexes, and that's the challenge I think. Spiders collect a LOT of data. Right now the closest thing we have to decentralized search, is metasearch engines like Searx, which query and cache results from all the major search providers that run their own spiders.
The bigger issue is consent. People on fediverse feel very strongly about consent, and search engines tend to just ignore it. Better do some serious research into consent to search on fedi before embarking on designing a search engine for fedi.
@rysiek
@f00fc7c8 @fediverse
I didn't think about that issue.
@rysiek I don't think Anders is asking about a search engine for the fediverse, this sounds more like a federated or P2P Google/DuckDuckGo replacement.
Ah I might have misunderstood, sorry.
@f00fc7c8
@rysiek
I meant a search engine for Fediverse :)
@rysiek (though if you'd like to argue that search/spidering requires opt-in consent in all cases, I'm happy to hear that argument)
I don't have to defend my right to decide how stuff I put out there can be used. Whoever wants to scrape my toots has to explain why they want to do so, and get my consent first.
And "well it's publicly available so it's fair game" is not enough of an argument. Just as "she was wearing a short skirt" is not consent to sexual advances.
It physically hurts to know that consent it such a controversial topic in tech circles, and it breaks my heart to hear people argue we give consent to invasive data practices just by existing on the internet. I've spent my entire life being taught by technology educators that I should expect everything I post online to be publicly accessible forever, and nobody every stopped to ask why.
I am one of those technology educators, and today I would still warn people that "Internet does not forget", and that they need to be careful what they put out there.
That doesn't mean we should not demand explanation from people who make it so, and that we should not demand them to ask for consent and respect our refusal to give it. I really appreciate how fedi culturally puts this front-and-center. I hope it continues to do so, and that this way of thinking spreads farther!
I agree that consent should not be a controversial topic. Regardless of how much it inconveniences techbros trying to "disrupt" yet another area of human endeavor.
@rysiek I was not talking precisely about scraping toots, I was asking whether you consider Google, Bing, etc uses of opt-out web spiders to be unethical, but fair enough. (Also, not interested in defending OP given the clarification that he is talking about searching the fediverse.)
I think search engines indexing plain old websites (blogs etc) are an importantly different case.
The nature of the medium in blogs/news websites/etc is way more public and way less intimate (in general…) than social media. Social media blur the line between private and public conversations, for better or worse.
Social media is like having a conversation in a public cafe; websites/blogs is more like publishing a newspaper or standing on the corner of a street shouting your message at strangers.
Making a public archive of newspapers or recording a person shouting at strangers is one thing. Recording semi-private conversations in a cafe is a whole different thing. Does that make sense?
@rysiek @f00fc7c8 The benefit model is different between the two too.
A blog's author benefits (in some small way) from being indexed, because it helps drive traffic to the content they're publishing for others to read.
The same can not really be said of toots and it's relatively rare that it leads to positive contributions to an ongoing conversation.
@rysiek yeah, that's the sort of distinction I was looking for. thanks!