Fediverse

just because the (corporate) internet works this way now, doesn’t mean it should

The web worked this way before there was a large corporate presence. Scraping was common during the blogosphere period and robots.txt was the solution everyone at the time agreed on and that's been the standard ever since.

I’m happy that you’re comfortable with this model, but I don’t want people who operate like this to intrude on the spaces we’re building to get away from it

We're not intruding on this space. We've been in the fediverse for just as long or longer; the fediverse has been scrapable since 2008.

I think Besse makes a great point here:

I think blurring the lines between public and private spaces is the opposite of informing consent. Cultivating unrealistic expectations of "privacy" and control in what are ultimately public spaces is actually bad.

I tried to single out the world wide web, as opposed to the internet at large, because the two are not synonymous. It's rather absurd to publicly serve webpages to any querying IP address and maintain that the receiving computer is not to save said pages to disk.

All this to say: I find it difficult to argue that web publications should or could be exempt from aggregation and archival (or scraping, to put it another way). I understand that the ease with which bots do this can be disconcerting, however.

If we stay with the cafe bulletin board, getting a detailed overview of all the postings on the board is akin to scraping the whole thing. If we extend our analogy instead to a somewhat more significant example, library catalogs do the same with books, magazines, and movies.

This is the cost of publishing, be that in print or online. It must be expected that some person has a copy of every- and anything one has ever written or posted publicly, and perhaps even catalogued it. A way around this might be to move away from the web to another part of the internet, like Matrix, as alma suggested.

I assume the non-consensual collection of various (meta-)data is what you refer to when talking about intrusion and money making. Lemmy, like many projects, seeks to offer an alternative to corporate, data-gobbling social media sites, but doesn't eliminate the ability to search through its webpages.

I would fully agree that other internet protocols are much better suited to information not meant to be broadcast publicly.

Civility is great, and should be highly encouraged. That's largely why I like Lemmy. Each instance can guide its community in line with its values, whatever those may be, block offenders, and generally forge the space it wishes.

However, I think Besse's comments on setting the correct expectations in the public sphere are worth considering.

For a different internet example: all the messages I send in any chatroom on an IRC server will inevitably be logged by someone, especially in popular rooms. Any assumption to the contrary would be naïve, and demanding that people not keep a log any of my publicly broadcast messages would be laughed at by the operators. It's a public space, and sending anything to that space necessarily means I forgo my ability to control who sees, aggregates, archives, or shares that information. My choice to put the information into that space is the opt-in mechanism, just how books or interviews do the same offline in print.

It's not so much the protocol as it is how making things public fundamentally works.