It's illegal when a regular person steals something, but it's innovation and courage, when a huge corporation steals something. Interesting how that works
this post was submitted on 07 Oct 2024
566 points (98.8% liked)
Technology
59669 readers
3970 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
Honestly it’s fucking angering. So much regulation and geo-restrictions and licensing schemes… but it’s cool that there are data brokers, and shit like this. On top of it all Chrome screwing us with manifest v3 and killing ad blocking on chrome. It’s already in canary build.
WHAT THE FUCK IS WRONG WITH THIS SPECIES?!
WHAT THE FUCK IS WRONG WITH THIS SPECIES?!
Yes.
WHAT THE FUCK IS WRONG WITH THIS SPECIES?!
Capitalism.
load more comments
(9 replies)
They're not stealing your data, they're pirating it.
They’re not pirating it. They’re collecting it.
They’re not collecting it. They’re archiving it.
Aaron Schwartz killed himself over punishments for less
load more comments
(14 replies)
We've had this thing hammering our servers. The scraper uses randomized user-agents browser/OS combinations and comes from a number of distinct IP ranges in different datacenters around the world, but all the IPs track back to Bytedance.
Wouldn't be surprised if they're just cashing out while TikTok is still public in the US. One last desperate grab at value-add for the parent company before the shut down.
Also a great way to burn the infrastructure for subsequent use. After this, you can guarantee every data security company is going to add the TikTok servers to their firewalls and blacklists. So the American company that tries to harvest the property is going to be tripping over these legacy bullwarks for years after.
This has nothing to do with Tik Tok other than ByteDance being a shareholder in Tik Tok
Also it doesn't respect robots.txt (the file that tells bots whether or not a given page can be accessed) unlike most AI scrapping bots.
My personal website that primarily functions as a front end to my home server has been getting BEAT by these stupid web scrapers. Every couple of days the server is unusable because some web scraper demanded every single possible page and crashed the damn thing
I do the same thing, and I've noticed my modem has been absolutely bricked probably 3-4 times this month. I wonder if this is why.
load more comments
(1 replies)
load more comments
(6 replies)
most
I doubt that...
People would be able to tell from the traffic on their websites.
load more comments
(1 replies)
Not surprising that Bytedance would want to gobble up every bit of data they can as fast as possible.
load more comments
(8 replies)
They're too late, there's going to be way too much AI generated garbage in their data and so many social media platforms like Reddit and Twitter have already taken measures to curb scrapers.
Like those platforms aren't already full of AI garbage as well. Training new models will require a cut-off date before the genie was let out of the bottle.
load more comments
(4 replies)
As for what ByteDance plans to do with a new LLM, a person familiar with the company’s ambitions said one goal has to do with the search function for TikTok.
Last week, TikTok released an update to its current search function focused on [keywords for ads], basically allowing advertisers to search in real time for words that are trending on TikTok. It allows marketers to build an ad with relevant keywords that would ostensibly help the ad show up on the screens of more users.
…
“Given the audience and the amount of use, TikTok with a search environment that is a completely biddable space with keywords and topics, that would be very interesting to a lot of people spending a ton of money with Google right now,” the person said.
A dark vision just flashed in my mind. And I am certain this is what will happen. AI-generated ads done in real time based on the latest “trending” thing. Presented to users basically as soon as the topic has the slightest amount of “trend”.
Just emitting untold amounts of CO2 to show you generated ads in near real time.
No wonder Google ex-CEO was saying fuck climate goals.
load more comments
(1 replies)
I can not contribute to anything here, I just came to say I really really like the phrase "gobbling something up" :D
load more comments
(1 replies)
from the article:
Robots.txt is a line of code that publishers can put into a website that, while not legally binding in any way, is supposed to signal to scraper bots that they cannot take that website’s data.
i do understand that robots.txt is a very minor part of the article, but i think that’s a pretty rough explanation of robots.txt
Out of curiosity, how would you word it?
i would probably word it as something like:
Robots.txt is a document that specifies which parts of a website bots are and are not allowed to visit. While it’s not a legally binding document, it has long been common practice for bots to obey the rules listed in robots.txt.
in that description, i’m trying to keep the accessible tone that they were going for in the article (so i wrote “document” instead of file format/IETF standard), while still trying to focus on the following points:
- robots.txt is fundamentally a list of rules, not a single line of code
- robots.txt can allow bots to access certain parts of a website, it doesn’t have to ban bots entirely
- it’s not legally binding, but it is still customary for bots to follow it
i did also neglect to mention that robots.txt allows you to specify different rules for different bots, but that didn’t seem particularly relevant here.
load more comments
(3 replies)
load more comments
(1 replies)
Another fucking CCP and PLA creation.
view more: next ›