abff08f4813c

And if you are backing up your content, are you checking the file afterwards to make sure it has all of the content you wanted to save?

I tried PDS and it actually overwrote some entries but then failed to save them, causing me to lose content. But it turns out I also tried from an older version.

Would love to hear about success stories from folks who have tried PDS, confirmed their CSVs have their content, and are now ready to start reposting some OC to the fediverse!

This is a followup to my previous PSA, https://kbin.social/m/RedditMigration/t/47320/PSA-If-you-have-more-than-1000-posts-more-than

That one warned you about the problem, which seems to have caught a lot of people unawares and caused a lot of confusion in general. (Thank you reddit!!! /s)

Now here's what you can do about it.

If you happen to know that you're only slightly over the limit in one category, odds are good that one of the modified versions of Power Delete Suite will work for you. (You want one with at least an additional 5.1s delay - the original PDS fails to delete/overwrite a lot of stuff because it tries too fast and reddit rejects the changes and imposes a cooldown period.)

PDS will delete your comments and submissions (posts), and it checks all four categories - new, hot, top, and controversial. So if you say only had 1001 or 1002, odds are good that the last few missing from new would show up in top (if old and upvoted a lot) or controversial (if old and downvoted a lot), and thus you still get everything wiped.

In theory you could get really lucky here. Say you have exactly 3000 comments - your newest 1000 have never been upvoted or downvoted, your oldest 1000 have all been heavily downvoted, and your middle 1000 have been heavily upvoted. Then PDS would get the newest, the most controversial 1000 - which happen to be the oldest, and the top 1000 - which happen to be the middle. All 3000 comments gone.

But what to do if you have A LOT beyond the limit? Say more than 4000? Also keep in mind, I was just shy of 1500 comments, and I wasn't so lucky - this method didn't get everything for me, despite not even hitting 2000.

A common tip is to search for your reddit self on google. This works really well to find more stuff but won't catch everything. I speak from experience here.

The right answer is to do a GDPR or CCPA request to reddit, and then they will have to give you your data, your complete history. You can then use this to wipe everything clean.

However, folks who started this around or after the blackout aren't getting timely responses. (As late as the end of last April, it came back in a few hours. Now, everyone who has requested has yet to hear back.)

I'm skeptical that we'll get timely responses here.

So the other right answer used to be to use the Pushshift API instead (hat-tip to 1chemistdown - https://kbin.social/u/1chemistdown ). That API archived a copy of reddit - and unlike reddit, it didn't have the 1000 indexing limit. So you could find even older content just by searching yourself using that API.

The bad news is that this no longer works as Pushshift was forced to shutdown by reddit, and they won't be reopening normal API access for regular users.

The good news is that before all this went down, Pushshift published torrents of their raw data. Up to the end of 2022. You can just download this, and then extract your own post and comment history.

The Pushshift torrent is 1.66TB, compressed. But the Pushshift team also released really good scripts so it's easy to extract your own info without having to uncompress the full archive like you traditionally would with a zip file. Instead, their scripts uncompress a small segment at a time in RAM and process that, so you just need enough space to store the compressed files and enough compute power to process it. (Spoiler alert - most modern laptops have enough compute power. My cheap <$1k laptop did.)

Even better - you can selectively download specific files from a torrent. Pushshift's torrent is broken up into a bunch of files, two each per sub (one for comments and one for submissions). So if you know which subs you frequented, you can just pick and download those instead of having to download the full archive.

So grab your reddit data from the Pushshift torrent, use their scripts to print out the complete list of every comment and every post that you have ever created on reddit, and then feed that to something like the reddit-migration script or a modified shreddit script to completely wipe everything. 100% of everything.

No need to depend on the benevolence of reddit to hand you your history back first. (For those in the EU or California, I'm sure reddit will get back to us eventually, but you'll forgive me for being more skeptical when these powerful laws don't apply.)

The specific details on where to grab everything, including links to the torrent and to the various github scripts and even patches that I wrote, is on my other article - https://kbin.social/m/RedditMigration/t/59451/Finally-Managed-to-erase-all-1477-of-my-comments

TL;DR - you can grab your entire reddit history from a torrent published by pushshift up to the end of 2022, and use that data to overwrite/delete your content beyond the 1000 index limit on reddit.

Reddit's CEO Faced Intense Criticism Over Killing a Popular Third-Party App, Apollo. His Response Is What No Leader Should Ever DoThe company's new API access fees are supposed to generate revenue. Instead, they're alienating everyone. Inc.

Okay, finally I am confident that I got them all!

Here's what I did

First, I got a torrent client set up, and went to

https://academictorrents.com/details/c398a571976c78d346c325bd75c47b82edf6124e/tech&filelist=1

It turns out that you can just download individual subs from the torrent instead of the whole thing. I know exactly which ones I first commented on, it was a small list and the subs themselves were pretty small too at the time. So downloading just those subs from the torrent was pretty speedy and easy.

After that I came up with a quick script (based on the example scripts by the pushshift team's github) at https://github.com/Watchful1/PushshiftDumps to parse and find my comments and spit out the comment ids into a text file.

Next, I modifed the super useful script at https://github.com/luphoria/reddit-migration one last time to accept a list of comment ids in a text file (instead of asking reddit for them), and then had it overwrite them all.

Looks like I won't be needing that GDPR archive after all, thanks for nuttin reddit!!

Hat-tip: https://news.ycombinator.com/item?id=36038684

Waiting for one sub to go public so I can finish erasing my content, then I can delete knowing I didn't leave a single thing behind.

Edit: Here are the patches btw.

For reddit-migration: https://kbin.social/m/DataDumps/t/65055/reddit-migration-patch

And for the pushshift scripts: https://kbin.social/m/DataDumps/t/65054/pushdump-patch

I thought about it and it'll be helpful a thread going of folks who made a request, and when they got their data. To help give other folks an idea of when they might here back, if they put a request in now.

I put my request in the day before the blackout. Still waiting to hear back from Reddit.

Edit: Just FYI I ended up deleting my account before hearing back on this. If I ever do hear back I will update on here, but at this point I'm seriously doubting that will happen.

Edit2: 2023-07-10 - they got back to me. Since I no longer have a reddit account this was emailed to the email address I registered the account with.

I just spent all day today fighting with reddit, trying to get all my comments deleted/overwritten: https://kbin.social/m/RedditMigration/t/45417/Anyone-have-experience-with-deleting-comments-to-see-older-comments#entry-comment-190482

It's not just me, someone else reported the same, though using a different tool: https://kbin.social/m/RedditMigration/t/46805/Strange-phenomenon-while-deleting-my-comments

Basically, reddit has the most ridiculous api ever! A 1000 limit on viewing .. well basically anything. Try to go further back, and you can't.

The tools and scripts and websites we are using to delete, they are hitting that limit and can't go past it. My own reddit is only 5 years old and I hit this. I imagine that many folks where, the ex-redditors who had 12, 17 year old accounts, you probably didn't get everything on your way out.

Unless of course, you had a data retrieval request made to reddit, and reddit responded with your data. Only then are tools like shreddit and websites like shreddit.com able to completely wipe out your history. Or else you knew about this somehow already and used an external manager like eternity - https://github.com/jc9108/eternity - to save a copy of your posts before they got lost to the 1k limit.

Worst of all, it's explained that deleting items does not rebuild the list - so you can't see the older stuff by deleting newer stuff.

I'm hoping that private/public transition is an exception to this and it'll rebuild my lists when that happens. Maybe then I can go far back enough to delete everything.

Edit: Nope, someone confirmed in a comment below that this doesn't happen.

Also looks like pushshift is not an option, as pushshift was shut down last month, https://old.reddit.com/r/pushshift/comments/13mhuzq/api_has_been_taken_down/ - and under the new deal, regular users won't be able to use it when it opens up for business again, only approved moderators can (and likely only for approved reasons) if i'm understanding https://old.reddit.com/r/pushshift/comments/13w6j20/advancing_communityled_moderation_an_update_on/ correctly.

Edit2:

Okay, finally I am confident that I got them all!

Got a torrent client set up, and went to

https://academictorrents.com/details/c398a571976c78d346c325bd75c47b82edf6124e/tech&filelist=1

It turns out that you can just download individual subs. I know exactly which ones I first commented on, it was a small list and the subs themselves were pretty small too. So downloading just those subs from the torrent was pretty speedy and easy.

After that I came up with a quick script (based on the example scripts by the pushshift team's github) at https://github.com/Watchful1/PushshiftDumps

I was able to modify the super useful script at https://github.com/luphoria/reddit-migration one last time to accept a list of comment ids in a text file, and then overwrite them all.

Looks like I won't be needing that GDPR archive after all, thanks for nuttin reddit!!

Hat-tip: https://news.ycombinator.com/item?id=36038684

Edit: confirmed with someone in the PSA thread that this doesn't work. See https://kbin.social/m/RedditMigration/t/47320/PSA-If-you-have-more-than-1000-posts-more-than#entry-comment-200913

Original post below:

I'm having the same problem as the commenter here: https://kbin.social/m/RedditMigration/t/41129/Just-started-the-process-of-editing-all-of-my-Reddit#entry-comment-180921

In my case I can see my comment history much further, but still about six months short of when I created my reddit account.

My post history seems go to back to my very first post, but I have a lot fewer of those.

For now I just edited my comments, but I'm wondering if I should also do a delete. If I delete say 400 of my newest comments, will I see 400 more older comments show up in my profile? And if so, how long will it take to refresh?

I'm willing to experiment and try and find out for myself, but was just curious if others had already experienced this.

(Of course, if it actually doesn't help - deleting new comments won't bring back the older history for me to backup and edit and delete - then I'd rather not delete anything at all and just leave my farewell message everywhere.)

I adapted this script to do what I needed, including printing out a count of comments found, as well as the permalink of each comment (so I could double check the date).

https://kbin.social/m/RedditMigration/t/16226/Wrote-a-script-to-edit-all-my-posts

So it's well known now that the developer of Apollo estimated the new API pricing would cost $20 million a year. For a source, see the title of https://www.theverge.com/2023/5/31/23743993/reddit-apollo-client-api-cost

But from https://apnews.com/article/reddit-blackout-steve-huffman-ceo-api-0a4f7b344ecfbf50c924b030c344c55e the price from supporting third party apps is $!0 million a year. And presumably this is all third party apps combined!

Huffman says the “pure infrastructure costs” of supporting these apps costs Reddit about $10 million each year.

Something's very not balanced here. That one app would have paid for Reddit's third party infra costs twice over.

I can not remember which ones now (can anyone help me out here actually?), but I think a few apps said they'd try to make it work with the new pricing.

Which means Reddit likely stands to make a huge pot of money once the new API changes take effect, in the short term.

Even if Reddit loses the best subs, the best communities, the best users, and the moderation goes to where the sun don't shine, I could see that new revenue boosting investors confidence enough to lead to a successful (if slightly smaller) IPO.

If Reddit goes downhill and loses lots of value afterwards, well, spez has already made his quick buck, so I doubt he wouldn't feel very sentimental about it.

Folks, please explain to me why I'm wrong. Please.

Does anyone know how to modify Shreddit?

Like from https://kbin.social/m/RedditMigration/t/34112/Updated-Reddit-is-quietly-restoring-deleted-AND-overwritten-posts-and#entry-comment-140501

https://github.com/jdbassa/Shreddit
https://aur.archlinux.org/packages/python-shreddit-git

I really want to use this, but I have a couple of additional requirements:

First, to save a copy of my content locally on my hard disk (somewhere, anywhere, any format is fine) before overwriting and deleting the comment/post. So that I can manually repost on kbin/fediverse later.

Second, one particular post and comment I'd like to exclude from deletion. I've already overwritten with my good-bye reddit message and - as it was my most popular post ever - like to leave it up so folks who used to go back to that post to see my advice will know where to follow up to find me again.

I'd actually prefer to leave all my comments up with just the modified message but I think I have too many for the tools to delete automatically - instead I'd have to delete, wait a bit for the queue to refresh, then delete again as older comments start showing up on the API.

And I'm not confident that Reddit will get back to me before July 1 about my data retrevial request, which might make automatically deleting much harder.

Anyone know of a modified Shreddit that can handle these two requirements?