Data Engineering

1 readers

1 users here now

Discussion on Data Engineering topics. Data pipelines, tools and technologies, databases and DBMS, best practices:

Rules:

Limited to data engineering, no general CS/programming posts.
No technical questions. Example: how to fix this bug in my code.
No marketing
No resumes, jobs
No PII

founded 2 years ago

MODERATORS

lucas50a@lemm.ee

-1

ARCHIVE_DT or when you can finally delete some shit (self.dataengineering)

submitted 1 year ago by greengnu to c/dataengineering@lemm.ee

0 comments fedilink hide all child comments

Knowing when a record can be disposed of in the future is deeply valuable to keeping database tables clean and containing only useful data.

31-DEC-2999 is pretty common for records that don't have an easily known value.

If you are in an organization that is trying to monetize your data, treat this value as the date when the user will no longer be able to see this record.

If you are in a more sane industry, treat it as the records retention date minus X number of days that match your _A table storage duration or if the record is already in your _A table, the date it will be deleted and no longer recoverable.

You may wish to put some thought to your database backup and rotation schedule so that those records cleared by that date as well but I leave that as an exercise to the reader.

no comments (yet)

sorted by: hot top controversial new old

there doesn't seem to be anything here