this post was submitted on 18 Nov 2024
171 points (97.8% liked)

Sysadmin

7682 readers
253 users here now

A community dedicated to the profession of IT Systems Administration

No generic Lemmy issue posts please! Posts about Lemmy belong in one of these communities:
!lemmy@lemmy.ml
!lemmyworld@lemmy.world
!lemmy_support@lemmy.ml
!support@lemmy.world

founded 1 year ago
MODERATORS
 

I was literally told to set up this new service as quickly as possible and it didn't need to be correct or best practice because this was just a proof of concept.

Here we are 6 months later and I'm still cleaning up my own mess.

top 26 comments
sorted by: hot top controversial new old
[–] TheBat@lemmy.world 5 points 3 hours ago

You fell victim to one of the classic blunders.

[–] slazer2au@lemmy.world 89 points 13 hours ago (1 children)

The motto I got from working at an ISP was "There is nothing more permanent then a temporary fix."

[–] DigitalNirvana@lemm.ee 28 points 13 hours ago (1 children)

Similarly, I’ve heard, “nothing more permanent than a 3/4 solution”, i.e. it works well enough that fixing other things always ends up taking priority.

[–] possiblylinux127@lemmy.zip 8 points 12 hours ago

You should read the Red hat article about decommissioning a data center. Over the period of 20 years there is a crazy amount of works well enough and no one documented it.

[–] friend_of_satan@lemmy.world 4 points 7 hours ago

This is one reason I love test driven development. Once you have the solution, there's somehow never time to write tests. If you write the tests first, then find the solution, there is somehow time to write them.

[–] possiblylinux127@lemmy.zip 36 points 12 hours ago (2 children)

It will be in production for 10 years

[–] seaQueue@lemmy.world 18 points 10 hours ago

In 25 years someone's going to discover a dust covered raspberry pi hosting the poc service in the back of a network cabinet and unplug it, bringing down the rest of production

[–] Valmond@lemmy.world 10 points 11 hours ago

Young people will be hired and spend months to learn "how it was done in the old days" (it was just old stuff piled upon other old stuff).

[–] wizardbeard@lemmy.dbzer0.com 33 points 13 hours ago (1 children)

Don't let the perfect be the enemy of the good. That said, it's vanishingly rare that I've regretted designing something for long term use rather than just hacking it together.

Taking the extra time to document stuff, to add guard rails/error handling, to make a piece of script easier to re-use, to actually plan something rather than building it as you go... almost always a good use of time.

[–] lousyd@lemmy.sdf.org 4 points 12 hours ago

Amen, wizardbeard.

[–] Fontasia@feddit.nl 25 points 13 hours ago

Always implement like you're going to turn it on tomorrow

[–] scytale@lemm.ee 19 points 13 hours ago

Same with “lift-and-shift now and fix after migration” . Everyone knows nothing will be fixed and the same problems will exist, just in the new environment.

[–] Davel23@fedia.io 18 points 13 hours ago (1 children)

Several years ago I had a client who needed to be able to receive files from a client of theirs. This was before Dropbox and the like, so I just went in and set up a quick and dirty FTP server. Worked fine, client got their files, all was good with the world.

A couple weeks later: "hey, we have another client that needs to send us files, can you set them up on the FTP?" Sure, no problem.

Repeat for about the next six months. The thing turned into this huge glob of a mess that miraculously enough still worked.

Finally, I call up my contact there and tell him if he wants this to be a permanent solution we should go in and set it up right, and to do that I'm going to need a bunch of information from him, who needs access to what, who should not be able to access what, etc. He says fine, why don't you come on over and we'll hash it all out.

I go on-site, we end up having a 2-hour meeting (billed, of course) where we go over all this stuff, plenty of notes are taken, decisions are made. We wrap up, I tell him all I need now is the list of users and their access needs and I can clean the whole thing up.

I never got the list. The thing just continued to grow and grow into some kind of unearthly abomination. Fortunately, I left that job before the thing imploded completely. Someone else got to untangle that mess.

[–] lousyd@lemmy.sdf.org 9 points 12 hours ago (1 children)

But somewhere inside of you, it hurts, doesn't it? One more pile of infrastructure and code that ain't right, and it could have been different.

[–] jonne@infosec.pub 7 points 11 hours ago

And of course, the next guy that comes in thinks the mess is your fault as opposed to an expression of organisational dysfunction.

[–] catloaf@lemm.ee 12 points 13 hours ago (1 children)

This is why I put an end date on anything temporary. If you want it longer than that, it'll have to be fully built out.

[–] possiblylinux127@lemmy.zip 5 points 12 hours ago

For me it is always the emergency script for fix a problem with production that ends up being forgotten about.

[–] hactar42@lemmy.ml 4 points 10 hours ago (1 children)

My boss convinced me to make a pricing calculator for our contracts using Excel with macros. I pushed back and said we need to use Power Apps, CRM, or anything other than Excel. He said it would just be a POC/wireframe we'd use to build a more permanent solution with later.

Well 5 years later, there are like 2 dozen versions floating around with different macros, pricing, etc. And of course I'm the one who gets messaged every time some makes a change and breaks the damn thing, or some sales person is convinced it's not calculating correctly. I fucking HATE that Excel sheet.

[–] TheBat@lemmy.world 1 points 3 hours ago

If it's on Excel365, you can edit permissions for specific cell ranges. That way your colleagues can only give input but they can't change formulas or fixed pricing.

[–] knobbysideup@sh.itjust.works 4 points 11 hours ago

This is why i always push back that if you want it, I'm going to implement it properly, or at least in a way I can change easily later. I'm not going to make my own life difficult on your whim.

Web developers are the worst. Especially the ones that call themselves full stack. </sysadmin rant>

[–] foggy@lemmy.world 6 points 12 hours ago

And this is why bug bounty hunters get paid.

[–] someguy3@lemmy.world 1 points 9 hours ago (1 children)
[–] lousyd@lemmy.sdf.org 9 points 9 hours ago (1 children)

"Proof of concept". Only meant to demonstrate something, not actually to be used.

[–] Kichae@lemmy.ca 5 points 8 hours ago (1 children)

You know things are going to get messy when the PoC are demonstrating. That's when the cops turn off their body cameras.

[–] UnderpantsWeevil@lemmy.world 5 points 13 hours ago

Even as a PoC, you'll want a strong foundation to build on. Even as a temporary solution, you want to know the fix won't fail the next time its under pressure. Even under time constraints, you want a solution that you are confident works when it goes live.