this post was submitted on 01 Sep 2023
329 points (96.1% liked)
Programming
17494 readers
121 users here now
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Rules
- Follow the programming.dev instance rules
- Keep content related to programming in some way
- If you're posting long videos try to add in some form of tldr for those who don't want to watch videos
Wormhole
Follow the wormhole through a path of communities !webdev@programming.dev
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
If I explain myself or add nuance, it won't be a "hot take" anymore, but here goes...
I definitely agree that they can be useful as both usage examples and as a way to enforce some rules (and consistency) in the API. But I'm not sure I'd go so far as to call that a "spec", because it's too easy to make a breaking change that isn't detected by unit tests. I also feel that mocking is often needed to prevent unit tests from becoming integration tests, but it often hides real errors while excessively limiting the amount of code that's actually being exercised.
I also think that actual integration tests are often a better demonstration of your API than unit tests are.
I haven't used any of these methods as much as I'd like to, but I suspect that each of them is strictly more useful than standard unit testing:
Makes a lot of sense. I figure contract tests is more or less what I have been doing then.
I think there is that misconception that unit tests are about validating each line of code and preventing logic bugs. Though obviously you understand that this isn't just about that, or not at all about that I would argue. Unit tests won't prevent accidental breaking changes, but you can at least add new tests every time this happen, so you're at least guaranteed that the next maintainer won't be doing the same mistake.
In an ideal world we could probably have nothing but integration tests, but in my experience if you only do integration testing you end up working really hard maintaining tests that are prone to break and costly to run. Unit tests are cheap to code and cheap to run, it is a great place to enforce your "contracts" with tests that are theoretically immutable. After those tests are out of the way, you can focus only on the actual interaction between your systems with the more expensive tests.
Anyway, you have a good take I am just blabbering. This is based on my personal experience as someone who only cared integration tests but was later converted by a person much smarter than I am. And then later on by joining a team that exclusively did integration testing, hundred of tests. It was hell (imo), we had to change dozens of tests for every little bit of modification. The rest of the team seemed fine with it though. We rarely shipped bugs but progress was incredibly slow for a module of such low complexity. Testing wasn't the only issue with that codebase though.
The one part of that that sounds weird to me is needing to change integration tests frequently when changing the code. Were you changing the behavior of existing functionality a lot?
Yes, and the test suites were insane. The program was outputting a lot of data, and we basically asserted on anything and everything for any given integration. I mentioned that testing wasn't the only issue, well there was a lot of issues. Unfortunately the behaviour changes were requested by the stakeholders and there was no way around it. That being said, had this thing we maintained been properly developed those changes would have been a breeze imo. The actual requirements were very simple.
But anyway, I realize this is maybe an extreme example to paint integration tests negatively, but the point remain. In this scenario, every time we changed a bit of code it broke dozens of integration tests instead of breaking just a relevant integration test, had everything that could have been unit tested been written that way. The integration tests could probably also had been less... exhaustive, but it was probably for the best considering the codebase.