• My observances over the last ten years of working in software with automated test suites is that broken tests get fixed quicker than broken docs.

    I shouldn't worry, I'm not a big believer in fixing broken tests for the hell of it, but I am a big believer that documentation describes intent better than tests do and that the code should match the documentation and vice versa.

    Now, I realise some devs will scream at what I've just said, but let me phrase it a different way...

    What does every software bug that has ever existed got in common?

    They all compiled, passed the type checker, were syntactically correct, passed the unit tests, and the integration tests, and the quality assurance.

    Every bug, passed every test.

    And once you deeply appreciate that, you have to concede that tests are only as good as the anticipation of every scenario, and that bugs will get through regardless. Far more important than being pedantic about creating swathes of tests is good documentation of what should happen, and the ability for deviance from that documentation to be treated as a bug of high importance. Either the docs are then wrong, or the code is. And in the case of an API with developers implementing against the documentation and provided interface, having their action be the test that needs to pass is critically important.

    It effectively declares that the documentation is the spec, and the implementations are the tests, and that if the code doesn't work that we either have a bug in the code or a bug in the spec.

    So whilst I have tests to act as an insurance policy against us breaking our contractual interfaces (the REST API)... I much prefer working to documentation and developer feedback for the raising and resolution of bugs.

    This is far more like behaviour driven development rather than test driven.

About

Avatar for Velocio @Velocio started