Gain CONFIDENCE With Approval Testing

No video

Gain CONFIDENCE With Approval Testing

Рет қаралды 8,945

11 ай бұрын

How and when should software engineers use approval testing? In this clip from Dave Farley's Engineering Room Podcast, Emily Bache, an automated testing expert talks about the core benefits of approval testing.
Watch Emily's FULL Engineering Room episode HERE ➡️ • All Things AUTOMATED T...
___________________________________________
🙏The Engineering Room series is SPONSORED BY EQUAL EXPERTS
Equal Experts is a product software development consultancy with a network of over 1,000 experienced technology consultants globally. They increase the pace of innovation by using modern software engineering practices that embrace Continuous Delivery, Security, and Operability from the outset ➡️ bit.ly/3ASy8n0
___________________________________________
🖇 LINKS:
Emily's NEW KZfaq Channel, show your support and subscribe HERE ➡️ / @emilybache-tech-coach
The Samman coaching website ➡️ sammancoaching...
Approval testing tool TextTest ➡️ texttest.org/
Approval testing tools Emily works with ➡️ approvaltests....
Emily on social media
Twitter ➡️ / emilybache
LinkedIn ➡️ / emilybache
Mastodon ➡️ sw-development...
___________________________________________
📚 BOOKS:
📖 Dave’s NEW BOOK "Modern Software Engineering" is available as paperback, or kindle here ➡️ amzn.to/3DwdwT3
and NOW as an AUDIOBOOK available on iTunes, Amazon and Audible.
📖 The original, award-winning "Continuous Delivery" book by Dave Farley and Jez Humble ➡️ amzn.to/2WxRYmx
📖 "Continuous Delivery Pipelines" by Dave Farley
Paperback ➡️ amzn.to/3gIULlA
ebook version ➡️ leanpub.com/cd...
NOTE: If you click on one of the Amazon Affiliate links and buy the book, Continuous Delivery Ltd. will get a small fee for the recommendation with NO increase in cost to you.
#automatedtesting #podcast #softwareengineer

Пікірлер: 21

@shashank.c 11 ай бұрын

I haven't watched the full version of this Engineering Room episode, but I can see what a gem this conversation was. I first understood what an approval test is from Dave's earlier videos and immediately knew how important it was for a team like mine which have been working with a lot of legacy code.

@JohnMitchellCalif 11 ай бұрын

I'm a DevOps Engineer with tons of Dev experience. In one of our projects the Devs had a Snapshot test which was failing. I thought: "hey, I can read code, I can read the test, I can fix it!" That did NOT work: I didn't understand the overall *application* well enough, so I didn't understand if I should fix the test, or fix the code. I did the correct thing: let the failing test continue to fail, vs burying a bug to explode later :)

@Dehibernation 11 ай бұрын

Sounds like snapshot testing, commonly used in React development. The downside I've seen working with it is that unexperienced developers can often just approve things blindly just to make the error go away. This can put more load on approvers to also make sure their diff is correct, which can be tough without explicit context about the components and how they work.

@sarabwt 11 ай бұрын

It is snapshot testing, just the implementation Dave showed is a worse version of the type Jest has. Devs blindly accepting the change is basically the same as updating the test to become green without asking yourself if that is the result that you want. However, I think snapshot testing for UI is not that great, because as you said, the approver has to know the context in order to be useful. I find it is much better used with validating API responses

@EmilyBache-tech-coach 11 ай бұрын

That's why I prefer to call it "Approval testing" to emphasize it's not just a snapshot, it's something you've approved. A "snapshot" implies it's here today, gone tomorrow, update when you feel like. An "approved" record or text file is something you explicitly decide whether it's good enough. I find junior developers just need a little bit of training and they stop blindly approving things in the way you describe. It's always in version control history who it was who approved a wrong result 🙂 The other aspect you're missing in React development is a custom 'printer' as I described. The text should be designed to be readable and possible to make an approval judgement about, if you understand the domain.

@lkyuvsad 11 ай бұрын

It’s more correct to say that React snapshot testing is an implementation of approval testing. Approval testing is a broader, and earlier, idea. Both snapshot testing and approval testing are things I tend to use sparingly. But they’re useful in their place. As Bache says, they are helpful when you want to test high level behaviour when there are a lot of outputs to test and they change frequently. Maintaining large, complex assertions is a pain. Blindly approving changes is equivalent to deleting tests when they fail. If devs started deleting tests, I’m not sure we’d say that was a problem with the tests. Why did they feel pressure to act without understanding something? Why is somebody approving or working on something they don’t understand without someone on hand to ask for help? Why didn’t they understand their responsibilities? Pairing seems like it would be helpful for both of the problems you mention- making sure inexperienced engineers understand what is expected of them, and that code reviewers actually understanding what they are reviewing.

@DmytroPoplavskiy 11 ай бұрын

It works with the code review when the code change is reviewed together with the approval test result change.

@manishm9478 11 ай бұрын

I watched Dave's earlier video on approval testing, but something about this discussion clicked for me :)

@AttilaButurla 11 ай бұрын

Interesting. I've worked for a company that would write unit tests for UI frameworks, such as React, where it would store the HTML/content that was generated as a string. The test would then compare any subsequent runs against that exact string. I wasn't a huge fan of that approach because anytime you adjust the UI, the test would fail. Yes, you could just update the tests, but it didn't seem like in this case the test was providing much value, because anytime the test would fail people would just accept the new output. Perhaps the scenario I'm thinking of isn't a good candidate for acceptance tests.

@dontanton7775 11 ай бұрын

We also use the term in games QA but in a different sense. It's the approval of the feature and quaility criteria on user level. Interesting to learn that this term comes from coding itself.

@barneylaurance1865 11 ай бұрын

AKA "golden master" testing or in the Micheal Feathers terminology "characterisation testing".

@jangohemmes352 11 ай бұрын

Golden Master testing is new for me, I like it most

@researchandbuild1751 10 ай бұрын

Used to call this gold testing. The result we would compare against works be the gold standard

@buddymoody 11 ай бұрын

While watching this I thought of something I often think about (and have never seen written about)... are these "compare output as text" tests good to maintain and keep in your build or are they possibly more useful just DURING REFACTORING. I am a fan of TDD (perhaps not the biggest but a fan nonetheless) but in my mind I do see a subtle difference between how I write tests during development and the tests I want to leave to help the next developer 1) change the code and 2) simply understand it.

@sarabwt 11 ай бұрын

It is absolutely good to maintain. It saves you the writing all of the assertions. It is equivalent running the code by hand and comparing the results, except that this is automated. If you like what you see, then you accept the change, otherwise, you change the code to produce different output. It allows you to move much much faster, especially if you have to validate large payloads. Also, when adding new features/removing old ones, the assertions are automatically "kept in sync", meaning that you don't have some tests that validate the new field for example and some tests that you forgot to edit. All of the tests will fail, and you will have to go through 1 by 1 to verify that the output was correct. This type of testing basically makes it so that you actually write the tests once and don't touch them again and you also don't need to be paranoid that you forgot to add some assertion somewhere, as any changes in the output will result in tests failing. I ditched most of the unit testing in favor of doing these kinds of API "e2e/integration" tests in test first style.

@EmilyBache-tech-coach 11 ай бұрын

Whether you keep the tests once you've done enough refactoring to get other unit tests in place depends on the cost-benefit analysis. If you've done a good job with the printer and the approved text makes sense, these tests can be cheap to maintain and good as documentation, as well as continuing to find future refactoring errors.

@Tony-dp1rl 11 ай бұрын

It's almost ironic, that Approval Tests seem to be just a higher level test, which we were already doing with test automation and integration testing without Unit Tests. It could almost be used as validation that Unit Tests at the code level are less valuable than advertised. i.e. Testing the results of a process involving several steps IS exactly what higher level tests do, and they do it in a place that is easier and less costly to maintain than at the code level.

11 ай бұрын

But the higher level tests won't capture all important details easily (too many inputs too indirectly related to the outputs).

@EmilyBache-tech-coach 11 ай бұрын

The advantage of the lower level unit tests is you potentially get faster, more fine grained feedback on your design.

@barneylaurance1865 11 ай бұрын

@@EmilyBache-tech-coach Yep, also the unit tests check that the individual units do what you think they do. If you're going to re-use a unit later when you're building a new or improved feature, you want to know that you understand the behaviour of that particular unit first.