the irony of listening to this while actively writing mocks and excessively unit testing

These blogs always pick an easy example. Most validation also requires examining the database. A common one would be, in this scenario, a requirement where user email address needs to be unique.

For testing library-type code (e.g. data structures) in dynamic languages, mocks can be useful for verifying that your fancy data structure isn't accidentally doing something untoward with the objects you're entrusting to it -- you can verify that only functions that are part of the interface that algorithm requires are called, and that things that are part of only _most_ objects (e.g. random toString logging calls) aren't used.

I find a lot of mocking comes from arbitrary code coverage percentages (80%-100% unit test code coverage). Also seen with: "unit test confirms exact log output", and "unit test getter/setter/properties".

The first time I heard about JavaScript on the back end, I laughed thinking it *was a joke*... I don't regret how hard I laughed

5:41 well that works until you find a compatibility issue because not all features are ported to SQLite in the exact same way as what PG did.

Great advice if all your functions are independent. It might be the case of 1% of all written code.

“… fixed TypeScript’s biggest problem” is the summoning ritual for Matt

I love Go's named returns. Keep in mind you absolutely don't have to use them naked. They are extremely useful as quick bits of documentation if you have helpful names for the return variables, especially if you are returning more than one variable of the same type. They also really help with reducing complexity because they guarantee those variables will be declared at the very top of scope, which can really help with refactoring and reasoning about the function. Kind of like defer guaranteeing stuff happens before exit, it makes it easier to write and refactor code while maintaining guarantees that your declarations haven't been moved around and caused some insidious bug. Even naked returns with stuttering names like "err error" can be very useful in private implementation code and should be used freely there. The only caveat with naked returns is don't add stuttering named return values in public methods: your public methods are primarily meant to be understood by the people using them, and stuff like (string string, err error) adds needless complexity to reading the function signature.

Hey Prime, not usually a commenter but I think this hate towards mocking is misleading to newer devs. Overall love the advice on code structure. Would really appreciate a response to the below :) In the example in the article there is a critical piece missing about how to glue things together. This is a transitive problem at each level of abstraction (normally several of these in any large code base), and these glue layers often contain small bits of logic themselves. Also, not all dependencies are deterministic (i.e. http timeouts, db trx contention, etc.) and it can be useful to model these situations to verify your code behaves as expected (i.e. propagating the right errors or retrying a certain way) For example, consider a common API handler function: 1. Stateless request validation (pure function) 2. Fetch some data (function with network dependency) 3. Stateful validation (pure function) 4. Example Twiddle: Some type mappings between request domain and data domain (pure function) 5. Store some data (function with network dependency) 6. Map stuff to a public facing response (pure function) This "glue function" has logic as to how it fulfils these steps and deals with potential failures (i.e. if we fail a validation step, further functions should not be called). When it comes to testing this, I want to test the wiring logic WITHOUT rehashing the behaviors of each of the dependent functions. I could do this by "injecting" my various dependent functions as well typed parameters and then "mock" the behaviors such that I can ensure certain branches get hit. This is all decoupled from the implementations; I can change the validation logic and it doesn't change the test. Call it mocks and DI or call it function composition, its all the same at the end of the day. You might argue all the glue branches get covered by integ tests, but this is almost never true at scale. These UTs are so easy to write too and in my experience have huge ROI for mature code bases.

There is a valid use case for mocked APIs: One time I was writing a shipment tracking service and it would connect to the API's of various shipping providers and check the status of packages and trigger notifications to users devices. While some of the shipping companies provided testing servers they didn't update the package status since there was no actual package and no real delivery process. So I couldn't use the actual API even if I wanted to. Instead I created a series of responses and stored them in a bunch of files. Then I served these files iteratively. A big benefit is that your test can go real fast without the network latency. But you have to maintain the mocked responses which is kind of a bummer.

No pgcrypt, clientside timestamp and hashing, no RETURNING id, no error handling (duplicate email? or transient error, so gotta retry?), no prepared statements, timeouts, stats collection. We don’t even look at the response. So what do we do? Extract two of the four non-db-related steps into a separate function of course! Now we’re cooking on gas! No need to run expensive integration tests to test those 2 ifs! The main problem of the project has been solved.

I highly recommend reading "Unit Testing Principles, Practices, and Patterns" by Vladimir Khorikov. It's probably one of the best software design books.

LOVE TO SEE IT. Thanks for the read Prime

Hello Prime, i dont know if you are ever going to read this, but as someone who just finished learning this job for two years in germany and tries to find her way through all this content and all, you are a great help. As a trainee didn't learn much of programming. I was basically just prepraed for the exams and that is about it. Your content helps me. If you or anyone here has some kind of help for me to what learn first and what I need to know, it would be sooooo helpful

How do you now test that the validation is actually performed before saving anything to the database without using a real database or mocks?

Many great points from you, the article, and many comments, but I don’t think mocks are a problem *if* you prefer, as I do, to test only public interfaces. Those in some cases while actuating underlying code, will need to call outside the library which shouldn’t happen in unit test. Mocking should be done only when necessary, but it is sometimes.

The one area I like to use mocks is to put the code in an unexpected state and see if handles the error and that it gives a reasonable error message. If the program is logging errors then I will mock out the logger to catch the error directly and compare it to what is expected rather than actually write the error to log. The other area is when I have to fix legacy code and I need to mock out the stuff that is irrelevant or that is creating expensive connections. Sometimes you can break the code out into a separate function and not have to mock it, but I have run into cases where that's not possible without a major refactoring of the code.

I would say there is one case where mocking is both trivial and definitely should be done: In Rust when parsing messages from some kind of data stream, when it is written correctly, it is easy to just get the test data and put it into a Cursor. That way you can just unit test the logic without having to do anything complicated.

I'm a desktop developer and disagree with this. Testing a desktop application without mocks is a nightmare hellscape where there is no return. I understand that backend is different but there are many more dependencies in this world other than db. What about calling other API's, report generation, interfacing with other applications or any of the many scenarios that would break a unit test without a mock?

I was taught mocking in school, it made a lot of sense to me. You want to decouple the code in tests. If module A depends on module B, you unit test B, then mock B for A's unit tests. That way, if a dummy breaks B, only B's tests fail and it's way easier to find the bug than if all tests fail together. Also, mocking is a necessity in Test Driven Development. Not saying if TDD is a good idea or not, I really don't know. It's a good exercice though.

100% agree! Mocks are a great way to fk up your test suite. Where i work, my boss mocked all his internals and taught the rest of the engineering team to do it. I never gave in. After he left i was finally able to convince the team to stop doing it. Your test should describe a narrative on how it behaves, not how it’s implemented.

4:05 One doesn't simply insert Screen Rant memes into dev content. Bootdev: Actually it's super easy. Barely an inconvenience.

as someone who uses C# professionally, why the fuck does someone want exceptions and try/catch in go? errors as values are far superior to debugger’s jumping in catch block at the slightest hint of trouble

I love all of it, from the article and your reaction. 🙂

Yo dawg, we heard you like mocks....

you gotta appreciate the Ryan George "super easy, barely an inconvenient" quote

Remember the old adage, “Use the right tool for the job.” Mock is just another tool in the development and testing toolbox. The bigger problem in the examples was the poor design of the system, coupling data validation to the save function.

Then how would you structure your codebase? Filestructure and MVC logic? Please inform me on whats a good way to develop a let's say a Rust http service?

This absolutely 👏🏿👏🏿👏🏿 As soon as I learned about mocks, I intuitively thought something was off about them. I came to see limited value due to the use of a very difficult coding framework (AWS SWF’s Flow Framework), but outside of that I saw a bunch of devs pretend that they needed to test that database and other remote call parameters were set exactly. If anything necessarily changed with the API functionally the same, you broke the test for sure because it was validating the internal procedure not the behavior. The most substantial benefit I see to mocks is when you need to test exception handling behavior. With a real database or remote connection, testing against unstable or failed network conditions is too annoying to simulate. You can have a remote call trigger an exception a number of times before returning to verify that internal retries are happening too.

How would you approach testing a function with multiple dependencies (database service, logging service, http request service, etc), where the code execution takes different paths based on various conditionals? The function's behavior might change if, for example, a validation step fails or if an HTTP request encounters an error so further code may not be executed but other will still be executed. I want to ensure comprehensive testing to cover all possible scenarios. Given the complexity and potential branching, how would you design tests to verify that the function behaves correctly in different conditions? I don't think its possible to achieve this without relying on mocking those dependencies, or at least i can't figure one out

Here's how I prefer to test my backend handles: 1)I use Docker (specifically docker-compose) to run an ephemeral database just for tests. 2)I create the database and run migrations at the start of the tests, and then destroy the container when they finish. 3)I test HTTP requests to a handler, which allows me to refactor the inner logic of the requests later. It's not exactly unit testing, but it works fine, especially in the microservice world.

But what if the "filddling" part requires querying the db for example? Maybe I need to validate something against some configuration or value not already in memory. Even if I'm not making external calls, I might call some other internal module which I then need to isolate from the system under test. Domain logic can still have dependencies which need to be mocked. What am I missing here?

can someone explain at what point this stopped being mocked? or is having a mock db the issue? cause I thought &User{ deats } is also a mock.

If a method depends on a integration point, how do I unit test that method without mocking the integraton point. Especially if the test environment will not have access to it. How will I avoid those null references?

How do I test the unit that calls saveUserInDB if validateUser was successful? At some point i need to moq those external resources.

90% of our test code is integration tests, and CI has to install MariaDB in the container. The main reason is that the code is ass, but the second is that there's tons of MariaDB-specific code that would simply fail on Sqlite or H2 and wouldn't be tested at all if we used mocks for everything.

Shout out to the author for using "Super easy, barely an inconvenience" 😂 Pitch Meeting has reached the far reaches of the internet 🙏🏿

I didn't think code coverage is problematic until I ran into a very pleasant and necessary test in a Django project.. basically the framework has a reverse() function, which takes the name of a view and returns its url, as configured in the routing layer, and a resolve() function, which does exactly the opposite of that. The unit test was taking 3 screens "testing the urls", ensuring that resolve(reverse(x)) == x. I'm so glad the url configuration had 100% coverage, I don't know how I would've rested if that test wasn't present in it.

In company I used to work, we tested for everything, even logs. Currently we only test functionality that means something, has purpose and needs to return/receive/modify values in certain way. Now I am not sure if author is talking about mocks. Mock is there to remove need for dependency in classes that have it while still preserving what function inside it will do (you can write whatever you want as functionality that mocked class does). In a way, you can write a mock that will behave exactly the same as class that you obviously wrote unit tests for and know that it work properly. Of course testing code in natural environment, some integration or interactive tests can be done on top of that but I usually find mocks and unit tests combined with CI Pipeline good enough and not face unforeseen issues (if tests and mocks were written correctly). You should always do stuff like stress tests etc. if your application needs to handle certain traffic as well and prepare for it beforehand.

How do you test a class (having internal state) with dependencies without mocking the dependencies?

1:36 minutes into the video, am already very much entertained. I love this man :X (Disclaimer: In a very platonic way)

Keep in mind, that while using SQLite for testing is great, it does not share all features with postgres. So then you have integration tests, which use different database than your production environment. I would sleep better at night, if the testing DB engine is the same as the production one :)

Check your unit tests privilege! Some of us are happy if there are any automated testst at all.

I think mocks are useful when your functions make calls to third party things. Like for example if you have a route that does some stuff and makes a call to an email client. You have to mock the email client response because you shouldn’t actually be calling a third party service in your unit tests.

How would this work for something like C# or Java? Traditionally, the Unit tests go in a separate project than the application code which means you can’t just split out functions and test them individually without first making those functions public? And making them when there’s no need for them to be public breaks best practices.

8:44 don't specifically need ORM, just an encapsulation function to not do it raw. A function which generates the SQL.

I wanted to skip the video when I heard “never mock”, but didn’t… and in the end the final thought was not to use mocks for integrated tests. And here are I 100% agree with you. But I do think that mock is a powerful tool for state testing. And to be honest you need to test the contracts and the states. If both are tested you don’t even need integrated tests.

Any idea how Comma AI does its (regression) testing? I assume they use an open-source tool (CppUTest, google test , catch2, doctest) but what?

Love the Ryan George "Super easy, barely an inconvenience".

What if the unit tests del and create a database? Am I sunsetting?

Imo, test writing is a team leader/project manager's job. Replace design briefs with actual specifications by limiting well designed tests. Would make it a lot easier to teach Juniors too.

Beeceptor has been a heavensend for mocking, debugging endpoints and timeout problems, testing adjustments pre/post production, CORS support, Customized Dynamic responses, Mock serving for OAS Specs. It's fantastic

The main problem is that people just don't realise what exactly they can test with different kinds of tests. It's damn useful to have unit tests on sql queries using mocks (sql.Mock) to test marshalling and unmarshalling of structures, to test passing parameters in queries. It's just hilarious how many silly bugs, panics, etc. there can be at these levels. But that only tells you that the data flow in your code seems to be correct. Apply encapsulation and be fine.

Great article

The lesson here is not that mocking is bad. The lesson is don't mix high level policy code with infrastructure code. High level policy code pairs well with unit testing, dependency inversion, and fakes of various kinds, including mocks when appropriate. Infrastructure code pairs well with integration testing and using real, as close to production as possible, collaborators.

Someone claiming to have "fixed" how any language does its error handling is hilarious. The hoops some people invent and then jump through just so they can use something they are more familiar with never fails to amaze. I had to work with Mocha/Javascript Unit Tests ones and they connected to and manipulated a MongoDB. The DB Setup and -Cleanup after every test made at least 75% of the test code. It would have been a lot faster to just set up a local docker container and do a quick reset before each test run (it's like 2 bash commands to do so). But nooooo, none of the other devs knew how to use docker, so it was not allowed.

00:00 - Introduction 00:09 - Know Your Language 00:21 - Mocks and Dependency Injection 00:31 - The Problem with Mocks 00:48 - The Story of Bill 01:34 - The Virtues of Errors as Values 02:04 - Unit Testing Database Logic 03:37 - Integration Tests vs. Unit Tests 07:03 - The Code is Ass 08:10 - Stop Mocking 10:45 - Separating Logic and Testing 11:17 - Conclusion

I love external service mocks in integration testing. I am using boto, localstack and docker compose to mock databases and aws service apis. This way I can develop a lambda function, ecs service or whatever locally and directly integrate it with surrounding services with the iteration time of a few seconds. It is awesome as it speeds up development significantly, makes me confident that the code and infrastructure code will run fine in cloud environments (yes you can run terraform or aws cdk against it too) and it is a clear working documentation of the API and how other services interact with it. It can even be used to test IAM policies locally without a single deploy to AWS!!!!!1111

I've run "unit tests" with in memory sqlite, it was super fast and easy and with much of the logic being in the SQL queries it added a lot of value.

i like to write in-memory repositories to "mock" the database, but actually i;m using them on "prod" when it's only a prototype as well. The key idea is that i can run bazillion of test within a second or three but if i want to test it with real database - also no problem, just it's taking a little bit longer. For APIs my team also wrote inmemory service to pretend it's a real service. Usually these in-memory things are enough but several times real database and real api discovered some bugs. We're running all of the tests with real things before merging to master, but these in memory are used in development and they giving us instant feedback. That's the compromise i believe.

I sometimes need to unit test my sql and Testcontainers is an excellent option to do that. Plus, mocking external dependencies is important. Example: Calls into your cloud provider. Mocking AWS SQS and asserting the data sent to it is extremely important. Internal dependencies? You need to have a good reason to have an interface and mock for an internal dependency. A real good reason because you usually won't find that reason. Mocking internal code usually just reduces your code coverage and forces you to write more tests. Why?

I love mocks in unit testing. if you use it properly you can test how your code should behave when different values are returned from mocked dependencies.

I only mock things that make http requests to external things like google services. 99% of the time you don't need a real response from google to test. Everything else is fileld with fake/incorrect/empty data, with a real database.

So basically you test your database access function, read or write. Connected to a real database, and do the Integration testing. Then modularize those fiddling functions, possibly pure so unit testing is easy. Then you have an Integration testing again with database for the entire api endpoints? Is that how you backend engineets usually test backend API? Just curious🧐

8:00 The sound of silence can be pretty intense, though notably he changed to more silence.

Have you juniors heard of in memory db?

One thing not mentioned is the "intergrated function" returned both an error for bad user/passwords and for database problems in the same variable... This is criminal but also the kind of code a try: catch: pattern encourages. Think about what the code looks like to handle the error of this function: (lifted from the article.) func saveUser(db *sql.DB, user User) error { if user.EmailAddress == "" { return errors.New("user requires an email") } if len(user.Password) < 8 { return errors.New("user password requires at least 8 characters") } hashedPassword, err = hash(user.Password) if err != nil { return err } _, err := db.Exec(` INSERT INTO users (password, email_address, created) VALUES ($1, $2, $3);`, hashedPassword, user.EmailAddress, time.Now(), ) return err }

We once had a devastating bug in production. And management was dumbfounded as to how it could happen as we had a near perfect code coverage in that servlet. ... Open up the tests for said servlet, meet a huge block of mocks at the beginning of every test. Proceed to inform our manglement that we don't actually have any test, just a whole bunch of mocks. Manglement did not like me pointing out problems, and we still have mocks.

Putting code that fetches data into separated function is a double edges sword, because some people don't care about the underlying code, so instead of writing their own sql query that does join, they instead use your "abstraction functions" and overfetch data af. Personally, I prefer always fetching at the function that handles the api request. All other functions that fiddle with data can be on their own, but once I'm passing db connection to a function, something, somwhere went terribly wrong xd

Smol Ame writing unit tests? Let's ground pound some bits!

I find the "abuse" of mocks an issue. But I do see scenarios where you simply don't have the real thing to test or you do have an unstable environment with no valid test cases for you to check contour scenarios. I was one of the guys who worked with a code coverage target. At a project I even got at 100% code coverage. I still look at code coverage, to see "hey, there is this scenario that we totally did miss in our tests". And there are situations when what we need is a failure. Which in some situations may be difficult to test. Such as if the code itself is a wrapper around a real database that is supposed to handle different database responses, or a production webservice that we cannot really mess with or the customer doesn't have a stable uat environment, or we lack access to that environment at some point. In that case, either we create a mock server. Or stop development totally. At this moment mocks comes in handy. IN the video itself, there is a mention on "using an SQLLite" database. Is it the real production database? No, right? I do consider that a mock. I work with developing a framework that other people uses to make real life integrations. I don't have nor want access to the customer database or services, nor do I want to test them. But I do want to receive a json payload or an invalid json payload os some times sobre issues with payload content to test how my framework reacts to that situation. For these there is no way around: I need a mock server. In short: I think mocks have a niche use. At least at some steps of development.

Prefer (in memory) fakes over mocks anyday. You should invest in a db fake and then you can test the write and read (100 percent of the backend) in unit tests like this: write op 1, write op 2, write op 3, read op 1 assert output. Of course in addition to the advice peddled here to break out the fiddling into isolated functions to test.

My last job had some points where things were injected 36 layers down and every layer wanted a test with mock injections no matterh ow useless the test was it was baddddd

God Bless You.

as an aspiring backend dev, I am so happy i understand this

hi, we have this situation. front end and back end is on separate team. in achieving the given timeline, development have to be in parallel, which means front end have to "imagine" what backend will response. Not applying mock in the backend for the consumption of front end will cause the delay in development time for front end. Therefore, I see the importance of having mock API. what's your thought on this?

I don't see any problem with using a mock in the described example. Everytime you use mock in your test, you just must be aware that the mocked part ALWAYS has to be tested in separate test. When you use mock or design your code the way it doesn't require using mock (e.g extract the problematic part from the code as it was proposed in the example), , in both situations you are avoiding using this probematicaly part of the code in your test, so the outcome is the same and what solution you choose doesn't really matter. I advocate the solution that you should ditinguish the "impure code" (= connection with external systems) from the "pure code" (testable, eg. data transform functions) but in some situations (e.g in OOP) you just need mocks.

Mocks are tricky for sure. Just a few days ago I ran into a set of tests that so much mocking going on, I couldn't figure out if there was any production code left that was effectively tested. Despite all that, I still think they have many uses. You just need to know when to stop.

Depends on what you're mocking, and how. I frequently mock network components

Are mocks a way for devs to make things fail in spite they failed to introduce bugs in the test functions ?

You gotta mock connecting to 3p SAASes, in integration tests.

Mocks are the part of that fiddling function you don't test, usually that function is not just a map, it would have some logic, need to fetch data from a dependency only in specific cases and stuff like that. You can solve it by having function composition or delegates too, but in the world of dependency injection we live in I believe mocks are prety usefull. Why would you test again a dependency that is already tested? Why would you take the burden of having to initialize everything that dependency needs in your test again? Just mock it and test the fiddling part, it's a couple lines of code versus many lines of initialization and the danger of having tests overlapping each other. When someone says that overused phrase "if you mock you are not testing anything" I always ask if they are doing functional programming or what kind of dependency inversion they are using, if they use dependency injection, that sentence makes no sense to me at all.

In agreement on mocks. However, my employer requires us to have 80% "code coverage". It does not matter to them that it's type strict and compiled. Since the "fiddly" bits are not 80% of the code base, I am either forced to mock, or add a file with an unused function that does nothing but exceeds the length of the rest of the project so I can fulfill an arbitrary metric for management. I am also in the camp of "unit tests are also still code, equally prone to errors, and often more lines than the actual code they are testing".

But sometimes I need to test a certain scenario that could happen in the DB but hasn't yet happened

I'm brand new to go, learning it now thanks to Prime's videos. Hadn't heard of Naked returns yet. I googled it after he mentioned it because I couldn't tell what they were from that brief mention. I literally stood up in my chair when I read the documentation. I don't like that. I will not be using that feature lol

I'm in the second month of my first Internship, things are a bit messy, and no job is being assigned so I decide to take a look at this thing called tests. "Well, this aint that complicated, you just have to run this framework with 'mocks' and be done." So, I offer myself to start implementing some unit test for the few first "services" we had (they only do some basic GET's and POST). Boy did i felt like an idiot, mocking data, just to test for a '!=null' or something like that. Lucky for me, something felt odd when i started mocking a db in memory and stopped myself.

I’m learning like totally 4 months in at this point and your videos are so advanced but still helpful. Just wanted to let you know that some of your viewers are totally ignorant of most of the shit you say 😂 but anyway I’m trying to understand Jasmine now

Meanwhile I had the joy of finding some tests in the java codebase at work where the database is mocked, and the database mock returns a mock of a plain java object. Sometimes I wonder where people got the idea that the new keyword is an antipattern

Oh and I would also add that many devs do not know how to responsibly write INT tests. A good integration test should leave no trace or at least try to leave no trace it was there, and not cost you tons of money by connection to a third party API or hammering a database when you don’t need to. Unit tests at the base of the pyramid help to ensure the integration tests on top are minimized and are as least intrusive as possible. Or am I crazy in thinking you can have unit tests and int tests and no mocks and mocks and be perfectly fine?

Tests don't test current code. Tests test future changes to your code. Your tests are bad if your mocks are able to hide future code changes. However, you can use them to also assure that future code changes don't break the expectations of current code.

I feel like people misunderstood the point of unit test. It's not suppose to test the real world use case it's a sanity for your small unit of code. Should be small and quick to execute and to catch unexpected edge case or when the 'unit' is somehow modified later on. It's only like the first step of testing

god to the jr... media personality to the sr.

I need to try doing this "never mocking" thing, generally i'm writing C# backend code, or Android with Kotlin for apps, and we mock the heck out of everything, wanna test this CreateRecipeUseCase? Uses error message internationalization, if so don't forget to mock StringLocalizer, oh but it also needs to know the user to check for access authorization? well then, mock the LoggedUserProvider, Oh and also the RecipeRepository so you don't access the database I forgot to tell you that you need to mock the UnitOfWork as well That is exactly how I code, but i don't know better, i should invest sometime refactoring my code to support the "never mocking" thingy

At my job, we work mainly in Rails... our biggest mantra is, "if you find yourself fighting rails, you're probably doing something wrong".... this is what I think of when I see someone circumventing a Go feature to add.... a tryCatch function? That just sounds awful.

I am beginning to wonder if Prime's self professed knowledge of development is actually pretty narrow. If throwing around blanket statements like 'never mocking' then perhaps you've worked on simple systems that never require mocks in the first place?

I'll argue also that you need to verify everything (pure functions) is called and in the order you expect...because it is great to UT but it also great to check what you tested is actually used. So for this you need to mock your pure functions. In short, yes when you mock too much, something smells in your code.

Oh man I laughed out loud at that hand on face joke. This doesn't happen where I live

The thing about mocks and unit tests, they look useless until you run them😂then you are happy you have wrote them

A RYAN GEORGE REFERENCE IN MY PRIMEAGEN VIDEO? IT'S MORE LIKELY THAN YOU THINK