Not merging for weeks is not called feature branching that's just working on a different project.

branching is like time travel, it's sometimes helpful but if you stay away for a long time the consequences are unpredictable

And Dave Farley said not to branch and there was no branching. And Dave Farley saw no branches, and it was good.

If you don't think a single developer working alone can benefit from feature-branching, I don't think you and I are talking about the same thing when we say the word "branch", which would certainly explain the discrepancy. a "branch" collects units of work (commits) together. Each commit is the smallest change which works on its own. Each branch is the smallest change which justifies its own existence. Within a branch, prior to integration, commits are not a commitment - they are rewritten / re-ordered regularly, as the discovery process continues. Integrating them doesn't make sense, because they aren't final yet. Forcing integration prior to the change being done makes the history unreadable because each change becomes separated from its justification. And of course, the one and only reason we're even using version control (instead of just sending each-other diffs against the last timestamped snapshot) is to keep track of *why* changes happen. I think we both agree that integration needs to happen very regularly, but disagree strongly about the acceptable scope of both feature branches and commits. I don't want to be "more continuous" than the smallest justifiable unit. I also disagree that "a branch which can't be integrated" is the same as "lost work", and I would have thought (based on your other video) that you would be a strong advocate against that sort of position. Sometimes code needs to be rewritten based on new information, and when that happens, it's always faster than writing the original code. It really sounds like you're talking about traditional, long-lived branches, rather than feature branches. As with so many discussions encountered on the Internet, I'm pretty sure I agree with everything you say, except for the specific terminology.

Once more insightful thinking of how sw can better made. Thanks Dave Farley. Since this is a so difficult practice to get the teams and/or Devs to move to, if possible I would love a future video of how to start with it. Something like what needs to be done from the beginning, and a possible adherence to other techniques, etc. Thanks in advance.

I recently retired after 47 years in the field, slowly trending from coding to writing tools to version control to configuration management to as close as I could get to continuous delivery, which wasn't very close because I could not convince my teams of some key points. I am hugely enjoying your videos and your explanations, and thought I'd share a few observations from my time in the field. 1) I haven't heard you say this actual phrase, but some major luminary very early in the software world's development stated "Nothing of any significance has ever been accomplished by a team of more than 7". I've heard you express much the same idea, though, quite a few times. More people than that exponentially increases the chances of miscommunication about the design of the system. 2) I used to sort of half-heartedly suggest feature branches for extensive, disruptive, dangerous changes, but my teams always resisted it hard for the reasons that you clarify in this and the other videos about "don't feature branch"; essentially, merge hell will invariably result. 3) I worked in about 9 different shops that all claimed to be doing Agile Development. In one of them, they were actually doing Agile Development. In the others, they were indulging in rituals designed to fire off the buzz words managers wanted to hear, but these rituals usually were bumbled so badly they didn't do much good and in some cases were actually harmful, like when the boss was the scrum master without training and did most of the talking.

- "When the information is modified in each place, then the long the time, the more those copies will diverge from each other". Well that is the description of entropy, it is so fascinating to understand that basic properties in universe affect everything even how we commit ...

Yay! Finally I got an excuse for committing directly into master: it's called CI

Hey who doesn't like the smell of a merge conflict in the morning standup meeting? 😜🤔

One of the issues I have around CI vs Feature branching is more related to maturity. As far as I can tell, it works great when you have a stream aligned team working with a code base where most of the work happens in parallel. When you have a monolith, the problem with CI (for me) is that integration events become unpredictable. One team may be testing a set of changes, when another team commits a change that is automatically deployed. All of the tests cases that had been performed at have been invalidated as a result of the change. That's primarily why we've used feature branches in the past, to isolate changes just long enough to test them.

I'm interested in what your opinion is on Code Reviewing practices, in light with your strong preference for CI workflow. Some companies I've worked at, have a pre-merge code-review process like Github's PR system that needs a review before being merged into trunk. A pre-merge code-review process or mindset would then only work in an environment that considers a pair-programmed changeset as properly reviewed? Would you consider a pre-merge code-review policy to be an impediment towards incorporating CI? In companies that have a post-merge code-review process this problem doesn't seem to exist, but one would probably require an adequate level of (non)functional coverage in their automated tests to catch mistakes quickly, such that code-review is only needing to (ideally) focus on design. Are you therefore of the oppinion that a post-merge code-review process (or pairing as mention before) are two key processes to have in place for enabling CI?

Interesting. We used feature branches at first - it is hard. Now we embraced, trunk based development and release branches. Release branches allow us to keep CI intact and enables patching. For new features, finished or not - we use feature toggles. Feature toggles allow several things: review partially implemented UX, understand complexity of database migration requirements. We also starting to embrace side-by-side approach vCurrent-by-vNext. You may say it is a lot of maintenance, on the other hand you are doing it anyways. Now we have much less feature branches, always up CI/CD, easier database CI/CD. Love the videos. Would like to see you touch code reviews topic. Often it is seen as one person offers ideas and knowledge sharing on optimizations or better code structure. But I think that it is more important to review how people handle exceptions, how people ensure that APIs are complete. Not all devs are experienced. I mean code reviews are not about beauty, it is about cleaning sh** left by others. This is more realistic and gives more value. Code reviews is about getting more experienced, not just about learning more patterns or algorithms.

Dave, I agree with what you say on so many levels. I am one of the people commenting on the other video. I would like to also refine my view. Please keep in mind, what I don't say explicitly is because I think we see things the same way. Let's recap what was the main issue in the other video's comments: the beginners of the team. Beginners are a healthy factor in a team for other reasons than the quality of the code. Technically, we tackle beginners by reviewing their code. Another reason to review the code is compliance. Either way, beginners will make mistakes that cannot be caught automatically, no matter how good your tests are. Let's add one more to the list of things to keep in mind: we fully agree that things should be integrated as often as possible. What we don't agree on is doing it at every commit. This is literally insane, for the reasons I hinted above. What we do instead is redefine the notion of a "feature". It's not an user story, but a subtask. Let's take an example. The junior on my team has been working for almost two months on a user story which ended up having about 30 subtasks. So on average, he finished (and integrated) one feature every 1-2 days. The reality is, some features (subtasks) were easy even for him, so some features were half a day, while a few took him as long as a week. I am fine with this, because we reduced the risk of breaking master, while also reducing the waste of effort (it was the beginner tinkering, so 1 week does not cost that much). Keep in mind, most of the time we were integrating every 1-2 days. We can also integrate all combinations of feature branches automatically and test them because since they're short-lived, there's less of them! We remove the branches after integration. So roughly we have only as many branches as the number of people in the team. Integrating at every commit would work only in a team of only seniors.

Day one: Checkout a new feature branch from develop. Day two: Merge said feature branch into develop. Rebase first if needed (if there are any conflicts). Day three: Merge develop into prod or staging or whatever deployment pipeline you have set up. Day four: Repeat. Feature branching and CI aren't mutually exclusive. You can do both. And I think you should. Keeping features small is just a sane thing to do and has nothing to do with CI or feature branching. If you can't keep the feature branch small, you should rebase it continuously (against the develop branch) until it's ready to merge. So, do feature branches against the develop-branch and CI/CD between develop and master with whatever steps in between you want. Ofc there's also other ways of doing it. Don't be dogmatic. Be flexible and adapt to needs and the preferences of your team.

Hmm, I'm still not 100% sold... Sometimes CI sounds like a really great idea, but the problem also is what do I do when I approach larger refactorings (like updating dependencies to a new major with breaking changes) that take time and will leave the project in an unworkable state until completely resolved or changes that we are not sure if we should carry them through... How do really revert a series of commits in a CI world, where they are mixed in with other people's stuff.

That is biased, in my view. What you call fundamental is not working in 100% real life scenarios. There are cases when it's useful to guarantee changes isolation for the sake of risks mitigation. Think of it as full transaction isolation in DB, like when evential consistency needs to be avoided.

3:20 I often feature branch when I'm alone. That gives me the benefit of switching features when I get stuck and I can come back later. You may argue that I should comment the changes out or implement a software switch when rewriting a function. That can indeed be a solution and I admit that I haven't tried it yet.

And that is why I enjoy the whole microservice hype in web development. It allows you to work in a small team. This way, you can have feature branches without worrying too much, given the tasks are also correctly planned in a way where the conflicts chance is very low. For such teams I find it very easy and enjoyable to have even bigger PRs, as they contain the full feature and you can observe all of the changes at once, giving you the opportunity to find more optimization routes.

I think big take away we need to remember is: “Continuos integration works by limiting the time period and thus limiting the fan out of the cone of changes, this reduces the risk of a clash”. That makes me wonder, if we use feature branches but could somehow guarantee the time period of unchecked code is the same or less as not using feature branches and forbid branching of feature branches to avoid the cone, are feature branches still bad?

Added costs of feature toggles is not always worth it. It takes a lot of upfront work to design and plan work on features so that they wont clash with each other. If not managed carefully, it can end up in really messed up spaghetti code. Especially in early stages of projects, when no one knows what they are really doing.

I get a lot of pleasure from watching these videos! The reactive system one has blown my mind and really got me thinking.

you cant just pick the worst examples of your opposing viewpoint and the best examples of your own and then say yours is better... sure if you are messy you can lose 12 months of work using feature branches. Another thing: Your method introduces loads of dead code into the production branch. If I look into the production branch, I want to only see code that is active, or about to become active. Not half written code for a feature that will be released in five months. For me personally, a clean codebase is way more important than occasionally having a merge conflict or two.

My favorite story concerning this topic was having a dev lead who had no concept of how to merge using git. The dev lead was responsible for insuring the integration branch was kept in order. Additionally the dev lead had to coordinate this for multiple pods. The majority of our pod was able to commit and merge to the integration branch on our own. The dev lead insisted on doing the merge for the more junior devs. I noticed that one feature branch worked on by a junior dev was spanning multiple sprints. I made sure to let the dev lead know that having that branch that long was going to be trouble. I was ignored and you can all guess what happened...The dev lead overwrote a lot of changes from the others. To end the story, that dev lead no longer works at the company.

It would be great to have a clear explanation of the hypothesis. It's not clear what feature branching is. I believe that CI is not alien to use your own branch to work on your changes and then allow peers review. I understand that short time feature branches could be used. It's not clear to me if those are not even accepted in CI.

How to use trunk-based development in a legacy system where there is huge technical debt and most of the code doesn't have good tests we can trust? How to ensure quality where each commit to the trunk can potentially break the system. Of course, we try to write more tests and do refactoring doing some new feature, but writing tests is not always possible when everything is coupled together.

Wow, 368 comments in a few days. You channel sure has grown, that's great to see.

At the company I work for, we implement CI/CD, but instead of master, we work on the UAT branch. Then once a week, we do a pull request to merge UAT into master. That way our production environment only gets clean and tested code, and hasn't crashed due to broken code in over a year now.

I think feature branching within a single repository can work, as they are visible to the team, but you need to treat each branch sort of like it's a trunk and regularly merge branches into other branches, not just mainline trunk. The advantage of this is if you do get some nasty merge conflicts you can put them off, let those branches diverge a little while you consult with your team about how to resolve them and prevent them in the future, all while continuing to make forward progress. Some of that progress may end up being discarded, but some of it won't, so overall you aren't completely halted by having to deal with the divergence.

I think the context of what kind of team and what kind of product you are working with will affect whether this is effective. When you have many disparate repositories of code, each with few changes over time, it makes more sense to use pull requests, since things move so lethargically. But if you spend most of your time in one repository, CI seems like a much better option. For example, you won't have to re-synchronize your feature branches to ensure you're on latest if you are always working off trunk.

Can you please further discuss the application of this in the context of open source software. Thanks

What about code reviews before closing an issue/feature?

My expierienced rule of thumb: smaller skilled good teamworking may start & come quit far with a strong disciplined checkin strategy directly on *the main* trunk/master branch, but those conditions tends to be not realiable with much more growing/larger projects teams especially including unskilled/newbies. Fitting VCS checkin & branch strategies depend of multiple aspects: a) project/task change-sizes & risks/fail costs (e.g.: not every deep re-design/re-strucure checked in as lots of tiny checkins but project-wide atmoic commit/submit), b) developers team's size, skills and discipline/motivation, c) surrounding IT & Development Infrastructure including build tool chain support, dedicated DevOps, Q&A and a good project (not directory) scaled merge/integration tool support (bare git is not enough)!

I’m trying to promote the term “reluctant integration” (RI) for those techniques that involve maintaining lots of branches...

Look at the Apache Maven project and how they manage all of their stuff, the same way I do, a single branch. Period. No need for a release branch either, to cut a release, tag it, build a release from the tag, and move on. Mainline development is what I recommend for all projects, as I’m sick of cleaning up left over branches from others. Git and Jira feature branches just makes things worse, as it tends to make the processes we follow fit the tools we use. The tools should suit the way we want to work, not the other way around. The number of times I’ve had to say “Son, just because you can do a thing, doesn’t meant you should do a thing”. Especially around Git branching models. The only time I use a release branch is when I have really long life releases (and I’m thinking 1-2 years here) and I have non backwards compatible breaking changes in happening. A good example of this would be a change of underlying middleware or java version for instance. Then each release branch effectively becomes its own master/trunk, and master fades into obscurity. I probably just triggered some with that last bit. But from first principles, it’s the easiest way to work - especially when the devs are inexperienced.

I have a shell script, git-sync, that replaces git pull --all + git rebase + git commit + git push, which mitigates some of these issues. Dave criticizes this approach at 7:21, but my script goes farther. It also checks for git conflicts across all other active features branches (we delete branches after merge), and list other branches. Finally, we have very small tickets, and we break them down even smaller to tasks which are safely atomically merged to trunk. We aim for "tasks" to last no more than 24 hours. In stand-up, we view branches older than 24 hours. Of course, none of that's full integration, but it gets us very close. Close enough to be better than having tons of feature flags, which is often required for trunk-based.

In general I'd agree, but in some cases it still seems useful to branch to me. Especially when doing some more experimental stuff, where you don't know, if you even want to add this change. But if you already know, you're going to merge this branch/feature sooner or later, it's always better to merge it now. It's much worse, when others already depend on a feature, which often changes, than when you are not able to merge this feature anymore. And you will always be able to merge your feature in most cases.

Also when working alone, branching is sometimes useful. I've already done it a few times.

Working in isolation in a feature branch keeps your unfinished code from being merged into the main branch until its completely done. If you check in code constantly to main, then you will have unfinished code or one big push with all your local commits. Keeping commits local without pushing is a problem because they can get lost if your machine crashes. You should always keep your feature branch in sync with the other developers by locally merging the main branch into your feature branch, atleast once a day. Though these features branches should also be very small. I'd would say that they should only live for maximum a day before it's being merged with main. Otherwise your features are too big and you need to break it down further.

I am working in a big software company. 6 years ago we had big problems with pushing features from the different teams back to master because of the changes by other teams as you explained. Now nothing changed except two things: 1) We must push any change as soon as possible back to master. Even if the feature is not completed if a pice of code, e.g. on function, is completed so it can be pushed to master without coder breaking it should be pushed. 2) Daily build We do not have any integration problems anymore. Because the discipline of the developers increased because they do not want to break a daily build code quality and efficiency increased. And the "old" developer who knows the "old times" saying that work is now easier.

This is an argument against distributed version control, not just branching. With DVCS, even if no one branches, separately developed code isn’t integrated until it’s all pushed to a shared repository. Merely committing won’t combine anything.

I think there is more to it than thinking ourselves as heroic loaners 15:01, one of the ways we breakdown complex problems and also with the code we write is the idea of breaking down a complex problem into specific concerns where we can think solely about each part in as much isolation as possible be it steps in a process or the responsibilities of an object. Isolate a single part of the greater problem at a time. Therefore, merge conflicts with someone else's code means we need to context switch to understand a potential different problem context in order to resolve the merge issue. Now in my experience its the guy who commits later that is responsible for resolving the issue (is this the unresolved special source that's nagging at the back of my head), and hence has to do this extra work of understanding an additional problem domain/context. Resolving who has to context switch stop the breakdown of their own feature and understand how their own problem context interacts with someone else's current work in the same problem domain ... should this be left to who commits second is seen as responsible for the merge ... the associated work is delegated on nothing more than who can smash that commit button before the other? It also means this is the point at which people's different coding styles interact, has the code been refactored or is still in a state of just putting together something to pass unit tests. I'm not saying leaving people in isolation till their own feature is complete before attempting merge with another complete feature, meaning there is a potential for a large divergence of code. I just think there is a social/phycological dimension to ... THE MERGE!!! I've found myself merging code ... before commit, "oh wait ... it's changed again" ... ok spend the time understanding the merge ... "ok right think that's my changes merged ... better do a pull form the remote" ... "oh jeez again some more small changes made before I was able to commit". This is doing my head in if only I had just been able to commit before I had to pull my hair out trying to understand this stuff written by someone else. Maybe it creates a feeling of winners and losers, and it's the speed at which you commit that is being rewarded not the quality of the code. Maybe I'm just crap and slow at merging peoples code, maybe needs ideas on "Pair Merging?", but does anyone else have experience of he who commits last is the loser??

What about changes that can't be completed in one commit and also can't really use tactics like Dark launching. In one of my pervious projects we had to make significant changes to multiple connected db-tables. I don't see how I can get around using a branch here. Yes the other developers will need to merke, but else we have broken code in the Main branch.

Excellent illustration of why the ingrained scatter-gather pattern of working is inherently flawed. We're experimenting with mob programming where all these concerns melt away, since no one is sent to their cubicle to work in parallel and in isolation.

Thank you for making the CD concepts presented clear and concise, that aspect pulled me in from busy, busy work. And much to my benefit :)

Branching does allow for an individual developer to create ownership over the feature they are working on. I'm not sure if it's always the "I'm the hero" motivation described in the video. Having pride over the work you do does create motivation to complete it which is important. It's also important to note that individual contribution does have a tendency to create tangents of thought that could become unpredictably useful or not useful. That variation can come in handy. That said, I do find it hard to argue against the idea of more closely collaborating to create better awareness of the application state. Maybe the experimentation and offshooting should just be left to a branch that's not necessarily meant to integrate for experimentation purposes; left to individuals who want to propose changes in structure and such.

I've really enjoyed your channel for a while now. I have picked up far more insightful ideas than I ever thought I would when I began watching. So my first serious comment on your channel is this... I absolutely love the shirt!

I don't agree with a lot of stuff he is saying but the comment section is great to read. There are some really interesting points of views.

Since you seemed curious about why people have an emotional response to this subject, I thought I'd chime in. What emotionally turns me off about ci is sharing my code before it's finished. Imposter syndrome is rampant in our profession and merging code into a shared branch before the feature is "done" make me anxious. Rationally I agree with you. Shorter cycles lead to better code and seeing everyone's else's "incomplete" might even reduce feeling of inferiority and serve as a learning aid for junior developers. Nevertheless the anxiety remains. Love the videos

What to do with code review? I can't imagine that there is no place for it in Continuous Integration ?

One thing confuses me with the “no branch” concept: in order to have a code review prior to merging, we *do need* a branch other than master (or whatever trunk) to be able to create a pull request, right? Or how else a code review would work if everyone is committing to the same branch?

This is pretty much close to how I always worked. I usually work on the master branch and push/commit regularly. Pretty much any change that brings a benefit while not breaking anything. I use feature branches for bigger changes that take longer, but I can't check in the intermediate work because it might break the existing state. It would not compile or break features. Usually a bigger refactoring. Where I want the intermediate states in the VCS but not break the master. As soon as possible I'd merge the feature branch into Master and continue working on it there. In some cases I combined it with the Dark Launch or Feature Toggles. Minimizing the time I work on a separate code base. Funnily enough, I rarely have merge conflicts as I usually merge Master back into my feature. My colleagues kept longer running branches and had to spend much more time on the merges. Mostly because they rarely pulled Master.

If you are dealing with a live system, with live customers who may report bugs that need to be fixed ASAP, maintaining an active "golden master" branch is the easiest, safest and fastest way to deal with it - and that means NOT having a bunch of barely tested or reviewed code go into the mainline. And even for an individual developer, I often use feature branches on my own projects, where I'm the only developer, if I want to experiment with different approaches that I may end up discarding later. I really don't get the hate for feature branches. I work with some trainee developers that need more review than others. I don't WANT their code to be continuously integrated - that would actually cause problems. And as for long-lived feature branches creating a growing tech debt/merge risk - just keep rebasing your side branch on whatever's been accepted into master, and you'll never be that far from a merge. Seems pretty obvious to me. To avoid branches and to just have everyone work in the master branch seems like chaos to me, and putting any customers of a live system at risk of not being able to provide them with timely bugfixes.

I think the concept of CI has a lot of merit. We had to change our concept of a 'feature' to help work towards this. On the basis that 'no one should commit code to master that breaks the build', we couldn't get an entire 'feature' developed quick enough to commit to trunk/master, so we used feature branches. But then we just found that if we broke down what we meant by 'feature' into smaller bits, we could do commit to trunk/master without 'breaking' anything. This did lead to some issues about testing these 'mini-features', but nothing insurmountable. And breaking a classical idea of a 'feature' down into 'mini-features' gave us a lot of insight into just what and why the 'feature' was in the first place. This worked well with a general 'Agile' thinking in that there were a few times our initial 'mini-feature' break down showed us where we had gone astray.

Thanks for the vid! It's very refreshing to see motivations for "why we do things like we do" other than: * that's how google does it * that's how good developers do it I appreciate you adding a scientific-like point of view. That being said, the context where CI really matters is almost clear, but not quite, so I'd like to understand: * Does CI matter more for companies where really big teams (or lots of teams) are working on the same code-base? * Does a microservice architecture make CI more relevant or slightly less important?

Not to be confused by not branching based on user stories. It is still helpfull to create isolated environment, a.k.a. branch, for user stories which takes couple of days only to implement and integrate. It is the smallest agile/scrum unit of work that validates a branch. It is more easy to discard a branch than to go back in history in case something goes wrong and developer has to start implement the functionality from scratch. Issue is, as pointed out in a video, a feature branch which takes several weeks until it gets tested and merged. Just my 2cents of course.

I'm curious about when/if Product testing fits in CI. Say that a developer (alone or working as part of a pair) is working on a feature, which was specified by a Product owner of sorts. There's always a chance of some level of misunderstanding between what was asked and what's being delivered. So it's now about whether the code works or not, but more about whether it does the right thing. In that case, being able to deploy a version of the code to a test environment that can be exercised by a Product person is quite useful. However, if we're always committing to trunk and the code is doing the wrong thing, it might already be too late. I can imagine one solution for it being feature flags for every single customer facing feature. While I can see that working, it can clearly add a ton of incidental complexity to the act of adding features and changing a product. What's your recommendation for that kind of scenario? Thanks!

Literally a day ago was talking with a friend of mine who is a QA lead at a big IT. He complained that it's a nightmare to deal with 10 teams pushing changes into the develop branch. He said that it is unmanageable because it's almost impossible to blame what feature, what change, and who introduced a bug. They spent hours just to identify this info. They do CI, but everyone understands and complains for that kind of workload branching model would most likely be much more manageable for QA.

He has this kind of voice where he could talk for hours about a wall and it would still be interesting.

CD, i'm new to programming and I just wonder, you often say you expect these CI branches to last rought around 15-20 minutes, or a day a most. So my question is: What if the feature I'm refactoring or working on is incomplete? like say we're making a video game, and I decide that the jump mechanic needs work - just doesn't feel good to jump in this game. Well 20 minutes into doing it over, i realize I need to spend some hours researching the subject and how other games did it. Do I commit, or possibly spend a day or two being thorough in my current approach to the jump mechanic? meanwhile, other devs are building level design choices based on the jump mechanic that depends on it being what they thought it was.

I think we can have the best of both worlds: consider the following workflow: The is a "trunk" branch called "dev", and each person creates a feature branch from it and then merge it back to dev after the feature is complete, with a pull request and ab associated code review. After an arbitrary number of features are merged, a new release os born, and it gets merged into "master", the production branch. If we create a "dev-ci" branch, and have the team muse feature branch, but at the same time rebasing/pulling from dev-ci and merging their changes into dev-ci frequently, we then achieve the ci goal, and the feature branch with commits filtered by author is still good for a traditional code review

As per your request, these are my thoughts on why I believe you are wrong. 1. Continuous Integration has nothing to do with feature branches. CI means merge and test often and has nothing to do with the branching strategy. You can merge your feature branches often and run tests to do continuous integration. 2. You said that a developer working alone, does not need feature branches. Wrong, a developer could and should work in a couple things in parallel specially if one the streams of work becomes blocked. He/she should not mix all the different things in the same branch, otherwise it can not be worked and merged individually. 3. Wrong, reducing the time of change does not means the change is small, in a few seconds you can do a refactor operation with your IDE of choice and change thousands of files. Short time != Small. Small = Small. One way of achieving small is by breaking down user stories into the smallest unit of change possible. 4. The fact is that more developers in the same code base adds complexity, not branches. More branches gives you isolation, which is critical when many people are working on the same thing. I hope this is constructive. Even though I disagree with the message, the content is well presented and you have your point of view.

I have come to love this channel. Thank you for the videos!

I've only rarely run into major conflicts when merging. Even when we had several people working on major additions to the same product. Most of the time branches are used to make changes or experiment without breaking the main branch builds. And since there are many things that we are unable to build and run locally code has to be committed to put it through the build, test and deploy pipeline. We could certainly investigate making it possible for each person to do that locally, but since branching is actually working it would be hard to justify the time and expense. Personally I'm not a fan of pair programming. We certainly use it on occasion, but I'd burn out in months if I had to work with someone looking over my shoulder all day. I've turned down lucrative job offers because the job would have involved working like that all day every day.

There may be an issue with semantics; TLDR; If we branch off of the trunk, but the branches are so small that several of them are PR'd in a day per person, does this still count as feature branching, or is it CI/CD as you describe? I run a small team of 3, sometimes 4 people, and we never work directly off of the trunk (which is actually our staging branch, but that is another story), as we want it protected, and instead, we make what we call 'feature branches', which is often small break downs of larger features into tasks (so maybe we should call it task branches), small enough to complete at least one, but usually several of in a day. We want the trunk protected as we want to review all the code via PRs before it goes into the trunk. We do this so frequently, that merge conflicts are quite rare, and our CD pipeline is often backlogged near the end of the day. In addition to this, we have small 10 to 30 minute (depending on what needs to be discussed) meetings throughout the day to discuss what we are working on and the direction we are taking, which helps avoid having to re-do a lot of work at PR. This strategy involves, more frequent, but much smaller PR that requires very little comment. Seems to be working great for us, except as the lead I need to work on getting my code reviewed by the other members more often and not just self approve. So basically, does this workflow still count as CI/CD as you describe, because it's still frequent, or is it disqualified because we are not working directly off of trunk. And if it's the latter, is it possible to protect the trunk in any way in this workflow?

I don't think your wrong, it's just that the software industry case been convinced that a feature is the smallest change worth integrating. This allows for all unit tests, functional tests, etc to be done over several commits that then mark this feature complete. We more use feature branches as short term backups then branches. I think where you would make this all one commit and the push it to the trunk, is really equivalent to a squash commit on the feature branch before merging. Also GitHub better supports the feature branch model with PRs, and other expectations of that system. Where would the review go in your CI model?

I tend to rebase on top of whatever the main branch is regularly so as to act as if I started my work after the current code base state. This isn't technically that different, but I find it easier to reason about. Essentially I agree with your video that a continuous straight line commit graph is preferable to me. At the same time, small merges are pretty inconsequential and easy to reason about. People do get quite heated about this so I leave it to them to deal with the worst merge nightmares while I commit on top of the latest when I'm done.

What surprises me about this guy is that, with all the experience he claims, he has never had to perform any kind of change that both crosses module barriers and/or touches one fundamental component of the system. Anyone who has done a database engine change (no, not a version change but an engine change such as from MySQL to Postgres) or a mejor framework versión upgrade already knows that there is no sane way of slicing these kind of changes into smaller parts. Also , he seems to ignore the not-so-revolutionary idea of simply performing dayly merges from the main branch into your feature branch. And no, nine months without merging is not an example of the dangers of feature branches. It is an example of spectacular stupidity and bad management.

I do get the overall idea but of course there exist several cases where feature branching is actually more productive and perhaps even mandatory. A few examples that come to mind: 1. perform a refactoring in order to replace an external library 2. Any form of "experiments" that are not supposed to be deployed in prod 3. large changes that are difficult to release "darkly" (or that can be released darkly, but with a high extra cost in coding the "darkly" (which is actually a feature itself). Again: I get the general advice but in my opinion "continuous" must be taken with a grain of salt, as with most practices.

I would really like to hear about how folks do code review in trunk based development. Years ago I worked for a place that did trunk based dev without code review. Integration worked well and the test suite was solid but the design of the code sucked because there wasn't a system for reviewing code design and architecture. I am wondering if it is possible to change workflow to accommodate this without having to have every change reviewed? I wonder if docs around feature architecture committed to source control coupled with well planned conversations would make sense? Anyone tried something like that?

You expect me to give everyone write access on the same repo? Certainly not? You mean everyone fork and PR into the default branch? Isn't that what everyone does? Branches are usually only used to track tags and releases unless it is a long running theoretical feature or overhaul that may only ever be used for demo purposes.

In solutions of tigthly coupled code, more monolithic, where boundaries are not clear, branching can cause problems and is expensive to manage. Simply because it is more difficult to understand the interactions within the code. However the alternative is much worse. Many arbitrary check-ins mean you may never achieve a stable build. The team just stops working until it is resolved - huge limits to team scaling. In practice you fall back to your local disk as your stable branch! The team don't pull everyone becomes terrified to push. The scope of changes becomes very very small until the code is refactored to allow multiple contributions. Contrast that to software with boundaries interfaces, services, single responsibilities then it becomes more clear which areas of the code a project, team or individual may touch. You know which component are likely to change for a given project or team due to the nature of the project. Then you get a better understanding of the changes across the various teams or projects and likely clashes. You may get a tendency for teams to focus or own specific components. In this situation, a more modern approach to design, branching makes much more sense. The answer to clashes may again be to refactor code. In this case the branching is easier. At this highly componentised end of the code base spectrum software is typically packaged. Adding tags to NPM or NUGET is like branching. So we also support a sort of branching of packages but in this environment it's designed to be short lived - pre-release labels. Sort of a clash with branches here btw. In all cases a stable version of the code, that a individual can build, is always required. If not you have no development. So a more abstract 'branch' is always present. So the statement "don't branch" is not a practical position at all (techi click bait and you got me). Where are the branches? What are the minimum branches needed? Are more sensible questions. My feeling is the answer to brancing nightmares is usually code refactoring such that branching in whatever form works better.

Thank you for the video. How would you go about prototyping different solutions whilst working on the same branch? It seems to me that rollback might be a really complicated in this case.

I've asked in one of your previous videos about the process of reviewing changes to a code base with continuous integration. I'm a bit confused about what the workflow would look like. I work with a small team, so cashes with merges are far and few between, but the most important part of our workflow is the review process we get when we ask for merge request. What does it looks like? How do you get that feedback from your teammates? Specially since that feedback happens when you got hands around the changes and it's all fresh in your mind, not 3 weeks from now when I moved on and a teammate happens to look at that part of the code.

What if one day your product owner decided not to release and undo that one complex feature (i.e. shared library change, db schema update) that everyone else had already integrated deep into the trunk?

I totally agree with fact that the longer you wait, the more clashes and conflicts may occur. The problem I faced with my team is that some time the work on a feature is incomplete before a deadline and we wanted to deliver some other complete features. Without feature branches we had to do all sorts of things, like turn off flags, hiding controls and even rolling back changes.

Interesting, I certainly agree with the principal, but think that using feature branch and PRs is useful. IMO I like to isolate CI from source control as much as possible, so there is one mechanism to trigger a build. Tags. A modular code base helps avoid conflicts more naturally also. Frequent conflicts between independent features is a sign that concerns are not well encapsulated or decoupled.

You mention the cost of CI, and IMO that's exactly what it's about: "cost". For many teams, industries, features, and contextual situations, being able to make continuous changes that are "ready to deploy" but not complete is pretty hard. There is little experience or insight on the strategies like feature toggles; branch by abstraction; dark launch; etc. But even if there is experience, every project and team always deals with something new, the team needs to know how to apply those techniques to their current situation, and that is not easy. For those teams, continuous integration is more costly than the pain of merge conflicts and the window of time where devs are "in the dark". This cost is understandable and at least easy to identify, so teams choose it and forego CI For continuous integration to get a bigger going, IMO that cost needs to be reduced, which I think is actually pretty large. In general, there are a lot of features; projects; etc; that don't fall neatly into a continuous integration approach, and they would require a lot of thinking and good design to be able to adapt to CI (with the strategies you mention). Basically, to be able to perform continuous integration you need all of your team to be extremely good at their job, you basically need all of them to be seniors, and good ones at that (at least in those cases). If we can reduce this cost, then teams would naturally adopt CI much more often IMO, otherwise they would ditch it the moment they encounter a big difficulty.

Generally agree with this precept to integrate small bits of code frequently. The only exception I can think of has to do with cybersecurity or life-safety system requirements that disallow dead code (i.e., can’t get there except in an unplanned error state) under all circumstances due to the risk of accidentally entering those code segments. In less critical codebases, I can easily agree that feature toggles (and similar control measures) are adequate and should be used instead of feature branches. If you follow a stratified development approach (e.g., Strata from Vitech for systems engineering), then you can, for example, stub the feature and integrate the stubs with the trunk, then add functionality as the feature is developed.

I had some time to think about your views. This is going to be longer comment, but I really hope I can get an answer from you, because I believe I have some strong arguments. I think you're loading into the idea of 'feature branches' something more, than I do. In particular, I agree completely, that the problem is in how OFTEN developers in the team integrate their work with the trunk. And that the less time they do, the better. I think nobody reasonable challenges this. Where the problem arises is that in fact NOT having feature branches does not prevent developers from diverging their work for too long and vice versa, using feature branches does not automatically mean long time to integrate (which is what I imagine is, what you believe the term 'feature branch' means). With trunk based approach, a developer could conceivably work on one commit for few days and create a humongous commit. In fact, sometimes there's just no way to avoid such commits. They are very rare, but they do come up, for example, if one needs to do large scale refactorings, that affect the whole code base. On the other side of the spectrum, if a developer uses short lived feature branches (say a branch contains maybe 2-3 commits and is merged with day or two after it was started), it poses about the same risk, as trunk based development. This approach of creating short lived branches, that are ideally small (2 or 3 commits), and are merged shortly after they're created (day or two after they're created) has many advantages over trunk based developments, with the primary disadvantage being, that it slightly increases the window for integration, but really only slightly (nothing in the realms of the horror story mentioned in the video). The primary advantage of this type of feature branches is, that there are plenty of tools (Github, Gitlab, Bitbucket, etc), that enable performing code review on a feature branch and this way code that is not optimal is not pushed to the trunk. There's one killer argument, that makes trunk based development completely unviable, unless there are special tools (that are not industry standard such as Github, Gitlab etc); With trunk based development a member of the team could conceivably push a commit, that fails to build. Now every developer that will checkout trunk will be unable to build their local branches. This is a disaster! And no, pair programming will not prevent this from happening. How does feature branch based development prevent it? By running automated build on the feature branch. Not sure about Gitlab and others, but Github can be set up in such a way, that the feature branch must be up to date with trunk changes prior to merge and automated builds have to pass in order to enable merging. This guarantees at least builds that will pass. And hopefully more people can review changes and if runtime errors would break the project at least more people have a chance to figure it out before it actually gets merged to trunk. I think if feature based development is done right (in the manner described above), it must be superior to trunk based development. I have not discussed code review, I know you advocate for pair programming, which honestly still should NOT absolve code from being code reviewed by other members of the team, so I think pair programming and code review complement each other, it's not one or the other. I'm really curious what you have to say about this and in particular how would you avoid problems with accidental push to trunk of a commit that breaks the project in trunk based development.

Hey guys! I just committed my half baked code that won't build to master. Off to lunch. Have fun testing your code.

That's why team members talk with each other. You can't say people don't know what the other is doing. If you work like that then you don't have team, but instead group of individuals or problems with project management. Pushing to master everyday means fixing merge conflicts everyday. Is it best way to deal with this problem? Even better, you can't decide feature one is ready to deploy, but feature two need some changes. How do you do this? Do you remove feature two from master and add it after release? Finally you chose the most corner case you could. Multiple big changes in the same code base at the same time. I don't know on what projects do you work but in my carier I didn't stumble upon that situation too many times. Because as I said at beginning, my team is there on plannings and rise concerns if someone tries to plan features like that. If it's needed you have more possibilities. You can create feature branch for 2 or more people and they work on the same code base but not on main branch. Still I needed this technic like few times in my carier.

I don't think CI or Branching address the real issue being discussed here. The question to ask is, "how do I decompose a piece of work into smaller deliverables that independently add value to the project?" Which is a design problem disguised as a delivery problem.

Excellente work as usual, I learn a lot and use it in training with my students. thank you

I agree with the principal that maintaining multiple branches can go wrong quickly but branches are not always evil. Here is my scenario. I have a production system pulling from master. I need to make large breaking changes to the API and UI. If i were to do this in the master branch I would lock myself out of maintaining the production system since I can't push or pull anything into production due to all the breaking changes in my code base. This has caused me not to be able to pull any changes into production for months until the dev changes were complete or having to directly modify the code on the production server. Not a spot you want to find yourself in. Having a single dev branch gives me the ability to switch to master, make a hotfix, commit the changes to master, pull to production and then switch back to my dev branch, pull from master to integrate the change and continue development on the features. Not branching in a scenario like this causes a seriously hard to maintain system since you end up having to patch all code changes directly on the server since you have now locked yourself out of your version control system due to all the breaking changes on master. Don't abuse branches but do not avoid them when you need them.

Branching with small FF trunk based development just works. Especially if you want to do different types of testing that is not similar to your main app (i.e. different frontend with a ephemeral env). This is something that doesn't have to even be defended. It's not either or, we can take parts of what your saying and branch as well. Branching is not guess work though and having everything in 1 branch stops dev testing and exploration. Testing one small change to code is testing more than just that code itself. You have to also test how it interacts with the env you have set up and I don't see how that would work with only 1 branch.

make sense to me, but I have this question, I have worked in real big companies with hundres of developers around the world, 24/7 coding and adding stuff, using the branching approach, some devs (no me :P) sometimes pushes their broken branch to master, blocking the rest of the team, with CI that will happen too, and even more often is my suposition, Thanks for the video.

A) We are pretty much stuck with feature branches on an organizational level - we are trying to keep our branches small and frequent though (still not perfect, but we are improving), in the end, communication is key here whether on the individual feature-branch level or when talking about the "complete" feature and the plans for it B) I fear CI is too straightforward (for a lack of a better word) - you either need really good pairs programming together to avoid silly and/ or hard to find bugs creeping in, especially as less capable developers are just as capable of breaking tests as any other part of the code - you miss that part of separation that a pull request and review gives you; this is personal, but I can better focus on reviewing changes and trying to figure out what they affect when I go into code review for a (small[ish]) PR. This is also true of my own PRs. This, I think, is especially true when your code deals with multiple concepts that you need to mentally context switch from. I'm fully willing to accept that this might be my prejudice talking, since I haven't really been involved in a real CI project thus far. edit: I do adore your videos though, including this one

What about the argument of atomic commits? Should every push to trunk be guarded by tests? If so, wouldn’t the scope of tests dictate the frequency of integration? Is it therefore better to base integration on a set time frame, or on set scopes? How would you determine scope?

I mean you can just rebase on the main branch after you finish working on your feature brench.

We have builds and test (functional and unit) runs on every commit of our branch. We do rebasing instead of merging of main develop branch into ours. We strive to make multiple smaller MR/PRs but they always have to be a finished working bit. Since codebase requires quite a lot of refactoring so that C++ api guarantees code correctness far more than we are able to achieve in Java version of the product even though Java version is younger... Which is really important for multi threading purposes (when using shared data). But the most important part are code reviews (no tests are not replacement for code reviews - and I'm talking about to some extent almost formal code reviews - they are additional help and documentation). Based on this and the fact that branch is developed for a week and then gets stuck in code review for about 14 days (including changes to the code) I would really like to see how CI and constant pushing of code would produce better code. Faster written for short term gains? Yes. Better... Please explain how as I can't see it work for anything more than service development where partial uptime is more important than stability of the whole.

Thanks, you correctly answered my arguments of pulling every day. I agree CI is the best way and committing to the trunk every day is the best idea. That graphics was useful. Only point is we must have test suite there that covers well.

So this topic makes a lot of sense in a lot of ways but one issue I see is with how my studio works. We have new programmers and interns from a local college working on projects often. This means that any one of them could break things on a moments notice with CI just due to discipline and skill being not there yet. On the flip side it would force them to pay attention and work more effectively though at a cost. This isn't a finished thought really and context matters but I felt like voicing it can't hurt, all while I continue to think about it.

So the team I'm working with now has three branches: master, validation, and integration. If I'm introducing a 'feature': I make a personal branch, complete the 'feature', and submit a PR to merge to integration branch. I'm putting 'feature' in quotes because I'm referring to any changes in the code base. We generally limit our changes to take no more than one week of work. Twice a day, integration is merged to a new validation branch (ie. validation-2021-05-27), which runs an enormous library of validation tests. If any of those tests fail, the culprit PR will be found and either fixed or reverted before merging to master. These fixes / reverts will then be merged back into integration. If your PR was reverted, you simply pull integration to a personal branch, revert the revert-commit, fix the issue and restart the process. Now my question is: is this CI or feature branching? There are separate branches for features, but each branch only consists of a single person doing a maximum of one week of work. Does feature branching turn into CI when the amount of people on a branch, and the lifespan of a branch, are limited? For instance what if we implemented the same strategy, but 'features' were split into daily deliverable chunks, as opposed to a maximum of one week, does that count as CI? Edit: Just found a video where you actually explain the difference explicitly, guess I'll watch that later.

I think you should start with a definition of feature branching because everyone has a slightly different one

I'm a fan of CI, but I'm having an issue with the idea that a feature branch necessarily means longer times between commits. If I were to run a development team, I would require the same interval between commits regardless ... basically when a small feature is ready. The gain (as I see it) from feature branching is the ability to merge from trunk and see any conflicts or failures before merging to common code.

In small projects your approach is doable and make sense but in bigger ones involving many people I don't beleave that it would be more efficient than feature branching.

Hello, I get the idea and I don't disagree. I think we'd need some detailed examples though, about how to do it. The rationale seems as correct as it is high level. The practical level is, I believe, what keeps many doubtful.

Great video Dave. I heard you first on the Changelog podcast, then looked you up here. You've spoken to my years of managing feature branched projects and suffering the inevitable conflicts - even in the very small teams I worked with. It's spot on. I'm running the rebuild of a Drupal project, which will initially be built without any automated tests and all in feature branches. In the very long run I'd like to move towards CI and TDD. One of many questions I have though: Our development so far has gone like this: A developer works in branch, I review the PR and test functionality and pass it over to another non-dev stakeholder who also reviews it against the spec and tests it on a staging set up. In CI, do non-dev stakeholders have to do their testing and reviewing directly on production? In our experience this non-dev stakeholder often catches mistakes where the spec was misunderstood, and we feel strongly that these mistakes should never make it into production. Sorry if that's been addressed already! Thanks for the videos.

In my experience, long lived branches are partially the result of poor planning and lack of design, and a proclamation that integration is difficult because the "complexity" of the feature doesn't lend itself to being worked in pieces. A lot of teams and their developers just want to jump into development of that next feature, rather than first reasoning about how the feature will be integrated, and the design considerations to incrementally build the code.

What about things like ROS where you have different distributions that behave differently? Typically, some packages will have different versions on different branches depending on your distro.