042 Splitting one Git Repo into Multiple Repos

No video

042 Splitting one Git Repo into Multiple Repos

Рет қаралды 21,412

Dan Gitschooldude

Күн бұрын

Пікірлер: 24

@michaeldajewski2938 4 жыл бұрын

Dan, Thank you for the video - drives right to the point. I did not notice any glitches in the video - for me the quality is perfect and it is understandable 100%. Interesting is a comment on repo organization towards the end of video.

@pig2008ex 11 ай бұрын

bro you are my hero

@christopherroth1702 4 жыл бұрын

My apologies if your video answered this but I'm not sure how if it did. I have two main projects: JP and MP. MP uses much of the components built in JP. In addition JP has several projects; let's just name two of them: AS and US, both of which are currently building autonomously. My task is to migrate these projects from Bamboo to gitlab. So far I've come up with three solutions to sharing common code between JP and P projects (as well as sub-projects within JP) 1. Have separate projects. This will require updating common code in each project. Too much over-head though. 2. Have a separate 'common components' project. This will require some sort of integration with each project that uses common files. Might be complicated to configure. 3. Have ONE project in gitlab (monorepo) for any project that uses common files. Each project will be built on a separate branch Before projects are built common code will need to be merge from the master branch and after each build common code will need to be merged into the master branch to keep it up to date and in sync with the other project. This introduces manual steps that are unattractive. What are your recommendations?

@DanGitschooldude 4 жыл бұрын

Great question. I usually recommend the mono-repo approach unless there's a good reason that isn't viable. Usually "these are different components" is not a good enough justification for splitting things into their own repos. What's more important is the co-dependencies of the components on each other for testing. If a monorepo is not viable for whatever reason, putting common code in a submodule that all projects can then include can be a good alternative approach. There are downsides to submodule usage ESPECIALLY if the code in those submodules changes often. Git can't merge at the submodule level so you typically want "slow-moving" code in submodules and "fast-moving" code at the super-project level. A monorepo bypasses this issue since everything is at the super-project level by definition. I'm not sure I follow your reasoning on how a monorepo has unattractive manual steps. The way you describe things, it sounds like you are worried about your branching/merging strategy, not the monorepo itself. Can you not have every user of the monorepo create MRs with the same target/integration branch?

@christianspringer4363 4 жыл бұрын

Thanks Dan The Git School Dude!

@Parm_newchannel 3 жыл бұрын

Thanks that was a great presentation can you please tell us the difference between monorepo and multirepo and which one is best

@DanGitschooldude 3 жыл бұрын

A monorepo is just a large single git repository that contains multiple projects that are loosely coupled or not coupled at all. I've never heard the term "multirepo" but submodules/subtrees are usually the alternative to monorepos.

@javakom 2 жыл бұрын

I think it is usable for a repo to splitting to submodules. A main repo has B and C subdirectory. B and C subdirectory history split by filter-branch feature then B and C subdirectory history is deleted in A by filter branch. so it can happen. :/ thanks @Dan.

@JoshPeak 3 жыл бұрын

Do you have any plans to demonstrate / document the git subtree split command and how it differs from the filter-branch approach?

@DanGitschooldude 3 жыл бұрын

I'd like to, but unfortunately I haven't had as much free time on the weekends to make videos recently.

@JoshPeak 3 жыл бұрын

@@DanGitschooldude thanks for the reply and thanks for the videos. I appreciate the content that you have produced and it’s been a treasure trove to stumble across right when I’ve needed it. I wish you well. 😊

@nikhileshsaggere 3 жыл бұрын

I have use case to create a repo2 tracking or derived from to another remote repo say repo1, and in the repo2 I want to change or play with the folder/directory structure as per repo2. However I continually want to be updated with the contents of repo1. How this can be accomplished, my opinion is to have the repo1 as sub-tree. Please advice

@ROCK3T96 4 жыл бұрын

Thank you for the video. I was wondering if you can help me with a niche case I am looking into as well: Suppose repo A is an existing remote repository which contains many subdirectories (namely i, ii, iii, iv, ...). I would like to create a new repo called repo B, which is a variation of one of the subdirectories of repo A - say A/i/. Therefore I am not interested in all of repo A, only the contents of A/i/. The idea is that repo B will operate independently of repo A but still connect remotely to repo A in order to merge updates made to the files shared by both repo A and B. At the same time, I still want to be able to edit the files in repo B so it becomes a matter of just pulling updates from the corresponding files in repo A and merging the updates with my changes (if all that makes sense?). I am aware that this is pretty unorthodox for most Git users so my questions are: - Is this possible or does this deviate from the purpose of Git? - What would be the best way to achieve this? I'm still starting out with Git so apologies for not being as sharp as most of the others in the community.

@DanGitschooldude 4 жыл бұрын

What you are describing fits better as a submodule or subtree rather than separate repos. If you were to split B from A in the manner described in this video, the repos would cease to have common history. Then, later when you merge A into B or vice versa, you will join histories, and have duplicate history for the subdirectory of interest. That won't buy you much. Check out my submodule/subtree vids on my channel, that's probably the route you want if you absolutely must have more than one repo.

@ROCK3T96 4 жыл бұрын

@@DanGitschooldude Will do, thank you very much! Really appreciate your advice on this. From my understanding, submodules are useful for 3rd party dependencies but I wasn't sure about whether I could change the scripts involved. Do you know if I can edit the scripts that my new repo is dependent upon specifically for my new repo?

@javakom 2 жыл бұрын

@@DanGitschooldude I was looking for a repo to split multiple submodules. I thought it is usefor for split but you mentioned that it cause duplicate history :/ filter-branch subdirectory for submodule folders but how to split main repo history from subdirectory. I will watch submodule repo. Thanks for videos. Your effort is really appreciated

@maeeshameem1578 2 жыл бұрын

I want to convert every commit of my git repository into separate folder. Can i do it by shell script? can you please help me telling how to do it?

@DanGitschooldude 2 жыл бұрын

See 'git rev-list' to get a list of commits and 'git archive' to take the code from those states. This technique will not have git history in each folder, I assume that's what you want.

@HansPeter-qg2vc 5 жыл бұрын

Sorry, but to me it's not clear what this command does. I'll test it for myself tomorrow but you should probably clarify for others whether this command makes sure the history of every file in the directory at HEAD is preserved, or whether the history of any file that ever was in that directory is preserved.

@DanGitschooldude 5 жыл бұрын

Excellent question! The history of the entire directory is preserved, meaning any file that exists or ever existed in that directory has it's history preserved as part of the 'git filter-branch' command. Here's another way of thinking about it. Imagine you re-wrote every single one of the 1202 commits in the repo and in each re-write you removed everything except the contents of the subdirectory (trick_source/ in this example). When you are done, many commits will have the exact same state (those that didn't touch trick_source/) and those get purged leaving 657 commits left (see output near 3:20). According to the documentation this purge comes from the --purge-empty flag. Interestingly, in my testing those commits get purged even without that switch.

@jondoough 9 ай бұрын

Started out well until you confused things about removing history, and you really didn't end up with the purpose. A good example would be something like drupal were a developer started making modules, and then you want to split them off. You forgot how to link the submodule back to the main repo. This would be a useful example.