045 Introduction to Git LFS (Large File Storage)

Рет қаралды 54,238

Күн бұрын

Today I'm introducing you to Git LFS which is a popular add-on to git that provides an efficient mechanism for tracking large (typically binary) files. The Git LFS server is already supported by many Git web tools, and in today's video we use GitLab as our Git LFS server in my hello world project which can be found here:
gitlab.com/gitschooldude/hello/
Additional documentation:
Git LFS: git-lfs.github.com/
Native Git Support for Large Objects: • Native Git support for...
If you are looking for an effective concise introduction to Git, I highly recommend my "Fundamentals of Git" course on Udemy: www.udemy.com/course/fundamen...
If you or your company is interested in live virtual software and tools training, see www.continuoustech.net/training/ for a list of training topics we offer.

Пікірлер: 50

@nikhileshsaggere 2 жыл бұрын

Excellent explaination and guidance

@rick7242 Жыл бұрын

This was an awesome explanation, thanks XD

@jacobjackson8619 3 жыл бұрын

im here cuz i couldnt upload my unity file to git. thnx for ur help

@JD-mv4rb Жыл бұрын

so cool!

@2175Wook 3 жыл бұрын

I have a Macbook Air laptop. What I am trying to understand is what folder should you install Git LFS to, is it my Macbook's applications folder. I am creating my react website from iterm2 bash command line with Visual Studio Code. The directory for my website is cd workspace/Personal-ship-project whenever I use bash and I am pushing it to GitHub.

@danielhoffmann7138 2 жыл бұрын

I was stuck around one hour wondering why my files were not showing up as being uploaded to LFS in "git lfs status" because I forgot to run "git lfs install" in my repo, thanks! For some reason I thought just doing `git lfs track "folder/**"` would work oh yeah by the way if you want to track a whole folder with LFS recursively that is the right way

@rs-tarxvfz 2 жыл бұрын

Also, from this tutorial I wanted to understand the behavior of LFS files on SERVER not just client.

@viswaviswa8616 Жыл бұрын

Hi Dan, thanks for your time, not sure git lfs will support azure repo, because i followed your video after pushing my changes into git azure repository, i don't see any LFS tracked file inside azure repo and also I'm trying to lock lfs files inorder to others can't be edited.

@yadali4833 Жыл бұрын

I am new to git, I did not understand what you did after the commit, you created a new branch and pushed to that branch?

@LDT7Y 2 жыл бұрын

I've installed LFS, but Gitbash seems to be ignoring it. I still get error messages about files being too big when I try to push my unreal project to Github. It suggests I use LFS, but I already am! I've even tried deleting the 3 files it highlights as being too big, but I continue to get error messages about them regardless. Not sure what I'm doing wrong.

@alexandremondaini 3 жыл бұрын

Hi I ran git add huge/* without first initializing the repository with git lfs. What would be the right way to fix this issue ? thanks

@DanGitschooldude 3 жыл бұрын

If you didn't commit yet, just 'git reset HEAD -- huge/*'. If you committed, you'll want to throw away that branch with 'git branch -d '. Then start the process again but using git lfs. Regardless of whether you committed or not, your local repo size will remain very large, since 'git add' is enough to put those binaries in your local history under the .git/ area. The important part is not pushing those commits with the huge binaries committed as non-LFS content elsewhere. Your local repo being large isn't a big deal (and will eventually get small again once garbage collection runs), you just don't want those unwanted changes to persist outside your local repo.

@nicoapinedo 3 жыл бұрын

Hi Dan, awesome video, just one question does using LFS add to the size of the remote repository? I'd assume it does because the files are actually uploaded there, if that is the case, how should a 10+ GBs project be stored in a repository? (it's a similar scenario regarding several large files)

@DanGitschooldude 3 жыл бұрын

Sort of. It adds to the size of the LFS "server" for that project, bit not to the git remote repo history itself. They are separate mechanisms. I have never used git LFS for 10GB+ of data myself but I believe it's intended to support that amount of data. Thanks for watching!

@nicoapinedo 3 жыл бұрын

@@DanGitschooldude thanks for the quick and clear reply. This will come handy!

@ithereos9554 3 жыл бұрын

Dude, your content is just amazing, do you sell courses?

@DanGitschooldude 3 жыл бұрын

Thanks buddy! Yes I have one udemy course and do live training sessions for businesses when time allows. See the video descriptions for more info on that. Thanks for watching!

@JohnGriffith1 4 жыл бұрын

I'm working on a machine learning project where we are using Git for the code repository. We have a lot of source data files that are used for training. These files may be transformed a few times to produce new files, but the source files are unlikely to change. I would not say the files are generally large - most are less than 1 MB. There may be A LOT of them, though. I'm trying to understand if LFS is the right thing to use here or whether I don't need to worry about that. Any thoughts?

@DanGitschooldude 4 жыл бұрын

What would the total size of the repo be on disk? As a barometer, at my day job I manage a repo that is several gigabytes on disk and we aren't using LFS (even though we should be). Ultimately it's up to you, LFS is probably the best approach assuming there's no issues with HTTPS access but it's not strictly necessary especially if you know the files won't change a lot over time.

@deimylaw5182 4 жыл бұрын

4:40 - adding attributes to the .gitattributes - does the last parameter -text means? to track the file as text file? what if we want to track the file as binary file? do we still mention -text parameter?

@DanGitschooldude 4 жыл бұрын

I don't know the details of what everything on that line means, git LFS handles the content of .gitattributes automatically. I would suggest using the 'git lfs track' command rather than editing that file directly. Thanks for watching!

@RSSpeacemaker 3 жыл бұрын

@@DanGitschooldude Hey how do you get out of the gitattribute menu anyhow? You exited, but whenever I type it in I get stuck. I can't leave or anything. I'm REALLY new to Git and have been largely using github, but I have not been able to transfer large files at all.

@DanGitschooldude 3 жыл бұрын

@@RSSpeacemaker In the video I'm just opening the .gitattributes file with the text editor vim. In vim you can type escape followed by :q to close the file and return to the shell. Use :wq to write the file and then exit.

@RSSpeacemaker 3 жыл бұрын

@@DanGitschooldude Thank you! I ran into the same problem when it was time to commit, and when I typed qw it allowed me to leave. Much thanks for the quick reply!

@mkklindhardt 2 жыл бұрын

I never get to the 6:05 point where the hook is intercepted ..

@scottisitt 2 жыл бұрын

Is this supposed to be installed globally, or directly in the desired repo? If the latter, do I need to install git lfs every time I create a large repo?

@DanGitschooldude 2 жыл бұрын

Both. git-lfs must be installed at the system level for all clients that need to use it, but it must be configured at the repo level as shown in my video. Once the repo is configured, that information travels with the repo, so it's a once and done process for each repo using git-lfs. Thanks for watching!

@scottisitt 2 жыл бұрын

@@DanGitschooldude Cool. Yeah, I looked at the git docs again and it made more sense the second time. Thanks for clarifying it! This is really cool to know about!

@mkklindhardt 2 жыл бұрын

Hi Dan Thanks a lot! Dan you are a little fast from 5:40 - 6:00 - Unfortunately I miss what you are doing here

@samueltamayogaviria3377 2 жыл бұрын

Hello Dan, I didn't know abot git lfs and i tried to push 2 csv large files and now i can't push any other commit, even knowing that I don't have those csv files anymore on my repository to commit. I don't know what else to do, would you help me please?

@DanGitschooldude 2 жыл бұрын

You probably still have commits on your branch referencing the large files. Look at 'git log' --stat' to see. You'll have to purge those commits by resetting your local branch to a commit before they existed, or otherwise rewrite history to remove them. Some of my earlier videos should help you do this.

@samueltamayogaviria3377 2 жыл бұрын

@@DanGitschooldude Wow, thanks Dan!! really helpful!

@dahwild 3 жыл бұрын

Hello Dan, thanks for all your videos. I have a question, To include this configuration as a dependency, what should be done?

@DanGitschooldude 3 жыл бұрын

I'm not sure I understand what you mean. If you are talking about the Git LFS package, it is installed by name as git-lfs on most linux systems. For example 'yum install git-lfs' on Fedora, and 'apt-get install git-lfs' on ubuntu. Thanks for watching!

@dahwild 3 жыл бұрын

@@DanGitschooldude Thanks, I have another question, I have a repo on github but some files now are huge files and I need to include them in my repo, when I track the files and push this changes to my repo I have an issue "error: GH001: Large files detected. You may want to try Git Large File Storage - git-lfs.github.com." I think that was because I had the same files on GitHub. Base on that, do you think that I need to run a different commant to migrate the files from Github ?

@DanGitschooldude 3 жыл бұрын

"error: GH001" appears to be github specific, likely a pre-receive hook looking for large files and rejecting them so that users can't accidentally bloat github.com. That means your large files were added as normal git files on your local branch, not git-LFS files. You'll likely need to re-base/squash the history to remove the large files from your branch history and then re-add it as an LFS file. Once you do that, the push should work.

@rs-tarxvfz 2 жыл бұрын

Hi, How to deal with large files without using git ? Is it even possible? Because github throws error for files over 50MBs.

@DanGitschooldude 2 жыл бұрын

You can roll your own process however you want, but version control is always better than a custom system in my opinion.

@corndoggydogdog 10 ай бұрын

Do other users need git lfs to retrieve resources? I'm guessing so, but it'd be amazing if only one team member needed lfs!

@DanGitschooldude 10 ай бұрын

Yes, they need the git lfs client. It's so widely used now it's available on practically every platform.

@Pboyyys 2 жыл бұрын

You sound like the buffdudes guy!

@DanGitschooldude 2 жыл бұрын

Lol had to look him up. I agree.

@mrhallodri_ 4 жыл бұрын

how do I do this in an existing repository? I tried some examples but ended up having my files stored in LSF but they're still in the repository.

@nicholasdalhaug481 4 жыл бұрын

I think what you are looking for is git lfs migrate: See link on git-lfs.github.com/

@DeadpoolWadeWilson 2 жыл бұрын

Hi, I have a project that weighs about 40Gb. Do you know any site that lets me host that amount of data and free?

@powil4ss 2 жыл бұрын

does every single file needs to have a unique name?

@DanGitschooldude 2 жыл бұрын

File name uniqueness requirements are the same as the platform git is running on. For example on Mac where the OS isn't case sensitive git may show weird output of 'git status' if the repo contains two paths only differing in case.