Great video, as always, Gary! I'm just beginning to learn BASH scripting and terminal commands, and so I *understand* what you're doing, even though I could never have come up with this process on my own.

Excellent info on every broadcast. And quick-moving without the fluff. Thanks.

Fantastic "one liner"! Just used it to find a ton of dups in my Documents folder - thank you!

Man, all of your videos are so good!!!! Thank you very much!!!! 👍⭐👍⭐👍⭐👍⭐👍⭐👍⭐👍⭐👍⭐👍

Thank you so much Gary! Very helpful!

Thank you for a very useful video, succinctly explained.

Just saved me so much time. Thank you!

Gary you ROCK

Hi Gary! Got my first mac 2 months ago after 20 years of using Windows. So now I'm blazing through your videos in search of useful tips and I really tell that you know a lot and you are a really smart guy! Meanwhile, in this video you show up a terminal script that search for the duplicates but it is extremely inefficient to me. It works for some small amounts of files but will take a really long time if you run it for hundreds of gigabytes of data, because it will take a ton of processing power and time to hash all the data. It will also get WAAAAAAY slower if you run same task for the files that are stored on the slow HDD storage. I'm talking hours and even days slower. It also put unnecessary wear on the storage since you have read every single byte of data to hash everything. You can optimize this process to be way more efficient by searching for the files of the same size first, then calculate the hashes of the same sized files and then compare the hashes. This way you will be hashing just the files that are the same size (files have to be the same size to be duplicates) saving just absurdly huge amount of time and resources. I'm not good enough with terminal commands yet, so I can't offer such command myself but you probably can remake it in the described way and make another video about it :) Thank you for your videos!

You are awesome! Thank you so much for this video!

outstanding! thanks Gary

Great video 🌺

You Sr. Have saved me some much time and space on my hard drive you are a life saver

Yes very useful Gary !

Unbeliveabe easy, and very useful.

This guy has a gift of teaching things, he makes things people like mnyself have no knowledge on and making the topic extremely clear and understanable. Many thanks @macmostvideo

Brilliant!

Neat, neat, neat! Thanks

Thanks!

Thanks bunches

Excellent video, as always. I would not stop on that, though. I would (and I will) make an application with this code using Automator.

Every person who owns a mac needs this as a pre-requisite. TY

A very useful and informative video tutorial today! Thank you, Gary! 👏🏻❤️

impressive. i have a demo copy of macpaw gemini. it will find the dupes but wants an activation code to finish the job so I was just using it to 'find' the duplicates. Yours is much better. Back in the 90s I would use tax software to do all the calculations but you had to pay in order to print the tax return. So i just copied all the numbers by hand to the paper return.

Gary, Great content and delivery! I am trying to figure out how to find duplicate files on multiple external drives. I want to clean out unnecessary files to make room for TimeMachine to do its thing. Any help would be appreciated. Thanks

I found it thank You

Hi, that's pretty cool. Do you think there would be a way to print /send the information to a file (in it is the checksum and filename at least), then modify that file to delete any of the obvious files? If there is a way, would you recommend it? Also, how would you do that? In DOSshell I can do all that but having long filenames makes it so much more specific and dangerous, at least in DOS.. Thanks

Hi Gary, thanks for your very helpful videos. I find my self in a situation where I made several copies of my entire hdd as a backup. Now I'm making a big storage and all my old hdd's will be copied there. This will take mote than 10TB of data and I know there is a lot of duplicates. Is there a way how to batch select all the duplicates and delete them? If I'm not mistaken I can't do anything like that in Terminal. But could this task be done in Automator?

Wondering if there’s such thing as a batch file on macOS, this way you could just drag the file into the directory you want and search it easily, no need to go through terminal and find the command to copy paste

Great video Gary ! How can I move those identified duplicate files into a separate folder where I can manage/remove files manually? Thank you

Hello Gary and thank you for another great video! I am wondering if there is a way to add a command that will delete the duplicates? Say I have 100 Giga of photos and backup photos and there are a lot of duplicates and I just want to keep the ones that are not duplicates. Thank you again, Yossi

on you mac the get info. How do you uncheck Multiple Boxes ? like The Locked in photos can you do a video on this

Great video, as always. BTW, on partitions formatted with APFS, duplicates don't take up extra space.

Most duplicates are going to be found in Downloads caused by people clicking on the same online document to view it. The filename of the duplicates will be followed by a number, like -1, -2, etc., or like (1), (2), etc.

duplicar]te file finder app is my best friend there!

Hello Gary, thank you for the video. I have a MacBookPro with El Capitan. What App could I use to find duplicates in the PHOTOS APP? I have too many pictures to do it one by one. THanks for your help!

A fantastic way to do it for free I use a paid version "Gemini". I did not realize the command line could do it thankyou.

When I back up my Mac using Time Machine is it duplicating the same files on the external hard dive every time? I have a 1 TB external hard drive that I only use for Time Machine. It is is full and my 500 GB Mac hard drive isn't full.

Hi Gary, I am using Numbers for my invoicing. Could I ask how can I add up only one cell in each sheet with number which indicates service fee? By end of financial year, the number of the sheets (invoices)could be over 300. Your help would be very much appreciated. Regards, Frank

can you add that string to the description?

So what do you do with the duplicates? Do these duplicate files affect performance in any way? I have what appears to be quite a few duplicates and wonder what I should do with them. Thanks man!

Hi Gary, I have a folder 📁 on my MacBook Air which contains family pictures and videos. The 📁 location is on the desktop and it's very large. I want to remove the duplicates before I copy the folder to the external hard drive. Can you please help me who to remove the duplicated photos and videos from a folder on MacBook ( please note that the files are not on photos app). Thanks for your time

3rd party software seems like a much more convenient solution?

Ok I wished you had included the terminal command in the description.

Does this work also with an external drive?

Hi. I thought you said the code would be in the notes for the video, but I don't see it. Is the code available somewhere? Thank you.

I either get a "permission denied" message, or nothing happens. In the "nothing happens" case, I have duplicated a file intentionally before starting the terminal command. 🤔

Gary I was waiting to see if you show a way to actually delete the duplicates. That is really the whole purpose of finding duplicates. I have1000s of pictures and I would say 30% of them are duplicates. I know there are software that you buy that do that but does apple have a way to delete the duplicates?

now how do you delete the 2nd copy and keep just one

why so hard a simple task that could be in finder easily?

so this would not work if you had two folders in different locations - say one on the desktop and one on a external drive. you'd need to copy the desktop folder to the external drive then run the command on the external drive. once you had the duplicates you'd need to check what folder they are in to make sure you deleted the old desktop version (or which ever version you did not want) unless I'm missing something with the find . command

I would use md5 instead of cksum. Even sha256 as md5 is considered weak.

Can you post the command please?

My bad habit is to take screen grabs e.g. of photos and then, a few days later, I inadvertently repeat this and end up with duplicates. I've used third party apps to find duplicates and they're pretty good. However they don't always pick up all duplicates e.g. if I deliberately save two copies of the same photo as a test to see if they're found. Looks like your method may be worth trying.

Confusing

Why am I getting "grep: -: No such file or directory" on this command?????

This would not work with duplicate photos as there are so many photos taken with the same file size.

I FOUND DUPEGURU...

Way to advance for my Brain.

Hello Gary, thank you for your great tutorials which have often been very useful for me. Now I am stuck with a duplicate file problem and I hope you can help. In short, I have a mid 2015 iMac running Big Sur. I use Mail for emails and I have nearly 20 years of email messages stored in different mailboxes. When I use Mail to look inside any single mailbox (e.g. WORK_2020) I can see a number (e.g. 3000) of email messages with their respective attachments. However, when I look at the same WORK_2020 mailbox folder within the ~Library/Mail/etc directory I find a much bigger number (something like 4800 or more) of .emlx files, corresponding to multiple copies of each email message. I suspect that this is what slows the Mail app and causes occasional crashes. My question is: is there a way to find and delete these duplicates ? I will add that I already tried a number of duplicate finder apps but none was able to solve the problem. Actually, only the "Duplicate File Finder" app does spot some but not all duplicate messages while other apps do not find any duplicate. Of course I followed your instructions in this video (kzfaq.info/get/bejne/o9t0m6ydvMmYiZc.html) but it does not work for me. I believe that in the case of .emlx files there is some additional complication, i.e. each multiple copy of the same email message has a different file size and different checksum. I can only confirm that these are the same email message by opening each one separately in Mail and check that the sent and received date and time are the same. Sorry for being so long. I hope you can suggest some solution. With my best regards Stefano Conti