Thank you Christian for going the extra step with the Databases. This kind of content is really missing on YT as most tutorials cover the "easy" stuff.

You’re doing great work man! Your videos are so easy to follow and helpful! Thanks for all your efforts on this channel :)

You earned a new subscriber. Your videos are better than a lot of tech guys out there. Respect from India🙏

SOLID videos. I've watched dozens of yours. I hope your channel explodes!

I was literally today contemplating how to backup my databases on my home server. Thank you for posting - your timing was perfect as was the info.

This video is great! I didn't know that I can back up docker volumes witt Duplicati. Thank you Christian!

A lot of thanks go to you for presenting this complete backup solution!

It amazing how much my home network project with docker, docker compose and all the containers I run line up with the videos you put out. Sometimes I get the feeling you are seeing my project notes. Just this week I started fiddling with backups and what do you know a new TDL video to explain exactly what I need to now. Keep up the awesome work man. Your videos really help me enjoy the process of creating and managing my own setup.

Database backups! I’ve been meaning to dive into this, thanks for the overview of how you’re managing this!

Just wow. What a video. Didn't know of Duplicati. This is just what I was looking for. Thanks!

Thanks for the demistification of backups. Great video.

very clear and well explained, thanks for taking your time making this video.

really love your channel Christian! thank you

So useful, thank you so much Christian! - The DB backup script, I can just read it.. but couldn't have written it.

Many thanks Christian, happy to sub!

Awesome video! Thank you!

The db backup script blows my mind! Thanks! Please more videos ámbito db restore!

As a Newbie I am able to understand and follow your explanations. Even though it's hard for me to deeply understand what's going on, I can still pickup on things to learn next. I would love to see a Linux survivor's guide by Christian TDL. I'd watch a nice, long, in-depth Linux introduction course by this guy.

Awesome Christian

Thank you for this great post, grateful!

very useful video!

Amazing, more video about Linux Administration bro, thank you:D

From the doc: > The Duplicati project was inspired by Duplicity and had similar functionality until 2008 I believe duplicati is an Italian word and you spell du-plee-ka-tee

You could also use the "--run-script-before" in Duplicati to run the database backups before the duplicati backup. Also you pronounced it wrong. Duplicity != Duplicati. there are 2 different products. Anyways - informational video.

Thank you Christian for this awesome video, it is really helpful! With the help of your tutorial, I am now able to backup the docker containers for my Zabbix installation. Nevertheless, during the setup process of Duplicati and your db backup script I had to overcome two hurdles and I would like to share how I solved the different challenges. The Zabbix installation has four different containers and the images are called "zabbix/zabbix-agent", "zabbix/zabbix-web-nginx-mysql" "zabbix/zabbix-server-mysql" and "zabbix/zabbix-java-gateway". In addition to these four containers, I run a mariadb container for the db. When the first line of your backup script (docker ps ....) is executed, not only the mariadb docker is in the output, but also "zabbix-web-nginx-mysql" and "zabbix-server-mysql", which both do not contain a database. So I had to adapt this line to the following: CONTAINER=$(docker ps --format '{{.Names}}:{{.Image}}' | grep 'mariadb' | cut -d":" -f1) With this, the output of the docker ps command is only the mariadb container. For the execution of this script, I use a cronjob. I also tested the restore of the mysql dump and since it is not explained in detail how the mysql dump is restored, here's how I do it: - Recreate the docker container with cli command or docker-compose file. - Mount the Backup directory from the backup script in the mariadb container (e.g. /appdata/mariadb/backup:/backup) - Go to the backup directory and unzip the dump file: cd /appdata/mariadb/backup && sudo gunzip name-of-dump-file.sql.gz - Connect to mariadb docker console: docker exec -it -u 0 name-of-mariadb-docker /bin/bash - Restore the mysql db with following command: mysql --user username-of-database --password -p name-of-database < /backup/name-of-dump-file.sql After that, I restart all Zabbix containers and also the mariadb container and everything is fine. Once again, thank your for this awesome tutorial!

Very good video, I like to see other opensource options preferably bacula in docker please

Duplicati/Duplicati is the official maintained Docker Image on Docker HUB.

Super video ! Thanks a lot. I've a just a question: What would be the fastest protocol to backup btw two server ? (Rsync is decent but for terabyte of data it get tricky, and on replicate there is a lot of options: SFTP/ftp etc...) Also an next idea of video would be disasters recovery. After a breach or a physical lost :)

I seem to have a problem with your script for restoring containers from backups. I used the scp command to send a backup of the bitwarden file to the /root/data/docker-data/bitwarden/backup.tar path on my other server, and then executed the following command, docker run --rm --volumes-from bitwarden -v $(pwd):/backup busybox bash -c "cd /data && tar xvf /root/data/docker-data/bitwarden/backup.tar/backup/backup.tar --strip 1" which reports an error, docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "bash": executable file not found in $PATH: unknown. ERRO[0000] error waiting for container: context canceled and I don't know how to fix it I don't know how to fix it, can you help me to see how to deal with it?

hello christian, great video! thank for this detailed explanation of database backups. i have a integrity check enabled on my nas and i read that this should not be enabled for folders that contains virtual machines and databases (maybe its the same reason like you explained in your video). so that means i can not protect my vms and databases from bit-rot or is there a other solution? what would you do?

Awesome tutorial! I had some changes on the SH file because now is Vaultwarden. I don't tried yet to restore the relative DB with the sql file to another Vaultwarden docker ( in order to simulate a disaster or just the brake down of my PI). I not that the SQL file is encrypted so I think I will have to decrypt that somehow, right? :)

Please do bash scripting tutorials, we need those a lot. thanks for the great job you are doing in your wonderful Chanel Christian 😍😊✨😎🙏

Hi Christian, it's an awesome explanation and video. However I think it's best to add --single-transaction --quick during the database dump especially on database that kept changing frequently especially when there's cronjob processes running (CMIIW)

Good video. In my opinion it misses one part: example of how to restore the backup. Also some backup scenarios can be done using btrfs filesystem snapshot (likely works with zfs, possibly some other filesystems). I already had some fun migrating few docker volumes and part of docker-compose from OpenSuse (docker) to Fedora (using podman). Hopefully podman has a command for importing volume from tar file.

yes please can you, do that video of comparaison between different solutions :'(

Great work and clear instructions. I use duplicati with BackBlaze on my server and currently rework my infrastructure with new servers. Good idea to grep the ROOT password out of the Container itself not having them at all in the script, will probably use your bash script for MariaDB . Learned something new about using systemd as timer , instead of regular crontab. As addition a quick guide on stopping containers (like gitea) before doing volume backup would great for others. Otherwise I assume you run it lock errors or data consistency errors ,too. Example I have is like Gitea, HomeAssistant (at least the .storage folder)

Very very cool Video. I want to suggest a Video about Automatic Backup Monitoring. Because ITS one Thing to have Automatic Backups, but another thing how to Monitor If the Backup hast been run completely without Errors. I would Love to See a Tutorial about that. Maybe with suggestions for Backup Monitoring ways (Email, snmp, log File Check...) Keep Up that amazing Work !🎉

Great video which helped me to optimize my backup. However, you could add something about the 1-2-3 strategy. E.g how to use it to save offline backup other places.

Hi Christian, will you please consider a companion video on restoring these database types (mariadb, mysql, sqlite) from the compressed file? There are different methods out there, but it would be great to hear your preference.

You are my God!

have you consider running mariadb data volume on a optane drive or in memory then running online incremental online backup routinely dump to a non-violate storage that also chks if backup is valid? cause when i playin around my photoprism instance , i found when db is runnin in memory is quite fast and rarely falls into their shitty sql-lock situation. they really should upgrade their ORM to gormv2

Hi, I have been following you for a long time and i appreciate a lot your contents. You are very clear and professional. I'm doing master thesis about docker containers and i'm dealing with redundancy in docker containers. Can you provide me more informations, books, readings, videos and so on, about this topic? Thank you in advice for the help. Bye!

Take a look into restic-backup

Thanks for this video, Christian! Two questions: 1) Why are you setting up individual volumes for your services? Wouldn't it be easier to just backup the entire /var/lib/docker/volumes/ directory and exclude stuff you do not want? 2) Does it make sense to also backup Duplicati's own data this way? I'm asking because when my system breaks completely, which is not unlikely for a Raspberry Pi running off an SD card, I would need a running instance of Duplicati to restore data from my backup.

this is amazing but i understand some of it basically i want a backup mysql docker container and run to another server so how i do it in ubuntu using command please explain to me 🥺

Awesome, do you know how to not expose port of the dockers when we are using nginx proxy manager like with Traefik

Right before it happened I thought to myself "Man that cap looks like it's about to pop.", then a second later it did. Stuff of nightmares right there.

You must try acronis cloud backup very easy to use :)

Hey guy this anachronistic (very old) dotnet thing is even almost 2 years later still in beta and the devs really struggle to migrate to ANY newer dotnet version. Coding wise this thing is a blast from the olden pre Natella evil Microsoft era.

Duuude, duplicati just saved me about 4-6 hours of lost work! Somehow I lost a large amount of untracked files that I had stashed in git, while I was doing a partial revert and interactively staging hunks and then popping the said stash after committing (I was distracted, I still don't know how they were lost). How could this happen to *me*? I "never" lose files :) I simply restored last night's backup, and copied them back into the repo. Done.

Can you create a video on creating a WireGuard network between servers at different locations so that they can communicate directly with each other while not disturbing their individual internet and container connectivity?

Is it possible to backup whole system? so i can restore it if something bad happends?

Maybe you can think about doing a video for setting up an external drive maybe on a friends Fritzbox as a target of a encrypted external backup target?

Would you recommend this if I were to backup the whole system? Linux and or Windows based? Where i can pickup right where i left off on a new machine?

This is great stuff but the volumes section is confusing me. A more detailed look of what is going on here would be helpful, it appears you have created a volume called 'duplicati-config' before setting all this up?

Hi, christian, I've been following you for a long time, I like your videos and what you do. I can suggest you something to see if you can make a video talking about RSNAPSHOT. Thank you very much for your time that you dedicate a greeting from Peru.

Hey, I get the error "mysqldump: Got error: 1045: "Access denied for user 'root'@'localhost' (using password: YES)" when trying to connect " 2x times when I execute the nextcloud script for the database. Have someone an idea how to fix this?

Great video, just would have been extra awesome if you could have finished with showing how to restore the databases. I figured out MYSQL, but I can't figure out sqlite for vaultwarden yet. I made the appropriate changes to your backup script, from bitwardenrs to vaultwarden, and it appears to backup, but I need to restore to test this. My current backup system for vaultwarden is, I'm just copying the entire folder from my synology nas with syncthing to my ubuntu docker server, and running a script to copy over the vaultwarden folder every 30mins. It appears to work perfectly fine with both source and destination containers running, but I know it's not best practice. I do this incase for some reason my synology goes down, I can just modify the URL in my app to access the backup vaultwarden on the ubuntu docker system, and still have access to my passwords. If you could just reply to this with the correct command to restore the vaultwarden database, that would be great.

What about a comparison of other backup solutions? Have you looked at restic, duplicacy, borg, arq, kopia? You seem to show the solution you picked, but not why you picked it. I'd love to see a breakdown of the pros / cons of the most popular solutions out there these days.

Any vedio about web based office like open office or something alternative to google sheets or 365 ???

Great video, learned a lot from it. But sadly i can not use your db backup script, because i use one maridb docker with 5 databases from differen dockers. And i realy don't understand how i can addapt your script to make a backup for every database.. :( Duplicat is great, will be my daily drive for backup.. Keep going with you videos, they are great..

Hey Dude, I have been following lot of your videos. You are doing an amazing job helping millions of IT professionals. Ah i just need an advice if you have some time. I have a cluster of 3 Proxmox servers which are on Hardware raid. i do not have ZFS storage and hence i am unable to configure HA in Proxmox. I have few VMs (Linux, Docker) which are having their disk on local storage on one of the nodes of Proxmox. I wish to understand is there a way i can setup a HA (using some clone/rsync? or sowhere the data from VMs of Node 1 or Proxmox are cloned to another Node. I may setup to clone it once a day, that should be fine but do you think any tool can help me on this. I do know about keeping the disk on shared storage (NAS) but then again that is a single point of failure. I am looking for a solution where if one of the node goes down, i can use the copy on another node to continue doing work.

I believe it is pronounced "Duplikati"

Nice video. I hope the SQL export you showed us at 19:30 is not from your real Bitwarden Database but just a dummy Installation! Otherwise you might want to change some of your passwords.

Fantastic! The script works well! The command: sudo systemctl enable db-container-backup --now does not do anything for me! That is a bit concerning! I seem to have the directory entered correctly and the user. Has anything changed?

Why does your website redirect to your git-hub page?

You are creating database dumps with date and time in the file name. How are you doing incremental backups? How does your smart retention keep yearly, monthly and weekly versions of the files?

thank you for the video but the issue with Duplicati is that it is "file" based and not block based. Proxmox backup server in my environment is a complete block-based backup.

I've come to this video as a person frustrated with Duplicati as my upload speed will NOT go more than 1.5MB/s (12Mbit), even though I have 500Mbit available. It's running on a Raspberry Pi 4 and SSD combo, and I've faced this same limitation whether uploading to Backblaze B2 or OneDrive, with or without encryption, and the Pi is hardly with any CPU activity, and all via Docker. Any ideas what I'm doing wrong? I never noticed your upload speeds.

The problem i have with duplicati is that the server which i want to backup needs access to the backup storage, instead it should be vice versa for security reasons. The Backup server should be able to reach the server, the server should not be able to reach the backup storage.

please make comparison video

couldn't we in theory just backup the data folder which was used for any database setup in docker container?

Anyone can tell me why he talk about duplicity and show duplicati in the screen? They are two different products.

Did I miss the part where he explains how to do incremental backups of a 100TB volume to a volume spanning set of 10TB USB HDDs?

but isn't this makes the volume and the database somehow have time discrepancies? as the database is backed up not at the same time of the volume backup run.

What is it now, "duplicity" or "duplicati"? These are not the same, but do the same...

its hard for me i need to take auto backups for docker database volumes only :D

Nice script for DB backups, but if on one container we don't have the name of Database in docker env the script will not work :( I know that the best option is one DB docker to one app, but propably You can add check [[ -z "$MYSQL_DATABASE" ]] && MYSQL_DATABASE="--all-databases" after geting from docker variable MYSQL_DATABASE, but for sure there is better way to handle it :)

time shift

Does anyone have tried Bareos? 🤔

so 3rd ty - I hate KZfaq Spam recognition which sees to contains words which are part of this software...

Duplicity is very easy to use but very slow to restore your backup. So be ware.

"doop - lic - ott ee" not "doop - liss - it - eee"

Your video is about Duplicati but you keep calling it "Duplicity" which is another different backup software. It's confusing and a common mistake I've found in other similar videos by different creators.

duplicati seems good, but restoring on a different computer ("disaster recovery") is insanely slow. like 20 hours to restore 1/4th of a 160GB backup. any ideas on how to enable faster restores on a different machine?

You are creating database dumps with date and time in the file name. How are you doing incremental backups? How does your smart retention keep yearly, monthly and weekly versions of the files?