Migrate Docker Volumes from one Host to another // backup and restore
Рет қаралды 78,650
Күн бұрынMigrate Docker Volumes, and how to access data inside a Docker Volume? I'll show you how you can easily back up and restore Docker Volumes and move the container data to another host. #Docker #Volumes #HomeLab
Backup in Linux Servers - Docker Volumes, and Databases: • Backup in Linux Server...
Docker Documentation: docs.docker.com/storage/volumes/
My Docker CLI Cheat-Sheet: github.com/christianlempa/che...
Follow me:
TWITTER: / christianlempa
INSTAGRAM: / christianlempa
DISCORD: / discord
GITHUB: github.com/christianlempa
PATREON: / christianlempa
MY EQUIPMENT: kit.co/christianlempa
Timestamps:
00:00 - Introduction
00:43 - How Docker stores data in containers
01:36 - Docker Volumes or Binds
02:05 - Some example Volumes and Binds
03:52 - Access files inside named Volumes
04:21 - Backup, Restore and Migrate data
________________
All links with "*" are affiliate links.
@ruwn561 2 жыл бұрын
You haven’t compressed the tar, you need the z options.
@christianlempa 2 жыл бұрын
Oh damn, you're right! I just updated the cheat-sheet on github ;)
@patricknelson 2 жыл бұрын
I’m so used to seeing “z” there that it stood out to me, too. For beginners: Don’t forget to then also suffix “.gz” to the file name (to indicate that it’s also a compressed gzip file that then contains a tarball).
@WokeSoros Жыл бұрын
How about a video about using Portainer’s backup feature?
@jazzyjay5062 Жыл бұрын
@@WokeSoros Does portioner have a backup feature?
@livingmadscientistlife 2 жыл бұрын
Outstanding mate, from a Computer Scientist and a Professor, you have a great talent and a great team. All my best.
@christianlempa 2 жыл бұрын
Thank you so much!
@dastiffmeister1 2 жыл бұрын
offen/docker-volume-backup is an absolute blessing to easily automate docker volume backups.
@christianlempa 2 жыл бұрын
Yeah that's a great suggestion! Maybe at some point it's worth another video ;)
@quintinfick8050 4 ай бұрын
Perfect Christian, exaclty what I was looking for! Will help me restore a clients data to their VMware VM containers. :)
@simonromanski401 2 жыл бұрын
You can't beat the amount of useful content that is packed into each episode.
@christianlempa 2 жыл бұрын
Thanks mate! :)
@marcelk.4371 2 жыл бұрын
Bester Mann 😀 Danke für all deine hilfreichen und interessanten Videos!
@christianlempa 2 жыл бұрын
Hehe gerne! ;)
2 жыл бұрын
One of the best videos ever! Very useful!
@christianlempa 2 жыл бұрын
Thank you so much 😃
@alfredoalarconyanez4896 2 жыл бұрын
Thank you very much for all your videos, they are awesome
@christianlempa 2 жыл бұрын
Thank you so much! :)
@mukhammadaminabdullaev4768 2 жыл бұрын
Content that you created is much clear and useful ) . Keep making more content. We will be wait content about `Kubernetes` for upcoming sessions.
@christianlempa 2 жыл бұрын
Thank you so much! And yeah more Kubernetes content is coming in the future 😁
@sawwwru 2 жыл бұрын
Exactly what I needed.
@christianlempa 2 жыл бұрын
Thanks!
@bc.export5542 10 ай бұрын
Clear and understandable. U r genius
@christianlempa 10 ай бұрын
Thanks 😊
@cybersecurehacks 2 жыл бұрын
oh yes.. Another awesome one.
@christianlempa 2 жыл бұрын
Thanks again!
@ThomasTomchak 2 жыл бұрын
Thank you for doing this tutorial. The more I learn about docker the more excited I get about what it can do. I was able to get your backup command to work perfectly and to scp that file over to the new host. However, I can not get the restore to work at all. In this video you stopped short of demonstrating the restore, I assume because it was obvious to most. However I'm falling short of knowing what's going wrong. I keep getting PATH errors related to "bash" executable, and I'm not even sure how to track that down since I don't understand what it's trying to tell me. I would be most grateful if you covered restoring using your one-line command in a future video. Thank you for the content that you put out. It's most appreciated.
@yousefpahang 2 жыл бұрын
Yeah Same for me !
@kittyyyyyyyy Жыл бұрын
It feels like clickbait
@arkanjo7509 Жыл бұрын
thank you so much
@joelfankam3719 Жыл бұрын
thanx , youe ease my job
@m0rthaus 2 жыл бұрын
@6:20 I just want to add that the PWD variable is an acronym that stands for Present Working Directory. I hope that helps someone remember the variable function, it helped me.
@alanjrobertson 2 жыл бұрын
Almost 😊 It actually stands for print working directory
@amm1go 9 ай бұрын
how i could make backup and restore when in 1 container i have many volumes. for example container with zabbix which consist from mysql\ssl\snmptraps\keys volumes???
@salimdawod9392 2 жыл бұрын
Thx bro for the great explanation, But i have a point in restore section which I've got while doing it on Mysql docker volume, unfortunately the restore command here won't work when use it on mysql container so I had to create a new volume and extract the backup file into it before running the new container.
@francoisvandermerwe551 Жыл бұрын
Hi Christian, thanks so much for this video, super helpful. I am however not seeing the devops folder in order to test the backup and restore commands you use. Could you please advise?
@rollbacked Жыл бұрын
(assuming everything is binded) Alternatively, can I just copy the directory where I made folders for my containers in a tarball and export my container config as stacks, then in my fresh install - copy over your existing directories to where they once were and re-deploy the containers with stacks (.yaml files) that I exported?
@kalaiyarasansivaji4005 Жыл бұрын
How can we run that migrate command for the container which is having more than one container path?
@mikeheck4998 2 жыл бұрын
Perfect Video. Thank You. I hope is corect wrided. My english is Not god. But i understand you.
@christianlempa 2 жыл бұрын
Thank you! That's great to hear :)
@ronald0122 2 жыл бұрын
You are my favorite bald online friend
@christianlempa 2 жыл бұрын
Haha, thanks mate :D
@incstudios5916 5 ай бұрын
or you could just compress the binded docker volume directory with tar and scp it to the binded location then run the docker stack with the binded volumes, i did that, after getting stuck. It worked beautifully, next create a cron script that backup the data to a backup server. Quick question if there are different addition of data into the container from different servers how do i combine the new changes into one server?
@EduardoRodriguez-fu4ry Жыл бұрын
I'm currently hosting a nextcloud container on a Raspberrypi and want to move it to a desktop computer that will function as my new server (both running Ubuntu). Is this the prefer method to move my data from one machine to another? Would you suggest just backing up the data and create a new container 'From the scratch'? I don't have any 'special' apps on Next cloud. I am using it exclusively as cloud storage
@ipelezikis Жыл бұрын
Can't see cheat sheet of the video at the description link.
@hatt5350 Жыл бұрын
Good day Mr. Lempa, I hope you are doing well! I have a question for you. Suppose I had a Docker Swarm that, at the moment, consists of 4 raspberry pi 4s each with a 500gb m.2 SSD drive plugged into them. Suppose further that I am also running Portainer to manage my containers. Now suppose that what I want to achieve is data persistence across these 4 Nodes so that if, say, the node containing the data for my Gitea instance goes down and maybe even stays down, another node should start up the service and also still have the necessary data. Any idea on how I could achieve this? You see what I mean?
@hamhumtube Жыл бұрын
what if container has more CONTAINERPATH?
@hansphung 2 жыл бұрын
This is really good, thank you :). Is there a way to migrate or clone an app running with docker-compose, which has both named and bind volumes?
@christianlempa 2 жыл бұрын
Thank you! :) You can use the same technique to backup binds as well. It doesn't make a huge difference if the container was deployed via dockercli, portainer, or docker-compose.
@theitzone01 Жыл бұрын
My Raspberry Pi Lite OS runs on a 64GB Micro SD card boot drive and I have a secondary 2TB SSD to store data in. I plan to install Jellyfin and PhotoPrism and NextCloud which requires installing Maria MySQL Data Base. What has me confused is how I specify all docker data storage for PhotoPrism, JellyFin and Nextcloud to take place on the 2TB SSD drive and not the 64GB Micro SD card boot drive? Do you have a video that shows correct docker Volume syntax that enables data to get stored onto a 2TB SSD secondary data drive?
@giftcp82 2 жыл бұрын
HI. I am getting the error "docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "bash": executable file not found in $PATH: unknown." when I try to restore. Please point me in the right direction
@christianlempa 2 жыл бұрын
Haven't had this issue before unfortunately :/ Maybe you can ask on our discord if someone has seen this before?
@aokijikuzan6453 2 жыл бұрын
Has the same problem, found the solution; Instead of docker run --rm --volumes-from CONTAINER -v $(pwd):/backup busybox bash -c "cd CONTAINERPATH && tar xvf /backup/backup.tar --strip 1" use docker run --rm --volumes-from CONTAINER -v $(pwd):/backup busybox /bin/sh -c "cd CONTAINERPATH && tar xvf /backup/backup.tar --strip 1"
@giftcp82 2 жыл бұрын
@@aokijikuzan6453 thanks I will try that
@TheDarkLordXXL 2 жыл бұрын
@@aokijikuzan6453 nice it worked for me, THANKS !!!
@midkaa 2 жыл бұрын
Just 30 mins ago i was thinking about backupping my docker databases
@christianlempa 2 жыл бұрын
Haha, perfect timing :)
@Christian_Ky 2 жыл бұрын
Hi @Chris, just wanna let you know, u had exposed you admin token @3:40 Thanks for the superB video!
@christianlempa 2 жыл бұрын
Thanks! That's on my old server, but thanks for the hint!
@ClAddict 2 жыл бұрын
One thing I don’t understand about Volumes is they seem built for hello-world sized projects on a single drive systems. Volumes seem neat, but a restricted alias of Binds which doesn’t allow you to specify which drive you want data stored. Most my docker use cases require data storage that exceeds my OS drive size, but that’s where volumes stores data because it’s built into docker, not on my Data drive. Is there something I’m missing about Volumes that would make them useful for larger storage needs?
@metaversegaming1577 Жыл бұрын
Step:1 Stop docker services sudo systemctl stop docker sudo systemctl stop docker.socket sudo systemctl stop containerd Step:2 make new root directory structure sudo mkdir -p /new_dir_structure Step:3 sudo mv /var/lib/docker /new_dir_structure Step: 4 Create daemon.json abd add the following sudo vim /etc/docker/daemon.json { "data-root": "/new_dir_structure/docker" } Step:5 Restart docker (Give 777 permission / change root ownership of the folder if needs) sudo systemctl start docker Step:6 check/verify the changed location by docker info -f '{{ .DockerRootDir}}'
@gdevelek Жыл бұрын
How reliable is the backup if the container is running and accessing it while you make the tar? Hmmm....
@sugizoakino2219 2 жыл бұрын
Is there any way I can do this on Heroku? It accepts commands by Shell. I'm using a bot, but the docker image was archived by the admin. Can you help me? I don't understand any of this. I just wanted to backup this docker and upload on dockerhub to help others who don't have a bot working before adm archived everything. Could you help me, please?
@camixxx 2 жыл бұрын
Looks like you need to update your cheat sheet repo's readme cause a lot of the links are broken.
@christianlempa 2 жыл бұрын
Yeah you're right :/ Still on my todo list
@Harmacist_TX 2 жыл бұрын
I'm new to docker, but could you not mount the volume inside a path instead of a named path and just use normal methods to back it up? Edit: Found your content the other day and want to give you props for your information and delivery.
@christianlempa 2 жыл бұрын
Thanks mate, yeah you can simply work with just binds, but I thought it might be useful for people to know how to migrate in case they use named volumes.
@TariqSajid 2 жыл бұрын
can you make tutorial on how to run multiple services in same docker container ?
@christianlempa 2 жыл бұрын
Not sure what you mean by that.
@tromtrom009 Жыл бұрын
Hi, wenn ich das auf meinem raspberry pi ähnlich mache wie du beschreibst (aber ohne busybox) und am Ende auf dem neuen Raspberry Pi "sudo docker load -i image.tar " eingebe, erscheint immer: no such file or directory an was kann das liegen? Ich starte meinen alten pi von SD karte. an USB hängt die neue M.2 SSD, auf welcher der neue pi schon funktionsfähig installiert wurde und welche einfach als usb-stick gemountet ist. dort kopiere ich die tar-datei hin. Dann starte ich von dieser SSD und will die tar mit dem genannten befehl wieder verfügbar machen..... :(
@ManfredRotgers 2 жыл бұрын
Hi, great video thnx I have a nginx container on a pi that I want to move to docker on proxmox, but this container has multiple paths. do I need to run the backup command multiple times? Host/volume Path in container /home/pi/nginx/data /data /home/pi/nginx/letsencrypt /etc/letsencrypt /home/pi/nginx/config.json /app/config/production.json
@christianlempa 2 жыл бұрын
If you can back it up on the host with one command thats fine, guess it isn't a DB.
@ManfredRotgers 2 жыл бұрын
@@christianlempa in the CMD you explain, only one path, I have three different paths. And the db is a separate container
@maddish1984 8 ай бұрын
Hi there - I'm struggling to get my portainer instance to run via the web GUI- I have no idea how its become the way it has, i've not changed anything's actively and can only assume an update has been done passively. I've had a look around and I can't see any ports applied to the container - this maybe a red-herring but wanted to address this first just in case. "docker ps -a | grep portainer 33f01ee62da7 portainer/portainer-ee:latest "/portainer" 3 months ago Restarting (2) 36 seconds ago portainer" This is what im seeing... Please help if you can.
@fauzifauzi2129 2 жыл бұрын
Hi there, I tried restoring compressed backup with your cheat sheet, but it doesn't work with busybox image. Then I tried using ubuntu image and it works.
@christianlempa 2 жыл бұрын
Can you open an issue on the github repo? That would be nice, I'll take a look then.
@shashankmarri1056 2 жыл бұрын
Hi, when I try to generate the tar file, I get this error can't open '/backup/backup.tar': No such file or directory could you please help.
@Seff2 Жыл бұрын
Same problem. Didi you find a solution? Edit: Never mind, I was using the wrong docker container in the command.
@damianortiz Жыл бұрын
no encuentro el Cheat-Sheets del que hablas
@arwendrew3393 2 жыл бұрын
What about if you have a couple containers being ran by docker-compose?
@christianlempa 2 жыл бұрын
Should work the same way, with docker-compose.
@arwendrew3393 2 жыл бұрын
@@christianlempa What if there is no mount point when I inspect the container?
@gavin4981 Жыл бұрын
Dust against the mailer in in time setting
@hohojimmy4443 2 жыл бұрын
It’s very simple,police finding me now
@moroccan2739 2 жыл бұрын
I can't find your devops cheat-sheets. Its just me?
@TheFriso1234 2 жыл бұрын
Why not just use something like Duplicati for this?
@christianlempa 2 жыл бұрын
The main focus is on migration, so it's important to know how to do it manually. But Duplicati is also a topic that I've covered in the past, therefore I've put it in the description and as a "watch next" suggestion ;)
@TheFriso1234 2 жыл бұрын
@@christianlempa I absolutely found value in the video. It's crucial to understand how these things are done manually for any kind of real understanding of your system. Thank you for the reply.
@jimmertzos3967 Жыл бұрын
Github Link doesnt work, the page you showcase is nowhere to be found !!
@damianortiz Жыл бұрын
Same
@Sagsibil24 2 ай бұрын
You run an image not a container. You start a container. And --rm remove the container file system after using it. It doesnt remove other containers
@christianlempa 2 ай бұрын
True, thanks for sharing
@Seff2 Жыл бұрын
you really should have taken the 3 minutes to also explain how to restore an image with the backed-up file. Becuase the command in your cheat sheat is really confusing, and its unclear if i have to create the image first, or what container names I should use in the command....
@christianlempa Жыл бұрын
Good point, I even looked at my cheat-sheets yesterday and had a hard time understanding the command again :D I'll give a quick update with more explanation on the docs, soon. Thanks for the headsup!
@t288msd 10 ай бұрын
@@christianlempa I created a pull request for this today (wth everything but the command to be added because I, too, don't know)
@howling-wolf 4 ай бұрын
Please do not use scp anymore. It is deprecated. Rsync is a more robust, faster and more reliable solution for copying files between servers.
@christianlempa 4 ай бұрын
Thanks for the heads-up! I'm going to have a look :)
@kittyyyyyyyy Жыл бұрын
Sir, half of you video got deleted in editing.
@blackburn116 Жыл бұрын
For Windows users without WSL2 replace $(pwd) with %cd% in the backup a container command: docker run --rm --volumes-from CONTAINER -v %cd%:/backup busybox tar cvfz /backup/backup.tar CONTAINERPATH