The Docker HOST and MACVLAN Networks - Docker Networks part 2

Рет қаралды 26,933

Күн бұрын

MANY THANKS TO ALL MY PATRONS on / onemarcfifty !!!
Part 2 of the Docker Networking series. The docker bridge network is fine for most tasks with docker containers. In this episode we will use Portainer again to define docker containers using the docker host network and the docker MacVLAN Network.
Breakdown of the episode:
01:26 the docker host network
03:30 Performance considerations: Host vs. bridge network
08:10 the Docker MacVlan Network
Portainer Documentation is here: documentation.portainer.io/v2...
sudo apt install docker.io
(on a Pi it may be called docker-ce)
One liner to start portainer:
docker run -d -p 8000:8000 -p 9000:9000 --name=portainer --restart=always -v /var/run/docker.sock:/var/run/docker.sock portainer/portainer-ce
(In portainer, click on "Networks" on the left, you'll see the networks)
(on the command line)
docker network ls
Commands that I use in the video:
(Pulling the image and running it)
docker run -d nginxdemos/hello
with the host network:
docker run --network host -d nginxdemos/hello
creating the macvlan network in a one-liner:
docker network create -d macvlan -o parent=eth0 myMacVlan
In order to execute shell (/bin/bash) in the container on the command line
docker exec -it CONTAINERNAME /bin/bash
Commands I type INSIDE the containers:
ip addr
ping www.google.com
apt update && apt install iperf3
running iperf3 in server mode
iperf3 -s
running iperf3 in client mode
iperf3 -c (serverIP) -p (portNumber)
apt update && apt install iproute2 dhcpcd5 iputils-ping
Please support me on patreon: / onemarcfifty
My youtube channel: / onemarcfifty
Marc on Twitter: / onemarcfifty
Marc on Facebook: / onemarcfifty
Marc on Reddit: / onemarcfifty
Chat with me on Discord: / discord
Licence-free music on / Lizenzfreie Musik von www.terrasound.de/lizenzfreie... music on / Lizenzfreie Musik von www.terrasound.de/lizenzfreie...
The docker logo from Wikimedia Commons By dotCloud, Inc. - File:Docker (container engine) logo.png, Apache License 2.0, commons.wikimedia.org/w/index...
The Thumbnail of this video is under the Apache 2.0 License: www.apache.org/licenses/LICEN...

Пікірлер: 110

@bitterrotten 3 жыл бұрын

Thank you for making these! I used your macvlan suggestion in another video’s comment section to host two instances of forked-daapd on one server but this is above and beyond. Very informative and I’m looking forward to what you’re going to do next.

@OneMarcFifty 3 жыл бұрын

Thanks a million for your nice feed-back! Stay tuned ;-)

Жыл бұрын

This 2 part tutorial are the first of your videos I watch. I have been self-training in docker for the last few weeks and these are the best introduction to the topic of docker networking I have found by far. Love the hands on approach with exercises that really help get the notions clarified. Bravo! subscribed now.

@OneMarcFifty Жыл бұрын

Hi Cédric, thank you very much.

@jairunet 2 жыл бұрын

Fantastic, it will be great to know more about the docker-compose features as it seems like a way to automate the deployment of the docker images. I also agree with creating a tutorial using two different physical hosts, docker hosts, and take it to the next level by simulating a larger project of containers. Thank you very much Marc! all the very best and until the next one.

@OneMarcFifty 2 жыл бұрын

Hi again, and thanks for your feedback ! Yes, docker compose is a great way to bring up the whole stack (Containers, network, Volumes...) at once - Especially great if you get a ready-made compose yaml file for the purpose. Kind of a docker version of turnkey really ;-)

@davidcrane7397 3 жыл бұрын

Really excellent videos - I finally think I'm getting to understand docker networking! I find your explanations and demos very clear (even if I have to watch them a few times). There were a couple of comments about you going too fast, I have to say I didn't find that: I stop try what you've suggested and replay - that works for me.

@OneMarcFifty 3 жыл бұрын

Many thanks for this feedback David - searching the right speed and depth is an eternal journey for me ;-)

@arnoldadame78 3 жыл бұрын

Love your videos! Keep up the excellent work!

@OneMarcFifty 3 жыл бұрын

Thank you very much!

@jeytis72 3 жыл бұрын

Of course yes! We are all interested into your next docker networking video!!

@OneMarcFifty 3 жыл бұрын

Thanks for the feed back ! I'll keep working on these then ;-)

@turbochip1208 10 ай бұрын

I've been fighting with this for 2 full days. Thank you, Thank you. Now I have the application that "HAS" to be on the same network as the host, there on a macvlan, and the other containers on a local bridge network they share with that app.

@DanielCruz-si9gf 3 жыл бұрын

Another great video! Thank you for the useful content and keep up the good work. I would love to see more docker networking as well as security. Would be great to see how to segregate services and and access control.

@OneMarcFifty 3 жыл бұрын

Many thanks Daniel. I will do more on those subjects, but rather every now and then because obviously I am a bit late to the Docker party on youtube ;-)

@shuvomist Жыл бұрын

Nice. Thank you & team for the efforts

@OneMarcFifty Жыл бұрын

Many thanks Mehedi, I'll pass it on to the team (Me, myself and I) ;-)

@doge1931 Жыл бұрын

keep it going my dude, networking is one of the most difficult things for me to grasp when it comes to docker/portainer

@OneMarcFifty Жыл бұрын

Thank you very much ;-)

@bahmanhatami2573 2 жыл бұрын

Great job. great content that you can't find maybe anywhere else; because he's mixing something enthusiastic with something scientific.

@OneMarcFifty 2 жыл бұрын

Thank you very much ;-)

@mgveloso1 2 жыл бұрын

You have discussed a very useful capability of a docker host/container in MACVLAN... I have not found this searching many tutorials in which I am sure there should be... Imagine a docker firewall inside a host, with all packets passing thru and processed by the docker firewall; even the host should use the guest docker firewall as its gateway to the outside world. The security possibilities will be endless in this arrangement, as all services can be made to only have a bridge internal IP, all using the docker firewall. ALL as in all packets inisde or outside should be made to pass only thru the firewall. Since this firewall is a docker instance, everything will be very flexible, easy and secure. That is why I have searched for this docker capability, MACVLAN will make this possible. Thank you for your good and clear overview. God bless.

@OneMarcFifty 2 жыл бұрын

Hi Mario, that is actually an awesome idea - I might pick it up ;-)

@TheAkabuck2001 3 жыл бұрын

thanks for the video! very useful and easy to follow for someone just starting out

@OneMarcFifty 3 жыл бұрын

Thank you very much - I am glad that you liked it!

@ubermensch-mne 2 жыл бұрын

I would like to thank you for this very useful video and wish you good luck in your future work.

@OneMarcFifty 2 жыл бұрын

Hi Omar, thank you very much.

@loaiabdallatif4947 2 жыл бұрын

Thank you very much , a lot of info , please more videos on Firewall for docker networking and docker swarm

@OneMarcFifty 2 жыл бұрын

Oh - Firewall for Docker - great idea! Thanks a lot for this. I’ll think it through. I am not sure if I will do anything on swarm though as everyone seems to go for K8 these days…

@nukelf 2 жыл бұрын

Usualy I don't hit the subscribe button very fast, but in this case 2 out of 2 videos where just amazingliy good. Thanks for this easy understandable video with with comprehensible examples!

@OneMarcFifty 2 жыл бұрын

Awesome, many thanks ;-)

@rvanwaay 2 жыл бұрын

I love your tutorials, thanks a lot from Enschede NL !!!

@OneMarcFifty 2 жыл бұрын

dank jij wel en groetjes !!

@sebastiantobiascastro9350 3 жыл бұрын

thanks for the video! very useful and super well thought out didactically. I already have material to play for the weekend. thanks again! (sorry for my bad English ;))

@OneMarcFifty 3 жыл бұрын

Awesome, many thanks!

@gotelldonn 2 жыл бұрын

I would like to see comparison of macvlan vs. ipvlan networks in docker, and some practical examples of each. Thanks again for your terrific videos!

@OneMarcFifty 2 жыл бұрын

Many thanks for the feedback - much appreciated ;-)

@pichonPoP 3 жыл бұрын

Interesting video. Thanks for sharing this.

@OneMarcFifty 3 жыл бұрын

You are more than welcome ;) Thanks for watching !

@vlados3966 2 жыл бұрын

Thank You. Your way of explanation make me docker networking more interesting.When could we expect more videos about docker swarm with two ore more hosts and overlay and ingress networks? Great content that we cannot find anywhere else.

@OneMarcFifty 2 жыл бұрын

Hi Vlado, many thanks for your feedback. Glad you liked the video. At the moment, time is a bit of a constraint to me but I do have Docker on my list for future videos. Just can't give a precise date for the time being ;-)

@Alex-Shadow 2 жыл бұрын

Great video, thank you!

@OneMarcFifty 2 жыл бұрын

Many thanks Aleksandr.

@abdelilah_hmidani 2 жыл бұрын

great 👍 thank you 😊

@OneMarcFifty 2 жыл бұрын

Thank you very much - glad you liked the video ;-)

@irtibatkisileri222 2 жыл бұрын

awesome !!

@OneMarcFifty 2 жыл бұрын

Thank you - glad you like the videos!

@dakzer55 Жыл бұрын

You have saved me, I have tried all the google tutorials online. But terminal did not work to enable access from other non docker machines on my network to my macvlan container. Thank you so much 🙏

@OneMarcFifty Жыл бұрын

Hey cool - glad it helped ;-)

@alfarahat 2 жыл бұрын

Very useful, please do how to make firewall containers

@OneMarcFifty 2 жыл бұрын

Hi Ismail, its noted. Many thanks for the feedback!

@stephengarrod360 Жыл бұрын

Hi Marc, Thank you for your wonderful tutorials, you have a great way of explaning things clearly! I have also been following the OpenWRT VLAN tutorials and, as I am running Docker on OpenWRT (on a NanoPi R5S), I wondered whether you would cosider doing a tutorial about that? Specifically how the networking in Docker relates to the VLANs in OpenWRT. At the moment, my Docker containers are accessible from all VLANs and I don't fully understand why and how to make Docker 'VLAN aware'? Anyway, thanks again for making these and sharing with us 👍

@OneMarcFifty Жыл бұрын

Hi Stephen, many thanks for the friendly feedback! I might actually consider doing something along that line.

@user-np2xe2ri4x Жыл бұрын

This video material should be on the official docker documentation website

@OneMarcFifty Жыл бұрын

Hi Aleksej, thank you very much - very kind of you ;-)

@Kzzzxr 3 жыл бұрын

Hello Marc. Nice two videos for newbies on docker. Keep it up. When possible do OpenWRT on docker with adblock and setup as a gateway pc.

@OneMarcFifty 3 жыл бұрын

Many thanks for the feedback - I'll give it a thought ;-)

@jig1056 6 ай бұрын

I’m loving this series. Thank you for putting it together. I’m very interested in learning about the macvlan, this was very helpful. The one thing that still unclear to me though is how do I take that macvlan attach it to vlan that I have configured in my router. For example, I have a VLAN that I use for IOT devices and now I want to associate a docker container with that IOT VLAN

@AntonPopov_ 3 жыл бұрын

Thanks for the great video! Considering all of the lately topics, have you considered a video on how to configure two routers for high availability? I think it would be useful when having your main router as а VM in Proxmox configured with a backup router when updating the Proxmox host or during debugging.

@OneMarcFifty 3 жыл бұрын

Awesome idea! I need to give this a bit of thought - not sure how many people would use a VM as their main router though.

@AntonPopov_ 3 жыл бұрын

@@OneMarcFifty, thanks for your time. Even if they don't I think a backup router is a very good idea. I've been trying to do so with keepalived but couldn't succeed.

@henkdevries5042 2 жыл бұрын

WOW!! Looked for promox and looked for hours!! Very good job, thank you!! Many questions but have to get new hardware first to install proxmox. First to ask is: can you convert a running metal machine and convert it into a virtual machine? Very curious in this item! Yes, I still use VMWare where 9 servers are running for long time. Also use Parallels on MAC and Virtual box for testing. I was Novell guy since 3.11 (1988) and grew along till 2008. Then changed job and country and got never really back into networks. Too bad, miss it! Time to retire soon and get back into networks and pick up some programming!

@OneMarcFifty 2 жыл бұрын

Hello Henk, Yes - I have made a video on that matter : kzfaq.info/get/bejne/jpmhaNF9s6ismZs.html

@bootifulghost8624 2 жыл бұрын

Awesome Tutorials, learned a lot in Part 1 and had my head explode in Part 2 :'D. Could you possibly create a tutorial, how to create and configure the docker networks correctly, so you could expose services run in docker to the outside world (ideally with a domain) - so basicly including portforwarding and a reverse proxy? I am mainly asking because I have a slight Network related issue or I believe it is. I have 2 physical devices behind my router, that I both want to expose to the internet. One is a Synology-NAS and the other being a NUC-Server with Ubuntu, Docker, Webmin installed (from your other Tutorial). What I'm currently trying is to use nginx or traefik to forward the incoming requests for services run on the NUC-Server correctly. However I might have understood something wrong how to configure the ports in my router or in the docker networks. Right now I have port 80/443 opened which point to my NUC, where I also run nginx reverse proxy and I'm trying to point to various services and the NAS as well. However I really never got everything to work and maybe a tutorial would help. Appreciate your tutorials, because they are well structured and very thought through! Thanks

@OneMarcFifty 2 жыл бұрын

Many many thanks for your kind feedback ! The network security aspects of port forwarding, reverse proxa and client certificates etc. are on my list for so long - I will definitely do something in this space. However, time is a big constraint for me at the moment ;-(

@facorner11 3 жыл бұрын

Nice

@OneMarcFifty 3 жыл бұрын

Thank you ;-)

@demanuDJ 11 ай бұрын

will you make videos about differences between ipvlan and macvlan in docker networking?

@WebystherNunes Жыл бұрын

Great video! Have any command to notify the router about the macvlan created with specific ip?

@OneMarcFifty Жыл бұрын

You mean something like "reverse DHCP", i.e. the router getting a notification that the IP address is taken ? No, none that I am aware of...

@WebystherNunes Жыл бұрын

@@OneMarcFifty there a few bizarre options, but works like a relay, I realize a better design for my network. Curious fact, I pissed off with you after buyed a mu 4a but after i discover that my wifi connections (drop) was because the wrong route to my dns. When configured correctly things works great! Hahaha

@tubejim101 2 жыл бұрын

Did you end up making a video showing how to import a disk image in a docker container?

@OneMarcFifty 2 жыл бұрын

Hi - actually no. At the moment I am more intrigued on how to move a rootfs between Docker, WSL2 and Proxmox ;-) But I might have another look into containerization of VM/ISO etc. at another point in time ;-)

@anonymuzz5102 Жыл бұрын

I just want to be able to have static ips in my lan by mapping mac to ip once router dhcp gives ip... but of course portainer makes this wonky for me...thanks for great video i will need to figure out a subnet solution.... UGH, why is this so hard i thought having docker container as static IP on LAN would be easy since many used that... but nope, so close, but no cigar...

@OneMarcFifty Жыл бұрын

Yeah - it's a headache ;-)

@twincitiespcmd 2 жыл бұрын

Hi Marc. Pretty late to the party but ran into a couple of problems with some of the steps in the video whose content I would say is excellent! I am running Docker in VirtualBox on Windows 10 with the adapter in bridged mode. I never was able to resolve the address of client on the Docker container in bridged mode when doing iperf3 -c . I also never was able to get Docker container to be able to ping an address on the macvlan even after successfully getting a DHCP address from my DHCP server even though the VirtualBox adapter was bridged and set to promiscuous mode. Any thoughts or pointers to where I might be able to resolve these two problems?

@OneMarcFifty 2 жыл бұрын

Hi Matt, even though I am not using the described scenario myself, here are a couple of ideas to help you troubleshoot: First, check outgoing traffic from the docker container to the outside world. (Ping, traceroute and the like) to verify that network connection is OK. Next try incoming connections from the docker host and then from the outside world. If those fail then presumably it could be the Windows Firewall blocking. From what you describe it seems like network connectivity is OK - did you get an IP address from your LAN ? Just trying to exclude that it's actually the docker host serving DHCP.

@Anonymouzee Жыл бұрын

"Ózóne"... Marc... let's go to the import of a vhd(x) to inside a container... ;-)

@OneMarcFifty Жыл бұрын

I am actually thinking of running kvm inside a Docker container and run a Windows on it ;-)

@Anonymouzee Жыл бұрын

that's my Marc...👌waiting anxiously!!!

@francocastilloAR 2 жыл бұрын

I would have liked to see how to connect to the containers from the host using macvlan, but I can understand that it was getting off topic.

@OneMarcFifty 2 жыл бұрын

Hi Franco - in essence as a host on a MacVlan would behave like any other host in the network, you should be able to connect to it from any workstation - actually including the host itself.

@ameenal-azzawi2180 2 жыл бұрын

A testbed of OpenWrt on docker would be great...

@OneMarcFifty 2 жыл бұрын

It‘s noted;-)

@migy220 2 жыл бұрын

Hey, would like to know what OS you're using for this demo?

@OneMarcFifty 2 жыл бұрын

Hi, I think I used Ubuntu 18 at the time - it turned out that the behavior of dhclient has changed since then. Your container would presumably need more privileges

@ivanhansson2195 2 жыл бұрын

Marc, I really like your videos. But! I have a problem when it comes to get an IP with DHCPCD every time and I went through that section over and over again but with the same results: dhcpcd eth0 eth0: if_init: Read-only file system eth0: interface not found or invalid dhcpcd exited I can't understand what I'm missing here. Please advice if you can.

@OneMarcFifty 2 жыл бұрын

That had worked with Ubuntu 18 but stopped working with later versions. You might need to give the container more privileges than the video describes

@rsporsche 2 жыл бұрын

I would love to know the reason portainer requires these 'config' macvlan networks. Does it provide any benefit or is it just to work around some issue integrating with Docker? Btw, when setting up the macvlan network you said that you configured an ip range that has nothing to do with your LAN but it was within the same subnet so I did I miss something? Honestly I don't understand what the purpose of the IP Range is since it doesn't seem to be possible to define an arbitrary range and I couldn't subsequently create the actual macvlan network unless the gateway address was within the 'IP range', even though the gateway address was within the subnet.

@rsporsche 2 жыл бұрын

Oh, I get the comment regarding addresses outside of your LAN now,, I was confused because I thought you were referring specifically to the chosen IP range but in fact I guess the IP range could have been left empty/172.21.0.0/24

@OneMarcFifty 2 жыл бұрын

Hi Robert, yes - I could have left it empty - but I just wanted to make sure (and also wanted to show) that the DHCP request goes to a different range.

@kristoffseisler2163 2 жыл бұрын

what if i want to use a socks5 proxy to route certain machines from a lan on one subnet through a different gateway on another subnet that only my router-server is connected to via its two nics? how would i go about doing that with docker?

@kristoffseisler2163 2 жыл бұрын

never mind i managed to do it with macvlan adapter combined with bridge adapter

@OneMarcFifty 2 жыл бұрын

It depends if your router-server is the docker host or not. If it is not then you would presumably need to use MacVLAN. You could then however create a different route into the second LAN from inside the container using ip route.

@marcosantucci7132 2 жыл бұрын

Macvlan is supported on cloud instances ex: aws?

@OneMarcFifty 2 жыл бұрын

Not sure - you would have to have a physical network available I'd say.

@mingkwanburckhardt 3 жыл бұрын

Hi, ich habe gesehen, dass du da openwrt als Container hast. Kannst du dazu ein Video machen? Ich bekomme das leider einfach nicht zum laufen...

@OneMarcFifty 3 жыл бұрын

Hallo MingKwan, schau Dir mal das Projekt von Jordan Sokolic (oofnikj) an: github.com/oofnikj/docker-openwrt

@oliviandosse2839 2 жыл бұрын

Hi,thanks for thé video. When I do the dhcpcd interface name It says interface not found or invalid dhcpcd exited My interface is enp0s31f6 and it is the same interface on my host machine with ubuntu 18.04. What could be the problem please

@OneMarcFifty 2 жыл бұрын

I will definitely have to take another look at this dhcp issue. I believe Ubuntu have switched to NetworkManager.

@dennisn9207 2 жыл бұрын

I liked the video, very informative. But in the video you mentioned about setting a defined mac address for the container, and you don't show how to do that... Such a missed opportunity. You should at least post the commands in the video description.

@OneMarcFifty 2 жыл бұрын

Hi Dennis, many thanks for pointing this out - I actually found this blog entry which nicely describes how to get this done: marcelwiget.blog/2018/10/01/macvlan-docker-interface-with-explicit-endpoint-mac-address/

@irtibatkisileri222 2 жыл бұрын

macvlan did not work for me.

@OneMarcFifty 2 жыл бұрын

Hi - it seems that things have changed in Ubuntu:latest with regards to dhcpcd - it's running in the background by default now. Maybe try with another Ubuntu container version (18 or 20). I haven't figured out yet what is needed to make it work on 21.

@technology_consultant 3 жыл бұрын

Really like the way you described it. But frankly while demonstrating, your pace is too fast. Please keep your pace slow, and explain it a little further in-depth.

@OneMarcFifty 3 жыл бұрын

Great feed-back, many thanks!!! How much slower do you think it should be - could you maybe give me a time marker, how long it does take and how long it should take? That would be awesome 🙄

@technology_consultant 3 жыл бұрын

@OneMarcFifty I just wanted to say that slower your video speed a little bit while you showing or demonstrating things so that eyes, mind can correlate with what you are saying. Overall I like your pauses and speed, just in this one you go little faster than usual, that's what I felt. :)

@OneMarcFifty 3 жыл бұрын

Oh OK I see - I will have to watch it again myself in a calm hour. Many many thanks for letting me know again!!!

@scratchbin Жыл бұрын

Hi, after im leaving the bridge and joinin the macvlan network. im gettin a ip from dhcpcd. i still cant ping google. i get "From 169.254.117.17 icmp_seq=1 Destination Host Unreachable" please help :)