Jugglebot Prepares to See: Installing My Juggling Robot's Depth Camera!

Jugglebot Prepares to See: Installing My Juggling Robot's Depth Camera! | PDJ #18

Рет қаралды 5,305

Күн бұрын

In this episode, we're focusing on giving Jugglebot the ability to "see" the balls it juggles! Join me as I explore different options for positioning the depth camera, ensuring the entire juggling path is captured and avoiding any blind spots. We'll dive into the modeling process for the ball's trajectory and the camera's field of view, discuss the pros and cons of various camera placements, and reveal the final choice for Jugglebot's "eyes." Don't miss this exciting step towards a fully-functioning juggling robot!
**** NVIDIA Jetson Orin Nano ****
www.nvidia.com/en-us/autonomo...
**** Github page for the project ****
github.com/Jugleer/Jugglebot
00:00 - Results from Last Week's Poll
00:18 - Where to Place the Camera?
00:36 - Where will the Ball be?
01:33 - ... and What can the Camera See?
02:08 - Nice Position or Clarity?
02:50 - What About More Cameras??
03:22 - Or We Could Try Planar Juggling?
04:54 - What's Next?

Пікірлер: 30

@harrisonlow Жыл бұрын

Hey everyone! Choosing the perfect position for Jugglebot's depth camera was a challenge. What do you think about the final placement? 🤖👀 Let me know if you have any thoughts about this!

@jabonet Жыл бұрын

Yo could put the camera in the moving part of the bot and substract it's position.

@harrisonlow Жыл бұрын

@@jabonet Interesting idea! My main hesitation with this idea is the vibration/motion blur that would occur if the camera were mounted there. The platform will be in motion pretty much all the time and I don't want to have to deal with that sort of compensation in software. Maybe it's worth trying though if this plan doesn't work. You never know!

@jabonet Жыл бұрын

@@harrisonlow have you thought of contacting "stuff made here" to do a collaboration video? He has done similar stuff.

@matslundberg1289 Жыл бұрын

attach long brackets to the bot and mount cameras on them. gives a fixed position and easy routing of wires

@PCBWay Жыл бұрын

What you have been doing is sooo GOOD! 👍

@esser50k Жыл бұрын

Man I complain about not gettnig enough views for the videos I make, but damn you deserve wayyy more!! very interesting project

@effgee123 10 ай бұрын

Maybe add a couple of protective bars close to the camera, in the gaps between the lenses so they're not in the field of view, but still prevent impact.

@LiatachMakes Жыл бұрын

I know it adds enormous complexity in terms of calculating vs. knowing the camera's own position but there is one more spot free of obstruction. right next to the hand, or mouth or whatever you're call the central orifice/launcher. Fab project, I'm well invested and only my second vid. :D

@harrisonlow Жыл бұрын

Interesting idea! My main hesitation with this idea is the vibration/motion blur that would occur if the camera were mounted there. Re. terminology, I'm calling the Stewart platform the "arm", and the central throwing bit the "hand" to go for the anatomical analogue. Thanks! I hope you enjoy what's to come 😁

@jabonet Жыл бұрын

Could you run the cable thru one of the carbon tubes? Also. What about putting the bottom 3 roller guides on the piston. That could make the movement much more stiff. At least in part of the stroke.

@Gulivver80 Жыл бұрын

First of all I find this project awsome and I am kinda bummbed that i found about this journey only now. Several things I have about the project: 1. The upgrade to other computer such as Jetson Orin is going to benefit you if you are going to use a different detection algorythm then just color threshold. 2. The RGBD camera that you are using is going to have a different FOV for the depth process then for the individual cameras since for the depth to work it needs to see the ball in multiple cameras at the same time. Also this depth has its limits and might be glitchy if the object is to far or too close or with weird lighting. 3. There are fudicual markers that could be used for easy initial calibration of the camera to the top part of the juggling robot if you decide to have it further away or even detachable. For this eventual other solution would be it have a rigid folding arm.

@harrisonlow Жыл бұрын

Thanks! Rest assured, this is still early days for the project; there's a long way to go! Some thoughts: 1) I agree! Thankfully the oak-d-lite cameras I'm using can actually run object detection themselves, returning only the "measured" position. I'm very keen to get an Orin Nano Dev Kit once they're available in Aus. 2) yep, I'm planning on doing fairly extensive testing of these cameras and their FOV and accuracy in different directions/conditions. Now I just need to find some time to actually carry out these experiments! Hopefully soon 3) Interesting. So you don't think it'd be too difficult to calibrate on boot-up if the camera were separate? Cheers for the ideas!

@Gulivver80 Жыл бұрын

Any calibration is going to add some complexity to the system. However I think it should be easy for you to run any of those markers such as Whycon/whycode or apriltag. Fast google shows it might be able to run directly on the camera itself. This could also give you proper feedback for the position of the throwing head instead of only the encoders which should show things such as loose strings. Also for the software side of things i recommend looking into something called Robot Operating System or ROS for short. It might help you with integration of other things as well such as camera distortion calibration or addition of second juggling machine for a tandem juggling" Also feel free to ask anything about the computer vision or software side of things since it is my background.

@harrisonlow Жыл бұрын

Thanks for the suggestions! Could be good to have tracking of the robot itself as a backup... We'll have to wait and see how well it works without that! Haha yeah I started looking into ROS2, and I think it's absolutely the way to go once I build up beyond 1 robot. I have no formal training in software and got rather bogged down last time I was looking into it. Thankfully it isn't critical right now and hopefully when I need it (when I build a second jugglebot) I'll be quicker to understand it. Thanks for the invitation for questions! I can't think of anything critical (software-wise) as I'm pretty busy with hardware and video editing right now. I'd love your suggestions on future videos though! I am certain there will be times when I'm doing something wrong/inefficiently/etc and would love to know when I am. Cheers!

@mikebrosius2939 Жыл бұрын

re video about string ends.. what about a small taper pin to jam the string at the motor pulleys?

@jabonet Жыл бұрын

I see you have a bit of backlash in the position of the actuators. Maybe you can implement something like the linear actuator of the "rise robotics" linear actuator to have more resolution.

@EricSampson Жыл бұрын

Which depth camera are you using? FWIW using multiple cameras is a fair bit harder than what you might expect. You may find that you need hardware synchronization between the cameras in order to get good results (if your camera even supports that). Just using multiple cameras over USB without considerations for accurate time alignment between the cameras will give…suboptimal results lol Personally for this reasons, I’d definitely suggest the KISS “single-camera with planar juggling path” approach to start with and get things working well first, unless you enjoy integration hell 😂 Life is SO much easier with a single camera. Then once it’s working well, go for multiple cameras if you want to get fancy with non-planar juggling.

@harrisonlow Жыл бұрын

I'm using the Oak-d-lite camera(s). Hmm yeah I can see how time alignment issues could crop up. I think using a system like ROS would relieve a lot of that, but I'm no software engineer and ROS was scary last time I messed around with it 😅 I agree on KISS-I'll be using 1 camera for as long as I can. I reckon planar juggling should be fine with just 1 camera, and there's a _ton_ you can do with just planar juggling. Cheers for the input!

@truegret7778 Жыл бұрын

Would it not be beneficial to have the camera, or "eyes", on the "hand" or at the top of the hand?

@aaronmarkstaller Жыл бұрын

Nice. Do you have a discord or github for following code or similar own builds?

@harrisonlow Жыл бұрын

Yep! I have a github page where I post any relevant CAD/code for each video. To save KZfaq from deleting my comment, I've added a link in the description of this video 😊

@00swinter21 Жыл бұрын

why the hell do u only have 300 views?!?!?

@gasparliboreiro4572 Жыл бұрын

do you a ctually want to keep working on this juggle bot? like, with all the potential you have you could have more than just 1 proyect

@harrisonlow Жыл бұрын

I do have other project ideas, and I'm working on other things that might not ever get shown on here. This project is certainly the biggest one though; the scope for jugglebot is enormous. Keep an eye out for my next video to see what's (eventually) in store ☺️

@chaosordeal294 Жыл бұрын

Your audio synching always seems several ms off.

@harrisonlow Жыл бұрын

Yeah my old editing setup made me completely unaware of this sync issue. Bought better headphones and now it should be all sorted 😊

@wolpumba4099 Жыл бұрын

Maybe you can use radar instead of a camera.

@harrisonlow Жыл бұрын

Yeah I've got two backup plans if this fails: 1) 3D printing a "cage" around a small tracker that works with SteamVR lighthouses. I've already got these and I think they'll work alright, but I prefer the portability of the camera. 2) If all else fails, I'll get a proper motion tracking system, like optitrack. Will be fairly expensive, but they're insanely accurate.

@Gulivver80 Жыл бұрын

@@harrisonlow For any exterenal trackign system you would laos have to calibrate it against the top of the juggler so i thing just using cammera shoudl suffice. Esspecially since you would loose portability of the machine which i thing is currently really large benefit of it.