there is no other way around but learning before being capable of creating interesting stuff. There's a number of courses on C++ and ML on the internet

If you want to get something new, do something new, then you need to learn something new. Otherwise, just keep posting cat pictures on Instagram.

Google Coral has python support, which makes getting into using it considerably easier

@@seagsmtrashseagsmtrash1906 Thank you for your message. I appreciate the sentiment behind "if you want to get something new, do something new, then you need to learn something new." While I personally have no issues with C++ and other programming languages, I have some concerns regarding the advertisement of the AI kit for the Raspberry Pi 5 in a way that looks like it is easy for everyone. All the videos I've checked on this AI kit don't mention the specific skills required to use it effectively. The Raspberry boards are predominantly used by hobbyists, and it's unreasonable to assume that all of them are proficient in C++. Therefore, it seems a bit too marketing-oriented to emphasize how easy it is to use the AI kit without clearly stating that a good understanding of C++ and the Hailo SDK is necessary. Additionally, there isn't a single video available today showing how to use the Hailo SDK or providing tutorials on writing code that leverages this AI kit on a Raspberry Pi 5. While some of the responses to my post seem to highlight "how good I am because I know C++," my intention was simply to direct potential buyers to investigate a bit further before purchasing this kit, which I consider amazing but.... again not for everyone.

True, I came to realize that luckily before buying. Unfortunately hailo doesn’t release the dataflow compiler, so it’s basically useless when trying to infer any „small-LLM“ like tinyllama or such. Currently it seems that the cheapest and most energy efficient alternative for any practical assistant-like LLM is the mac-mini, but the price tag - oof. Perhaps the orangepi would be a bit more accessible, but I have yet to see any LLM on it

I’ll look into that shortly Jeff - what equipment do you use to test the power draw?

Ah, I may have been reading the specs wrong-realized they are using "1.5W" on their site to indicate typical usage at 30-60 fps of a standard image processing model. It can use more than that when it goes all-out at 13 TOPS. Wish I could edit my video ha!

@@kevinmcaleer28 I use a ThirdReality Smart Outlet for measurements over time, it's connected to my Home Assistant. I have also used a few other power measurement devices, but the ThirdReality one's been the most reliable and seems pretty accurate!

Agreed. I was waiting for a promising tensor accelerator for the pi for a while. I just bought this to try it. A possible advantage is that you can use the ram on the pi, and offload computation via the PCIe3.0 accelerator. This could allow to use larger models, a problem I have with e.g. the oak d lite is the 2GB ram limitation. Something I want to try is to run a local LLM on the pi using halio acceleration. This could give a low latency LLM for things like e.g. Speech to Text to Speech. it's something that works poorly on the cloud due to the latency involved.

Cool! Yes, if I added a camera and ollama back end it could detect a person and then say something spooky about what it sees!

Yes I’ll be following up with more info on that now that the full software stack is released

@@Playfool I haven't tried this, but I'm very skeptical. Since there's no onboard memory all the data needs to be transferred back and forth over the PCIe 3 connection which is going to be a huge bottleneck. That is, assuming it works at all.

Thanks - its back up!

machine learning is not about any considerations, it's just a pre-trained model that understands nothing, but is trying to categorize objects based on the training it was given before.

Nope it won't. If you're on a Pi 5 inferencing speed on the CPU is the same performance as the Coral TPU.

Great question - unfortunately it doesn’t work like that, that programs need to be specifically away of the accelerator and use it to run the ML models

AI is part of the programming skillset. You would have to expand upon the hobby question. You can make a living from AI in many’s, but the programming side requires advanced skills and knowledge of algorithms to make a living from. I mean you can make a living in many ways from AI so you’d have to be more specific

Same question… not sure if I should go for a Jetson Nano, Orin Nano or a PI5 + AI Kit. I want to run a RealSense + RPLIDAR for object tracking and SLAM.

how are you running llms on a rpi?

This chip appears to be low on memory, and not fitting at all for LLMs. You want to look out for Hail-10H though (not released yet, no release date either)

@@galdakaMusic be careful running frigate on a pi as it does not have a compatible encoder and will eat your cpu too much

​@@realivanjxI have Home Assitant with Frigate Addon in a Rpi4 with Coral USB. With 4 cameras al 50 Zigbee2mqtt devices without any problem since 4 years. I ask if would be posible migrate to Rpi5, pimoroni nvme dual with SSD and Hailo 8L. Runs Hailo 8L with Frigate??

Yep - Raspberry Pi 5 plus a tower case (SunFounder have launched one recently)

Thank You Kevin😀

its called DSLAM, using pictures to map the environment in real time while you navigate it.

Not with this Hat, but you can use the Pimoroni Duo to have both at the same time.

@@kevinmcaleer28 could we use the Pimoroni Duo and the Hailo accelerator with the same software, etc?

www.raspberrypi.com/news/raspberry-pi-ai-kit-available-now-at-70/

@@kevinmcaleer28 Do you know any reseller in the EU?

@@eMgotcha77 there’s links in that link

@@eMgotcha77 Pretty sure my friends at Pimoroni ship to the EU!

Thanks, I know. All shops, even Pimoroni, show either nothing or "pre-order, coming soon". Well i have to exercise patience meanwhile 🙌

You'll need a Hat that can take 2 NVMe - such as the Pimoroni NVCe Duo

@@kevinmcaleer28 isn't the Raspberry Pi NVME hat stackable? so why cant we have two?

@@bern047 The Raspberry Pi M.2 Hat has one PCIe connector and one M.2 port. The GPIO pins are stackable, but you only get one PCIe connector. Pimoroni Duo has two M.2 ports. There are other Hats available that have multiple PCIe connectors so you can daisy chain them - check out PineBoards website for more info on those.

@@kevinmcaleer28 Thanks, Pimoroni have done it again for me, will order one thanks

Yes - you'll need a Pimoroni NVMe Base Duo though

@@kevinmcaleer28 Thank you.

Don't know about Ollama, but Hailo, the company behind this M.2 kit (Hailo-8L), the kit has enough processing power to be able to do so.

This is not like Rabbit or Humane, it actually does the work locally. I’ve made a lot of content about using object detection previously for various projects so this will take that workload off the main CPU freeing it to do other tasks.

Sure is - in fact that’s the one I demoed it on

Not without mods. It uses "PCIe Gen 3" connection on Pi5.

@@AerialWaviator Technically it could be made to run on a Compute Module 4-though I haven't tried yet.

@@JeffGeerling Was wondering. Perhaps a custom PCB where both plug-in. ;)

The RPI4 has a PCIe 1x2.0 connected to a USB3 host controller The CM4 has the PCIe 1x2.0 lane free for the designer to use Hardware wise, it should work, a compliant pcie 3.0 peripheral should communicate with a 2.0 pcie host at reduced speed. One has to see if the halio pcie silicon interface supports that. The RPI4 has the BCM2711 SoC, the RPI5 has the BCM2712 SoC. One has to see if the provided Halio driver works on both SoC, or if it is specific for the RPI5 SoC. I'd really like for the Halio M.2 accelerator to be compatible with CM4, there are lots of CM4 boards out there with a M.2 expansion slot already exposed. e.g. "EdgeBox-RPI-200 equipped a M.2 socket of M KEY type. ONLY 2242 size NVME SSD card is support, NOT mSATA."

According to the specs, yes!

@@kevinmcaleer28 Hi Kevin, Do you know of any good getting started guides for it? The demos are fine and the hailo site offers very little. I am interested in a guide to loading models and running inference in python so that I can make some particle use of it.

What actual uses would you find useful?

@kevinmcaleer28 anything besides imagine detection or recognition.

Haha