Good catch, thanks for clarifying!

Thank you! I spent way too long trying to figure out why the algorithm must work knowing the factoid about prime numbers was mistaken.

I asked GPT Chat about this, it gave an example and stated that 3 is greater than the square root of 21: Prime factors of a number can indeed be greater than its square root. For example, let's consider the number 21. The square root of 21 is approximately 4.58. If we prime factorize 21, we get 3 * 7, where both 3 and 7 are prime numbers, and both are greater than the square root of 21. This illustrates that numbers can have prime factors that are greater than their square roots. It's just that, as per the fundamental theorem of arithmetic, every composite number must have at least one prime factor lesser than or equal to its square root.

​@@UmVtCg I apologize if this is a dumb question, I understand everything up to both 3 and 7 being larger than the square root of 21. We esteblished that 3 and 7 are prime factors of 21, and that the square root of 21 to be ~4.58. Where I am lost is you are saying both 3 and 7 are greater than square root of 21 (~4.58). Indeed 7 > 4.58 but 3 ! > 4.58?

@@alexcourchesne2051 You may have misread. They are saying that 21 has at least one prime factor (7) bigger than its square root, not that all of them are.

Wow, thanks!

Mostly due to general computing on GPU being a relatively new thing, also CUDA only works on nvidia hardware, so you become super platform dependant

I guess it is due to GPU programming being too proprietary. Also only in just recent years GPUs got REALLY that fast. CUDA is an NVidia only technology, There is an effort by AMD called HIP that strives to copy the API of CUDA and make it applicable to both AMD and NVidia GPUs. Maybe it will help change something. Also most of the stuff you would use such a thing is mostly for highly advanced stuff such as solving partial differential equations, simulating fluid dynamics and similar physics related areas so it is a much more niche/domain specific knowledge than just a general computing skill that you learn in Computer Science.

@@astroid-ws4py True, in college I learned the basics in a computer graphics and simulation course and some in a machine learning course. Other than that it wasn't particularly needed for general understanding of programming fundamentals. It was treated more like something you learn if you need to.

It's possible to drive them in HLSL / GLSL aswell but these are explicitly graphics focused languages and mostly game / engine devs talk about them in their specialized forums not in general programming boards etc. Half the knowledge isn't even written down anywhere, being tribal knowledge only that is.

I guess this thread could use a gentle reminder that OpenCL, OpenMP and SPIR-V exist. You don't have to write everything in Glide. ;) I'd consider prototyping with CLyther or Futhark.

These scams are everywhere

It's all over KZfaq. You can hit the three dots next to their post and select [🏳Report], but they'll just create another disposable account. It's like swatting flies on roadkill.

Lol

Thank you too!

Glad you enjoy it!

These two people helped me understand computers way more than my teachers in school ever managed.... In retrospect: my former teachers seem lazy compared to the work these legends post on KZfaq....

@@lofasz_joska 100% Makes me wonder why I bothered to pay for the piece of paper on my wall to begin with. haha

Just imagine how cool it was in 386 days when we got a 387 "math co-processor" to do finite element analysis that would still take a day for the simplest task.

So with CUDA a GPU can do everything a CPU can?

@@liamconverse8950 No it can't, but it can multiply a lot of numbers really fast and that is mostly what the program I had did.

@@kamtschatkas What can't it do?

@@liamconverse8950 It _can_ do everything the CPU can, but it will not always be faster. (Well, it can do anything the CPU can in a computing sense, it can't process mouse movement or open files on a USB drive, since it lacks USB ports). Many programs have large sections that are sequential in nature (the next step depends on the current step in some way), and in those cases the GPU will be a lot slower than the CPU.

AFAIK integer math is faster than floating point math? Further floating point units are internal to the CPU (and GPU) nowadays. In the past (like math coprocessors) that wasn't the case.

Or Col Sanders of KFC.....

He isn't fat enough for Santa.. good beard, though, agreed 😋

​@@SeanBZA i was going to say 'or a chicken salesman' ;)

My uncle would grow a Santa beard every fall, he had the belly too, 😆

I’ll sit on his lap for a 4080🤓

The mystery was not revealed... Still a very informative video. I may need to do some Googling about how VRAM works now.

Hierarchy is basically right. Every compute unit has its own local memory, and there's a cache hierarchy before reaching the memory controllers, which are typically interleaved across the address space and use a whole bunch of distinct channels. Even within the compute unit, the multiprocessors often have operations across parallel threads of execution, like bitwise or. When you do get congestion (quite easy to do), your accesses will stall and stagger, which is often hidden by running other contexts at the time. Remember how the CPU runs a couple of threads per core? The GPU might do hundreds.

@@0LoneTech This raises the question would this code actually run faster on some of the older data center GPUs that have HBM2 memory? Might be interesting to try it on an old P100 or V100 and see how that compares. If memory bandwidth is a limiting factor, these cards even though they are much older might perform better.

@@stefanl5183 It might, depending on which cards you compare. Radeon 6900 XT has about the same theoretical VRAM bandwidth as Radeon R9 Nano (500GB/s), easily beaten by e.g. MI250X with 3TB/s. However, local memory is far faster still (one or two orders of magnitude again), and the host bus (PCIe, now up to about 60GB/s) far slower. The block version of this program should essentially operate wholly in local memory. Similarly, when running on CPU such a blocked version can keep all the bit clear operations within L2 cache. There's no need for off-die RAM round trips where HBM's shorter bursts might help.

Awesome, thanks!

Thanks! I was hoping the grid would make it clear!

@@DavesGarage Yes, that grid really did explain it and that along with a few details in the comments and I now see how the square root works here as well. It's amazing how the right diagram or comment can clear up most any question. The real trick I've found is listening for that comment.

Not really, it would have been better to run the first step on a GPU as well to avoid a significant PCIe transfer latency in between. It's quite common in GPGPU computing to have steps running on a single GPU core (very inefficiently vs. a CPU) simply to avoid costly transfers.

@@vitalyl1327 PCI gen4 X16? Do the math to see the bandwidth. As far as I understand you don't get DMA with a GPU. Am I wrong about this point? The initial data transfer to the GPU would be TRIVIAL, so much so it's not worth mentioning. If it were an issue the results would show the latency problems. But, when you can clock a single PCIe lane at 2GB/s with gen4 and 4GB/s with gen5, no, just no. Latency isn't an issue. Either way, the CPU is going to call data before it goes to the GPU. Isn't that what DirectStorage is supposed to change? And what kind of decision making can the GPU do? You read about GPUs and they can't go general purpose computing. Well, that's typically a lack of ability to make decisions or very limited decision making. It's a ONE task system. It can process data and give that data back to you, but it's not making decisions based on that data. This is why the GPU cores, in this case for Nvidia CUDA cores, are very tiny. They exist to calculate and pass data on. So I don't see how you can get the GPU to do this entire task. But please this is a lack of knowledge on my part if I'm incorrect about this. Have access to the dev. tools and can list out the huge instruction set for CUDA? All the branches it can do?

Can you build a PC with just a GPU and no CPU? Then I'll run it on that :-)

@@DavesGarage I'm not being critical, I'm just pointing out that on a PC to run such a task you need both, that's all, because a GPU isn't a GP compute device, at least not in a PC. I don't know all the details of server GPUs. I'll ask you the same question though with a slight modification, CAN you run a PC with just a GPU? In other words does a GPU actually have an ISA that allows it the full complexity of running an OS and loading applications? I understand a GPU to be a single task device with tiny cores which do math and that's about it.

@@johndoh5182 bandwidth is irrelevant for small buffers, there is a cost of about 1ms to just initialise a PCIe transaction. If you have many CPU steps in your pipeline, it's not worth it and better to do it slower on a GPU. Not sure what you're talking about regarding "decision making". There is a performance penalty for a divergent branching on GPUs of this architecture, but in this scenario only one GPU core is working, so divergence is irrelevant. It is just a slower CPU.

Same. But all power to him if he did - it’s a great choice and it’s his channel.

Duh. What MishaMingazov said...

Correct, thanks for the correction!

Have at it! The memory contention can be a bear though.

Got a contact at AMD? :-)

There is no newer threadripper non-pro though. The market segment seemed to be dead but Intel has finally released some hedt stuff

There should be an absolutely Giant sign on every introductory tutorial on GPU programming to read up on SIMD architecture. Even an amateur who understands the architecture will create more performant code than someone who has decades more experience in the language used. One can't parallelize if one doesn't understand what it actually means in terms of the hardware it's running on.

@@RiversJ Do you have a good resource for understanding VRAM? Particularly: How do thousands of CUDA cores all write to memory at once? I wrote a comment with some guesses elsewhere in this thread. I could paste it here if you are up for it.

Do you know SIMD? Why haven't you contributed a SIMD solution to the Github Primes project?????

I believe nowadays math coprocessors for floating points operations are part of the CPU. GPU is different kind of coprocessor optimized for matrix multiplications and stuff like this, that can be easily parallelized.

Exactly that, they are big paralllel FP32 & 64 processors with input/output for specific instruction sets. Nvidia has considerable driver abstraction, but because you address via CUDA it's only an academic objection. OpenCL is similar but is GPU agnostic, all major players engaging with the Kronos workgroup. There have been some crazy advancements in thoughput, doing one GPU's worth of FP grunt 25 years ago would have been a supercomputers worth. The throughput enables more granularity/resolution for a lot of calculation types. I appreciate the "decade or so" :D

good old days of 8087, 80287, 80387, 80487s. I remember piggybacking a 287 so I could raytrace back in the day. GPU's started out as 3D accelerators for vector matrix math - T&L (transform and light). More features were added by Nvidia, like CUDA, which supports image processing, PhyX, etc. I think everything Pentium onwards had built in co-pro, ending the whole SX / DX chipsets.

FPUs were merged into the CPUs, and extended for streaming and vector operations - e.g. MMX, 3DNow!, Neon, SSE and AVX. GPUs are more closely related to DSPs, extended for embarrassingly parallel tasks. There's a bunch of overlap, of course; we've had accelerating processors on graphics cards long before GPUs existed (nvidia marketed the term when they put transform and lighting into graphics accelerators), and FPUs optimized specifically for 3D, like the IIT 2C87 which can do 4x4 matrix by 4 vector multiplication. The Raspberry Pi uses a DSP called VideoCore IV to do its GPU work.

No. The FPU or math coprocessor got incorporated into the CPU with the 80486 although there were cheaper 486 SX units sold that had the internal FPU disabled. Anything from a 486 DX up has the FPU incorporated. GPUs started out as 3d accelerators and were mainly focused on 3d rendering or drawing polygons on screen as they should appear in a 3D environment. But many of the early "gpus" were not really much like what we call GPUs today. For example the famous 3DFX voodoo cards were basically just 2 asic chips that did drawing and texture mapping. Anyway, the modern era of GPUs was usher in by Nvidia with CUDA.

Then you have to worry about getting data to and from each core. Most CPUs and likely most GPU cores are limited by memory speed on many real-world problems.

Anyone can answer obviously.

I'm also a self-taught programmer who's story might be interesting to your viewers. I worked in blackhat hacking making bots for a while in the mid 2000's but eventually settled into a full-stack programmer role for the last 15 years.