Graphcore has Float8 being considered by the IEEE.

Same as Tesla's Super computer that has a custom number format.

I know 😃. A lot of optimizations have been done since then to squeeze out even more performance. (A.I)

Are 50 GHz of modulators available on Chip ??

@@kalukutta Yes, 50 GHz has been available for several years both in electro-absorption and carrier depletion modulators in SiP, so both amplitude and phase modulation. They're even mature enough to be available in MPW PDKs, so you can just include them in your layout without designing anything from scratch. However they are too large for dense integration of meshes like the ones here.

ASICs, FPGAs, GPUs do parallel calculation too. That is no special attribute of photonics.

@@crackwitz True, but in a fundamentally different way. Those devices need to perform a long series of logic operations to get a result. A photonic interferometer mesh is configured to be the multiplication itself, and the result comes out as soon as the input light has passed through it, about a tenth of a nanosecond. It's limited by scaling the chip and how fast you can switch the input/read the output. A reasonable parallel would be that to calculate shadows with ray tracing you need to compute a ton of stuff on a GPU, but the photonic approach is to shine light at the object and look at the shadow.

not useful unless they manufactor in mass quantity. its probably incredibly slow to make wafer scale chips

Yes, essentially, inference of smaller models can be done locally, in the browser (WebGL or Webassembly). At larger scale, it's always a limitation of memory bandwidth because no hardware can keep billions of parameters in cache. The caching locality of GPU breaks down pretty quickly and the 24GB of VRAM in top tier consumer grade GPU is still far from enough. At the end you give up and rent Google Colab to run your large models.

@@harrytsang1501 True, I don't think even any university will invest in specialized hardware, most cloud providers also simply give credits to use their cloud services, but for the cloud provider itself I think such chips can provide significant power savings perhaps and worth it in the long run.

What do you mean by "regular"? Regular GPUs? For certain things like NLP (stanza, Marian, etc) CPUs can be orders of magnitude slower than GPUs, making them totally unrealistic for running inference, so regular GPUs sure, but not CPUs!

"64 KB of memory ought to be enough for everyone" 😏

Real time 30FPS image processing of a HD camera input is very demanding.

Curious, what makes you believes this claim?

yeah, same reason amd came out on top of intel but cant do the same with nvidia no matter how competitive the hardware is. nvidia is just lightyears ahead of everyone in the software stack.

I suspect part of his secret is that he simply loves to learn.

@@RoderickJMacdonald it's not that hard if it's your passion

He's a true academic, there's no doubt about it!

Honestly the whole idea that in order for AI to work, thousands humans have to manually classify each picture is just so debilitatingly stupid... It's like having a cheat sheet with all answers for all tests and instead of understanding the question and thinking, just guessing the closest answer without any understanding...

@@DanOneOne It's less of a cheat sheet more like doing practice problems then checking your work against the teacher's answers

I used Xilinx chips years ago. You are correct.

You’re telling this to the guy who pronounces “Dee RAM” as “der-am”

The most egregious mispronunciation in this video was of the great John Von Neumann's surname.

It's almost like this channel take some sick joy at trolling tech enthusiasts with improper pronunciation of industry jargon. He's never corrected any of his flubs.

wish i bought NVDA when you posted this comment

​​@@vijvalnarayana5127 I wish I bought Nvidia when you posted your reply

Just a parameter whose value is used to multiply an input. They're called weights because they describe how much you weigh a given input, for instance, maybe a cat face detector might have high positive weights for a cat eye and cat ear input, and perhaps a negative weight for a dog snout input.

Only if you implement your design in VHDL which can be synthesized. If you're coding up specific logic functions which exist in the FPGA vendor supplied libraries then you're going to have a problem. And it's not a case of whether ASICs are in demand, it's simply a case of performance and cost and the volume of sales.

I would have expected at least a brief mention of Analog computing, using resistors as weights and adding currents together.

Something like memsistors seem like the real future of neural network hardware as network complexity outpaces how many compute units can be put into a package. Programmable resistors could instantly apply a neural network by sitting between the CMOS and typical ISP.

Problem with analog computing is luck of design tools and knowledge, hard to debug, and many more. Phonic is interesting but same neuromorphic chip design. Because NN are something you know where each memory need to be, so don't need to use RAM, but put memory next to compute and load with pre program sequence etc.

@@m_sedziwoj We already have put the computations besides the memory -- the GPUs, with their megabytes of registers and caches right next to the ALUs. But this is still not nearly enough and doesn't overcome the huge overhead of the classic discrete binary computing. Computation and memory must to be fused into a single functional structure, similar to how the organic neurons work, to get out of the power overhead trap. Probabilistic computing also could be significant contribution to ML, since most of the training doesn't need precise results or use of strict data formats.

@@RoyvanLierop forget electrons, imagine doing computations with photons on the fly.

That would be nice, but I'd also just like to see more dedicated components that supplement the CPU and GPU. There are a lot of things that a dedicated AI chip or two could do that would reduce the need for such extreme horsepower. Any efficiency gains we can make are going to be important, and I think we're simply at that phase where we should be creating a new pillar of components to do that. Photonic computing will be great, but even a breakthrough in that space won't make it to the end user for another ten years. But AI chips are almost reaching a point where they can be introduced as a standalone part. Even in something as pedestrian as the gaming space, I could see AI chip applications all over the place. It could produce a lot of streamlining in the design and development phase. It could create better variability in the game, which is honestly just a perk. Most importantly, it could be used to create a healthy number of assets in-game, which could reduce the overall _file size._ And I can't stress enough how important file size is becoming. Many video games take up an enormous amount of space and it's about to skyrocket soon. Just look at Unreal Engine 5. It has great potential to reduce GPU usage, due to its ability to render _a near infinite number of polygons_ without breaking a sweat. But all that polygon data still has to be stored somewhere... assuming it has to be stored at all. Now if a dedicated AI chip could be utilized to create the majority of that content in the end-user's computer, while they're playing the game, that would allow for game designers to deliver lush realism without crushing our drive space with >1TB downloads. Level design, texture design, NPC randomization, NPC dialogue creation, truly sophisticated enemy AI, there's a lot of stuff this could be used for. It's an utter waste of electricity to always depend on the GPU to do these tasks. And then for production workloads... man, there are just so many applications for machine learning here. It's only limited by one's imagination. Applications where the end result doesn't need to be _exact,_ but it just needs something convincing to fill in the blank and round out the rough edges. Adobe image processing, video color correction, pattern recognition, animation, 3d modeling, predictive 3d modeling, etc. Just tons of stuff that we're kinda already dipping our toes into, but the current GPUs are just too slow to reliably carry the load without bursting into flames. And of course anyone with Excel wizardry could probably think of an infinite number of potential applications there too.

@@pirojfmifhghek566 UE5 allows the use of Nanite, which is getting more features in experimental 5.1, and the assets are compressed, in Lumen in the Land of Nanite most of the space is taken by high res textures

@@JorgetePanete It's highly compressed, but it's not nothing. There's still a natural tendency for game designers to push file sizes to their limits. The difference between a AAA title with static assets and a procedurally generated title can be enormous. Even if nanite could shrink the static assets down by 70%, it doesn't hold a candle to the potential of procedurally generated design. I see it as a type of low-hanging fruit. Texture creation based off of smaller seed files would also be a helpful use of AI. You are right that they take up a crapton of space. Sometimes the bulk decompression of texture files alone is enough to make CPUs and SSDs weep. That's a bottleneck we could do without. "There's got to be a better way!" I shout, with my fists raised to the skies.

@@pirojfmifhghek566 Seeing how massive each cod warzone update is, when there are people with dial-up internet makes me sad, I hope the community stops just saying "meh" to all the bad things big companies do

Yes that thing is the coolest

As well as hydraulic presses and moltern metal

Photon based computers are on the way

I believe that's what they're gunning for. I saw another startup where they were hacking with transistors to change them from simple 0/1 to something like a sliding scale, and then doing math with those values. I think that's extremely similar to how an organic brain works

@@AlexK-jp9nc wait... transistor are analog parts, you know. It's how you use them that makes then digital or analog, when you saturate them or not. Analog computing with discrete transistor is an old art.

What are you opinions on Tenstorrent?

Luckily accuracy is measured as the accuracy of the inference as a whole rather than numerical accuracy of individual calculations.

heard about RISC-V ? Well, ok, look up "F-CPU", started in 1998.... and the "Libre-SOC" started in 2008 🙂

what you mean? these neural network will just pump up these numbers. these technologies can produce content and pump it out in the millions of articles a day.

@@herpaderppa3297 There's this problem with images and video on social media where algorithm simply cannot recognize something prohibited and the only way it is handled right now is by human labour. This type of labour is quite traumatic psychologically since people have to look constantly on something you wouldn't want to watch.

I was really against the proliferation of AI. The visions into the future brought by movies can possibly come true. But as I see divisions within our country and the greed by our politicians doing best for them with little consideration for the masses, I have come back to let technology push through. There will be a point when we will let the AI write it’s own program to make itself more efficient and less wattage drawing and so on and so forth. Maybe AI will be compassionate to our childish needs and help raise us into the future. Maybe AI will come to the conclusion that we are just parasites to the energy resources that are limited to the planet that AI and humans inhabit and will deplete. I am very sure that safety measures will be in place so AI will not go rogue like. But what if even the safety measures are countered by Ai? I guess time will only tell. Would we have androids whose sole function was to take care of humans eg…the episode of Logan’s Run (circ.1970’s) or the terminators and hunter killers from the Terminator film series? Perhaps I’m completely wrong and it’s definitely the Hunger Games scenario when the elites have complete control over the remaining resources of the planet and govern human existence. I will need more whisky and cigars to think more thoroughly about this subject matter! Good day and be safe!

if your system is electrons, that could make sense 😀

It depends on use case. Training takes way more computation, but you only have to do it once. Neural networks applied to images taken in smart phones are never trained by phones, just by the people that make the phone then that model is distributed in the OS.

kzfaq.info/get/bejne/i5eTeZyCqNuVooU.html I saw it in this video

John you're a little mistaken. Dalle Mini doesn't draw or paint or sketch anything. It compiles images as a result of instructions. So it's not drawing - for instance - Johnny Depp eating a carrot; it's compiling images of drawings of Johnny Depp from internet browsers like Google, Bing etc. It isn't painting Trump eating Nancy Pilosi, it's finding images of "paintings of Trump" and compiling them into multiple images based on instructions. Nonetheless it is cool. But at the end of the day what you're seeing are human-created images compiled into some dream-like result.

@@vanillavonchivalry6657 that’s not true.. I mean it is somewhat. But you can say “Johnny depp as a angel eating a carrot in heaven drawn in the style of the Simpsons” It has some reference for all those things but has to get creative to make something new

@Cancer McAids Look up Dall-E 2.

@Cancer McAids You haven't visited internet in a while, I see.

yes, that would be for post-processing. The pro camera is here only to capture the most accurate data as fast as possible... unlike a smartphone, the photographer wishes to retain the artistic and technical grip on the result and can spend more time post-processing on their computer, than shooting.

could also big giant thermistors, for short circuit protection or something

looks like "polyswitch" to me, they exist as round and square shapes, for overcurrent limiting. Ceramic caps. do not look like (or this large) that for more than 60 years. They would use SMD "bricks" as can be seen in other pictures.

@@leyasep5919 Yeah, that's them. I know I've seen them somewhere before, probably in data center computers. I spent a long time looking for images of the TPU v2 and v3 board before I commented that and its definitely not labeled "C" anything like the electrolytic capacitors but I couldn't find any images where I could make out what it was labeled. Its weird to see what looks like pretty low quality components for something as expensive as that. Not necessarily the fuses, but usually in expensive stuff they spring for the more expensive long life and low ESR aluminum caps.

@@htomerif Maybe it's a prototype.

@@leyasep5919 Well dang. I had like a whole smart comment wrote up, but I searched again for "TPU v2 board" and I guess I missed it before but there's a 2000x1300 image where you can clearly see those things labeled "F10" and "F7", so yeah, they're PTC thermal fuses. As for whether its a prototype or not, it looks like there are zero unpopulated pads and literally hundreds of test points, so that's pretty consistent with a prototype. It also has an SMT micro RF connector on it. I wonder what that was for.

I just pronounced his name this way in another video. You’re gonna love it

@@Asianometry , hahaha, nice! 😄 Thanks for the reply, and I will very likely watch that other video some time soon. Keep up the good work!