You are such a beautiful woman.😍

Do you think we will reach singularity 2045?

Reducing precision is a standard trick, for AI chips. Though it's usually used for inference, when things are already "baked in", and can even deal with "compressed" models. For training, the problem is you might need to "range" (in precision) for exploration. This is why GPU's are still largely used, while one can get away with alternative (lighter weight) architectures for inference. For training, one can argue reducing the precision can also have a regularizing effect. I suppose atomizing the dot product as a basic operation (which is a sub-operation in matrix multiplication) maybe one way of having little of both. There is still the issue of data transfer between memory and the computing part, which is one of the limitations of GPU's (GPU ram). This is why things are done in batches, in training algorithms. But data transfer is one of the bottlenecks. In the case of neuromorphic chips, memory and compute are married together, for each "neuron". It minimizes data transfer. It is more energy efficient. But it would require a very different paradigm, to make this work (and work well), the current approach representing everything in maths, specifically matrices. This is esp. true, for online training. One can try to shoehorn traditional approaches into this architecture. But there are limitations, and doesn't take advantage of the inherent structure of this type of "neuron". Same with quantum computing (if people can ever get the thing working, when scaling up the qubits, and adequately dealing with the noise). This is a fundamentally different beast, taking advantage of the quantum properties, superposition and entanglement. You're basically trying to solve problems by composing interfering wave functions. Not an easy task. One can shoehorn some traditional algorithms into it (e.g. RBM). But this is not the best use of this type of computer. Much less than it can handle the current best of the class architectures, with its complexities, such as "transformers", from traditional deep learning.

AI is only as good as the data we feed it (and complexity of the model). AI can't really be "innovative" and only really reproduces the data given to it to begin with. Really well, but not human. Not actually "creative" like human.

is IBM a Fabless company? who makes IBM CPU chips?

Her explanations are so easy to understand

Can u recommend channels to learn hardware and software??

This is how Skynet happens

@@kob8634 Subtitles are available.

You can tell she is really smart because she understands it on a level to be able to explain for us more normally gifted individuals.

I have been around longer than that.

I think 2028

As someone in the field, we are ages away. We don't even have any idea of the challenges and limitations ahead.

@@Peter_Lynch What do you mean by ages away ? Currently a majority of experts are predicting 2030 for AGI. What is your timeline ?

@@alteredcarbon3853 Real actual AGI not before 2050. Of course you can possibly create a chatbot by 2030 that passes the Turing test for most people in short conversations but the leap is huge. Gpt3 for example doesn't even have a state/memory right now. Additionally the structure of the models are all stable and we have to actively retrain them to even adjust the weights.

The Nvidia H100 is just another fancy GPU product. The latest-and-greatest, the usual improvements in performance, power, efficiency, cost, hype. Another fab shrink, more transistors, more memory, more bandwidth. Along with the usual new firmware and software capabilities. It's an exciting new product in the GPU/SPU world. But it's not a fundamentally new paradigm in computing. It promises to do all the old things better - maybe it can even handle the same workload as two or three other GPUs. It can't do new things in new ways that computers could never handle before. There's no reason (other than cost) that a datacenter or supercomputer couldn't use both hardwares in the same machines as necessary. Though I suspect these chips won't be available (or necessary) in consumer devices for some years.

I think it's fairly easy to cram a bunch of TFLOPs into a small area. The problem is that a good AI design requires a careful balance of resources: compute, L1 (local cache), L2 (shared cache), memory bandwdith. If the design is lopsided towards one resource or another, the design will look great on paper, but actual perf on many real world networks will be quite bad. So, perf per mm is in some sense a terrible metric to compare chips. The second thing to note, is that exploiting a novel hardware architecture takes a hell of a lot of ninja programming and is a massive software effort.

@@nemmart Isn't the whole idea that these magical AI synapse chips will be able to program themselves? That they may start poorly but they can "learn" how to improve their own performance over time and eventually evolve/converge onto optimized hardware performance?

Nice hair, terrible accent. Had a hard time making out what you were saying…

@@Henkvanpeer Indeed, but she provides a transcript! I follow other youtubers still harder to understand and they don't have it.

*This*

Keep listening. Enable subtitles. Your brain will figure it out.

You might find Yannic Kilcher interesting as well.

@@deang5622 I literally don't know what you didn't understand about my comment but let me break it down for you. First I am a scientist and what I am saying is that the way in which information on such a level is conveyed not only displays communications skills but a deep understanding of the field itself because in order to communicate effectively you need to understand. So you can view both in isolation both are valuable skills but their interplay is highly non linear. I know many brilliant scientists who cannot convey the information on the level she does the inverse, however, I have never encountered her. As soon as somebody can convey such topics on this level you know that the person knows what the fuck they are talking about. if you take offense to my swearing I don't care, the level on which she conveys information is rare really rare, and suggests a deep understanding of the field most scientists cannot even come close to. So if you don't mind please refrain from comments which do not carry any meaning.

@@Peter_Lynch Indeed I do guys brilliant! Thanks for mentioning him anyways!

Asianometry is like this channel

@@deang5622 According to Cambridge Dictionary, swear words can be used to "intensify what is said", in this case the rarity of combining hard and soft skills. I see no offensive use of language here, so no harm done.

IBM was undeniably the world leader in technology for many decades. But I get the impression that most of their "innovations" after around the 1990s was managerial and investment stuff. They left the consumer market. They left the enterprise market. They make tons of patents but they don't seem to make much real tech anymore.

@@pwnmeisterage it does seem that way. I developed software on IBM Midrange systems for 45 years, and it seems that the AS/400 (now the 'IBM i' - whatever marketing decides to call it this month) was their last real hardware/opsys success.

Man, von Neumann model is far from physical configuration, we have tons of cache now. So logic sit next to the memory, and these blocks are repeated hundreds times. Computations are so dominated by weight memory footprint, that images and generally tasks are procesed in batches: few layers process hundreds of images, then we load new layers and process next step, and so on. There is a direct support for int4 in new amd and nvidia chips.

@@peceed Thank you.

I agree, and in this one it seems it got worse. I've listened to few previous videos and it was relatively easy to follow.

a shame, yes i understand only 50% of what she says :(((

Perfectly said my brother.

Try switching on English subtitles and muting the sound. I found that useful.

Apologies Bill, this has nothing to do with Anastasi use of the English language as beautiful as she uses it for a second language or eighth for all I know but my understanding or lack of for technological matters.

Don't worry, if the likeness of becoming mad is related to the cube of the intelligence, we will only be able to build super AIs that become super mad too... well, that's not seems very reassuring.

Thank you !

@@AnastasiInTech you so beautiful , and your work is amazing thank you !

I'm die. Thank you forever.

@@AmstradExin kzfaq.info/get/bejne/atKqhdd8u53Yeo0.html

Sorry YT removes comments with links. I will add some to the description tomorrow

lmao based AvE reference