100%

I really appreciate the article + video release. I've never been able to enjoy podcasts due to constant context switching in every part of my day.

Yes!

I like how the author dumbs down things into a mnemonic, perfect for me, as I'm no longer in this field and skimming and hand waving is just fine.

Here Here

Wow! Thank you so much!

Interested to hear your thoughts on the proposed OCP standard MB form factors!

intel was the future that never came, they are stuck in 1970 ARM was always the better design!

Still a long way away from this in the server space.

@@ServeTheHomeVideo Well, it's the new kid on the block, but it seems to be running quickly. Give it a few years and maybe…

@@roysigurdkarlsbakk3842 As far as I know there is current,y no real push into the server space (and considering how server companies make buying decisions based on maturity and support, this makes sense). RISCV is currently trying to conquer the low power sector, especially ML on edge and there are some efforts to bring RISCV to low end consumer desktops and similar sized computers (think Raspberry Pi or Intel Atom).

@@PhilfreezeCH I'm aware of this, but given the lack of licensing cost to whomever wants to make a RISC-V chip, I would be surprised if noone catches the chance and tries to make some for the server market.

@@roysigurdkarlsbakk3842 if you need custom solution for very specific use case then going risc v could be worth it (then you still need to have that custom solution added on).

thanks for your reply very helpful to understand the market

Intel went there then backed out of it, putting an Stratix into a Xeon package.

Lessons were definitely learned on the 6138P. It will be interesting to see what happens with FPGAs and CXL.

@@hikaru-live It was Arria, not Stratix, in the 6138P.

FPGAs don’t necessarily speed up the processing but they are definitely the most efficient

They would plug the FPGAs to Infinity Fabric; that way they would play nice the CPU/GPU ecosystem. IF is AMD"s secret sauce

That would probably be its own series if we do it.

T-Rex the series - Tiny ARMs. :)

@@HiltonT69 lol

RK3588 has Cortex A76 cores from 2018. - IPC around AMD Zen 1 or early Skylake - this core is used in Ampere Altra and Graviton 2 - much smaller than x86 core, at 7nm each A76 core has about 1.2 mm^2 incl 512 kB L2 cache . today's new Cortex X3 has 76% higher IPC than A76.... - it's shame RK3588 in 2022 has 4 year old A76 when 5 gen newer X3 is available - BTW the Cortex X3 has 30% higher IPC than AMD Zen 3 in Milan EPYCs.... which means 18% higher IPC than new upcoming Zen 4 (1.3 / 1.1 = 1.18) Graviton 3 - uses 5nm TSMC and Neoverse V1 core (derived from Cortex X1 with added 256-bit SVE SIMD extension) - Neoverse V1 core has about 5% higher IPC than Zen 3 (in general code) - SVE is for HPC (competitor to AVX512) and supports vector and MATH math (AVX512 lacks matrix math) - SVE supports FPU width scalable from 128-bit up to 2048-bit (AVX512 has 512-bit fixed) - SVE was developed for and 1st used in Japanese super computer Fugaku (Fujitsu ARM A64 FX cpu) - Fugaku was the fastest supercomputer on the world (only CPU, there are no GPUs used at all) now in 2022 dethroned by GPU SCs - still today the fastest CPU-only SC Interesting bits: - since 64-bit ARMv8 ISA is not compatible with 64-bit ARMv8/9 ISA (different encoding) there is needed two decoders for backward compatibility - since Cortex X2 designers in ARM Holding removed 32-bit compatibility and saved transistors (x86 cannot do that due to using extensions in variable length CISC coding, sure this has pros and cons) - pure 64-bit CPUs for max performance and IPC

@@arm-power While everything you said is true I don’t think the newer cores could compete from a price perspective (or even be available) for consumers. I just replaced my x86 home “storage” server with an RK3568 Board and am still able to fully use the 2.5GbE-Nic while cutting power draw by at least 3x on full and up to 10x on idle.

Cloud providers. The Enterprise server vendors (Dell, HPE, and Lenovo) are no longer setting direction in the market.

I'm not this will be true when raptor lake comes out. Seems like they are leaning into cache at least from the rumors

@@technicallyme Actually it will be MORE true with Raptor lake. I was talking about Zen 4 vs. Raptor Lake. Intel is adding more e cores if you're talking about desktop parts, AND they are clocking the CPus even higher. I just saw a leak which says the 13900K will use up to 350W as opposed to the 240W for the 12900K. And that makes sense and is what I expected. If you add 8 more e cores, that's about another 50W right there when running all-core loads (really that's the only reason why you buy CPUs like this). And NOW you clock 8 p cores and 16 e cores even faster, but Intel was already at the border of efficiency with the 12900K, in fact most reviewers felt Intel had already gone too far with it. So that puts you in that 340 - 350W territory. I don't see anything good about that. When METEOR LAKE comes out, we'll see, but Raptor Lake is just an improved Alder Lake. Meteor Lake is supposed to be on "Intel 4", and if it is and Intel adds more performance cores then they have a much better part. And I was really referring to all-core clock behavior. AMD is going to have much higher power limits than they did before which is going to allow their all-core clock speeds to be much faster than for Zen 3. This is where AMD is going to shine, because what it will take Intel to do with 350W, AMD is going to do about the same thing with 235W.

​@@johndoh5182 We can't really know till it happens. It's easy to speculate but how long have people been saying this. When I built my PC everything seemed to say AMD was the better choice factoring power etc. In reality Intel offered lower cost and higher min frames and performed better across different memory clocks. I know that's not relevant to what these CPUs are meant for but it does tell you that power level and cores doesn't tell you everything. For constant CPU loads eg. Cycles the CPU I got is identical to the AMD one it was typically compared with. If your load doesn't fully feed all your cores all the time the e cores could be a big deal. You've kind of just said Intel has just consumed more TDP with extra cores and higher clocks to get performance and then that AMD has done exactly the same. The difference will probs come down to cache. AMD having extra room to bump up wattage might just lead them to hit the same wall Intel did where it gains them basically nothing.

In theory, that is where Intel's acceleration helps

TrueNAS might be good one day on Arm, but for now it is x86. TrueNAS Scale may be the one that jumps more easily one day to Arm. Many storage vendors use QuickAssist for compression/ encryption as mentioned.

It is finally happening. That is something that I discussed when I spoke at the Cavium ThunderX2 launch in San Francisco. It is just taking a LONG time. That said, you are right that there is still a gap. Even yesterday's Solidigm D7-5520 NVMe SSD review we did on the main site yesterday there is a difference between Intel, AMD, Arm (Ampere/ Huawei), and IBM Power9 in terms of NVMe performance.

@@ServeTheHomeVideo Graviton 2 itself created 10% server market share in just one year. Cortex A76 core in Graviton 2 was the break point for ARM, because it has IPC similar to Intel Skylake or AMD's Zen 2. That's huge performance jump from Cortex A72 used in first AWS Graviton (A72 has IPC of AMD's Bulldozer or K8 Opterons which is really bad). A72 is 3rd core from A15/A57/A72/A73/A75 1st "high performance OoO" line up where A15 was fairly terrible. I have A72 in my Raspberry Pi 4 and it is very inconsistent, pretty good in scalar integer code (running Gnome linux desktop), while very slow in everything FPU/SIMD intensive (no wonder because A72 needs to break every 128-bit NEON vector into 2x 64-bit because it has 64-bit HW SIMD only (A76 has 4x more performance because it can compute 2x 128-bit FMA per cycle), also crypto SHA instructions are very slow which was probably OK for smartphone but it's pain at server/desktop). Simply put A72 is missing some key instruction features due to mobile focused design. A76 has double IPC than A72 and it is 1st core from completely new "desktop class" core line up (A76 / A77 / X1 / X2). Engineers learned lesson since A15. I have A76 at Linux desktop Debian/Ubuntu and it's HUGE jump from A72. X1 core in Graviton 3 is still A76 line up but 50% higher IPC and 4x 128-bit FMA doubles the A76 in Graviton 2. X1 has IPC at Intel's Raptor Lake level (little better than Zen 3). X3 core is 1st from entire new line-up. At least Graviton 4 and Nvidia Grace server CPUs should use this core in 2023. Cortex X3 has 20% higher IPC than X1 which put X3 as the most performant server CPU core in 2023. I really like how many executing units X3 has: 6x ALU + 2x Branch units = 8-wide integer scalar core. For comparison Zen 4 has only 4x ALU + 1x Branch unit and Intel Raptor lake has 5x ALU. Raw performance is about 50% higher for X3 on paper so those 20% IPC advantage is just beginning. X4 / X5 / X6 has pretty solid base to build on. Another topic would be special instructions for AI/ML like Intel's AMX. ARM has matrix support already in SVE from 2016 but there are SME (Scalable Matrix Extension) and brand new SME2. This is heavy focus to matrix math from ARM. Intel Sapphire Rapids with AMX is 1st gen and it's not even at market yet. ARM has 1st matrix gen out in HW (A64FX and Graviton 3), 2nd matrix gen is about to come out soon (in Neoverse V2 core in 2023?) and 3rd matrix gen has already closed specifications (CPU cores are developed right now, it takes 3 years at least). I do worry about AMD because they didn't mentioned when they are gonna support AMX. If AMX support will take them same long time as AVX512 did (almost 10 years delay after AVX512 announcement), then AMD is in trouble because ARM is already working on 3rd martrix gen HW and AMD has nothing.

This next-generation adds more caching plus transitioning to DDR5 w/ 8-12 memory channels per CPU, so memory bandwidth will go up. On the Intel side, there will be HBM Sapphire Rapids parts.

They are mostly focused on ensuring a slow market share loss rather than being parts to gain share.

Yes. Their idea is more that people will use RPi or cloud instances.

Just trussing

Thanks for including Ampere in this roundup! That stuff is really heating up.

Im looking forward to the cpu market being more betterer

Maybe more around OCP Summit 2022.

Not just turbo. Having SMT on does impact consistency.

@@superkoopatrooper4879 of course I can do anything to my CPU in the bios. But I can't enter the bios of my VM on public cloud

Yea, I know. Hoping to get the Altra HPE soon.

Wow! You are fast.

Maybe one day the More Bestest too. :-)

hell yea!🥳

Xeon Phi is basically what you are seeing as NVIDIA Grace, AMD MI300, Intel Sapphire Rapids HBM

@@ServeTheHomeVideo I suspect E-core is related, too

That is a cool idea.

@@ServeTheHomeVideo thank you :D sitting in the dilemma now and thought "hey getting one shouldn't be hard" oh boy I was wrong, they have like 4-5 different models on the market that practically look the same some managed, some unmanged, some "smart" some "prosafe", think they even have some PoE model but not sure if the switch is just powered by another PoE switch or if the switch itself delivers power to 4 of the 5 ports, but you can't even compare them on their site to get a list of features one has that the other doesn't have and so on. So you atleast got 1 viewer for such a video :D and I hope you folks can help us "simpler" folks figure out what is up and down with all these models.

I did a Power10 video some time back. Actually, we used one of IBM's last systems to target this type of market the LC921 (Power9 PCIe Gen4) in a SSD review last weekend where we used 10 different PCIe Gen4 architectures to test a drive: www.servethehome.com/solidigm-d7-p5520-7-68tb-pcie-gen4-nvme-ssd-review-on-x86-arm-and-power9/2/ IBM is not really a competitor in this market anymore. Instead, IBM is in the market of providing upgrades to those customers running Power.

Probably not in mainstream sockets unless it is part of an accelerator tile.

I don't feel like an overclocker is the person to look to here. He only tested Cinebench (small overlap with cloud apps) and he also calls E cores ERROR cores and has insisted they're a waste of silicon.

@@LunarLaker Thats the thing, he tries to paint them in the worst light, yet his own data shows that they are FAR more efficient than the P cores, and a 40E core 12930E, if scaled perfectly, would be better than a 3960x, and use as little as 140w, but more likely 165w

I like your point since it's something I've given a bit of thought to myself. Not only are the performance questions centered around P-cores versus E-cores interesting, but the whole use of P/E cores per se (by both Intel gen 12/13 as well as Apple M1/M2) raises a plethora of what, for lack of a better word, you might call philosophical questions. Sometimes I like the P/E cores idea and sometimes I hate it. A reason to like it would be when you have a mix of work that can run multithreaded but the needs of one type of thread are more intense than the other type of thread. Often the reason I hate it is because I just want a uniform core like AMD gives you. Whereas on Intel 12th gen, I ask the scheduler for 12 equally fast cores, and instead it gives me 4 cores plus 8 pieces of crap. If you wanted to take the most cynical attitude possible toward Intel, you could use the cheeseburger under the hot lights analogy. If Intel were making burgers like MacDonalds, it's like they realized they were way behind AMD (Burger King) in technology and couldn't match their sale specials of 8 Whoppers for $10. So instead, Intel (MacDonalds-like) offers you (as a $9.95 special) 4 fresh Big Macs plus 8 free cheeseburgers that have been sitting under the hot lights for five years. Yum! 5 years - that's about how old the technology is on those E cores. Intel is counting on non-technical customers who might think 4 Big Macs plus 8 stale cheeseburgs is a better deal than 8 fresh Whoppers. Yet (as you point out) for some apps that are conducive to multithreading solutions, an ostensible all-E-core chip might be a nice thing to have. If you search a bit, you might be able to find a recent podcast by one of the major Mac review channels in which they note (quite amazingly to them at the time) that you can take a brand new Apple M2 Air with 4 P cores and 4 E cores, and if you set the macOS bit to run at low power, it actually runs on the 4 E cores plus the 4 P cores only downclocked to the E-core clock rate, and for normal usage that doubles your battery life to absolutely insane numbers with no noticable reduction in humanly perceived performance (until you go export some video, that is). Here's another good one for ya - no tests to prove it yet, but I conjecture there are some model-number-versus-performance inversions within the P series of 12th gen Intel CPU chips. That is, I think some of the lower part numbers may be faster (or more horsepower actually) than some of the higher part numbers with more P cores. I'll just say, choose wisely among 1240P, 1260P, 1280P etc.

@@jimatperfromix2759 This is my whole problem with comparing Apple M1 and Windows laptops. The other part that bothers me is that they never take into account Apple's use of 5nm, while windows laptops use a mix of 14, 10, 8(which is just a hot 10nm), 7, and 7++(6nm). But more to your point, Apple has designed their OS to run on the E cores at the lowest speed possible, only ever using the P cores when the E cores are so slow, or so congested with other programs, that it would be more efficient to use the P cores. Windows on the other hand has a schedule for 1995 that struggles managing more than 4 cores, let alone many differnet kinds of cores as well as power and cache states. The Steamdeck is an oddly good example of how efficient a properly tuned OS is on X86 . I honestly expected far less performance because, on paper, based on windows performance of the equivelant to this hardware, it should have performed about as well as a 4750u with the GPU clocked at 2.1Ghz. Instead i would put it on par with a theoretical 4750u with 12 compute units instead of 8. On paper, based on windows numbers, RDNA2 should be about 30% faster than the VEGA2 found in the 4000G/5000G processors. Given they both have the same CU cound and clock speeds. But the steam deck is running at 1.6Ghz max, where as the 4000G/5000G can often run at 2.4Ghz. Based on windows numbers, the steam deck should be equivelant to the VEGAii 8 running at 2.08Ghz Yesterday, i was playing batman arkham origins a game from i think 2013 so its not like its new, but still at 800p30, the whole SOC was only using 1.4w. If i had a way to turn off the display and use an external display i'd probably have a battery life that approaches 20 hours with things like haptics and wifi ect off. Its just too bad i cant test because this thing seems to just turn the screen black when using an external display, and there is no 'off' backlight. Then if i turn it up to max, the thing is able to push 4K30(limited by my dock or the USB-C port) on Elite dangerous, using only ~16w from the wall, which probably means the SOC is using ~12w Now imagine you were not playing a game, and instead doing something highly parallelized that scales effectively perfectly with core/CU count, say, video editing, or blender, or a specialized server application. Sure it would be "efficient" with this 15w 4 core. But it would be a heck of alot more efficient if you had a lower clocked 16 core that still ran at 15w. Heck with 16 cores, you could run it at 30w and it would still probably be 2-4x as efficient.

I think that's the same idea as AMD zen4c skus with more cores, less cache, and lower frequency for each

@@chengcao418 Exactly right. As for being better/worse, it depends on the workload

Ampere (using ARM64) has incredibly low power draw compared to x86 based Intel and AMD

Sierra Forest mentioned in this video

Sure, but both Intel and Arm folks will be adding cores by then too. By 2025 a 128 core per socket system will seem small.

@@ServeTheHomeVideo The way Intel is executing at the moment we might be lucky to have Sapphire Rapids by then :-) Whilst I have you Patrick, I would love for you to do a 'deep dive' on Submer. Their cooling solutions look really interesting.

worth noting Zen 4c is smaller than Zen 4, and the V1 in Graviton3 is much larger than the N1 in Altra...

One needs to remember some basic facts - (pretty much up til now anyway) a core is a core is a core, and a thread is a thread is a thread (hyperthreaded or not), and a thread must run on a core, that is, a core must be allocated to a thread for the thread to run. The advantage of hyperthreading (two-way, say, for simplicity) is that it seriously speeds up the context switch from one hyperthread to the other. It's as if (an oversimplification) the OS kernel allocates 2 threads to be brother/sister hyperthreads on some given core, and then those two hyperthreads get scheduled round-robin on that core. But hyperthreading only minimizes what is always an overhead, namely context switching between sibling hyperthreads. You can think of the optimal context-switch overhead reduction as simply replacing a single pointer word in the core to point to the thread-context register file of one hyperthread or its sister. Fast switch at the expense of one level of indirection. (As far as I know) up til now, no hyperthreading architecture actually runs both threads at the same time. The latter is possible in some future architecture. After all, we have instruction lookahead with multi-issue instructions within a single thread; What's to prevent one from having multi-issue across two hyperthreads? However, there is no free lunch. One can sanely argue that if you're gonna do that, your chip real estate would be better spent just doubling the number of cores (and while you're at it, why not quadruple the multithreading level too). So as far as your question, no, same core count but double hyperthread count does not, in general (all other things equal) give you a 2X boost. It might give you a boost of 1.y where y is some small fraction probably not greater than 0.1. As pointed out, the true decider of which chip is better is both a function of the application (or application mix in the case of timesharing use or virtual sharing of cloud resources) as well as the "fit" of the instruction-set architecture of a given CPU to the mix of instuction needs of the application. For a given app, a better fit might be provided by an Epyc chip or a RISC chip of some sort, depending on chip architecture.

@@jimatperfromix2759 on the "both threads at once" point, I think VISC was aimed at doing that (they've since entered the shadow realm of course.) They didn't necessarily mean the performance was doubled with SMT for twice the threads. other than that, 🧱 📖

being a cloud service provider is also about cost of vCPUs to your customers. Ampere provides much more efficient chips, thus lowering costs for general cloud tasks

high core count IS effective on cost, especially when you're running them on 40% less power than the x86 competition

Ampere is an American company. They are headquartered in Santa Clara, California.

There is similar Chinese ARM server chip but they may not be sold outside China. Alibaba's chip designing department has designed an ARM v9 server chip Yitan 710. It runs neck and neck with Amphere Altra Max. The chips are being used in Aliyun's cloud servers.

The Altra Max is not really designed to be 128x cores running nginx. It is designed to be 128 cores running 16x 8 vCPU VMs running nginx as an example. AMD will be at 128C with Bergamo and Intel will be (much) higher when Sierra Forest arrives with E-cores. By then this will be an entire industry segment.

@@ServeTheHomeVideo Then I do not see really a point of mentioning "per-core" licensing of some software. It is a real niche of a niche.

The STH main site (non YT) is ~10x as big as the YT channel, so it has to match. Home is the /home/ directory in linux.

Listening in the background [?] who r u kidding. mb

sir, this is a Wendy's

Ha! Thank you.

Im looking forward to the cpu market being more betterer

@@__--JY-Moe--__ One hopes that will be a case. I still haven't even managed to start building my 5950X system.

I can try to film after I have coffee in the morning if that will help speed things up

excellent analogy

will this cut down on : U'v lost that loving feeling?😁