I wish I was genius so I can make chips faster

rip gamiing fps stuck at 240fps

Very helpful, thank you very much!

Excellent content. But...I'm pretty sure you have already achieved this: kzfaq.info/get/bejne/qNqRlLKVrNuxc5c.htmlm54s

Thanks for the great content it feels nostalgic. I studied everything in Electronics and Communications Engineering(ECE) bachelor degree I'm still having knowledge of EDA, Virtuoso, Verilog, VHDL, Chip Designing and Embedded systems. In India, there are quite fewer opportunities for graduates most of the companies require post-graduation(masters).(Nvidia,Synopsys,Intel,Siemens,Qualcomm and STMicroelectronics). Because of some financial conditions, I studied Computer Science during my final year and changed my whole stream to get a job in a product based company. My Batch had 60 Students and 40 among them joined a CS/IT product based company during campus placements. But I don't regret my decisions as competition is quite high in both fields it helps me to improve but I hope one day I may work on the conjunction of both or pursue higher studies to become a design/verification engineer.

When chips get bigger it’s almost always 80% verification. Too much money at stake

"Automatic trace routing".... good lord, does anyone actually use this stuff? Even the high end ones are just so horrendous.

You needs compilers, not only reuse, but automation. Of course VHDL is kind of the same idea we programmers had with Fortran in the 1940s. The problem is that VHDL is horribly dated, we made much progress with formal proofs in programming languages, they need to catch up. Actually, my PhD thesis on computing science will be about compiler hardware synthesis, but totally automated.

@@yum33333 yes, when you have a billion transistors, you use it.

@@yum33333 also, whats "horrendous", if its ugly, but works, who cares ? does it respect the ERC ? then that's enough. assembly programmers used to say the same thing about machine generated machine code...

But I know routing could be better, but that's usually a P-Space problem in computing, its really hard to solve it very efficiently. But it doesn't matter much, you aren't usually constrained by area in silicon, not the same way as you are in a PCB. So, that's not a problem for in-chip routing. Imagine routing a 10cm PCB but now in 1m of space. There's plenty of space on die.

I'm curious. How difficult is it to set up an automated test for those? When you talk about the manufacturers cutting into the parameters. Do they cut the safety margin to the bones now. I know an old chip designer that got 10x advantages back in the day by seeing how close he could cut into the safety margin on hand designed chips and tiling software.

@@waynemorellini2110 the parameters i was referring to are the actual transistors performance metrics. How much charge needs to build up before they flip, how fast that charge can build up, and once flipped how fast the charge drains. All of those characteristics vary within a window in a normal non process controlled production wafer. As for automation you basically have to automate everything. We have entire engineering teams dedicated to making our test infrastructure, and test cases. not to mention running the testing.

@@kensgold Thanks. I was wondering how much time goes into preparing the test code and rig?

@@waynemorellini2110 It very much depends on the company, and their validation strategy. Generally after tape out, and rtl freeze, there is pre production. This is where we simulate the chip on fpgas, so we can port our functional tests, and system tests to the new chip. During that time we design, and manufacture custom pcbs that we use to do validation testing. That can take anywhere from 2-3 months depending on how much changes between the old chip, and new chips. That time estimate is very dependent on the chip however. If its a generational change it will be longer than a simple rev A to rev B of the same chip.

I worked for company (nameless but huge and begins with letter M) that verified hamburgers... :)

@@prioris55555 Nameless but huge, when a company moves so much product that makes everything work that nobody really knows who you are by a brand name. Unacceptable in todays shareholder farms. They could had chosen a better name like Corsair as a flanker brand, as they failed to move retail products Ballistixally. LOL

@@prioris55555 Melanox?

After the Verilog revolution? That was a long time ago. I got into chip design after helping to create that Verilog revolution. It is truly incredible how massively the transistor budgets, or design sizes, have grown. This video does a good job. But it doesn’t convey just how incredibly complex the corner cases are for chip design as compared to SW. A SW programmer would start to get a sense of what it’s like when they go from single threaded to multi-threaded programming. The amount of pipelining and concurrency and speculative execution in hardware designs is enormous. This video also doesn’t mention formal verification. Those tools can do things like exhaustively tesingt a 64-bit adder while any brute force or constrained random testing cannot achieve the same in any of our lifetimes.

@@ttb1513 The last company I work at silicon design for, Seagate, was emblematic of the basic problems with ASICs. It was 1995 and they were new at ASIC design in house. And they weren't very good at it. It was random logic design before Verilog. An example of this was our timing verification. We had timing chains that were too long for the cycle time, and thus they simply alllowed for the fact that the signal would be sampled in the next clock cycle. Now if you were an ASIC designer back then, what I just said would have made you reach for the tums, if not a 911 call for cardiac arrest. Its an open invitation to metastability. And indeed, our AT&T fab guys were screaming at us to stop that. I got put in charge of hardware simulation for the design, and I have detailed this fiasco in these threads before, so won't go over it again. The bottom line was that ASIC process vendors were loosing trust in their customers to perform verification. The answer was that they included test chains in the designs that would automatically verify the designs at the silicon level. It mean that the silicon manufactured design would be verified, that is, defects on the chip would be verified regardless of what the design did. My boss, who with the freedom of time I can now certify was an idiot, was ecstatic over this new service. It was a gonna fixa all o' de problems don't ya know? I pointed out to him, pointlessly I might add, that our design could be total cow shit and still pass these tests with flying colors. It was like talking to a wall. In any case, the entire industry went that way. Designs are easier to verify now that the vast majority of designs are in Verilog. I moved on to software only, but I can happily say that there are some stunning software verification suites out there, and I am currently working on one, so here we are.

software fuzzing comes from hardware testing.

so thats why video games are so buggy these days

@@Blox117 it's actually the reason they're only as buggy as they are. A million chimpanzees at teletypes will do things in your game that no human would ever think to do.

@@Blox117 Game developers aren't given enough time to add all the features let alone fix most of the bugs (bug fixing can easily take up 60%+ of the dev time) because you can always just do both of these in post-launch patches and charge extra for the former as a result. Game projects also tend to have lower testability due to the intense time constraints, the fact that it really doesn't matter if they are buggy or fail as they are just entertainment software, and the fact they can be patched at a later date so there isn't any incentive to spend extra time getting it right now vs spending a little bit longer to patch it later.

Fuzzing is a less constrained input approach. More constraints are applied to limit “crazy” combinations to that of actions customers will actually take. For the keyboard example, it might be better to constrain the simultaneous key presses to a max of 10 and not allow for nearly all the keys to be pressed at once all the time in all of the tests. This lets you focus on things that are likely to happen in the course of normal use rather than finding obscure issues in corner cases that won’t happen. The config space for ASICs is wide enough that full fuzzing would be time intensive and difficult to debug. It would reveal probably more problems with the test environment than the design I’ve been interested in a fuzzing approach in verification but like I said, tracing back misbehavior is often harder and the scenarios may not even be possible on the bench. It is more valuable to find bugs that impact the major use cases and can be replicated on the bench

Throwing more people at projects was what my former employer thought to be the best solution... In all cases. Needless to say, it didn't end well for them.

It’s so annoying how capitalism hinders growth, like rather than these companies all fighting and trying to screw each other over they could all work together to make the best product possible

@@ThylineTheGay i dont think you can find a more growth-oriented system than capitalism.

@@felixknight2278 If you actually look at historical sources (primary and secondary), their scientific complex was not efficient at all. Take their rocket program for example: In the Space Race the extremely bureaucratic nature of the Soviet's development agencies were so restricting and inefficient the top scientists in the various agencies made an "underground" board to handle matters for themselves rather, calling themselves the "Council of Chief Designers". For example, when discussing the practicality of reproducing the A-4 (technical name for the German V-2 rocket), the men in NII-1 had to report to the chiefs in the Ministry of Aviation Industries, who then proceeded to deny the military applications of rockets. The team at NII-1, in violation of their orders, still decided to document and analyze the A-4 rocket in secret. Had the team not defied instructions, the development of the R-1 would have taken significantly longer. Additionally, during this time, the three major agencies involved in rocket-related technologies (NII-1, OKB-SD, OKB-51) “worked independently of each other, despite the fact that … they were employed by the same ‘ministry,’ the People’s Commissariat of Aviation Industry.” This led to rampant conflicts-of-interest and duplicated work. Source: Challenge to Apollo - The Soviet Union and the Space Race, 1945-1974 by Asif A. Siddiqi (2000)

Theres serious lack of Massive Death Count during non-war situation for Capitalism, but such list are surely exist for Communism.

GIGO

Botched the wording a little bit, not to speak of the kind of vocabulary not used by lead representatives of a polite, child friendly company. Shigeru Miyamoto, quoted regarding N64 delay: "A delayed game is eventually good; but a rushed game is forever bad." Interestingly reviews don't even play into it, as they knew they had young loyal fans who would be buying the product without consulting any reviews. Disappointing them would burn this relationship forever, which is much worse than bad reviews. Remember, people didn't have Internet access back then. Magazines were around, but their reach was in tens of thousands copies while product reach was in millions for the same given region.

Seems like a good place for a A I.

Tell that to Cyberpunk 2077 and GTA remastered trilogy.

See this quote everywhere

That assumes delayed production does not incur further expense. In reality, workers still have to be compensated for the extra weeks/ months they have to work on the delayed project.

Amen

Unfortunately, just like artists, our talent tends be recognized only after we're gone 😅

Hello! I am a beginner in coding. I would like to get better so what do you mean exactly as ‘artist?’ Thanks!

@@qk.6535 basically, it's when you care about using elegant code instead of just "good enough" code. It's also when you come up with funky algorithms that can speed your code a thousandfold. Like FFT. When you see your data the way Neo sees the Matrix at the end of the movie, you'll know you've become a code artist and not just another StackOverflow leech 😄

@@qk.6535 This isn't a topic that could be described easily, as there are many aspects to it. For example, there are people who own 10x, or even 100x the workload of a typical code monkey, yet aren't stressed out, nor spend extra hours at work. They spend much of their time thinking about the problem, and proportionally *very small amount* of time actually coding, and practically no time debugging (at least their own code). Also, when new requirements come up (new features, specification changes, etc.), they take very little time updating their existing code to support them, because it's almost as if the code was written to support the new features. It's almost like magic, like an ability to predict the future, but it really isn't. One of the "art" aspect of chip verification is the ability to verify it for the actual functionality in real life, rather than against just some specification. For example, the specification might contain a big table of what the inputs and expected outputs are. Validating the hardware against the spec is not even a half step. Understanding the reasoning behind the entire architecture, and verifying for that is many classes beyond just standard verification.

These are also the most common errors in advanced math textbooks too. I have a keen eye for errata so I notice when it's a 1 instead of an I. It's why I'm so fun at parties.

If your conception is that Intel engineers are good programmers, I have some bad news for you. Hardware engineers in general are pretty poor programmers, most having mostly written C and perl code, without much or any experience with modern software engineering practices

@@TDRinfinity They literally make the best optimizing compilers and performance tooling. Apparently they know a great deal about modern software engineering.

Also consider there are likely malicious actors who are part of designing the chips for most big enterprises.

@@wfjhDUI The performance, compilers and tooling team are not the ones involved in Verification.

what do you do with the knowledge you acquire?

I do have conversations with industry professionals on occasion, but mostly on background. They mostly ask not to be quoted.

@@Asianometry Oh yeah that's totally fair. I was thinking it would probably be a nightmare to get any to actually come on and be frank about stuff. I can only imagine how hard your research portion is as it goes now! Love the videos man, I get hyped everytime I see a notification.

Interviewing people who have to deal with NodeJS and NPM (and web dev in general) will just get you hours of someone going between laughing and crying without a word uttered. 🙃 (And after the interview, they’ll immediately run off all enthusiastic about this new JS framework that they _need_ to rewrite their project with.)

_Cough_ Log4j _cough._

@@Olivia-W How could you do this to me on this day of days, fucking Log4j. Ridiculous situation and I feel horrible for anyone dealing with it in a complex corporate environment.

*Tips fedora*

Intel also had to keep a bug in their chips for generations due to it being a "feature" that was baked into Windows etc & fixing it would have made MS recompile a lot. Pretty sad...

Is that the IEEE 754 non-compliance ? IIRC their hardware floating point support was notoriously unusable in anything critical.

@@TheNefastor nah it's the Pentium division flaw. en.m.wikipedia.org/wiki/Pentium_FDIV_bug

@@MrGoatflakes that's exactly what I said. Faulty floating point support.

That's amazing and makes so much sense. The difference obviously now is the scale of the software and hardware designs.

It amazed me how they still didn't introduce more formal verification, like we have in modern statically typed programming languages. There's an entire volume of computing science tricks that could be applied to electrical engineering. There's an entire PhD for me on that.

@@monad_tcp Go, Luiz! You'll discover if you're onto something rather quickly in an engineering PhD program. The real world is very unlike a KZfaq comment thread -- no BS allowed!

@@kwgm8578No "BS" allowed? Only PhD’s? 😂. Or other type of BS?

@@monad_tcpFormal verification is used in chip design. As a trivial example, a 64-bit or 128-bit adder can be formally verified quickly to be 100% correct for all input combinations. Look up BDD’s for a starter. The controlled randomized testing or even doing exhaustive sweep testing at the unit or block level would never come close in our lifetimes. There is inherently just a huge amount of concurrency in a hardware design. And race conditions galore. A lot of people will take away that copy-paste errors are a common error and be astounded. Those are rare. I’ve never seen that. Intel’s ancient ‘divide’ bug was due to such an error: only part of a table of values was copied over. What’s much more common (and can be included as a typo) is using the wrong delayed version of a pipeline state bit. A design can work often, but if the timing of some event shifts by one clock cycle, the design can break. Possibly because of using "state_bit_d2" instead of "state_bit_d1". That’s really a logic error though, not a typo. It would be like in this video’s traffic light example. A possible error could be a race condition where the 2nd car arrives at just the right time as the 1st opposing cars exits the intersection. The light could fail to change for the 2nd car, getting stuck. It is a contrived example, but it amounts to the decision to change the light as car1 exits mistakenly looking at whether there was a car2 already there 1 clock cycle before it showed up, so that action decides no change is needed. And car2 arrives and sees, say, that a car1 is in the intersection and that it will trigger the light to change when it exits (which it mistakenly does not). This is contrived, but it is actually a better example of a bug. A bug where the design does not meet the spec. The video mentions a spec error, not a design bug: the spec failed to state the light must be fair when congested.

DRAM

I do disagree about one thing, is that in web development if the library doesn't work it's mostly your problem if the guy doesn't feel like fixing it, while if you buy an IP from ARM, Cadence or Synopsys and show it doesn't follow the spec they will fix it and apologize. Also I would also mention there are a lot fewer critical bugs that happen and they do clear notes about what was changed and how it can affect you. It is very unlike companies like Google that change stuff and break your code.

Honestly that's only true if you suck at _originally implementing_ or you're working on old legacy software that's full of bugs already. Bugs happen obviously and they can be hard to solve, but if it's taking you more developer time to test than to actually implement things it means something's gone badly wrong. In a properly designed codebase 90% of bugs should be caught during implementation by the developer himself, and by the time you push to the test environment the application should be almost bug free. The only bugs that should be left are integration bugs that cannot be discovered untill all the other teams working on other systems have _also_ deployed to the test environment, and those should cause fatal errors right away and be easy to track down because you _should_ have extensive validation on all inputs and service responses. If you don't, you're asking for it.

@@demoniack81 The key word in your reply is "should." Many of the things that "should" happen, in practice, don't. Even in the case of faultless coding, many bugs are a result of a poorly written or incorrectly interpreted specification. Or of bugs in the tools being used; the tools are themselves typically "legacy software." The only cases I've seen where debugging and maintenance did not consume the bulk of development time were cases where the developer either couldn't be bothered or was cut short. Counterexamples welcome.

How is formal verification different from what is currently done?

@@abuttandahalf Basically there are mathematical proof finding algorithms already in use in academia. The gist is that the formal verification engine takes in "assertions" that you write that establish that "condition A must occur after condition B in X clock cycles", allowing individual line item requirements to be broken down into checks in the code. This is what is done in UVM and constrained random flow, but there are quite a few limitations and expansions that are far less hardware and more mathematical. Here's two statements we can use: p until q - if q does not happen,p holds forever p s_until q -q must eventually happen in the proof Leading to statements like (@ posedge clk) start ##1 !complete[*] ##1 complete |=> done ##0 eventually finished Which means "on each clock edge, if start condition is valid AND on the next clock cycle complete is NOT valid for an in indeterminate amount of clock cycles AND complete is valid on the next clock cycle, then on the next clock cycle done MUST be valid AND finished must be valid *at some point*". It's got the same level of density as regex, only now it's time based and you can overload the formal verification engine by implying testing conditions that explode exponentially. Yippee.

It’s cool to be called a popular channel. Thanks

@@Asianometry 100K subs means top 10 in my country :)

which country?

@@jonathanthomas7287 Hungary (Central Europe)

Ha, when I made this video three months ago - I thought I would NEVER get to 100K ever.

Also, relating to the keyboard testing example 13:40, this video did not mention doing unit tests of blocks of the design. At that level, it is much easier to control and sweep through interesting cases, often exhaustively. Something that usually cannot be done when the design block is exercised within the context of the entire chip. Additionally, formal verification tools are also used. They can verify properties exhaustively that cannot even be tested with functional tests. A simple example is that a 64-bit or 128-bit adder can be exhaustively tested with formal verification tools. Constrained random or brute force sweep testing cannot do the same in our lifetimes.

Got to have something to show people

The story of RISC-V is still developing. I feel I wouldn't have much to say here.

AI isn't magic

The one suits all approach is hard. Taking time to make an ai might not be worth it and instead paying a team of 10 to do the testing comes cheaper. We have seen an increase in ai used for the architecture of chips but even then you spend years developing the parameters and code for the ai itself. AI shouldn't be seen as a way to put out chips at a faster rate, they screw up too. I think we just need to understand that all aspects of the chain need time to catch up, we have research projects that show we can shove more transistors into a smaller area but so what, if the machinery isn't caught up and only a few men are knowledgable enough to put it into practice it is all for nothing. Customers want to hear that the next product has a newer chip with faster speeds, they have grown accustomed to getting a new phone every year with some sort of improvement. This can only last for so long, the chain is bound to collapse at some point. wouldn't it be a waste to invest billions into a technology you know will only be relevant for a year or two. Any ai you make will lose its efficiency as soon as the chips get an upgrade and now you are left with a useless ai that needs to be changed to suit the new chip. People with proper teaching are just more cost efficient for a business.

I’ve been trying that yeah

I have a feeling that it can be harder to do the formal verification than the actual chip. And then it will also be a bigger risk for errors in the formal verification than the actual product. It must be so hard just to define exact how a chip should function.

As a software person dabbling with chip design I initially assumed by verification the hw people mean formal verification. I was so surprised to find out that verification for hw is just a different word for testing. Honestly having worked on a few open-source hw design projects the test coverage is often worse compared to sw projects. Even talking to a few industry folks it seems like hw verification is a lot more manual and inconsistent than software test cases. Now obviously the big chip makers can afford to hire huge teams of verification engineers but it looks pretty weird to me. For software projects pure test engineers are pretty rare as just writing tests all day long would be pretty demeaning to most software engineers.

Let’s say you had something simple, like a counter with set, reset, and clock. Now make that counter 64-bits wide. It’s impossible to check the transition from every state to the next, for every input sequence; maybe take 100 years even in FPGA verification. With formal verification you can do it in a few seconds. Ha ha, don’t ask me how it works. I just know you mathematically prove it works, and it takes the same time as to verify a 4-bit counter.

@@rfengr00 Well with formal verification you do mathematical proofs on the structural properties of the circuit. While I do think this would have a lot of potential for both hardware and software engineering the reality is that it requires quite specific skills and has not really caught on for centuries. Aside from security critical applications that may require formal proofs like cryptography it seems like the effort is too big for broader adoption. However I do hope this changes as for hardware we should insist on correctness.

@@rfengr00 It becomes more complicated on more complex systems. Its not realy my field but if your code or harware has to handle a lot of edge cases it also becomes hard to describe them for formal proofs.

I made the video 3 months ago.

@@Asianometry Thanks for the reply. More importantly, your channel is fast growing ...well deserved.

I think for a time we'll find using machine learning AI techniques mixed with traditional we'll find a lot of short term efficiencies but long-term it'll be rendered moot if we ever get a solid grasp on quantum computing. That's a ways off though, for now, best let the algos have a crack at it before we drive our small minds insane 😂

how did you even comment

Why is this comment from 2 months ago when the video just came out a day ago lmao

Wtf

@@skazka3789 Yeah I thought it was a bug with the new KZfaq Android update, seems not.

hi im EE student graduating this year. how do you think about verification engineer as career path compared to design engineer?

@@valentinohose1723 there is more demand for verification, because at least 2 verification engineers are needed for each designer, ideally 3 (which is not often the case, unfortunately). However design is usually preferred because it's seen as more creative, and has less pressure: the deadlines are beared by the verification team. Verification closure is the last deliverable before moving on to layout/fabrication for IPs/SoCs

@@bearcb thank you for your answer. as someone with 20 years of experience in the industry, would you recommend pursuing a career as a verification engineer? I'm interested in understanding the potential career advancement opportunities in this field. I've found verifying 32-bit RISC-V with UVM in a school project to be quite enjoyable.

@@valentinohose1723 it's a personal decision, you have to balance pros and cons. Demand will always be there. If you enjoyed your verification experience, I'd say go for it. But again, it's very personal.

Thank you

A 50% rate of re-spins sounds right to me (maybe 65% is closer to the truth). The fact that slang terms exist for "a chip which came back from the fab and does nothing" tells you a lot. . . I also suspect that many of the new versions of a chip which are supposedly "to add features" were actually involuntary bug fixes.

No just by being open source doesn't reduce the verification required

​@@slicer95 I think you oversimplified my question, it's not about "just being open-source".

@@avamander. RISC-V by itself doesn't change anything for the verification. RISC -V is just an open instruction specification, x86 and ARM were proprietary. The instruction specifcation and verification are orthogonal. For the second part even if you get reusable and mass tested cores, they still have to be verified when they are combined which is non trivial.

Dogstyle Joe Biden

But FPGA is also an SRAM device, which makes it somewhat slow, space and power inefficient and cost inefficient compared to dedicated silicon at sufficient volume. Back decades ago, you'd have antifuse FPGAs which were essentially programmable ROM based configurable logic devices, but those seem to have died out, having been in a bit of a no-mans-land between in-system programmable devices and dedicated silicon. Before that, there were ULAs, which were mask ROM configurable logic devices. And then microcontrollers and configurable peripherals such as DMAC and if you look at something like RP2040's PIO peripheral probably eat quite substantially into CPLD/FPGA territory for a lot of tasks these would have been used for prior, as they're much easier to program. You're also correct in recognising a configurability increase in insanely complex circuits, and that it's absolutely vital, like PC CPUs contain fixed ALUs but the actual software facing instruction set is implemented as "microcode", a rough and usually barely functional version of which is on the silicon, but it gets updated from the PC firmware on every boot. Spinning up such a device costs a fortune and they are prone to bugs being discovered in the field which need to be fixed. Also so many hardware designs have leaned into SoC territory as time went, like "oof, this is hard - let's stick a processor core in there", and thus everything even devices which you think of as fixed functionality like SD-Cards and just about every USB peripheral have become firmware driven. When you buy a USB-to-serial dedicated function chip today, you find inside a 8051-compatible core and a ROM. As universal as FPGA are, they have also always been a niche device. The promise of FPGA everywhere somehow is always nearer and always further away and this has been this way for like 20 years.

Have you seen the Xilinx Zynq chips? They have an ARM core with an FPGA attached. I think Xilinx's MPSoC products also have hardened elements separate from the processing cores, but I don't remember what.

@@theInsaneRodent Yes of course. As I was clicking the submit button, I kind of anticipated that someone would make the observation. The thing is, today FPGA manufacturers often build in microcontrollers as sub-components on primarily FPGA chips. But what I am imagining is more along the lines of device manufacturers starting with a custom set of hard microprocessor cores and peripherals, and then adding an FPGA module as a sub-component. Apple for example, started with an ARM core and then added their own 'hard' IP to create a custom chip. Apple is big enough, but still mistakes could be very costly. By doing essentially the same thing, but instead building in a soft FPGA module, the chip could be less risky (no pun intended), and could be marketed to other device manufacturers as well. Mostly semantics I suppose, but I have a feeling this is where things are heading.

One man’s hardware is another man’s software

Yes you’re correct. Today, it’s usually the smallest feature on the device (may or may not be gate) is taken into consideration while deciding on a naming convention.

Yeah my bad.