C++ is like chemistry. You spend a respectable amount of time learning the rules and a ridiculous amount of time learning about exceptions and edge cases.

I myself program in C++ and what you did there is just amazing. Honestly you're approaching legend status in the C++ community. Keep up the incredible work!

Watching this video made my will to learn C++ go from 7 to 0.

Messy or no, the fact that you managed to set a goal and achieve it with such unusual means is much more impressive than you give yourself credit for.

OH MY GOD HE POSTED

Well done! I've been doing C++ for over a decade at this point, and even participated in the standards committee for about 5 years. Standard reflection is still an ongoing discussion (with multiple drafts and Technical Specifications, and several draft implementations), and it won't even include runtime introspection, so good on ya for not waiting until C++26 (or later! 😅). You mentioned this is on 8 or 9 months of C++ experience? It's impressive 🙂

Let's see... *Builds an OS from scratch *Has an affinity for old-style cpu's *modifies a popular programming language to suit his needs You're becoming more and more like Terry Davis with every passing day.

Dude, you’re actually a freaking legend. I code in C++ too; around an year ago I’d asked a question about reflection on SO. Turns out it didn’t exist. ‘Oh well’, I thought to myself and gave up on the thing I needed reflection for. But you actually adding reflection TO THE LANGUAGE. LEGEND FR.

JDH, you've made an extremely janky, bodged system here that's "temporary." And I love you for it.

yep, this will totally work in the long run and wont cause any problems at all...

writing a parser for c++ would be a video series of its own

Wow. I personally am a JavaScript and C# programmer, so I guess I never really thought about all the work that goes into this stuff. I'm honestly really impressed. Honestly, you're one of the best programmers I've seen in a long time. I think we could all learn a bit on your approach to problems. Keep up the good work!

I've been working with C++ since at least 2003 and I can say that this was quite impressive to dive into the intermediate language of the compilers. Some of the best developers and engineers are those who design their own tools! Job well done!

I went with the parser approah for my serialization needs. It worked relatively quickly for the only reason that I only parse header file, and I use the double square brackets to know what to parse [[custom_tag]]. I'm amazed that you went through the compiler route, because when I looked at it, oh boy it was scary. All and all, it seems we (read gamedevs) need a better, more suited langage for games. In the meantime, I guess we can still use C++. Also, it's another banger video. Good job!

In one of those ironic life moments, i actually just spent a month working on a reflection project in c++... and im genuinely impressed you created an even more convoluted solution. Brilliantly done!

I don’t know what I was expecting you to use to make a save system, but I definitely know it wasn’t this, incredible job!

I literally made a serialization library 3 days ago, i went with parser approach (didn't have the gust to go the compiler route), leveraging clang's -ast-dump. It only parses header files and does not parse functions and I use __serializable__ attribute to know what to parse. The one you made though is on another level, great job!

I've actually also written a janky reflection system too, but my method used extreme preprocessor abuse rather than anything remotely smart like templates or your method of automatically fetching data. I had a macro that took in the name of an existing class, and then a varadic collection of other macros. That main macro created another secret class with a static constructor and some metadata variables. The varadic arguments in that macro then had to be yet more preprocessor macros which were primarily used to list off fields that could be found in the actual class, as well as give them metadata. Through abuse of static constructors (Which ran before main() ), and the typeid(xyz).name(), offsetof_s() and sizeof() functions, as well as just horrific abuse of casting, I had a workable reflection system that met the needs of the project, with each macro essentially building up part of an internal type database that will be fully populated with all relevant types once main() itself runs. Since typeid().name() gave me a string name of the field's type, I could then also slot some type converters into a hashmap to handle serializing/deserializing anything that wasn't an int and void* casting it into the class instance at the right spot thanks to my knowledge of the field offsets and sizes. Later on, the field definition macros even let me add extra metadata that let me add networking support by marking replicated fields, from which a subsystem would grab, compare and generate packets automagically! It even sort of worked! Ah, university... Got to love being given a complex 3D project where the only library you're allowed to use is DirectX and yet it needs a load/save system... No idea if my project would have worked on any compiler other than MSVC, but it got me a high grade, that's for sure.

Your dedication to doing things yourself instead of settling for what others have already made is truly remarkable! Excellent video!

man this was so overkill for what you're planning to achieve. and i mean this in a good way. i honestly admire the sheer amount of willpower to sit thru this, learn the inner workings of the how the language works. i thinks its great what you're doing and you should keep doing this. if you were able to reach this level of divinity in c++ in just 7 month, i cant even imagine what you'll do in couple of years.

It’s like my favourite tv series posting a new episode this game dev vlog

I love that this guy took a simple idea of saving game state, and went off the deep end with it. Kudos my crazy ass hacker brother!

"Babe wake up new jdh video just dropped"

Your breaking the laws of physics for a game that might not be released amazing

doing metaprogramming in c++ with templates involved must be one of the most terrifying jobs in CS history..

Wow, this is amazing. I've written C++ for 10 years now, and would never have come up with generating code with a compiler plugin. Amazing work!

That’s awesome!! I love when folks are able to go “there’s surely an easier way to do this…” and then roll-out a completely custom system to accomplish what they’re looking for. This “meta” programming is incredibly powerful and I love seeing it when it takes place. Thanks for such a well-documented version! :3 Is this something like Unreal Engine’s reflection system under the hood? Or is this this completely different :o

00:23 a gamer and a philosopher! you're fully of surprises mr jdh.

I haven't seen anything like that in my life. This is what I call persistence, dedication, and devotion. I think you're a genius man.

What would be useful to have in your serialization is to address an occasion when you have a class, and some time later in the development you change field names or datatypes. After that old save files will be unusable. You can serizalize some version number for each type, say by default each version 1, then for each changed class you increments it version. Then in your code you check if version for a class from a save file matches your source code class version, and if not you could implement some per-class conversion function to keep backward-compatibility.

I am beyond excited for this game! I’ve always wanted a game like this!

I've enjoyed C++ with reflection for 15 years or so now -- Qt's "moc" (meta-object compiler) generates reflection data for classes. I used this back in about 2008 to build a tool to transparently connect signals to methods on a remote instance over a network connection. More recently I've also used it to automatically bind C++ classes to Lua, to allow scripts to control C++ software. And in another project, moc-based reflection lets me expose methods as web API endpoints.

I've been programming "professionally" (getting paid for it lol) in C++ for a bit over two years now, I really like this horrible language because every single day I learn something and I never stop feeling like a beginner, when you said you look at code of programmer 10 times better than you I realized I am indeed a beginner still lol, amazing channel my dude and amazing work.

This game is looking amazing! I'm really looking forward to seeing it

It's funny, you're a big inspiration to me in terms of skill but it's also intimidating to see just how far away I am. I still need to learn so much to be able make the software that I dream about making.

This is incredible. I don't have anything really productive to add, but I just wanted to say how impressive this is.

I appreciate that you aren't afraid to show the technical side of game development, and really try to explain your code. I think too many tech related channels really try to hide it.

The more I watch this video, the more I understand, and the more I commend you for attempting something like this.

Ok this was just absolutely ridiculous and amazing thank you for blessing us with this

Really impressive, keep up the good work 💪💪

This is dope. Plugins are crazy powerful and crazy tedious so good on you. Not sure if you’ve seen the circle compiler… tbh I haven’t looked into it much and it of course doesn’t have the backing of the llvm/clang community, but I believe the madman that made circle implemented reflection

About three months into learning c++ I started writing a gui program to display all the little code snippets I'd written. I wanted the program to run the code in one frame and display it simultaneously in another. I spent about a day trying to figure out how to use reflection to do this, asked about it on stack overflow and then promptly realized it was way beyond my skill level and so copied all my code snippets into strings... it works great.

omg this game came so far since i last watched one of your devlogs!!!!!!

I've not followed it In a while, but c++23 was supposed to get reflection. At least it was one of the more popular proposals when the c++23 spec was first being announced. But this is awesome and truly amazing work, looking forward to what you do next

You’re my hero if you can do this in so much short time

Omg relatable hyperfixation keep doing what u do

This is exactly why I respect game devs. Appreciate the video being seasoned with memes💯

Your work is incredibly impressive

Can't wait to get this game and have it break my computer due to some weird memory leak or something

amazing work dude

This is such an insane amount of work to do something you'd get for free if you started in C# or Java or Python or something. I salute your dedication to insanity

Crazy skills needed to pull this off in C++. At university way back in the 90s I did make a C-interpreter in a compiler course using lex and yacc. That was pretty fun as I could modify my code in real time as it was running to experiment myself towards the output (but often messing up the data in the process), C is way simpler though so making reflection support for that would have been way easier for sure.

I'm in the process of writing a remote DWARF debug info server/parser. Stack unwinding wasn't too bad even if I couldn't use libunwind's built-in remote unwinding for... reasons. Then came the part of parsing actual debug info so I could see where and how variables are stored and... Your comment about edge cases upon edge cases is soo true for this as well 🤣

I went back and forth on reflection for the past year and a half precisely for serializing components in my tiny game. At first I did it all by hand, but it became too much repeated code (serialize in, serialize out, add to the imgui inspector...) so I said to myself, it can't be that difficult to do it automatically... Well, down the rabbit hole I went. I looked at all the macro approaches first, and left it like that for a while. Then tried the clang library but was too intimidated by it. Now, since I only need reflection for component data, which should be literal types, I use something similar to the PFR library which decomposes them into structured bindings which can be used like a tuple. You can iterate over them while compiling and save/load everything. The only missing part were field names, so I wrote a quick and dirty python script that generates a header that is optionally included where the components are declared and creates an array of names that you can use in the reflection library. It's not perfect, but it's the best I could do and actually I'm really happy with how it works now for my needs, specially compared to other previous solutions. However, I must say, what you did here is black magic. Knowing firsthand the hurdles of implementing reflection, this was amazing to watch. And everything you accomplished is seriously mind blowing (as are all your c++ experiments to be honest). So wow, thank you so much for making this incredible videos!

Don't forget that if you incorporate a library that is GPL (even v2) and distribute it (freely or commercially) you need to make available to the end user any change you made to the source code.

I haven't used C++ in years so i honestly can't remember much but i know this wasn't easy, amazing job

I'm genially impressed by your approach, dedication and knowledge. All that said, I don't think writing compiler extension it the best idea :) I think one of the approaches you could take is just use some interaction with Rust, because it's already has decent serialization without much of boilerplate. Also for my personal needs I implemented somewhat similar to Unreal Header Tool, which just scans file for macros/tags and then do runtime serialization, so you not writing whole C++ parser, but a small part of it. I did a very very basic stuff, so only few types and containers are recognizable, it doesn't work well with templates and custom spacing/aligning etc, but I think you can make it pretty generic too.

I think I would have written a custom serializer instead and just used reflection for property attributes, but bravo... you're a better man than I am!

Good lord... This is giving my PTSD of when I worked with C++ xD The absurd complexity of everything yet the surprising lack of "common" features and those compile times... The compile times especially hit me hard, after getting used to regular C, where quite often I'd just recompile whole projects while testing, just because it was fast enough to not bother me. Was also WAY less error prone to the kind of wacky compilation anomalies only C/C++ have (as far as I know).

you are a genius holy lord of c++ 🙏😭

I've been waiting for your video where you recreate the entire universe and make a computational machine out of photons and gluons, but this'll work for now.

Glad to see you back..

You are so incredible and inspiring! Mega fede videoer!

i wish i could understand anything said in this video, being able to do that myself would probably bring me a lot of joy

Why am I even here? I mentally checked out like three videos ago. I have no idea what's going on. So why am I still so captivated and entertained by this?

I love this solution. Re-inveting the wheel is somewhat frowned upon by programmers, but I think it should be the opposite, it's precisely what's necessary to get a deep understanding of things.

You are crazy haha, good work man!

So informative, thanks a lot!

Before I start waffling: I guess you didn't have to bother handling anything too weird (like SDL objects, or whatever graphics interface you're using) as you only want to save game data, so I'm guessing you can ignore pointers to external library stuff (more weird than STL) ? Or did you have to write edge case stuff for all those too ? Now to the waffle: Genius ! I looked at the LLVM backend thinking it'd be a great tool to write my own language, but then I decided I didn't have the time (the skill or the insanity). C++ is the first language I learnt (I hesitate to say learnt, does anyone know all it's knoocks and crannies) , it has made learning everything else real easy.. Anyway some fantastic craziness there, delivered with perfect understatement.. well done !!

I salute this level of insanity! My solution to this was just to store the game state in a single struct and then I can do a single file read or write to save or load. I really wish C++ would add reflection, every now and then I want to do something that turns out to be completely impossible.

15:30 wow that was a blast from the past

YOOOO YOUR BACK!

I program in C++ and find your work incredible.

you should take these detours when writing code! if your goal is to learn and become a better dev, you shouldn't be ashamed of them. you learn a lot more stuff and it's *way more* fun to write code and contribute that way, leading to you becoming a better dev. keep it up jdh!

I did something similar but for C. I added a NotNull macro to C and then made a python script that uses libclang to check at compile time if a pointer is known to be able to be NULL when it's passed to a function or used. Libclang is pretty cool.

You should add all kinds of lights and cool crafting recepies for furniture and stuff

Anything that has to do with parsing the mountain of syntax that is C++ is an absolute feat of software engineering. You have my respect. 40 years of stacking features on top of features have made it virtually impossilbe for a single person to know the full syntax of the language.

Ooh! I did this too, but for typescript which was waaay easier. I don't think I would have done this for C++ due to it being too fragile. So I have reflection for serialising data but it's purely runtime (and much easier because I define specifically how to save each type, I just made it so I only have to decorate types to make them save/load properly). I also have a compiler plugin that adds proper mixin support. So I feel like I've done both sides, but in easy mode.

Thank you jdh you inspired me to write C code.

Serialization by reflection is only cool on paper in my experience because it adds a big headache on top of the already existing headache that is migration / data layout changes between versions. thats why i prefer explicit serialization amd for the reflection part i usually just use an enum based type registry.

This is, by far, the worst, most convoluted, most unmaintainable c++ solution for anything I've ever seen... and it's beautiful. I have no words. They should have sent a poet.

I wonder, could you take a snapshot of the memory as a dump of sorts, and then allocate that same state again? Another idea could be the use some sort of custom base class or template that could record all needed data. Perhaps with a touch of boilerplate code generation. Not saying every language couldn't use proper reflection features, especially cpp. Anyways, very impressed that you got it working, well done!

Very cool results. I have personally spent too much frustrating time with LLVM/clang and trying to make sense of the mess that is C++. Also, imagine doing this in Rust and being done in an hour using procedural macros lol

I hear more recent programming languages like Zig let you do this sort of metaprogramming very easily, and at compile time rather than runtime. Even in C/C++, I've seen people successfully solve this exact problem with code generation (writing C programs that _literally_ write out C programs). This avoids having to update serialisation code manually every time e.g. an entity gets an extra field added, but without having to massively bloat your data structures with runtime type information (which is what I understood to be your approach). Either way, very entertaining video as always.

Most great coders make things look easy. JDH makes things look hard.

Welcome back

Impressive as always

thanks, it actually let me through so i could download it.

It worked! Tank you sir.

I really would like a good reflection-extension for C++, but as we also needed to serialise out data (in multiple different formats) we opted for sourcecode-generation for the data-classes. It also has to work together with the database, so now we have code that generates database-sourcecode, which is then used to generate C++ source for the data-classes (and you can also configure them via an xml). And for serialisation we then have an xml/xsd-combo that generates the mapping/marshalling classes for us. The benefit over reflection is that we could just easily change the generated source by hand and we have full IDE support once the classes are generated (with other languages that offer reflection it can be a real pain - if you ever have to debug java and can not find a single functioncall in the entire source cause the method-names are dynamically dispatched -.- ).

I had a similar but less successful story 20 years ago. I was making a game in C++ and wanted to save & load all the different objects. The system I built first was inflexible - any time I added a member variable that needed saving, any old files would no longer work. So I thought I'd try saving everything as the code needed to recreate everything. I tried making a scripting system for this purpose but took ages and got nowhere. Then I found an existing open source C++ scripting system, improved it with some asm to do the function calls I wanted. I used plenty of macros to shorten all the things I was doing in every class. These days I use JS and other modern languages. I'm throwing JSON data around all the time. Luxury!

I'm rewriting all my C++ in Rust. It's hard at first, joy afterwards. Serialization in Rust is very easy.

You did in a few weeks what hasn't been done for years by the C++ standard committee. Very impressive!!

Love the content :D

This is crazy. This man transcends us.

My kind of person. A few months ago I wanted to make an audio processing program because the current ones werent good enough for me. Long story short, as a direct result, im now trying to make my own little x86-64 desktop operating system from scratch.

I do this in C, but I just generate my structs with the preprocessor, which allows me to also generate a reflection database. This allows me to easily iterate over the fields of a struct, query their names and types and call serializers and deserializers on them. This entire system is under 100 lines of code and completely standards compliant, so it'll work on clang, gcc and MSVC.

The game looks so pretty :O

This will probably not be backwards compatible with older serialization dumps. Unless you also added [[version=...]] tags

hey i was just binging your videos (again)!