Coding A Lexer Part One

Рет қаралды 2,920

2 ай бұрын

In this video we are going to begin work on coding our lexer. We will first tackle the 1 character tokens, as they are they easiest to part.
As a side note, in the code shown I have forgotten to include a default case for the switch statement, you may view it in the repository linked below.
We are going to dive into one of the toughest subjects to crack in Computer Science, Compilers and Interpreters. Don't dig through exhausting documentation or dense textbooks. Lets build one together in bite-size chunks!
Link to code:
kzfaq.info?even...

Пікірлер: 15

@selvamp5775 17 күн бұрын

Excellent 👏

@Ma1ne2 2 ай бұрын

Awesome content, great speed, clear explanations, keep it coming!

@Logan-mj3wx 2 ай бұрын

Thank you! ❤️

@marmont8005 2 ай бұрын

Nice video bro

@Logan-mj3wx 2 ай бұрын

Thanks my dude

@xetta8805 2 ай бұрын

this is super informative, thanks

@Logan-mj3wx 2 ай бұрын

Glad you think so!

@CSDex 2 ай бұрын

Good stuff

@cosmowanda6460 2 ай бұрын

By the way, you haven't added this video to your compilers playlist. Awesome content! Do you plan to go through parsing and code gen?

@Logan-mj3wx 2 ай бұрын

Absolutely going to go through parsing! In terms of code generation, we will execute the code and make a full interpreter, as for optimizing the source code given...probably not.

@James-zn8nz 2 ай бұрын

Currently coding my own lexer, and I'm struggling to figure out the best way to handle multi-character operators. Preferably, I would like to do this without using nested match statements (/switch statements). Other alternatives have been finding the longest matching prefix, or searching though a Trie, however these seem OTT and I haven't seen anyone else doing it this way. What would you suggest?

@Logan-mj3wx 2 ай бұрын

Ill cover this in my next video on Tuesday.. The way I have chosen to do it in this course for multi-character tokens is an if inside of the switch. Not the prettiest, but good for beginners. Ill reply with a link to two lexers I have written in Go/C++

@cosmowanda6460 2 ай бұрын

@@Logan-mj3wxLet me know when you reply 👋🏽

@fltfathin 2 ай бұрын

Never actually wrote it but if i do it i'd make state machine and decide what token to add to token list depending on what the next character match to. Tracking current state, current token text, and next character in every loop. its simpler to write if you think about the flow as railroad diagram and try to make state machine out of it.

@tylerlaceby 2 ай бұрын

Use regular expressions. This is by far the easiest way to handling Lexing. You can construct an array of regular expressions and handlers.