The questionable is that “include functor” sounds like functor include, not functor applying include. As for me, much better to have some open recursion with special keyword like This or Self that can be used as any module argument of any functor or in other modules construction.
@nuisho.studio4 күн бұрын
But looks like it will be impossible to implement this sugar without required module type annotation after This because of recursive module syntax (under sugar). include F(This : T1)(X) include G(X)(This : T2)(Y)
@RichardEisenberg-JS3 күн бұрын
Yes we've considered something like that. Maybe the conversation around the upstream maintainers will push us in this direction -- we're definitely open to the idea. But the existing syntax is simpler and works fantastically in practice. I don't immediately see why we'd need an annotation in recursive modules.
@alexanderlea22936 күн бұрын
That's cool. I haven't done enough OCaml to know if it's necessary, but I can definitely see it's utility.
@kernel58897 күн бұрын
Please tell me what theme you use, I can't sleep until I find it!!
@ivanjermakov11 күн бұрын
Useless, complicating the syntax just for this, not worth it. Or it's just a bad example.
@RichardEisenberg-JS3 күн бұрын
This is a small example for demonstration. But even in bigger examples, this feature doesn't save all that much typing.... yet it's gotten tremendous uptake within Jane Street, without any encouragement from us language folk. We simply made the feature available, and now it appears thousands of times.
@JeanNoelAvila12 күн бұрын
Why not just declare Rev_array_Map as Mappable_minimal and apply the functor on it in order to get the module you want? Is this feature syntactic sugar?
@JeanNoelAvila12 күн бұрын
OK, should have read the description first...
@RichardEisenberg-JS12 күн бұрын
Yes, this is just syntactic sugar. But it's very sweet!
@tsunningwah347112 күн бұрын
支
@user-kn4wt13 күн бұрын
VERY nice
@user-xo9yk1nt1h13 күн бұрын
What theme is this?
@alurma13 күн бұрын
I really need to apply to Jane Street. Haha
@alurma13 күн бұрын
Very nice
@moazzammoriani1656Ай бұрын
Thank you for this series! I'm curious about the speedup and performance gains that modal types offer. Are there any benchmarks currently available?
@RichardEisenberg-JS29 күн бұрын
I don't have benchmarks to report. Part of the problem with benchmarks is that it's hard to make any that are not misleading. That is, I bet I could get an arbitrary percentage speedup by switching to locals, if I have the freedom to write my own program, so writing my own program is not a good way to address this. I could use existing programs, but because using locals well requires some annotations, those programs would have to be edited. How much should we edit? Now that locals exist, we might be able to make large restructuring to a program that will make it clearer without losing performance.... but how does the benchmark take that into account? I'm not trying to say that these problems don't have solutions, just that "measure the perf improvement" is never quite as easy as it sounds. Incidentally, the biggest place within Jane Street that makes use of locals isn't around performance improvement, but is instead around code improvement. That is, there are some applications with performance requirements. Previous to locals, those applications used other techniques to meet their needs. Now with locals, programs can be written more idiomatically (making them easier to maintain), all without losing performance. So the biggest win -- maintainability -- is even harder to measure.
@bjzabaАй бұрын
Great series, thank you, this feature is very exciting! It would be nice to get some clarification on what modes are compared to modalities (this came up in passing when talking about the global field). I was thinking this whole video series that "modes" were a Jane Street-ism for "modality". I had felt it was a little bit unfortunate, as there is already a notion of "mode" in logic programming which is in relation to how the parameters of a predicate can be instantiated. If you're familiar with bidirectional typing, this is the same as how you can mark parts of the typing judgements as inputs or outputs - I even think some authors refer to these as modes as well.
@RichardEisenberg-JSАй бұрын
Yes, there are lots of different uses of the terminology out there. I don't particularly defend our choices, but instead I acknowledge that I think any choice of term would have thrown off some potential learners. As for modality vs mode: We use modality to mean a function from mode to mode. Suppose we have a record r at mode m whose type has a field x with a modality f. Then an access of r.x is at mode f(m). The `global` modality is the constant function that always returns `global`. But we have imagined other modalities that actually depend on their argument.
@bjzabaАй бұрын
@@RichardEisenberg-JS Cool, thanks for the explanation! Yeah figuring out terminology for new things is definitely hard, and I don’t envy the compromises you have to make. There's always going to be people confused or frustrated and it's hard to predict that in the future - it could be fine, or it could be grumbled about for years, and there's only so many words to choose. Anyway, thanks again for your work and your efforts in explaining this stuff, it's greatly appreciated!
@huge_lettersАй бұрын
thanks for the series :) still hard to wrap my head around most of the concepts, probably will once I get to actually use this but was very interesting to watch
@12332aАй бұрын
Heyy can u plzz tell me that a B.E Cse graduate can apply in jane street or not..
@edwardc.21352 ай бұрын
"What have i done wrong"-Everyone have been to this on any language
@alurma2 ай бұрын
Interesting.
@didyoustealmyfood87293 ай бұрын
Nah bruh too much yapping where the good docs at
@zyptoskid2 ай бұрын
hey now that's rude. respect your programming elders especially when they volunteer their time to show you stuff that entitled behavior is no good
@mndrix3 ай бұрын
It seems like a lot of type annotations. I wonder if the escape analysis could be done beneath the covers so it doesn't surface through the types. I recall Go's escape analysis being pretty good without guidance like this. I'm probably missing something.
@RichardEisenberg-JS3 ай бұрын
Most of the annotations here aren't necessary. They're there to make everything more explicit for the video. The `exclave_`s are necessary, but nothing else. Marking lcoals in mli files is necessary though, to enable separate compilation.
@edwardc.21353 ай бұрын
Interesting, but i couldn't understand how that changes the fact that computer memory break when you attempt to read and write the same variable at the same time.🤔🤔
@anshuman19875 ай бұрын
I am currently 15 yo already started my basics in Ocaml , already have knowledge in fintech,risk analysis and will do much more...In 2028 u all will be interviewing me and i will be joyous to work st Jane Street my dream company❤️ I am Coming Soon , See you
@theCornellJerald5 ай бұрын
check out cornell's cs 3110 with professor clarkson! i am taking that now lmk if u have any questions.
@bhavyajoshi76204 ай бұрын
In which year are you going to give JEE Advanced exam?
@cryptodone39065 ай бұрын
ocaml is functional, so will be relatively easier to write elegant code and do the compiler do optimization specific to JaneS needs. at the end the machine code generated on the CPU that matters.
@cryptodone39065 ай бұрын
i hoope microsoft do the serious works with F# on CLR, it's ocmal inspired language basically.
@cryptodone39065 ай бұрын
the lesser the language can do in term of syntax (concise/suctinct), the more optimization can be done. as it's narrow down the possiblity of developer `creatitivy` that sometimes abuse the language features
@thomasmeslin83995 ай бұрын
Interesting
@groovediggr87775 ай бұрын
Looking forward to the series
@Winium5 ай бұрын
"Performance is really predictable" Haskell folks shaking rn. Does branchless programming help (against branch mispredict inconsistency)?
@RichardEisenberg-JS5 ай бұрын
Branchless programming definitely helps! But of course isn't always possible.
@MirceaPricop5 ай бұрын
It's very interesting that this also makes lists allocate closely together in memory space, thus potentially leading to much better cache locality in addition to saving garbage collection costs.
@RichardEisenberg-JS5 ай бұрын
Yes that's a good point! I wish I had made that point in the video. :)
@sami93235 ай бұрын
Wonderful video with great explanations and code. Love the work going on at Jane Street with OCaml!
@MirceaPricop5 ай бұрын
Why do the contents of a local_ list have to also be local_? I feel like that was glanced over but it's not obvious to me. What would be the problem if the argument to iter leaks a reference to a list element? We can still clean up the list itself at the end of the region, right?
@RichardEisenberg-JS5 ай бұрын
Spot on, yes. In general, we have a free choice here: we could say (A) that the contents of a local list are local or (B) the contents of a local list are global. (A) means that we can store local things in a list but we can't let the contents of a local list escape; (B) means that we can let the contents of a local list escape, but we can't store local things in a list. We can't have both. But we can, actually, via a thing called the global modality. We'll get there in a few videos. In brief, it allows you to label a field of a structure as acting like (B). So you get an (A)-list (the normal list type) and a (B)-list (something you'd have to write yourself). These would be different types. This is a bit annoying, for sure, but having one type that can both store local things and then let them escape is definitely worse!
@adicide90706 ай бұрын
Do you know if this stuff will be part of the language?
@RichardEisenberg-JS6 ай бұрын
I know we will work with the OCaml community maintainers to get this to be a part of the language -- but not for a while. Our approach is to experiment locally (ha ha) here at Jane Street, gain experience about what's good and what's bad, and then to work to upstream when we have more confidence. In the end, language design is hard to get right, and we want to take advantage of the fact that we can somewhat easily undo mistakes before this is all part of the official language.
@adicide90706 ай бұрын
@@RichardEisenberg-JS aaah the puun! :D nice to watch though. not sure if OCaml can get more popular and widely adopted at this point but ML is the bomb, so here's hoping something ML does!
@a_external_ways.fully_arrays6 ай бұрын
Hmm, so this was just to solve for simple functions like iter and init, where even here, I find the process too complex by itself - imagine doing this for complex functions, and depending on libraries that doesn't support locals. Also, my guess is that this will make people copy other libraries code to their own codebase, to add locals support - so it breaks the modularity of libraries
@RichardEisenberg-JS6 ай бұрын
Decent points, indeed. (Though these functions have the complexity that hits locals -- bigger functions aren't necessarily harder in this respect.) We're currently working our way through `base` (our open-source standard library), adding local annotations throughout. It's not particularly easy! And yet the benefits we're seeing seem worth the cost. A big question we're thinking about is how this will all shake out in the end, which is one of the reasons we want to develop this on a branch of the compiler instead of pushing early for inclusion in general OCaml.
@alicaglayanrulzok6 ай бұрын
So is the plan to make this inferred by the compiler? Otherwise it seems like a lot of hassle unless you have some hotspots you want to speed up.
@goodnight_noom6 ай бұрын
But then does it mean that for functions we want to explicitly type we wont benefit from the inferred locality and have to go through the hassle anyway?
@RichardEisenberg-JS6 ай бұрын
This is a bit subtle. We're not going to infer exclaves, because doing so can change the asymptotic runtime of your program. (Actually, I should make a video about that. But not for a few weeks!) We can infer everything else -- but (as @goodnight_noom suggests) every time you write a type, you potentially interrupt that inference. So it really is some hassle. And yet my colleagues here at Jane Street are happy to put in the time adding the annotations to get their speedup. The whole system is opt-in, so indeed you'd probably only want to do this where it really counts.
@pdp115 ай бұрын
@@RichardEisenberg-JS Why can't there be mode inference even if you specify the types? Why does specifying a type without a mode annotation mean it's global?
@RichardEisenberg-JS5 ай бұрын
@@pdp11 I suppose we could have something like `_ t1 -> _ t2` that indicates we should infer the mode. (Well, not that syntax.) But we need to know what `t1 -> t2` means: is it saying global implicitly? Or does it want inference? Right now, if you write a type, its modes are considered known. (This has to be the case in interface files, regardless of what we do on function annotations.)
@karavanidet6 ай бұрын
I am subscribed, I have a bell - I only see this now
@user-lq6fx6sk4j6 ай бұрын
does anybody know that colorscheme. its so clean
@RichardEisenberg-JS6 ай бұрын
It's One Monokai in VS Code, with a bunch (~15 lines) of customizations.
@vinylwarmth6 ай бұрын
@@RichardEisenberg-JScould you share please? 😁
@kozmicluis25525 ай бұрын
Looks like a custom high-contrast monokai but I use 2 similar high contrast themes that I am satisfied with: Nigh Wolf - No Italics Dark Moon
@moonbird22666 ай бұрын
What an awesome video!
@adicide90706 ай бұрын
you trying to do some rust here now?
@RichardEisenberg-JS6 ай бұрын
Well, yes! Rust is an inspiration for what we're up to, but we're not trying to mimic it directly. Instead, we see how Rust uses lifetimes to effectively allow stack allocation, and we want that, too. :)
@adicide90706 ай бұрын
@@RichardEisenberg-JS, sure, I get it. but as a dude that is trying to get into ML, with OCaml already being a hard sell -- queries like "Why Ocaml sucks" bring up interesting, if sometimes outdated, results -- it looks a bit cumbersome to throw this in the mix. just like the sentence I just wrote, inadvertently!
@RichardEisenberg-JS6 ай бұрын
@@adicide9070 Makes sense. There's always a delicate balance between expressiveness and simplicity. We're hoping to gain experience by targeting primarily fellow Jane Streeters, figure out where the pain points are, and then try to make it so that all of this becomes simple. One nice bit of feedback I've gotten roaming the office is that some programmers here don't even really know what locals are -- that means that you can successfully use OCaml without knowing about this niche feature.
@paulspl25816 ай бұрын
Don't know why youtube recommended me this, but really cool video man ! I started out programming with ocaml and this is making me nostalgic ! Your explanations are really clear
@VIDEOS-qf5jk6 ай бұрын
Please sir give me LinkedIn link I can follow you
@huge_letters6 ай бұрын
Is there any difference between these? 1) let f: local_ 'a -> local_ 'a = fun x -> x (* I think I need to put exclave here somewhere - assume the function returns a local *) let x = f "str" (* x is local? *) 2) let f: 'a -> 'a = fun x -> x let x: local_ string = f "str"
@RichardEisenberg-JS6 ай бұрын
It depends on what you mean by "difference". Your two `f`s (I'll call them f1 and f2) have different types. Because f1's argument is local_, the compiler will try to treat its argument as local. While explored more in the next video, this means that allocations that happen in the argument can be made on the stack, instead of on the heap. (This is good.) In the actual code here, though, the argument is just a constant, so no allocation is necessary. In both cases, `x` is local. (But actually this won't compile as written, because the syntax you've used is for elements of a module, and a variable in a module cannot be local. You don't need exclave_ because there is no allocation in your f.)
@huge_letters6 ай бұрын
@@RichardEisenberg-JSyeah, sorry - I was mostly interested in the difference for the variable x declared on the 2nd line. It should be local in both cases - so I was curious if there were any advantages to the first option over the second one. Maybe the first option doesn't require a local_ mode on the function parameter for this example, it would probably make my question clearer then - my intuition on all of this isn't that good yet.
@huge_letters6 ай бұрын
How would something like let y = "string" in let local_ z = y in y work? Doesn't this kinda escape its region - but y is global so it should be fine but it's not in terms of z being local also being equal to y? What about if it's not just a string but a ref?
@RichardEisenberg-JS6 ай бұрын
Global/local is something that applies to expressions and values, not memory locations. So in your example, z is local (and would not be allowed to escape) and y is global. So returning y is just fine -- even if y and z are aliased in memory. The same is true for a mutable reference: a local variable can be aliased with a global, and the global one is allowed to escape.
@huge_letters6 ай бұрын
@@RichardEisenberg-JS I see I was just thinking in terms of your previous example with a file handle. Say, a function creates and returns a file handle - wouldn't it be able to retain it through those means? Or it then just guarantees that the ref is local, not the underlying value? Anyways I'm glad to be learning OCaml during such resurgence :)
@RichardEisenberg-JS6 ай бұрын
@@huge_letters There's a subtlety in the way that question is phrased, mentioning returning the file handle. If a function returns a file handle, then the handle can't be local -- there's no way to control where it will end up (as you rightly surmise). But we can still protect file handles by using a callback (or continuation, if you prefer that term): `with_file : string -> (local_ handle -> 'r) -> 'r`. Once something is local, it can't be made global, so that file handle will be protected.
@Metruzanca6 ай бұрын
Would love more of these ocaml unboxed
@JeanNoelAvila7 ай бұрын
Very interesting! Maybe a dumb question: why annotate when the compiler can check whether the variables do not escape their context?
@RichardEisenberg-JS7 ай бұрын
By annotating, we can check whether our belief meets reality, turning a mistaken belief into a compile-time error. Annotating also informs future readers of our code and makes the code easier to understand.
@JeanNoelAvila7 ай бұрын
@@RichardEisenberg-JS Thank you. Is the stack allocation not implemented if it is not annotated?
@RichardEisenberg-JS7 ай бұрын
@@JeanNoelAvila Inferring stack allocation (not covered in depth in the video, but it will be!) works just fine. You don't need to annotate for that.
@hsyl207 ай бұрын
It would be nice to have this in Haskell too! We could assert that a param isn't stored in a thunk (think `DynFlags` in GHC itself). How difficult would that be to implement?
@RichardEisenberg-JS7 ай бұрын
I think thunks should be fine; it's just that the thunk itself would have to be local in order to capture local data. It would be a pretty major lift, though -- getting this working in OCaml wasn't easy. (I don't take credit, as most of it was implemented before I started at Jane Street.)
@jslee09097 ай бұрын
So it's like Rust lifecycles without that manual memory management. Nice
@vedthiru5757 ай бұрын
FYI, there's a typo in the link to the compiler (it takes me to the branch "with-extepensions" instead of "with-extensions").
@SkinnyGeek_10107 ай бұрын
What an excellent video! I fully expected to watch a dry whitepaper type of video but this was easy to absorb :)
@jameslew72697 ай бұрын
all fun and games until you can't unpack GADTs bc they'll escape );
@RichardEisenberg-JS7 ай бұрын
Do you have an example? Should work fine with unpacking GADTs.
@jameslew72697 ай бұрын
@@RichardEisenberg-JS i had an issue with unpacking GADTs where their type escaped the region. it works as intended but have to be careful bc so many ways fail. which is good since thats partially why local_ is there, to spot mistakes
@drew-et1mm7 ай бұрын
its a lifetime annotation!
@RichardEisenberg-JS7 ай бұрын
Indeed! Well, it's a little different from Rust's interpretations of lifetime (ours are simpler; Rust's are more expressive). But I didn't want to start by assuming knowledge of Rust.
@dmmulroy7 ай бұрын
This is cool, great intro to the topic!
@mathiassven7 ай бұрын
Glad to see you back on YT! I might pick up OCaml just so I can follow along, your videos are always incredibly insightful on whatever topic you speak on
@donbasti7 ай бұрын
Eisenberg - "IO monad is nothing but passing in the World as a parameter to your function" (or smth like this) <3
@worgenzwithm14z7 ай бұрын
It's the guy from the Tweag videos 😃
@RichardEisenberg-JS7 ай бұрын
Happy to be back on KZfaq. 😁
@SlickMona7 ай бұрын
FYI the Github link is currently broken.
@RichardEisenberg-JS7 ай бұрын
It seems to work for me, even when I log out. Looks like YT redirects to snag some analytics; maybe that's the problem somehow? If you search for "Jane Street opam repository", you'll find it. You might need to follow a link at the bottom to get our compiler extensions; should be hopefully clear from context.