I worked as admin in a small lab, the Resourse Kit seemed to be standard must have at the time. The books filled up half a shelf, and occasionally helped, but that's not why you bought it.

I remember the NT Resource Kit well. I was an NT admin back in the 3.51 / 4.0 days and that kit was indispensable. The book was also riddled with errors - documentation that was flat-out wrong. "It works like this!" No, Microsoft, it does NOT work like that.

NT has a native subsystem capability that was leveraged for this. Microsoft simply added another subsystem, although it certainly took a long time to get it going. Maybe there were plans to improve it, maybe there wasn't Microsoft replaced it with "Windows Services for UNIX", which was SUS compliant (probably not getting away with that half baked POSIX bs anymore), and then WSL1, and then WSL2 (which is literally a linux VM running inside hyper-v with some clever stuff to get it communicating with windows) Btw, NTFS is also case sensitive, fwiw, windows just...kinda kludges around it

The devs probably not only knew but fully supported developing this "malicious compliance" POSIX subsystem.

Those "many hours" were spent figuring out how they could do as little as possible

It's been a long time since I had to fool with it, but iirc "enterprise support" didn't get you far and in our case, lead to migrating to an entire MS shop against the wishes of everybody who didn't have their hands in the piggy bank. Which if you boil it down, that seems to be the intended result anyway.

@@randomtidbits7695 can confirm that this was exactly how it went at the law firm that I used to do IT work for

Enterprises which wanted to use POSIX software on Windows NT were very rare and those that did used one of the variety of 3rd party solutions on the market and not the POSIX subsystem. These 3rd party solutions mostly targeted software companies looking to provide their product to Windows only shops. Most POSIX software running in Windows back then was 3rd party software from these companies and the terms of the POSIX solutions allowed distribution of their software with the POSIX runtimes.

M$ does not support. There were some major bugs reported to Microsoft in their SNA implementation (another embrace and extinguish, but then again, who really wants SNA). Bugs were send with fixes and reproduction instructions to their enterprise support. 6 months later a reply came that they found it was a bug and that they will look for a solution. That's how enterprise grade their support was.

@@bloepje SNA Gateway was a particularly bad product. Many of my employer's customers at the time unfortunately didn't discover this until replacing the reliable Novell SAA with it. It was so bad but also so key to my employer's customers accessing our primary product that eventually my employer was able to strike a deal with Microsoft. They loaned some of their developers to Microsoft to help improve SNA Gateway. Besides the poor quality of that particular product my experience with Microsoft support was ok, but at that time my customers were also very large enterprises or large government agencies which had large expensive contracts with Microsoft in order to get a high level of fast support.

This beast's still walking in the forest, that's how unbaked it is.

And it wasn't even a dessert.

It only existed to appease some bureaucrats. It was never meant to be used and no matter how much work they'd put into it, it would never be reasonable to use it anyway. The whole underlying architecture of the operating system is different.

Its not even there its literally just assorted piles to create a beast to walk around

@@RemnantCultThe beast's name? Windows NT.

Yeah, WSL sounds like a Wine competitor, not the opposite 😅

Yeah, I was going to say... a Windows subsystem for Linux sounds like a way of running Win32 applications on Linux.

technically i think the grammar is correct, but regardless it does intuitively sound backwards indeed Windows Subsystem for Linux the Windows Subsystem (the subsystem that exists within windows) that is used for running Linux

@@shallex5744 yeah, I get that’s what they were going for, but it’s still backwards. Anywhere else, the noun adjunct to “subsystem” describes what the subsystem does, and “for” indicates what subsystem is meant to be a part of/run on. It’s not a Switch emulator for N64, it’s an N64 emulator for Switch, for example. Even if the grammar is technically ambiguous (it probably is, English is not especially precise), WSL is still strictly backwards to the standard convention used in any other similar context.

@@Thanatos2996 yes, it is a very strange name choice indeed

I worked on a project in 1995 where management had the vision that we should have a common codebase for Unix and NT. The team actually tried to use the POSIX subsystem whit the same success as showed in the video... Our solution was to write our own libraries as a version of pthread for NT and not to tell management. 😁

Embrace Enhance Extinguish

tbh I'm not surprised this is such a mess, I've never had POSIX accomplish what it was supposed to. Anything beyond basic apps has always required a pile of ifdefs even for stuff like memset/memalign/malloc. They didn't even add endian swapping functions until *2014*... for a standard which is literally centered on standardized IO.

@@ShinyQuagsire it’s way better than nothing though

POSIX is C source code level. Unix API is some kernel calls wrapped by standard library.

@@ShinyQuagsire I think you are conflating Windows adherence to the POSIX standard with the actual POSIX standard. The reason why WSL exists is because of Windows attempting POSIX compliance.

Yup, posix compat means compiles on both AIX and Linux even though AIX feels like a barren shithole compared to gnu linux

Ironically Windows NT is a fork of OS/2 there joint project with IBM. It's likely it's posix compliance was also this messy

@@JeffreyPiatt NT is not a fork of OS/2, its a nearly 100% independent codebase

@@hye181 Sure. Independent for legal purposes.

POSIX itself is an example of malicious compliance though, since it was all about taking the UNIX API and whitewashing it as a standard for all OSes despite it being obviously intended only for Unixes. They even made the acronym rhyme with Unix, just to rub it in. Which is why POSIX is so limited, once they removed all the bits that were too Unix-y, there wasn't much left. Microsoft made a good call by implementing just the standard to point out the ridiculousness of the standard.

@@jankrynicky no, it is literally a fresh reimplementation, the only thing taken directly from OS/2 is the command interpreter

My short attention span appreciates these live stream summaries, too.

Yeah, I remember M$ bragging about that C2 compliance from the very start during the presentations of NT 3.1, and how hard work it was to get that certified. Later I found out that this approval was only valid for a certain subset of hardware/software components, and anything with a network (even just LAN) was _not_ C2 compliant. Even back then running without some form of network was pretty useless.

"Computer Necromancy", I love this term and will be stealing it 👍

The linker flag specifically sets the subsystem file in the PE header; I don't think it affects anything else besides that, but I'm not 100% certain.

"POSIX_ME_HARDER" is the funniest variable name I've ever seen.

@@LonelySpaceDetective For some reason, I want to add DADDY at the end of it.

A couple of Linux distros (specifically, EulerOS from Huawei and Inspur K-UX, both RHEL derivatives) got UNIX 03 certification at one point, proving that Linux indeed _can_ be made "fully compliant" if needed.

There have been 2 UNIX-certified Linux distros tho.

There were few Linux distributions that passed Single UNIX Specification tests and were registered as UNIX systems.

How’s POSIX on Mac?

@@MaxOakland _so_ much better. fully functional and works like you would expect it to.

@@adora_was_taken That’s awesome!

@@MaxOakland macOS is weird since it combines Old Macintosh, NeXT, Unix, and Mach operating system heritages. Macintosh gives you the GUI style and file-systems, NeXT gives you the premier programming API, Unix gives you the more standard C API and file-system structure, and Mach gives you some of the kernel and executable file format

This lawsuit eventually brought us C#.

Oh awesome, which group? I went looking but couldn't find anything but references to references.

I worked in IT in this era. I remember the 9x ones. The NT one was rarely, and it didn't note that the POSIX tools were included. The book itself doesn't even list that; I had to check the disc myself. The book just lists traverse checking has to be disabled, and nothing else.

@@NCommander I worked in IT during this era. Your recollection maybe that the NT resource kit was rare but it certainly isn't mine. It was everywhere.

My brother and I had the NT 4resource kit though he had to order it. A few years later I was able to get the 2000 resource kit but that was a lot more common..

Because, for all intents and purposes, nobody did.

I saw one extremely edge case once, but that's it.

Also, it was executing a file that was not .exe

Windows NT is still Posix compliant like it's sibling os/2

@@JeffreyPiatt OS/2 was not POSIX compliant. "it's" is a contraction for "it is". "its" is the pronoun.

@@JeffreyPiatt NT had a os/2 subsystem, BTW

@@szt1980 Only for 1.x text mode programs. Which was actually super useful because OS/2 1.x was a better DOS than DOS, much better for software development. Many dev tools were bound to work under both DOS and OS/2 1.x.

I'm pretty sure cc.exe and make.exe exist entirely to run the NIST test suite.

It's a miracle those aren't just one massive if-else block that spits out fixed machine code that passes the test depending on the input file name.

@@NCommander running NIST tests suite incidentally is a good candidate for “something to do with it”

That Windows NT networking stack was in fact copied directly from the BSD one.

Windows has the equivalent of fork(), but it is expensive to use. For example, back in the days when Git was becoming really popular, Mozilla were looking to switch to a DVCS (I think they were using SVN or something). But they had a requirement that whatever it was would work well on Windows. And Git, at that time, made too much use of fork/exec to work nicely on Windows. So Mozilla went with Mercurial instead. Git has been modified since then, to tone down its appetite for spawning processes, to cater better for Windows installations. On Linux, of course, there was never a problem.

Cygwin has a spawn() function that basically emulates fork/exec with native Win32 calls in the event that that's what you really want to do. But it does provide an implementation of fork that's kind of hilarious in the lengths they had to go to. Obviously it's incredibly slow, but the best part is that because of address space randomization it can sometimes just fail if Windows by chance decided to stick a DLL in the wrong place first. It will retry a finite number of times but there's always a remote possibility that it will just have to give up eventually.

@@lawrencedoliveiro9104 On early Unix systems fork() was also an expensive operation. Since every writable memory page in the parent's address space would be copied to the child. Hence there was vfork() which would share the address space and suspend scheduling of the parent until the child called exec() (or exit()). In more modern systems writable pages are shared and marked Copy on Write.

I put a pinned comment about this.

A TechNet subscription (for SysAdmins, not developers) should have also provided the necessary resource kits (but not SDK's).

The real punchline is that Microsoft developed and sold a very popular commercial UNIX versions of the 1980s: Xenix. Even considered pushing it as a replacement for MS-DOS... But then AT&T started selling UNIX directly, and Microsoft recognized pretty quickly it couldn't compete with the company it was already paying licensing fees to, and decided to work on a different successor to MS-DOS - OS/2. As for Xenix? That got sold to a company that was co-developing Xenix with MS: The Santa Cruz Operation, or SCO...

@@watchm4ker I would love to get more history on the NT name, some say it actually came from a version of Windows developed for the Intel i860, code-named N10 ("N-Ten")

@@Rouxenator That, I don't know. I know it used to stand for New Technology, at launch. And that it's... Basically the original version of OS/2 3.0, after Microsoft decided to abandon THAT partnership.

@@watchm4ker There's no relationship between OS/2 Warp and Windows NT.

@@watchm4ker Ah yes, back when getting a Unix box at home was scathingly expensive. Linux really did change everything.

They do not. And haven't had a CLI text editor since 'edit' in the 16 bit days No wonder WSL is a thing, CLI workflow in the most basic sense is basically non-existent

Strange and incorrect characters in the stream? Sounds like logging into a non-Microsoft IRC server using MS' Comic Chat.

@@RadikAlice Yes they do have a terminal (console host, using pre-11 nomenclature), a bunch of improvements was added to conhost in 2019, and now Windows Terminal is default on W11

That was Services for UNIX and Windows Subsystem for Linux v1.

Wsl is a more 21st century solution with containers and virtualization

SSH wasn’t enabled by default in PowerShell until W10 1809. It was a huge leap forward.

WSL1 isn't binary compatible, it is syscall compatible. It also wasn't the first time it had been done either. Circa 2002 out of MS Research a similar project was done which could run Linux ELF binaries on a Subsystem. To demo the ability they were running an Apache webserver and had Firefox accessing it. It was never going to see the light of day with Balmer leading the company. WSL1 is more like WINE in that the syscalls were handled by Windows and translated on the fly to an equivalent set of Windows API calls. WSL2 is more like And Linux or CoLinux where the kernel is actually running in a VM and is isolated in its own Hypervisor. Each have their own strengths and weaknesses, but WSL2 with how it visualized the file system and hardware is more resound and can keep up with Linux kernel changes in a way that WSL1 can't. The magic is how they've bridged WSL and Windows so that they are both working together with only a few seams if you know where to find them. For most of my Linux needs, Windows is the best Linux system I need and it is easy to switch between different Distros. POSIX on the other hand. It was available for very custom work and not expected for more than supporting those very bespoke customer solutions. The subsystems were initially designed to support more than the Win32 API, but with IBM and DEC usage being significantly anemic, OS/2 and VMS compatibility wasn't a priority. NT 3.1 and 3.5 were intended to offer more, especially on the POSIX side of things seeing as Dave Cutler and many early NT engineers had built VAX for Digital before coming to Microsoft. The bones were there and have been from the beginning -- NT kernel was going to run Win16, Win32, DOS, OS/2, and POSIX applications, each in their own environments -- it was soon clear that Win32 was the Subsystem to design around. I think it would be interesting for Microsoft to revisit supporting more subsystems. There'd be a great MSR project to make modular environments to host things like a Node/Deno container, one for .Net CLR, Android, or a modern LISP machine, with a Win32 host machine providing a way to keep those other systems running in isolation. It would look a little like Docker Containers in some ways, or a NT-based Kubernetes OS. I think Windows 10X was sort of heading that way with Win32 apps running in their own container, and I think it is still a concept worth going after. WSA and WSL demonstrate that it is possible, but it still needs refinement.

It's on my TODO list, although I've covered xenix before I did youtube: casadevall.pro

I dunno. I don’t trust them with Linux. I’m sure they’ll ruin it to promote their own stuff if people get too comfy with Linux Subsystem

that is cursed

Sounds like a super convenient way to pull out your hair at some undetermined time in the future! Thanks Microsoft!

@@lucass8119 It's actually really useful for debugging when you are doing iteration on Windows but the target device uses a case-sensitive filesystem. You can set the folders as case-sensitive and catch those bugs early before trying to deploy and debug on the target hardware.

Now we just need ReactOS to proceed past pre-Alpha stage...

“Unix” is a trademark that only proprietary products can afford to license. And all those products are either dead or dying right now. What matters nowadays is not “Unix”-compatibility, but “Linux”-compatibility.

@@lawrencedoliveiro9104 While I understand your point, the wording made me roll my eyes and kinda want to say _something_ out of spite It feels insufferably pedantic, but maybe that's just a gut reaction from a bad first impression

No, that's completely fair. I think it got left out of the actual cut, but in the original notes, I actually did compare it to the OS/2 one, which can access NetBIOS, and IPC structures, and has more integration. Like, you could actually use the OS/2 subsystem as a migration path from OS/2 server to NT, and Microsoft's own pre-NT 3.1 early NT toolchain was OS/2 based and ran through the subsystem.

@@NCommander Totally. Around this time we were building Window 95 with direct X, then merging all of that code into the NT codebase to produce Win2k, figuring out how to service the OS, doing the IE thing, I mean the origin of Windows was a rocket ship. It was a very diff env from today, tbh noone particularly cared about POSIX. Linux was nothing even close to what it is today (a trajectory which changed dramatically when Google invested in Android) and it was definitely not a high priority. The situation has changed 1000% and WSL is a much more committed attempt and IMO absolutely critical for Windows' - survival given the importance of Linux today dev machines need to be good to work with Linux and OSX has an advantage there. But IMO Windows has made great strides here e.g. passthrough GPU virtualization for WSL usage is a major deal for machine learning workloads. Anyway im rambling.... Awesome video, awesome channel, thank you - subscribed!!

100 times this ^ POSIX isn't there to run some nerd's *nix nethack, it's there to run POSIX compliant software. That's all. I think folk confuse _most *nixes are POSIX compliant_ with _all POSIX machines should be same as some *nix_ POSIX is designed to be _Unix-like_ , not _to be_ Unix. It's very basic, and that's the point of it. [ and aye, as a software dev since the 80s, every company I've worked for from aerospace, thru shipping, to pure IT, has had a full MSDN if they were an enterprise business, it's the way it was - I've got folders and folders of the stuff in my spare room :) ]

Technically for a short while, Windows had: NTVDM - 16 bit, POSIX (lol), Win32 together. Technically also could run DOS apps.

Back in the day I used active desktop to have a live stream of some kittens playing in a cat shelter stream from a webcam as wallppaer. It used flash and you had to restart explorer everyone once in a while to fix the memory leaks.

Thanks for watching!

I think they were using HPFS at the time

Don't even need cygwin. Lookup "Link Shell Extension". NTFS supports both hard and sym links. And it has for quite some time (NT4!). That plugin has some cool functionality they built on top of the base functionality, so it's definitely worth a look.

@@ccoder4953 hell you don't even need to install anything, the standard command prompt windows ships with has a mklink command for doing this though having it as a right-click context menu is _very_ handy if you make links a lot

Symlinks were not added to Windows until Vista. Even now, they require special privileges to create.

@@lawrencedoliveiro9104 Actually, we're both sort of wrong. According to Wikipedia: "Symbolic links to directories or volumes, called junction points and mount points, were introduced with NTFS 3.0 that shipped with Windows 2000. From NTFS 3.1 onwards, symbolic links can be created for any kind of file system object. NTFS 3.1 was introduced together with Windows XP, but the functionality was not made available (through ntfs.sys) to user mode applications. Third-party filter drivers - such as Masatoshi Kimura's opensource senable driver - could however be installed to make the feature available in user mode as well. The ntfs.sys released with Windows Vista made the functionality available to user mode applications by default. " As far as the special permissions thing: "The default security settings in Windows Vista/Windows 7 disallow non-elevated administrators and all non-administrators from creating symbolic links but not junctions. This behavior can be changed running "secpol.msc" the Local Security Policy management console (under: Security Settings\Local Policies\User Rights Assignment\Create symbolic links). It can be worked around by starting cmd.exe with Run as administrator option or the runas command. Starting with Windows 10 Insiders build 14972 the requirement for elevated administrator privileges was removed in Windows "Developer Mode", allowing symlinks to be created without needing to elevate the console as administrator. On the API level, a SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE flag is supplied for this purpose" So, the policy can be changed and it appears Microsoft will soon make the change the default.

We've been having a long discussion on it on Discord, and how it works (which is abusing cygwin). It's on my to cover list as a stream if nothing else.

Yes, my thoughts exactly. And didn't they send their own people to the standards meeting just to vote for them, and never show up again? I remember that causing a problem because they couldn't get a quorum or something. IAC, what they shipped was the original version without any of the feedback fixes needed by the eventual standard.

Microsoft itself does not support ISO 29500 (OOXML) properly. Microsoft -Office- 365 still only implements it in “Transitional” mode, since “Strict” mode apparently causes too many compatibility problems.

For a while back in the early days of those standards I remember discovering that Word's implementation of the ODF appeared more true to the official standard than OpenOffice Writer's was. This was because Writer had already progressed to a later version of its native ODF format by the time it became standardized, which had more extended/updated features. I cannot recall that it had any way of saving/exporting to the older but standardized version. Word's way of supporting ODF was rather cumbersome though, as one had to install and deal with it as a separate plugin. Conversely OOXML has always been a mess from its very inception, allowing for including BLOBs in proprietary formats that only Microsoft itself ever has a chance of fully supporting.

SFU is absolutely much better than this. Wow.

SFU is much more complete and better than this. Not only it provides much more Unix utilities and POSIX API but later versions even contains X11 libraries. SFU is much more close to something made properly than this thing ever was.

You know, this is a fair point.

It all came from the KZfaq Music Library

That's what a lot of people have told me. SFU was really when it was taken more seriously.

@@NCommander I've been in software for 40+ plus years, an I have been burnt many times by emulating other peoples environments. Its always a moving target, and its really amazing how common it is for programs to skirt the rules of an interface and take advantage of undocumented quirks of the interface. This is one reason why I am not a fan of either Wine or the Linux on windows subsystem. If I want the Unix/linux environment, I'll just go use Unix/Linux. If I want a program to port to a different environment, then recompiling it for the new environment is the way to go. For programs that cross multiple platforms, that's why god made cross platform porting kits. Put it this way: a program designed for porting via a kit had portability in mind from day one. A program written for a given interface (Windows/Linux/Mac) was designed to be bedded in cement from day one.

Indeed. I understand why they went with WSL2 in the end, but WSL1 seems more ... elegant, for want of a better word ...

I will at some point. It's on the topic list.

There wasn't a lot, most people complaining they lack parts, or being told it was useless.

It actually doesn't work correctly with Traverse Checking left enable. sh.exe doesn't load its prompt properly and there is a bunch of weirdness.

It's on the topic list.

its funny you say this because there are actual email exchanges dated 1990 between the posix committee and the NT team in leaked windows source which basically go like this

I've been livestreaming, but making video content has been difficult. I've made some headway on a new video, but its been very up and down in real life

@@NCommander It makes sense, don't let my comments build any pressure. I really just wanted to say your videos are awesome!

The original Windows NT 3.1 TCP/IP stack was a direct port of the one from BSD 4.3 by Spyder Software. I've always assumed it was a holdover from that, although having a hosts file is de facto standard (and might even be in one of the RFCs as required).

@@NCommander The hosts file predates DNS.

On GNU systems, whether you need a hosts file or not actually depends on settings in your nsswitch.conf.

Remember Exceed for X11, but I mostly used Cygwin and Mingw32 for terminal stuff.

@@butsukete1806 MKS Toolkit was another early player. I think Cygwin became popular a bit later (and obviously I used that once it did).

This was part of the original architecture of Windows NT: there was a core kernel, on top of which you could have multiple “personalities”, each with their own userland API. Win32 was itself one of these “personalities”, and so was the POSIX layer. In the early days, Microsoft did let the documentation get out on how to write your own personality, and the Interix folks got hold of this. Then management changed their minds and decided to restrict this info. So they bought up Interix to prevent the details getting any further, and the Interix product became Microsoft SFU. You’d think present-day WSL would be built on the same architecture, but it is not. WSL 1.0 might have been built that way, but they could never get it working right. Which is why WSL 2.0 falls back on good old virtualization instead. And still doesn’t work completely right. So one could say that the original “personality” architecture has pretty much bit-rotted away.

@@lawrencedoliveiro9104 You meant I can't run my OS/2 1.1 and 1.0 apps anymore? And yes. That was another personality. And yes, I'm still bitter about OS/2.

Need DEC Alpha hardware. The compilers are all native.