"Programming is just Excel without the boxes" - Real Civil Engineer

Spreadsheets DO have a use-case. Just not as a back end or as a stand-in database.

I remember watching a manager at work, enter a bunch of values into a column. Then get out a calculator and calculate the total and put that number on the final cell.

"Excel is not a database" Hank Hill: "If those governments could read they'd be very upset

During the COVID pandemic UK government contractors used old version of spreadsheet to do COVID tracing. They lost tens of thousands of records because they reached the limit of 65536 lines.

I work on a mobile game played by millions. They made the specific decision to move TO using google sheets as the source of game data like assets, IAPs and so much more. They were so proud of it they wrote a blog post about it. Now if anyone updates one of the spreadsheets it forces every player globally to restart the game. As a previous data engineer... it makes me so sad

If you ever find yourself linking a spreadsheet to another spreadsheet, stop immediately and atone for your wickedness.

19:52 dev: "But it works on my machine" company: "but are you going to give your computer to the client?" dev: "yes, I don't want to rewrite this thing"

Excel, the most commonly used distributed database.

This author clearly has never worked at a company where it takes 6 months to get IT to setup a database.

We used to scrape Windows Stores front end as another team WITHIN Microsoft. We’d scrape the store instead of some sane data delivery, pump it into several excel workbooks, load those into Access after review, and then run a rube goldberg machine of powershell scripts to execute PowerPoint desktop app instances on the cluster of windows servers to generate reports that nobody looked at. I shit you not, this abomination was written by actual Microsoft employees.

I’ve done things with Excel/VBA that many people aren’t aware are possible. Chiefly because clients have back themselves into corners and had no other reasonable recourse. Terrible decisions. Many terrible decisions. Spreadsheets have their place; running businesses off them entirely is an egregious thought that I’ve seen too many times. I have the thousand-yard stare, boys. I’ve….seen things…

"I spreadsheeted really hard" The past tense of spreadsheet is spreadshat

You just angered the "Watches tutorials on Microsoft Office applications" audience that accidentally got recommended this.

Having worked in ETL and having been told that this business important, daily update data, was coming from an excel sheet on Bob's laptop that Bob manually updated by pulling data from our database systems... I fully support this gate keeping.

For a software development consultant such as myself, Excel/Access/Power Apps/ etc. are a godsend. Literally any business application you can devise could be an Excel workbook. That makes them the ultimate MVP. Once a company gets dependent on them, then scales out of them, you the developer just swoop right in. No selling required, the client already knows exactly what they want, and understands the business value.

This is why game devs know the difference between prototype and production. Every experienced dev says to never ship prototypes because you will always regret it. And often times you need different people because some people are tinkerers others make finished products.

This hits home so hard. I'm finally attempting to learn programming at age 32 almost entirely because I've been so completely victimized at a small business run on spreadsheets from vendors (that need to be updated manually every time you want to look up their current pricing), internal spreadsheets, and quickbooks running off a system of spreadsheets, tubes, and pulleys. The Inventory is a 204,000 line spreadsheet that needs to be updated manually every time you use it. I need an adult, and the business needs an exorcist.

The issue with most of this is the distinction between hindsight and foresight. If you start out to get something done, you don't know yet where you'll have to go. Therefore you just can't start out by building THE ONE proper perfect database for the task. There is much exploratory work in many cases. Most of the article seems to come from a point of looking back at what has already been built. Sure, then the necessary structure becomes obvious. But in a business environment, who is gonna pay you or give you the time to completely scrap a running and functioning feature just so that it becomes more "elegant"?

18:40 --- here's the problem... you haven't worked in goverment. Do you think we're just allowed to install cron and go onto our PC.

I work in manufacturing IT/OT. One of our manufacturing lines used paper forms in Excel to write down all the collected data for assembled products. In my tenure in charge of that line, I said "This sucks", set up some automated data-collecting routines in Excel to let us at least chart the form content over time (because each form tended to have the same tags next to the associated data) then transitioned to onboarding a better record keeping software. HOWEVER. Before I could finish bringing it online, the new engineer in charge of the production line switched the entire layout of the form so that batches of product could be squeezed on the same form. When they did this, instead of just adding new columns to the right, they completely re-wrote the entire form and added tons of merged cells in a way that made it impossible to calculate which data was associated with which product. This broke all of the process control charts, and when we went to do failure analysis later I had to tell them "Sorry, you're fucked. I can't back calculate any of this without someone manually writing it in by hand"

People at my office use sharepoint because there is no better option given by IT. The reality is it's not even that hard for a company to give everyone the access to create and manage their own databases, but no IT department allows it, so the result - everyone uses sharepoint lists, and if they don't have sharepoint lists, they use spreadsheet software. IT need to get out of their arses and allow people to actually maintain structured data if they want people to make structured data stores.

Working with Microsoft Access 2007 in 2024.. not ideal, but VBA is still a great way to learn how to code and come up with solutions to specific business requirements. Nothing better than an Access Form calling a batch script which calls a Python script which is listening for pngs to appear in a folder, so it can apply a frame to them and the end result gets printed via the Access Form. 😎

As someone who manages a data warehouse, spreadsheets are the bane of my existence. Avoid them like the plague.

Make a robot that moves and rearranges the pebbles in your lawn, and get the binary value via a webcam

Everytime someone says to use Excel for a process as a stopgap or temporary solution, I remind them that there is nothing more permanent in business than a temporary solution

"It feels so good [to hit the limits of a concept]" yeah, if you're not the one who has to refactor around that limitation...

What happens at every company everywhere is: Developer: "I can make it work in time, but we will need to spend a few weeks after release to implement a real solution." Manager: "OK, great! As long as we can release in time." Developer: *Releases dirty hack* Manager: "OK, now that Feature A is finished, let's start working on Feature B right away. Chop chop, no time to waste! It's getting released tomorrow!"

whoever wrote "hawktuah( )" in chat is my hero of the day xD

As someone who used to work with spreadsheets for so long (translation prep and QA somehow always boils down to spreadsheets, idk why but most clients translate those lol) this is always one of my first thoughts as well. Tho i usually go with csv instead Edit: rants incoming lol Now instead of Excel, in the new company, we use Google sheets and i wanna die. Day one, they told me that we need to check generated strings's (purposefully keeping it vague) uniqueness and length via ctrl+f in a column woth almost 10k results, and i died inside. Few minutes later, i learned Google sheets and added a conditional formatting to that column. And what did i find? Several dozen, if not more, repeated strings lol A few days later, we had to keep track of some categories' frequency of use, and i was like "great, I'll just write a few simple formulas to count them!" Co-worker looked at me and said, "wait you can do that? I used to count them by hand for months..."

This is my life. I just wrote a 2k line VBA application in MS Visio to parse .CSVs and draw diagrams.

"Every last line of code you will ever write is one layer on a shit onion that is slowly being added." Never have truer words ever been uttered.

having to repeatedly ask Flip to take something out to be clear that it's not a joke is so funny 😂

The team I'm contracting with is doing exactly this. I wonder if the author is at the company. > I wrote a Python script that took a spreadsheet as an input, ran regular expressions in it against a database, spat out ten more spreadsheets, then used those to power a generic PowerBI template which had to have all ten spreadsheet targets repointed by hand. This became one of the most requested services my team performed for several months.

its honestly so fun to have an entire business just relying on this one computer with some port forwarding tucked away somewhere

Spreadsheets are evil. In the army while working with spreadsheets I've witnessed utmost nightmareous files Randomly merged columns, zero-width rows, among many other things

Unless somebody here can offer a good suggestion for getting a somewhat-layperson to enter data as easily and efficiently as they can to a spreadsheet, I'm not fully convinced.

Ours is an iMac. Guy wrote scripts that have to fire on his desktop for the company’s daily pricing

You can do databases wrong. Believe me. You can absolutely make them a umantainable mess. I've seen people create subsets of a database with millions of entries in a text file.

7:00 My department currently utilizes a dashboard built on about 100k lines of VBA. It's technically on Excel but disables all of the Excel features for performance reasons. I spent 3 months trying to refactor it before giving up because it was such a deeply entangled mess.

Depends on the spreadsheet. I worked at a shop once where they had a mainframe job whose //DD * records they would fill out manually by retyping from a spreadsheet. When I suggested they simply reformat their spreadsheet in a very specific but easy way and upload it to the mainframe, then change to //DD DSN=??? (I think that was the syntax, 20 years is a bitch on memory), IT screamed that it was impossible. So I showed it to the poor lady responsible for this transcription, and she loved it. I did not like their IT, which refused to move out of a box very carefully constructed in 1977(this was in 2006). And I was in that IT department, too.

20:15 a server that is a random computer running an important infrastructure… At Apple they're called closet servers… but you may find them in somebody's office, behind a desk… hanging from the ethernet outlet.

My friend made a JRPG in C# (XNA), using excel exported to csv.. all from stats, to dialogs to events etc

I feel like getting to know Excel, VBA and Power Query gives you better idea about how to write business requirements. So many devs complain about operations not being capable of giving clear requirements. How they suppose to know if they cant write a basic working logic themselves?

It's actually neither hang or hung, but hanged

I am a programmer. I use a spreadsheet to wrangle data all the time. It lets me do stuff I would never do by hand. It is just a tool. Don't really care if someone uses it as a database. If you have toy sized data you can use a toy sized database.

MS Access is what got me started in IT.

I worked for Nasdaq around 2000 and their entire payroll system was on a VAX that was in a closet since 1990. Nobody knew how to use it and no one dare touch it.😅

it's why old MS Access was great. small projects that need a portable database. now that spreadsheets can handle a million rows, it's not as attractive

Unpopular opinion: most developers know fuck all about databases. I met actual 'professional' software developers who didn't know what an index is.

As a federal employee, I'm starting to understand why nothing ever functions properly. We use spreadsheets, Access, and SharePoint for everything.

The simulation in which we live is really spreadsheets all the way down.

When I worked in cable, we had to manually upload a CSV every few days to feed the TV Guide's data.

10:38 - there is a joke pattern for this and it is called annual rings: each year we add another layer to decouple us from the crap we built last year.

Oh man I just binged on this guy’s articles and spat out my coffee. 11/10 strongly recommend.

I maintain major companies excel sheets that my brother wrote for them 12 years ago and they have no idea how they work.

No way punk. No way will I be separated from my spreadsheet. It's not going to happen. It is the ultimate DSL.

Yes, I agree that spreadsheets are not databases , but , For data science nothing beats a spreadsheet. Sure you could program something better but that takes time , and time is money. 16:39 once you're writing Vbasic scripts with the above point, kind of goes out the window..

omg he just called our entire MDM out! There's a mac mini in our Corporate Office's Tech closet.

And I need to define a process to automate data processing using Python. Ah yes, the data is spread across a dozen spreadsheets, Word, PDF, manually entered by colleagues and business partners with limited data validation and recognizable patterns. You guess it, only a hell of condition branches and loop can help you here. One automation (and there are more than 10) can take more than 40 minutes, but it can save weekdays of labor work. I sympathize with those who needs to manually process the data entries before.

Here's the kicker, an excel spreadsheet is free. A database and client web client costs a metric tonne of money to make, and metric tonnes more of money for all the little changes that are required when the product doesn't do what it's supposed to. And then the website gets bloated and slow. Malformed Excel spreadsheet that requires manual input, no validation, and likely not great for making statistics from. Or expensive slow website that's stuck at being 80% done?

all corporations run on spreadsheets, u wont change that, by seeling them some bloated overhaul over Spreadsheets, they will still download a spreadsheet and use it as such. I've seen 100+ MB spreadsheeds used as database in almost every job i had. and im the one who comes over and tells them this should be a database, and they will still use spreadsheets, and i will work on spreadsheets doing what they asking me to do. because that's what they know and can understand.

When the company has the technician doing a task that can easily be automated but they forbid any kind of developing tools or third party applications, that Excel file with a VBA macro is the go to solution. It might be what saves them from burn out.

7:10 "I know a lot of you are pumped up for government healthcare" assuming admin systems around private medical insurance and healthcare is any different or better. I'll never understand STEM guys who don't bother to look at the numbers and just say stuff like this. The American healthcare system is a textbook case of monopoly, rent-seeking, price gouging, pretty much all the distortions and excesses you get when you let the market provide a vital service to the public. The RAND corporation found that single payer would save billions of dollars by REDUCING administrative costs and controlling arbitrary price rises (what we are seeing now in housing, food, etc with "greedflation") People obscure this stuff by calling it "political" but in reality the numbers are clear. The political part is the influence of the medical lobby in addition to vibes-based ideologies like the one Prime unfortunately seems to have on this issue. That's what keeps us all from saving a lot of time and money.

I love me my spreadsheets. This is the hill I'm willing to die on.

When your IT department flat out refuse to install Python and your BI department take 2 months to build a basic dashboard, spreadsheets are great.

I freaking love this channel. It's one of the only places where smart people are, and it's one of the few places where I'm decidedly in the bottom quantile of skill/IQ. I find myself watching it even when I have no direct interest in the subject matter and when said subject matter flies straight over my head. Jacobins mentioned around 2:05. So few people would get the reference to who the Jacobins were and what they were all about. Thank you. Please keep streaming.

Remember that time Public Health England lost 10’s of thousands of COVID test results because they used a spreadsheet instead of a database? Excel is not a database. Also ChatGPT isn’t a search engine. It scares me that people use it as one.

Spreadsheets are awesome, it's how people use them that is cringe. Sharepoint lists are awesome, it's how people use them that is cringe. There is no reason to not use sharepoint lists as a simple request system. It takes like maybe a couple hours to make one and Ive done it for requesting label printing and managing requests for an internal machine shop and it works well. These lists have worked for 3+ years without issue ot much modification as things were thought about *in depth* up front, they arent exposed beyond one building, they dont track sensitive info, and they are used by very few people (less than 100 for the most part) You know how long it takes to ask IT to install a database, to write code to access that database, to write a front end to abstract all the query language? It takes much longer than a couple hours. Not to mention, databases have their own issues with being *EASILY* improperly designed, multiple key structures for query optimizations, changing table schemas affecting *the entire db and all its applications because of foreign keys* The argument should be problem scope, not use an M-16 for deer hunting. Remember, its perfectly fine to use a screwdriver to drive in a screw, even though an automatic drill exists.

I can't hate excel too hard - Where else do you get reactive programming... a visual repl... with 0 new dependencies... and usable from the start by a non-techical audience? Version control however...

I'm a civil engineer working in the Transportation field. Never got the time to learn programming well enough to apply it to our niche problems so I end up brute forcing a solution through an excel worksheet. Sometimes learning python or whatever just doesn't feel worth it when you can easily do the task with a couple of hours of grunting and head scratching while staring at a spreadsheet.

I was on a team developing a SAAS for shipping and logistics. Of course we needed truck drivers so we started a separate business to hire truckers to field test the software. We never intended the test business to be our primary source of revenue, but COVID happened and shipping went crazy. Next thing I know, were running 100+ truckers in 48 states delivering well over $100 million in products annually and the backend for the whole operation was three guys and an Excel spreadsheet that was stored in box, but could only run on one specific machine because all the macros used fixed paths. It was SOOOOOO janky, everyone was afraid to touch it. It frigging broke at least once per day. It was a literal Rube Goldberg machine.

So, do you really want to have some person from HR or worse from bookkeeping, come to your desk bugging you to do some stuff for them on pretty much a daily basis? Instead of them being able to do it themselves via spreadsheets and the occasional VBA (ok, sometimes you will also meet the accountant where his VBA script is in the tens of thousands of lines)? Until they get a frontend in which *they* can write their own scripts and visualization in, spreadsheets are more of a necessity.

The fact that you can copy tabular data from a spreadsheet and paste it into a web page and it keeps its structure is super slept on.

A place I worked at had a BSD machine that ran our local Cygwin mirror. Our entire build and dev process depended on custom Cygwin packages on that mirror. It never went down and nobody in our 15 engineer team knew where the box physically was. The landlord sold the building, and as part of some new remodeling the new landlord found a doorway to a small closet that had been plastered over. Our BSD Cygwin mirror was humming along on a shelf in that closet.

Funnily enough as a mechanical engineer we also had a lecture where it was emphasized that excel is not a program for leadership and data between different levels of organisation. It is not ERP or SAS or whatever there is. On the other hand for some group work I made a table of multiple values and scaled it to thousands of entries and made a graph that would show a cut-off point. A classmate copied that graph to our report in Word. We had to abandon the report file because nobody could load it up successfully for those massive data entries directly linked to the small graph on one page. The one thing I actually wanted to do with Excel was a transformation from a drop down option to actual value. Like you'd choose in your gym program the % of 1RM and it'd convert into an actual weight in kilograms in that cell as you've chosen it. Without additional number and value comparisons. But apparently that required VBA and it got way too messy and I decided to never go that far with Excel. It wasn't even necessary to do that way, but I already had way too many sheets of numbers and calculations of % of 1 RM and corresponding max reps, I just wanted it to look neat in the logging side.

Programmers often forget that the main purpose of the program is to solve a problem, and to solve it now.

Spreadsheets are great. I made my business analysts write the tests, thus shifting left the responsibility for bugs. I can't make my BAs edit sqlites, but excels, no problem!

I feel like actuaries and accountants are the prime targets for this rant.

I can tell you why this happens. The spreadsheets are built by people that can't get access to servers or other it infrastructure. I built some great stuff in excel only to be told by IT that I could have used another program for them not to realise that we didn't have access to those programs. Even if we asked IT would have cancelled the whole thing.

My best work at my last job was making a complicated front-end for a database application inside Excel. We did that because a) I was not allowed to code something from scratch, b) rolling out software throughout the company was a nightmare and Excel was already working on all machines, c) users rejected the jerry-rigged web interface of a completely different system for it, and d) the hairbrained decision to do complex calculation logic as stored procedures inside the DB instead of literally anything else was made against my strongest objections already.

5:57 imagine living in 2024 and suggesting Index/Match or VLOOKUP over XLOOKUP.

I once created a million cell spreadsheet to implement a an algorithm I used in my undergrad thesis project to calculate electrical properties of custom precision wound coils for work. It took about 1.5 seconds to refresh every time I edited a cell. So satisfying!

Me reading this yt vid title, having just made an excel exporter for finance dept: *nervous laughter*

I just made an automated process using a spreadsheet a couple days ago. It was soooo easy to set up and it just works. There are not that many projects, especially outside large corporations, that need a millions of rows that would choke things up. The real problem was the difficulties he ran into dumping it to a database when they got too large.

spreadsheets are great and anyone who is THIS against them is an idiot. They are so flexible, they work in any business team as people understand them intuitively, they transfer across platforms and clients as everyone supports spreadsheets. being a purist sure avoid them but you will never remove spreadsheets for how flexible and powerful they are and easily they integrate every where. edit: also they're cheap, another thing in their favor. Oh I can pay this one accountant team member 50k a year to spend time building this OR I can try line this up with the IT team with an engineer who is paid 6 figures but they have a full roadmap until next year and they need to do budget calculations/adjustments and then there's multiple license fees on top I have to pay to get it going? no thanks. ill take the spreadsheet and get the job started and worry about that crap later.

The pebbles in your front lawn thing, is REAL. Ethiopian shamans would use binary multiplication using pebbles put into two rows of holes to tell people how much money a herd of animals was worth. It worked because the first row was used to convert one number to binary digits (by halving the number of pebbles in each slot in the row and then seeing if the result is odd (1) or even (0)), and then the second row (which instead doubled the number of pebbles in each row) was basically multiplied by the bits calculated in the first row, which calculates the partial products that just need to be added together. The example used in the article I read was 34×7, which looks like this (zeros added so things line up nicely): Row 1: 34, 17, 08, 04, 002, 001 Row 2: 07, 14, 28, 56, 112, 224 After determining which columns in row 1 had an odd or even number of pebbles, and then removing the pebbles from the bottom row corresponding to a value of 0 in the top row: Row 1: 00, 01, 00, 00, 000, 001 Row 2: 00, 14, 00, 00, 000, 224 So then they'd just count the remaining pebbles in row 2, and there's your answer: 238. It's worth noting that the binary value of 34 is used, but is technically backwards; 100010 is 34 in binary, but when actually performing the calculation I have the bits backwards. That's simply because it's annoying to halve or double a value from right to left while typing.

I spent over 1000 hours building process infrastructure for my company to never have this problem again. Well engieneered solutions are now easier to set up, deploy and maintain rather than any VBA or Excel based solution to any problem. I also built components to ingest and export Excel content dynamically described in a single schema validated file per job.

I will effin laugh at you all when you need a solution quickly and the dev guys say "come back next year" and I'll build it for you in one afternoon in Excel! 🤩 There's a time for proper development - and there's a time for spreadsheets. I'm sorry.

Sometimes i use google sheets as a temporary database 🙂

Holy shit, Order of the Stick mentioned?! That article just became a favorite 🤣

I'm a former Metrologist and used to program automated calibration procedures in Excel using VBA to acquire measurement data back in the mid 90's. We've come a long way since then.

I mean, for completely automated tasks, a spreadsheet might be superfluous. For cases where a user might need to check the data, or even calculate something, having it in a format that many people understand how to modify is invaluable. Putting data in SQL can make it inaccessible for many to even select, let alone update. Furthermore it might just be simply easier than developing all the needed calculations and possible calculations and graphics in code.

did prime make an 'Hawk Toah' ? :D

SQLite is a file, with no UI. Excel is a file, with a billion dollar UI. I feel like this is an awesome example the normal distribution of skillz meme, Noobs use Excel and Staff+ Engineers use Excel (and Windows with WSL in general).

I feel this article so much... I work in a laboratory and I worked in another lab before that (material scientist turned software engineer). Labs are the manifestation of "temporary" Excel spreadsheets. And there is a simple reason for that: Lab software is unbelievably expensive and still they are all without exception complete garbage. The developers have never been within 10 km to an actual lab so they get advisors from a university. But newsflash: universities are not how labs operate. It's like asking a dutch riding his bicycle to work about advise on driving 300 kph on the german autobahn. So every lab has that one guy/girl that "knows Excel" and thus creates all test evaluations in it. I have been that guy. I have been so good at it that I learned to hate Excel with a passion. It is a great spreadsheet programm but complete and utter garbage for everything else. I hate it! Microsoft should roll out an update: "F*** you! We removed everything that isn't relevant to spreadsheets.". I managed to convince "that guy" in the lab I now work at to learn Python with my help. And if he ever creates an Excel spreadsheet again I swear I will use all my non existent martial arts experience to beat him until he sees reason!

using a spreadsheet to show customers their revenue is really nice. Also using spreadsheet to show stats to product owners is also nice

A sane engineer?? We have those?

The moment your first instinct is to use a spreadsheet as part of your process, is the moment you evolve to a principal engineer.

From Google Sheets, they now have named functions, various array functions and lambda... Even using only built-in functions to play with data can be so handy and so much fun!