AWK is not just a text processing utility, awk is a complete programming language!!

At 8min, when you pipe uniq, note that 'uniq' does not detect repeated lines unless they are adjacent. You may want to sort the input first, or use 'sort -u' without 'uniq'. so it would be | sort | uniq OR | sort -u

In the square root example, very time DT said square root, he means square! Great video, DT!

I just came to say that I can see that you listened to the comments and now you are sharing your knowledge on simple but technical stuff that's really helpfull. I apreciate that and also can feel that effort

This is a great tutorial about awk. I've learned quite a bit. The only thing that red flagged me was that NONE of those were square roots. They were squares.

awk is a must. So powerful. Btw. You actually did just square. Not square root :)

DT: AWK is important for linux users Luke Smith 1 hour later: Why AWK is useless for linux users

That is the most clear demonstration of AWK I have ever seen. Amazing.

14:00 - /^[b,c]/ matches anything that begins with a "b" or a "c" *or a comma*.

Glad you made this video. I was stumbling upon awk in everywhere and wanted to learn it but didn't have time to do so. Thanks

Great TUT. I ran into an issue because I don't use uniq and sort often enough; uniq doesn't work if dupes are not adjacent so you need to sort first.

Had to process a 120k line csv at work the other day. Awk was a god send

Learned more about awk in 20 minutes than in the previous 40 years

Good video. Some of these, I didn't knew. An error in the video: it wasn't the "square root", but just the "square", or "power of two". The square root of 4 is 2, the square root of 9 is 3, the square root of 25 is 5, and so on.

Thank you Derek I appreciate the time and effort you put into making these videos. You help us "middle users" take things up a notch. Middle as in we are no longer beginners but won't ever be sysadmins. We do however want more out of their Linux system.

This is one of the best explanations of awk I've seen! I'll save this one for some people I know that are still just learning Linux.

Sir, your a wizard with awk, i only knew half of this 😂

Great tutorial DT! AWK is very powerful!

I have been using linux for years living on a few commands. I had only a faint knowledge of the command/programs such as awk or sed. Mainly because I had no ideas of their capabilities and use cases. Now I can see how useful they can be. I usually recommend your channel to my PhD students and observed a real improvement in their computer skills. IMO your channel is vital for linux wannabe and "familiar but not expert" users. So thank for your work.

This is great, this video arrived just in time since currently I am on problem automating a lot of stuff with bash scripting and using awk for text-processing. Thanks for the content!

Good introduction to awk. Back in 2013 i had to parse huge log files that had no field separator, even worst each line could be completely different depending on certain conditions. Awk saved my life.

Nice, thank you. I am beginning my journey on bash and this is awesome.

Great video. An accessibility note: Your console’s active line is at the very bottom of the screen, which is where KZfaq on Smart TVs puts the subtitles, so the subs block the thing you’re typing. It would be better if your terminal was sized so that the bottom 10-20% of the screen isn’t used.

awk is awesome, but if you just want the number of lines, using wc is less characters to type :)

I bet someone is already rewriting awk in rust for the memes.

The simple fact that fish is your default shell deserves a like and a subscription.

I love you man, you're the only one who was able to explain awk in a simple yet effective way. I finally understand how awk works!

I have always had trouble with awk because of all the syntax involved, this helped sooo much. When you pressed enter on the square root part then said that the output was correct, this is the task list I had to perform to get back on track: 1. Facepalm 2. Pull fire alarm 3. Crying session 4. Shower with clothes on 5. Call Mom 6. Eat some sweets 7. Finish the video

Great video DT! Thanks for the awesome awk tutorial Except: s/square\ root/square 😀

DT, you are amazing man. Yours are the only videos I have been watching on a daily basis. Thanks for sharing all the knowledge!

Great tutorial. I learned more in 20 mins w/ awk then in 7 years in IT.

A little bit of Awk can be useful. My standard example: I have a list of shares I track. The website I use lets me download data, but holds more data (open, close, max, min,volume ) than I want. Awk is perfect for filtering a single column (close values).

Straight to the point & excellent as always. Keep up the good work DT ! 👏

Needed a quick refresher on awk as I had no used it in years. This was a great video, covering the essentials. Cheers.

small suggestion, bring the bottom of your terminal window up a bit so the youtube progress bar doesn't cover up the command you're entering when the video is paused. Thank you for making this!

I think a very good example of how good & useful is awk is the example of a log like the apache access log (in the default format) and properly filter the requests that responded with a 400 or 503 response, given that you can find 400 and 503 in the request address, or in the size or the time it took to complete. Also to filter all requests that are longer than a second for example and many other things like that. It's basically Excel powers in CLI over column-structured data/files (which are common in commands outputs or log files)

I have been putting this off for a long time thanks for encouraging me to finally learn it.

Thanks buddy for such clear explanations. Now I'm comfortable to start write some crazy scripts.

This is really helpful and practical, thank you DT!

My payment will be available in 2 weeks, then I'll be a patreon. Your content is amazing.

Awk ... very meaningful name !!! Looks like an assembly instruction.

As I found out, you have to sort _before_ running the _uniq_ , otherwise it wouldn't work.

That was some awesome awk-fu bro, thank you!

I learned so much more here in 20 mins then I did in my last two classes about this

Hey DT. Great review of the power of "awk". You cover all the basic functions for a new user. I use "awk" in a shell script to sort through my datalog's results that i need. Then dump it to a csv file.

Hell yeah!!! AWK arrived at the right time!! Thank you Derek!

Nice Luther insult at 8:20! I for real laughed out loud on that one. I'm surprised no one called that out in another comment...😁

This is truly amazing, hope to have another new Awk video in the furure!

Thank you professor DT for the lecture .

A couple months ago I wrote a little shell script to gather system information such as CPU model, memory, computer model, shell, WM (using lscpu, dmidecode, etc) and I had to learn a little bit of Awk, Sed, and Grep to clean it up and make it look pretty. This video would have came in handy back then, instead of spending 30 minutes reading StackExchange articles on them!

Thanks for this video. It helps me understand how to use awk better.

First: a good video, thanks! But it might need a "part ii", going deeper into what makes awk special among other languages. Awk is what at least I call a "pattern-action" language. So, the thing before the braces is a pattern, a kind of conditional structure telling for which input rows to execute that given block of code, and the block of code is then what to do when the pattern does match. Pattern here refers to a much wider set of conditions than just regular expressions; it can e.g. be a comparison against some variable you set yourself within the program, based on some previous data, and so on. So, if your statement within the braces is just "if (some condition based on the input row)", you can just lose the "if" and move the condition outside the braces, to have the same effect. You also briefly show that the pattern can specify a range of rows, by having two expressions to match the first and last rows of the data to process for that particular action. This could be for example "/start-marker/,/^$/" to process all sets of rows that begin with a row containing a "start-marker" and end with an empty row. Whatever comes after the empty row before the next "start-marker" is ignored (by this action, but you can have other actions in the script to do something else for those rows). The patterns for the beginning and end of the range do not need to have any relation to each other. These together make ti possible to very rapidly build "state machines" for processing text with some more complex but still regular structure (IBM JVM garbage collection logs being one example of such data format; I've done some awk pieces that pull out numeric data out of those and then output that data in a "only numbers" CSV form that can be pushed out to a spreadsheet for producing graphs out of the data). It is of course possible to do the same with any general-purpose language, such as Python or perl, but they don't lend themselves quite as well to building such processing state machines as awk does, it takes much more time to have the equivalent Python/perl code look anywhere as neat and readable.

This is the kind of stuff I love to watch

Thanks for the share. I used use text editors like Textpad to do these things with regex, but this is so much better since not just find/replace, but perform operations on each result, and being able to easy have in a shell pipeline of commands. Always too lazy to learn it for some reason, but now I'm motivated.

DT, this is an excellent video! Thanks!

Very useful info DT! I only knew how to print columns in awk.

Heaviest 20 minutes in my life. Worth it

love both Sed and Awk

Your audio sound 1000x better then before

Very nice introduction to awk features. Thanks for sharing this useful knowledge!

Yup, it's a complete language, as it is taught in CS degree courses (when taught). However this guy, otherwise, is a far more interesting teacher. Thanks, DT! This app/language confusion here is, like with Python, there is the language, AND there is ERIC, a free IDE that comes with Python. ERIC executes Python, but it's NOT Python. The AWK IDE is more invisible, but there. You can write independently, then execute.

Yup, awk is cool. Just as an aside that was squares, not square roots. For square roosts you would need something like (n^(1/2)). AWK does have ^ which is raise to the power function. Raising a number to a fractional power is taking the root.

This is a great video on AWk..thanks

This couldn't have come at a better time! I was seeing this command everyone in people's dwm configs and had no clue what it is.

The problem with awk is that most people know it but all seem to think its whole purpose is just to extract columns. This Video gives many practical example that it's so much more.

Great tutorial! I do prefer the WTF command. It covers all my confusion when trying to navigate the command line. 😜

Okay so nube here. I need to either awk on windows--what's the Linux on widows thing---or htf do I export *that* ou for awk to process a "computer accounts" move request the admin is clueless about

This is a great review of some practical usages.

Excellent presentation!! to the point and straight forward! great job!

I really need to push myself to use awk more. A lot of this can be done by just combining grep, sed, and cut, but it’s kinda nice having it in one process.

To print the last words or specific words we can use the cut -d "." -f3 or -f3- -f3 prints only 3rd word -f3- prints rest of the strings including 3rd word

Great introduction of `awk` 3:30 can be done with 'getent passwd' :-) 13:45 you print the square of, not square-root of.

Your example for the line count can be reproduced with the command "wc -l /foo/bar/file_a /foo/bar/file_b"

Awk is awesome, great vid I learned a lot

Great video, helpful comments and even better resourceful and corrective people in the comments

I wish to have a series on awk,sed,grep from you dt just like Emacs and vim.

Please do jq tutorial next: it's so awesome to manipulate json data, combined with curl it's very powerful for working with REST apis

i always found it easy to use because i'm AWKward.

Thanks for making this video! I've tried a couple of times to add AWK to my arsenal, but gave up halfway. This has inspired me to give it a more serious go. I'm generally pretty comfortable with grep and sed but looks like I can make this work for replacing those two for many use cases.

I am actually going through the famous Awk, the programming language for the last two weeks. And, I spotted a least 1 verboseness. When you are trying to format the output of your shells, you don't need the regular expression to look for slashes. Not sure why you did that. It might also be helpful to people to know that you are turning NF(which counts and stores the number of fields) into a $1, $2, however many fields are in each line, that is what $NF becomes. If it is 4 fields, then $NF becomes $4 and so on.

For getting the last column, we could simply do : awk -F "/" '{print $NF}' /etc/shells. Why is pattern matching required?

I enjoyed your presentation on awk and I subscribed to this channel. I have a question: Can awk to the following job -- send the first line of a file1, delimited by default space, to another file2, file2 that would take its name from the first few fields of the line 1 just transferred from the file1? Thank you

Never needed awk in last 6-8 years or so... but thanks!

14:24 Why did it print the indented line? I would have expected it to not do it, based on the behavior of 11:04, which counted indentation as characters and this one checking the first character being 'b|c'. Are whitespace characters only pseudo-characters or something like that? It would sort of make sense, because of the whole column thing, but I'm still quite a bit confused.

Thanks! What mic are you using? Also 7:40 , you could use awk -F "/" '{print $NF}' /etc/shells for the same result Your df command gives output equivalent to df -h , I suppose you configured an alias?

Is there an easy way to make the output columns TRULY line up? I don't consider your \t output to be very readable either, since it didn't really align the output columns.

thanks awk command that help me a lot in my project after watching this video ^_^

04:40 min. Pipe the awk to column command. Like: | column -t

How on earth do you remember the AWK syntax? Each time I have to use AWK, I have to look it up again. Even when I am using it, it has been very basic. I’m not very good at reading the escape characters and anything in slashes. Thanks for making that clear.

Hi, thx for ur great videos reagarding Linux! I have a question: r u still using trackballs?

Amazing! Basically I can programming using awk. Thanks

Good Morning Sir, I just wanted to know which Distro and terminal you're using here?.. it's so dope! 🔥 Also that fun message after entering wrong syntax LOL that's where I lost it..

Hi! You need a sorted input in order for uniq work correctly

For now I am more comfortable with sed, cut, tr . Surely awk will some day be in my tool belt , just can't find a reason to spend time learning it.

3:00 `cat /etc/passwd | cut -d':' -f1` also works

Missed an opportunity to show how to run calculations on numbers over different rows. With AWK you can basically have spreadsheet calculations.

you dont need to put the double quotes. you can also do awk -F: '{print $1}'/ /etc/password

Dude, plz do Udemy or any full courses, I came here from some lame udemy course, because there was no deep dive on awk command like here love Ur content, awesome !

Great vid DT!

I don’t tend to fool around with people who can write complicated awk/sed. :)