UPDATE! As for 2023 May, you'll have to change #courses for #cscourses, otherwise the code will return an empty array on 14:00.

As usual another content so we can learn new things. Thanks Brad for your dedication you help us so much not only on how to code but with your open minded regarding all the aspect of programming and all the possibilities that contains. 🙏🙏

I watched and coded along 4 videos and finally, thanks to this one, I can understand this topic. Very clear and concise! I am working hard to become a front end developer and I have a test ( for a job) on web scraping next week. I feel ready now!

Thanks for the video! This is great stuff. I used Puppeteer for a project at work, scraping charts from a web app, taking screenshots, and saving data into a .csv file. Very handy tool.

I wish everyone can make tutorials of this quality.

This is a good intro. If you look into regular Puppeteer for scraping tutorials they often go into major projects which end up going out of date very quickly; I really enjoy your review of the basics, it makes it very accessible and easy to refer to.

A great video. One of the best 'scraping videos I've seen on KZfaq which starts from the ground up.

Super awesome man, i searched the whole you-tube, but i found your explanation the best.

If you take a look at my search history, you'll find out that I was searching for scraping tutorials 2 days ago. I'm super happy that you released this video today. The timing is just perfect. Thank you so much!

Awesome video Brad. I wanted to comment because web scraping has great use cases in the real world. I am a BA with an agency that works with a very large client in the news/journalism space (one of the largest, in fact) and I'm working with a developer that usually creates new story feed ingests for their API so they can sell/syndicate the stories out to other news outlets all over the world. This is usually done via an XML feed but this one in particular is just links to HTML pages so the developer is scraping the stories from HTML and adding them to the API (this one for soccer/football stories related to the 2022 World Cup). So creating your own API by scraping data for a particular niche or use case is quite a valid skill set to have.

Congratulations on 2 million subscribers, Brad! The whole tech community is proud of you.

This is so great to see! Not too long ago I got my 1st dev role that required the use of puppeteer the majority of the time. Knowing absolutely nothing about it, I was pretty much thrown in with the wolves. But it was such an awesome learning experience!

Thanks, Brad! An "advanced" puppeteer tutorial would be awesome. I've wanted to combine scraping with a database that updates on cron to create a custom, one-off API with just a couple files. Just an idea. Thanks again for all your content!

Awesome! Best tutorial about web scraping. We need more about this topic Brad!

This is a great video. It's easy to follow along and understand.

Brad Schiff introduced me to Web Scraping. Great vid.

Such a quality content you're providing for free thanks brad sir ❤️

This can change a lot in how the migration of sites happen esp on UI. Happy to learn this

This was amazing thank you so much Brad hope all is well with you and the Beautiful family,

Thank you Brad! I appreciate you so much. Thank you for your dedication to helping others.

Thank you for the tutorial! In my case when creating the PDF, I included the 'fullPage' option to make it work.

Thank you so much. I was finding it really hard to find error in my code, and as soon as I saw the screenshot method in first 5mins of your video, I tried it and got to know the error and mistake I was making. :)

Hi Brad! how are you? Great to watch your video after long time. You still inspire me.

Brad, I just bought two of your udemy courses - Node masterclass and react front backend 2022. I just came here to say big thanks man so far I am super satisfied. Thanks for everything you do !

My man is back with the tutorial I wanted !

Thank you @Brad for this awesome video.

Great tutorial for a really useful library. Thanks. Also....for anyone getting time-outs due to slow connections etc.....add this line before the 'goto' await page.setDefaultNavigationTimeout(0);

Thank you, Brad. Super easy video to get me started with Puppetter.

That was a really well made video as usual, Thanks Brad 💯👍

Congratulations on 2 million subscribers.

just realised you've gotten fit. Nice work man!

another fantastic vid, brad! you're a real one. i wanted to point out on the fs.writeFile() part, you can make it so that the JSON.stringify() method will automatically format the output in a readable way by including a third parameter; in this case, the length of array of objects. null is used to bypass the optional second 'replacer' parameter: JSON.stringify(courses, null, courses.length)

Great tutorial. Thank you for uploading this.

Thank you, Brad! You are Rock, as always 👍

I have been trying to build a crawler for a long time but no success. With this, the possibility is endless. Thank you Brad. Your tutorial is always a top-notch. God bless

Whoa! That went smooth.... Thanks for the tutorial..

Thanks.I am automating my work with beautiful soup.

Wow puppeteer is awesome! Will definitely be playing with this soon ❤

Great tutorial! Very useful indeed. 😊😊

great video as always. personally, I think there are loads of great scraping tools already invented (web scrapper io, octoparse etc) both free and paid that do pretty great job of scraping all kind of content and even allow one to create spiders & schedule scripts. everyone should know about those too! 😍

Nice video. Hope to see more about this topic. It's not easy to find good content about it

thanks, a very simple explanation !

Awesome tutorial Brad🤘 Your follower from Afghanistan😊

Awesome video as always!

I love puppeteer, I made an actual product for a company that uses it and dang it’s so cool what it can do

Gracias excelente video y muy bien explicado... ganaste un suscriptor de Latam...

Awesome, I'm waiting for in depth course.

Awesome as always ☺️

Great Tutorial, Thanks

Nice explanation. Thanks :)

Thank you, Brad, very much.

Great value, appreciated

You're looking healthier, Brad. Hope you're working out and staying strong.

Thanks a lot for your content!

Awesome video!

Right on time 🤘

Really Great Video

good video, subbed!

Great Tutorial : )

I WANT TO USE THIS OPPORTUNITY TO SAY A HUGE THANKS BRAD, YOU'RE THE BEST, YOUR TUTORIAL IS AMAZING AND EASY TO UNDERSTAND, YOU'VE HELPED A LOT OF PEOPLE WITH YOUR TUTORIAL, AND IM NUMBER ONE. PLEASE MAKE A VIDEO FOR RESET PASSWORD. I HOPE YOU READ MY COMMENT. THANKS BRAD

Hey Brad first of all thank you for you videos,skills you give us and I would like to ask solid.js crash course,thank you.

Awesome. Thank you very much :)

Brad is the best!

Incredible - thank you. Completed it. I plan to go into this further until one-click integration. One question - how would you scrape through all of the website pages in an index with Puppeteer, from page 1 - page 100 - scraping the content? And then adding this to. a CVS file later - pandas? Would appreciate your help.

thanks for the guide

You surprise me everyday.

Such good marketing of your courses:-) You should put a puppeteer course on your website, and then basically redo this video😅Might have a nice conversion🙂

I JUST used this for some critical css scraping. What are the chances!

how can you scrape handlerbars injected values into HTML? thx for the help

great work , can we download pdf files or videos from website using this nice tool? thank you

No "promos". Yet awesome. Thanks, Brad. P.S. (Dec 2023) #courses > #cscourses P.P.S Advanced scrapping tutorial will be amazing.

Interesting package. Maybe I trying using this for laravel ;) If I can...

Awesome thanks 😊

3 minutes and 240 views... early early. Good stuff as always, Brad!

Would you be able to put together a crash course on DynamoDB and HTTP Module?

Love u sir

Hey Brad, long time viewer and have taken a few of your udemy courses. Any plans for a deployment series explaining how to properly set up and deploy full stack apps across hosts like AWS, Azure etc...?

Thanks, Can you make more courses on puppeteer scrapping in detail + there are ni convincing courses for developing chrome extensionsin market. In you can make in-depth course for Chrome extension. Thanks.

Thanks for talking like a normal person. Refreshing

How would you scrape dynamically created classes?

never forget to close the puppeteer browser. I had a web server constantly crash because we didn't close the browser when an error occurred so we kept opening browsers without closing the ones we stopped using, the server's memory eventually saturated. Lesson learned : always close the browser in "finally" in the "try/catch/finally" blocks

Is that new "Traversy Media" animation? :D

Is there any tool or technique for automatic web scraping without directing towards any specific website. And Is this concept crawling or scraping in which automatic website elements are being returned without specifying towards any single website.

what is the name of that vs code theme. I like that setup

wow amazing

How can we use this to build a bot

how can i do this but with websites that have "paste URL here" with my own URL and get a screenshot of the new page.

Can this also scrap meta data of the websites??!

I am just wondering that instead of doing puppeteer on nodejs, can I do this on nextjs project? I believe Next.js is sort of combination of React + Express; I can write backend-related logic.

Anyone know how he got the PDF format list to popup?

♥

i have tried this too many times but i still get a timeout error. can someone help me fix this? 'TimeoutError: Timed out after 30000 ms while waiting for the WS endpoint URL to appear in stdout!' is the timeout message from the command prompt

Does anyone know how to easily have a variable copied to clip board from the .js and posted into a website?

After 11:00 whatever I'm trying to do I get the following error node:internal/process/promises:288 triggerUncaughtException(err, true /* fromPromise */); Can someone help

i wonder if you can use this to scrape data from google maps?

can i open puppeteer in my used browser

How could you do this same thing but scrape an entire site, instead of just 1 page?

Can we scraping data from Facebook ads ?

can you please continue this series?

Why not playwright instead?