You Should Use CSS Selectors for Web Scraping.

Рет қаралды 15,620

3 жыл бұрын

Lets talk CSS Selectors. I demo the most common, and most useful ones to help you extract data using your web scrapers as easily as possible. I'm using requests_html in Python, which can be pip installed if you do not have it already. It's a great libary and well worth learning
-------------------------------------
Disclaimer: These are affiliate links and as an Amazon Associate I earn from qualifying purchases
-------------------------------------
Digital Ocean (Cloud Servers, Affiliate Link) - m.do.co/c/c7c90f161ff6
Sound like me:
microphone amzn.to/36TbaAW
mic arm amzn.to/33NJI5v
audio interface amzn.to/2FlnfU0
-------------------------------------
Video like me:
webcam amzn.to/2SJHopS
camera amzn.to/3iVIJol
studio lights amzn.to/3aBpKik
small lights amzn.to/2GN7INg
-------------------------------------
PC Stuff:
case: amzn.to/3dEz6Jw
psu: amzn.to/3kc7SfB
cpu: amzn.to/2ILxGSh
mobo: amzn.to/3lWmxw4
ram: amzn.to/31muxPc
gfx card amzn.to/2SKYraW
27" monitor amzn.to/2GAH4r9
24" monitor (vertical) amzn.to/3jIFamt
dual monitor arm amzn.to/3lyFS6s
mouse amzn.to/2SH1ssK
keyboard amzn.to/2SKrjQA

Пікірлер: 33

@mohitkumar-tv5ye 3 жыл бұрын

You make one of the best tutorials mate. you simplify it and explain it so well, can't thank enough. Keep it up buddy :)

@celerystalk390 3 жыл бұрын

The most informative and concise video on this topic! Thanks again John.

@zhangxueyan4684 3 жыл бұрын

Great video! Would be great to have another video comparing CSS and Xpath. Differences in their time performance, generality, compatibility with browsers and etc.

@polakpuckstopper 11 ай бұрын

Fantastic overview. Thank you much!

@sdriding 2 жыл бұрын

This is gold. I’ve watched so many of your vids but missed the foundational stuff like this. If I’m ever in the UK I’ll have to buy you a beer!!

@sdriding 2 жыл бұрын

I don’t yet work professionally as a developer, and while I really enjoy scraping I wonder if I’m pursuing a route that has a low pay ceiling. If anyone has any opinions on that I’d love to hear.

@JohnWatsonRooney 2 жыл бұрын

Learning web scraping is a good way to teach you skills that will transfer over to any web related job - keep learning until you find what it is you want to do, even if that’s not a tradition dev job. There is money to be made with scraping data but taking those skills and building web apps that connect other software via an API is a good place to look

@movieblast1259 Жыл бұрын

Best video on css selector

@higiniofuentes2551 2 жыл бұрын

Thank you for this very useful video!

@MichaelShingo 2 жыл бұрын

so easy to follow thank you

@ferilukmansyah3037 3 жыл бұрын

thank for the best tutorial

@muhammadatif3003 Жыл бұрын

thanks for making simpler

@alessandroceccarelli6889 10 ай бұрын

Amazing! A cheatsheet would be appreciated too

@the1gofer 3 жыл бұрын

thanks

@mattmovesmountains1443 3 жыл бұрын

This is huge. I spend the most time trying to grab the correct elements, rather than the actual logic of the code, often coming up with None types. This is going to be my new reference guide

@mattmovesmountains1443 3 жыл бұрын

@Shepard Kingston this reminds me of the time I went to the grocery store and one of the aisles had Spam in it

@reshamchawra1598 3 жыл бұрын

Please tell me the difference between select and find all method and which one to use when

@ukaszwilk3138 3 жыл бұрын

That was amazing tutorial but I have a problem. I've got a class that has many values however a specificl value is changing in diffrent websites I mean that for example in page one I have my value in name = r.html.find('.sc-1bker4h-4.llfiOB')[0].text but in other I have that value in name = r.html.find('.sc-1bker4h-4.llfiOB')[5].text. Could you help me? Thanks in advance

@technicallymind201 3 жыл бұрын

Can you make one video on login in amazon by using requests

@absoluteRandom69 3 жыл бұрын

Hello John, Could you make a video on Scraping Tweets form Twitter?

@tubelessHuma 3 жыл бұрын

I like your video 👍 xpath seems easy than css. Which is the best option?

@JohnWatsonRooney 3 жыл бұрын

I prefer css as I am more used to it. However once you learn xpath properly you can do some cool things

@DrJohnnyStalker 3 жыл бұрын

xpath is more powerfull

@stephenwilson0386 Жыл бұрын

This is a dilemma for me. CSS selectors seem much easier to read and are more straightforward, but XPath seems a lot more powerful for selecting exactly what you need within complex nested tags. I guess it works to mix and match though, like use CSS by default and switch to XPath when I can't get what I need with CSS?

@JohnWatsonRooney Жыл бұрын

I’ve always got by with css selectors. I learned them well when I first looked into scrapy as they were in most of the documentation. It’s absolutely fine to use whichever whenever as long as it works for you and you get the results you want!

@stephenwilson0386 Жыл бұрын

@@JohnWatsonRooney Thanks John! Between your channel and a Udemy course I got on sale, I'm getting the hang of this stuff quicker than I thought. You're definitely the go-to resource on KZfaq. Kudos and keep it up!

@sounakchatterjee9059 3 жыл бұрын

wonderful video just wanted to ask how to select a text from an element?

@JohnWatsonRooney 3 жыл бұрын

With requests-html we just add “.full_text” at the end - you will also need “first = True” in there too

@sounakchatterjee9059 3 жыл бұрын

@@JohnWatsonRooney Thanks John. Also wanted to know is there any forum (like discord) where I can post issues or share my code? Like I want to scrape the country name, cases, death etc. for eg. how to scrape the country name? covid19.who.int/table . Can you please assit?