Selenium Course for Beginners - Web Scraping Bots, Browser Automation, Testing (Tutorial)

Рет қаралды 785,113

Күн бұрын

Learn Selenium by building a web scraping bot in Python. Selenium is a powerful web automation tool that can be used for browser automation, to test front-end code, and create web scraping bots.
💻 Code: github.com/jimdevops19/Seleni...
✏️ Course developed by JimShapedCoding. Check out his channel: / @jimshapedcoding
⭐️ Additional resources ⭐️
🔗 Python Download: www.python.org/downloads
🔗 Pycharm Download: www.jetbrains.com/pycharm/dow...
🔗 Selenium Documentation: selenium-python.readthedocs.io/
🔗 Copied and Pasted during the video:
‣ www.seleniumeasy.com/test/jqu... (Section 1&2)
‣ www.seleniumeasy.com/test/bas... (Section 3)
🔗 Chromedriver download website: chromedriver.storage.googleap...
🔗 All the Explicit waits: selenium-python.readthedocs.i...
🔗 My Personal website for code snippets: jimshapedcoding.com
⭐️ Course Contents ⭐️
⌨️ (0:00:00) Getting Started with the basics
⌨️ (0:16:44) Explicit vs Implicit
⌨️ (0:28:11) Sending Keys & CSS Selector
⌨️ (0:43:42) Structure a Bot Project
⌨️ (1:03:13) Deal Searching
⌨️ (1:44:38) Booking Filtrations
⌨️ (2:07:24) Execution from a CLI
⌨️ (2:21:31) Deal Reporting
🎉 Thanks to our Champion and Sponsor supporters:
👾 Wong Voon jinq
👾 hexploitation
👾 Katia Moran
👾 BlckPhantom
👾 Nick Raker
👾 Otis Morgan
👾 DeezMaster
👾 AppWrite
--
Learn to code for free and get a developer job: www.freecodecamp.org
Read hundreds of articles on programming: freecodecamp.org/news

Пікірлер: 489

@jimshapedcoding 2 жыл бұрын

Third time is a charm! Thanks FCC, I am glad to share this with the best coding community. Hope everyone will have a great time learning Selenium! :)

@NAEEM_MALIK 2 жыл бұрын

Your work is *COMMENDABLE* I appreciate your *EFFORTS*

@moviesnight248 2 жыл бұрын

Congrats Jim🎉

@__________________________6910 2 жыл бұрын

@JimShapedCoding I already sub your channel from last 8months

@landsfadern2 2 жыл бұрын

sadly wont listen to 3 hour video! :(

@victorwestmann 2 жыл бұрын

You guys are doing history here by democratizing high quality education for free!! 🙏✅

@arxoidwrld4766 Жыл бұрын

for those who don't know, the newer version of selenium has removed the driver.find_element_by_id and similar functions, the new way of writing the same is- driver.find_element("id" , "") hope that helps

@yassinedghoughi9500 Жыл бұрын

Thank you for the help, I just want to ask you, where did you find this info? I can't find a new documentation for selenium.

@nikitakuznetsov3418 Жыл бұрын

Thank you , really appreciate for this information! :3

@Programlama101 Жыл бұрын

@@yassinedghoughi9500 python gives warning when you use old version of code.

@Laizin Жыл бұрын

How can we extract a word with a tag of yt- formatted -string And im scraping data by looping through 50 yt channels. Sometimes it stops looping with an error of unable to locate But when im executing them individually all 50 channels are able to scrape well. What might be wrong

@NayyarAbbas-sh1vw Жыл бұрын

its more like beautifulsoup now :D

@vaibhav31 2 жыл бұрын

9:15 - r as the prefix to a string indicates a "raw string"; that is, character sequences like will be treated as literals and not as escaped control characters (as they normally are in a string.) It's useful when you expect your string to include backslashes, for instance Windows paths, or certain regex expressions.

@riemanngalois7383 2 жыл бұрын

Thanks you

@zainali-pm1mn Жыл бұрын

I am glad I chose this video to learn selenium scraping. I learned a lot from you!. You are really a great teacher. Hats off to you!

@geoffkelly262 2 жыл бұрын

Just started using Selenium and this will save me a bunch of time trying to figure out all the methods, thanks!

@mikepenprogrammer2652 2 жыл бұрын

Many thanks for this amazing tutorial. Not only was it great intro to OOP, it also provided enough info to build my own app from scratch: Automated my timesheet entries at work. It now takes no more than 2 seconds to open browser, login, navigate to desired page, enter all work and time related values, and to pass the submit process with approval from management :) Feeling good while conscious of how much more there is to do and to learn :D

@akirablac Жыл бұрын

use this for browser to not automatically closing: ` options = webdriver.ChromeOptions() options.add_experimental_option("detach", True) options.add_experimental_option('excludeSwitches', ['enable-logging']) driver = webdriver.Chrome(options=options) `

@workoutinspirations9990 Жыл бұрын

Hey, how can I put it in code from "structure a bot project" (as a part of the booking module)?

@DimaShvetsov Жыл бұрын

Thanks a lot!

@yangwilliam3137 Жыл бұрын

thanks you're amazing!

@apxmvrc 10 ай бұрын

@@workoutinspirations9990 Have you found out the answer?? 😅

@apxmvrc 10 ай бұрын

I am also stuck there

@red_cape. 2 жыл бұрын

Great video, to the point and explaining the details for those who are unfalmiliar with Python. !

@federicopalacio5349 2 жыл бұрын

greaaaattttttttt, just finished valentin's postman crash course and willing to learn automatization for testing. gj

@wakko3wb 2 жыл бұрын

One of the best and most complete tutorials, thank you for this.

@jimshapedcoding 2 жыл бұрын

Glad to read this :)

@rstark 2 жыл бұрын

God finallyyyy FCC uploaded selenium tutorial, not 1000th tutorial on react or HTML 😁 I'm very happy! Thank you lovely FreeCodeCamp ❤️ hope here will be more videos with test automation library's, such as playwright and selenium!

@srdjanst1 Жыл бұрын

Great content. One of the best explanations in a coding video that I've ever seen.

@kyawheinhtut3728 2 жыл бұрын

Thank you so much guys, you guys are helping us to continue learning during these covid times. THANK YOU

@aryavihat7098 2 жыл бұрын

They are doing really a great job.😘.We all should appreciate them sothat they always get inspired by this. 😘👍.Carry on guys .😊😊

@sandglokta4699 2 жыл бұрын

This is gold. Please more info about testing automation 🙏🙏🙏

@KimKim-fu7zs 2 жыл бұрын

Thanks to your coaching, now I feel more confident in my coding skill a bit, haha, thank you, Jim!

@xilllllix 2 жыл бұрын

great tutorial! explains everything in a newbie-friendly way...

@ideepakmathur 2 жыл бұрын

Started listening salenium today, and this video tutorial is gem 💎. This guy is really awesome at teaching. #ThanksMuch

@zenabmohammed1156 2 жыл бұрын

Awesome Content thanks Jim shapedCoding channel we wish to introduce more series in this field of python in web scraping 🥺🌸+ thanks FCC for sharing this awesome content on web scraping with us 👏👏🌸

@worthsalive 10 ай бұрын

Thank you so much for this video. I really find it very helpful and easy to follow along. I was able to build bot that is able to scrape huge data from multiple paged website and dumps the record to json while keeping track of records that has already been scraped incase the bot fails at any point it will be able to continue from where it stoped when you run it again and finally merge all dumped json files together and generate an excel sheet. But I got my basis from this video all thanks to you man. I deeply appreciate.

@maratin93 Жыл бұрын

Great tutorial. I find a lot of interesting features in this video, which I am going to use in my automation project. Thanks.

@jatinvashisht4293 2 жыл бұрын

After watching the tutorial: This is one of the best tutorial for beginners, after this you will be able to scrap easily and make your own bots and can understand the documentation easily. Thank You Jim and FCC for this amazing course 🙏

@lbb2rfarangkiinok Жыл бұрын

It's "scrape/scraping". "Scrapping" means something else. :)

@barrientoscardenaslinofern4717 2 жыл бұрын

This guy learn me Flask and Web Scraping with Bs4 for free. THANKS JIM PD. Your KZfaq videos of your canal help me to my carrer

@muntaface 2 жыл бұрын

Another amazing tutorial. Thanks Jim

@dattran9665 11 ай бұрын

the first time I watch a excellent tutorial video. thank you very much from Vietnam. wish you all the best

@ilCvlto 2 жыл бұрын

Jim is always a great teacher ✌🏻✌🏻

@Flo84swiss 2 жыл бұрын

Your tutorial is amazing. One of the best coding video I ever saw on yt. There are some elements which changed in the meantime, but I could manage it by my own. It help me to build ezze a bot on another project. Thank you for that!

@Stopinvadingmyhardware 2 жыл бұрын

What?

@thecryptoindia Жыл бұрын

I am not able to send_keys, can u help?

@emmanuelace1979 2 жыл бұрын

Finally...I have been waiting for this course all my life😂😂👍🏾

@switchi8663 2 жыл бұрын

You guys have been dropping bangers

@crosshawk1944 Жыл бұрын

This is the best tutorial I have ever seen. Thank you very much!

@Superdupafool Жыл бұрын

how? it's literally outdated. there's no way you learned anything useful with this.

@satyamraj17 10 ай бұрын

@@Superdupafool wow

@RobsonDev 2 жыл бұрын

Amazing course! Thank you for share

@murtazaburhani4022 2 жыл бұрын

I loved his beautiful soup tutorial too 👌

@HerozTech 2 жыл бұрын

I love this course ❣️

@ahmedqureshi4831 2 жыл бұрын

Great Project! Loved the accent. However, there is one thing i would request for the next selenium tutorial, I am would like to learn more about multiple browser window handling and switching. Thank you for this amazing #CrashCourse

@saburiyusuf7451 2 жыл бұрын

This is a great Selenium Tutorial.

@mohammedbadi6938 2 жыл бұрын

where was this 8 months ago Greatest channel ever

@Brocollipy 2 жыл бұрын

Really helpful stuff, many thanks!

@Nandhis 2 жыл бұрын

Great tutorial! Thank you!💐🎂

@judahnat6529 Жыл бұрын

Great tutorial and great and clear teaching

@hmm7458 2 жыл бұрын

i needed this😭

@maherhanna2432 2 жыл бұрын

This is an amazing tutorial thank you very much

@gurjot2318 2 жыл бұрын

brilliant... been waiting for selenium tutorial for ages

@tahagoren7011 2 жыл бұрын

Please keep continue this diamond tutorials thanks a lot

@manjunath7497 2 жыл бұрын

😫guys please.. I beg u, really u are making much efforts. Giving all these for free is not a small task.

@_mytube_ 2 жыл бұрын

Nicely explained man, thanks

@Rohan-bg8ci 2 жыл бұрын

And as always thank you FCC

@____kklw7148 2 жыл бұрын

Thanks for teaching. I have learnt a lots

@kamiln851 2 жыл бұрын

This tutorial is great!

@grumpy653 2 жыл бұрын

Amazing tutorial!

@SkWebStudio 2 жыл бұрын

Thanks for the video!

@neillunavat 2 жыл бұрын

Finally a tutorial I will surely watch.

@neillunavat 2 жыл бұрын

I have an issue called 'procrastiantion'. I just don't end up watching most of the vids I wanna watch...

@sinbad2597 Жыл бұрын

You watched?

@neillunavat Жыл бұрын

@@sinbad2597 hell no 😂

@codesuki3005 2 жыл бұрын

Please upload more of this tutorial and also teach automated data collection. Your style of teaching is nice

@user-rk7sx2uu2l 11 ай бұрын

"Thank you sincerely for creating and sharing the video. I'm incredibly grateful for the wealth of knowledge I gained about coding through your content. The concepts I learned, particularly finding new locators and changing the codes, have provided me with valuable opportunities to put my skills to the test. I can't express enough how much I appreciate your efforts in helping me expand my coding abilities. Thank you wholeheartedly once again!"

@aben6717 2 жыл бұрын

Thanks for the great content. Just one thing : Regarding selenium links: the pages aren't found. Anyone could help? thanks!

@yorukama 2 жыл бұрын

hey its this guy. Love this guy.

@AznBMan75 2 жыл бұрын

Hi! Can you use Expected Condition to wait after I go through login page, and then after the next page to perform web scraping on the 3rd page? If yes, then I would have to refer to the 3rd page and methods in the Expected Condition?

@sampasomnathvloger7781 2 жыл бұрын

Beautiful 🙏🏻

@exspider64 Жыл бұрын

I needed to add ; before the driver path

@realbutters 2 жыл бұрын

Was getting 'chromedriver' executable needs to be in PATH despite following along with the recommendations and I figured out the issue for both Mac and Win. First, I added a print statement right after the os.environ path setting piece: print(os.environ['PATH']) I noticed it was missing a ; in Windows (: in Mac). So, I modified driver_path append step to this and it worked: os.environ['PATH'] += os.pathsep + self.driver_path os.pathsep adds your OS specific path separator character.

@realbutters 2 жыл бұрын

There are other ways to do fix this problem, but this is what I did in a pinch to see what I could do to make it work like the teacher was doing it.

@rejkee Жыл бұрын

Thank you, it worked for me.

@MiturGrunge Жыл бұрын

Great solution! Thanks! I figured out that simply adding a ; in front of the driver path works as well, so my line looks like os.environ['PATH'] += r";C:/selenium_drivers"

@bugrahanozcan8682 Жыл бұрын

@@MiturGrunge what is meaning of semi-colon?

@adrianford468 Жыл бұрын

Can you share you your code at this part for an example

@mayurasandakalumsellapperu6778 Жыл бұрын

Thank you 😍🔥🔥

@mainbotpy Жыл бұрын

It was a great one for sure

@davidwang9827 Жыл бұрын

It's helpful for me. thank you very much.

@user-cq5hr2zg1u 2 жыл бұрын

Videos on this channel are just free gold mines, can’t believe you can get so much knowledge for free.

@erikmedeiros6124 Жыл бұрын

hello, the video is very good, very intuitive, however in 2023, many of the examples used are invalidated, mainly due to the attributes of the elements of the sites, such as the booking one, they were changed which made it very difficult to scrape the website and follow the progress video was very difficult because of this.

@faiqjavaid4398 5 ай бұрын

i am also stuck with that. I am unable to select currency in booking website. If you have solution for this let me know please.

@sunday-ucheawaji7966 2 жыл бұрын

This is great!

@amanshaikh9900 2 жыл бұрын

Thank you so much guys

@trido3815 Жыл бұрын

Thanks, Jim

@sovietcat4825 2 жыл бұрын

this is awesome

@Palmit_ 2 жыл бұрын

Almost all the tests links are dead AND the code is outdated. Selenium now uses find_elements() NOT find_by_id() or find_by_class_name() etc.

@Vyzinn Жыл бұрын

How does find_elements() work?

@reklamsz3m3t Жыл бұрын

@@Vyzinn class = .classname, id = #id etc.

@Palmit_ Жыл бұрын

@@reklamsz3m3t a belated but sincere thank you to you :)

@shivamgupta-hn2nf Жыл бұрын

Can you please tell me How to find this attribute name losser-id = "name-of-loser" in selinium

@sinbad2597 Жыл бұрын

@@shivamgupta-hn2nf self.find_element(By.CSS_SELECTOR,'tag[losser_id="name_of_loser]') Here for tag,you have to find which tag this stuff is located in . Like div or a or li. (if you aren't using classes , use driver instead of self )

@tristan2338 Жыл бұрын

yes we are using this for 'testing'

@dimazepam 8 ай бұрын

great course

@kluchtube7042 Жыл бұрын

great video learn lot from you!

@KevinOgutu-cx8ix 10 ай бұрын

I really love the tutorial, I have been folloeing your tutorials always and thumbs up. I have an enquiry; suppose I want selenium to be refreshing a specific website at an interval of 2 seconds without stoppind, and if a certain link pop ups on that website, click on it. What is the procedure for that?

@TK-mf9dz Жыл бұрын

rather not for the begginers even thought I am following the explanation rather ultra basic, without going into details, just 1 senctence as if you have a mission to squezze in 3 h with everything max. Great idea to work on Booking Thanks anyway

@jeenitprajapati3449 2 жыл бұрын

Wow! Thanks

@bobeo56789 10 ай бұрын

Thank you for your video🎉

@user-el9ju6fn7x 7 ай бұрын

Спасибо! =)

@JoseLopez-or8xw Жыл бұрын

Thank you!

@xinyuecao9130 Жыл бұрын

Help! the selenium easy links does not work! It said: "Page Not found"

@et8175 2 жыл бұрын

Anyone else have an issue with the os.environ['PATH']? I had to add a semicolon for it to work: r";C:/SeleniumDrivers". Anyone know why it doesn't add it automatically? Thanks.

@et8175 2 жыл бұрын

Are the seleniumeasy links working? They seem to be removed from the website.

@Doom_C 2 жыл бұрын

Wow, posted 1 day ago, and fixed my exact problem. No idea why this works.

@emimartin7044 2 жыл бұрын

Thanks for sharing this, i didnt know i should add semicolons there.

@muhmmedarbee8014 2 жыл бұрын

THANK YOU FOR THIS!!!!

@Manikanta-ko7qy 2 жыл бұрын

@@et8175 thanks man..I am searching for this..

@littleshort95 Жыл бұрын

Hi, the seleniumeasy demo site is no longer working

@soyedafaria4672 Жыл бұрын

One of the best tutorials for beginners.

@benukapunchihewa4125 2 жыл бұрын

Thank you very much!

@willbutplural 2 жыл бұрын

the course we all needed 😍

@xaviersalgado2197 10 ай бұрын

45:44 file making 49:19 import libraries

@eugenekwaka Жыл бұрын

Great tutorial Jim and the FCC team👌. I just have an issue with the links provided for the websites to test on. They seem to have been moved or are not working anymore. Please fix them for devs to use for learning.

@alejandrodelriosalas3627 Жыл бұрын

@vanshjagyasi4004 Жыл бұрын

@muhmmedarbee8014 2 жыл бұрын

Hi Could you provide me with the version details, what version of python you used, what version of pycharm you used, etc etc

@sayanjitdas7087 2 жыл бұрын

for some of you concatenating the driver path string showed may not work specifically to windows user , for me I had to prepend ; to driver path to make it work like ";C:\seleniumDriver"

@abdulazizabdullaev8931 2 жыл бұрын

Thank u worked for me too

@joyedet1363 Жыл бұрын

It doesn't work for me

@jihenbm6443 2 жыл бұрын

nice video very helpful thank you

@Zeix02 2 жыл бұрын

I hope you make with the new version of selenium a tutorial. A lot of things is changed.

@alvaromorales6828 2 жыл бұрын

Thanks. we also need an R version!

@rstark 2 жыл бұрын

Thanks!

@rstark 2 жыл бұрын

Please create for us modern JavaScript test framework course with a tool such as playwright, it's so awesome ❤️ open source and free

@Amit-jb5xt 2 жыл бұрын

Thanks ❤️

@matthewtalbot-paine7977 2 жыл бұрын

I've just started using selenium and I've created a bot that collects from a bitcoin faucet using a vpn and I'm currently running 10 accounts at once although I need to investigate headless and I need to add some asynchronisty to it.