Selenium Course for Beginners - Web Scraping Bots, Browser Automation, Testing (Tutorial)
Рет қаралды 785,195
Learn Selenium by building a web scraping bot in Python. Selenium is a powerful web automation tool that can be used for browser automation, to test front-end code, and create web scraping bots.
💻 Code: github.com/jimdevops19/SeleniumSeries
✏️ Course developed by JimShapedCoding. Check out his channel: kzfaq.info/love/U8d7rcShA7MGuDyYH1aWGg
⭐️ Additional resources ⭐️
🔗 Python Download: www.python.org/downloads
🔗 Pycharm Download: www.jetbrains.com/pycharm/download
🔗 Selenium Documentation: selenium-python.readthedocs.io/
🔗 Copied and Pasted during the video:
‣ www.seleniumeasy.com/test/jquery-download-progress-bar-demo.html (Section 1&2)
‣ www.seleniumeasy.com/test/basic-first-form-demo.html (Section 3)
🔗 Chromedriver download website: chromedriver.storage.googleapis.com/index.html
🔗 All the Explicit waits: selenium-python.readthedocs.io/waits.html
🔗 My Personal website for code snippets: jimshapedcoding.com
⭐️ Course Contents ⭐️
⌨️ (0:00:00) Getting Started with the basics
⌨️ (0:16:44) Explicit vs Implicit
⌨️ (0:28:11) Sending Keys & CSS Selector
⌨️ (0:43:42) Structure a Bot Project
⌨️ (1:03:13) Deal Searching
⌨️ (1:44:38) Booking Filtrations
⌨️ (2:07:24) Execution from a CLI
⌨️ (2:21:31) Deal Reporting
🎉 Thanks to our Champion and Sponsor supporters:
👾 Wong Voon jinq
👾 hexploitation
👾 Katia Moran
👾 BlckPhantom
👾 Nick Raker
👾 Otis Morgan
👾 DeezMaster
👾 AppWrite
--
Learn to code for free and get a developer job: www.freecodecamp.org
Read hundreds of articles on programming: freecodecamp.org/news
@jimshapedcoding 2 жыл бұрын
Third time is a charm! Thanks FCC, I am glad to share this with the best coding community. Hope everyone will have a great time learning Selenium! :)
@NAEEM_MALIK 2 жыл бұрын
Your work is *COMMENDABLE* I appreciate your *EFFORTS*
@moviesnight248 2 жыл бұрын
Congrats Jim🎉
@__________________________6910 2 жыл бұрын
@JimShapedCoding I already sub your channel from last 8months
@landsfadern2 2 жыл бұрын
sadly wont listen to 3 hour video! :(
@victorwestmann 2 жыл бұрын
You guys are doing history here by democratizing high quality education for free!! 🙏✅
@arxoidwrld4766 Жыл бұрын
for those who don't know, the newer version of selenium has removed the driver.find_element_by_id and similar functions, the new way of writing the same is- driver.find_element("id" , "") hope that helps
@yassinedghoughi9500 Жыл бұрын
Thank you for the help, I just want to ask you, where did you find this info? I can't find a new documentation for selenium.
@nikitakuznetsov3418 Жыл бұрын
Thank you , really appreciate for this information! :3
@Programlama101 Жыл бұрын
@@yassinedghoughi9500 python gives warning when you use old version of code.
@Laizin Жыл бұрын
How can we extract a word with a tag of yt- formatted -string And im scraping data by looping through 50 yt channels. Sometimes it stops looping with an error of unable to locate But when im executing them individually all 50 channels are able to scrape well. What might be wrong
@NayyarAbbas-sh1vw Жыл бұрын
its more like beautifulsoup now :D
@akirablac Жыл бұрын
use this for browser to not automatically closing: ` options = webdriver.ChromeOptions() options.add_experimental_option("detach", True) options.add_experimental_option('excludeSwitches', ['enable-logging']) driver = webdriver.Chrome(options=options) `
@workoutinspirations9990 Жыл бұрын
Hey, how can I put it in code from "structure a bot project" (as a part of the booking module)?
@DimaShvetsov Жыл бұрын
Thanks a lot!
@yangwilliam3137 Жыл бұрын
thanks you're amazing!
@apxmvrc 10 ай бұрын
@@workoutinspirations9990 Have you found out the answer?? 😅
@apxmvrc 10 ай бұрын
I am also stuck there
@vaibhav31 2 жыл бұрын
9:15 - r as the prefix to a string indicates a "raw string"; that is, character sequences like will be treated as literals and not as escaped control characters (as they normally are in a string.) It's useful when you expect your string to include backslashes, for instance Windows paths, or certain regex expressions.
@riemanngalois7383 2 жыл бұрын
Thanks you
@mikepenprogrammer2652 2 жыл бұрын
Many thanks for this amazing tutorial. Not only was it great intro to OOP, it also provided enough info to build my own app from scratch: Automated my timesheet entries at work. It now takes no more than 2 seconds to open browser, login, navigate to desired page, enter all work and time related values, and to pass the submit process with approval from management :) Feeling good while conscious of how much more there is to do and to learn :D
@zainali-pm1mn Жыл бұрын
I am glad I chose this video to learn selenium scraping. I learned a lot from you!. You are really a great teacher. Hats off to you!
@federicopalacio5349 2 жыл бұрын
greaaaattttttttt, just finished valentin's postman crash course and willing to learn automatization for testing. gj
@rstark 2 жыл бұрын
God finallyyyy FCC uploaded selenium tutorial, not 1000th tutorial on react or HTML 😁 I'm very happy! Thank you lovely FreeCodeCamp ❤️ hope here will be more videos with test automation library's, such as playwright and selenium!
@kyawheinhtut3728 2 жыл бұрын
Thank you so much guys, you guys are helping us to continue learning during these covid times. THANK YOU
@wakko3wb 2 жыл бұрын
One of the best and most complete tutorials, thank you for this.
@jimshapedcoding 2 жыл бұрын
Glad to read this :)
@red_cape. 2 жыл бұрын
Great video, to the point and explaining the details for those who are unfalmiliar with Python. !
@geoffkelly262 2 жыл бұрын
Just started using Selenium and this will save me a bunch of time trying to figure out all the methods, thanks!
@xilllllix 2 жыл бұрын
great tutorial! explains everything in a newbie-friendly way...
@sandglokta4699 2 жыл бұрын
This is gold. Please more info about testing automation 🙏🙏🙏
@srdjanst1 Жыл бұрын
Great content. One of the best explanations in a coding video that I've ever seen.
@KimKim-fu7zs 2 жыл бұрын
Thanks to your coaching, now I feel more confident in my coding skill a bit, haha, thank you, Jim!
@aryavihat7098 2 жыл бұрын
They are doing really a great job.😘.We all should appreciate them sothat they always get inspired by this. 😘👍.Carry on guys .😊😊
@maratin93 Жыл бұрын
Great tutorial. I find a lot of interesting features in this video, which I am going to use in my automation project. Thanks.
@zenabmohammed1156 2 жыл бұрын
Awesome Content thanks Jim shapedCoding channel we wish to introduce more series in this field of python in web scraping 🥺🌸+ thanks FCC for sharing this awesome content on web scraping with us 👏👏🌸
@muntaface 2 жыл бұрын
Another amazing tutorial. Thanks Jim
@Flo84swiss 2 жыл бұрын
Your tutorial is amazing. One of the best coding video I ever saw on yt. There are some elements which changed in the meantime, but I could manage it by my own. It help me to build ezze a bot on another project. Thank you for that!
@Stopinvadingmyhardware 2 жыл бұрын
What?
@thecryptoindia Жыл бұрын
I am not able to send_keys, can u help?
@emmanuelace1979 2 жыл бұрын
Finally...I have been waiting for this course all my life😂😂👍🏾
@barrientoscardenaslinofern4717 2 жыл бұрын
This guy learn me Flask and Web Scraping with Bs4 for free. THANKS JIM PD. Your KZfaq videos of your canal help me to my carrer
@worthsalive 10 ай бұрын
Thank you so much for this video. I really find it very helpful and easy to follow along. I was able to build bot that is able to scrape huge data from multiple paged website and dumps the record to json while keeping track of records that has already been scraped incase the bot fails at any point it will be able to continue from where it stoped when you run it again and finally merge all dumped json files together and generate an excel sheet. But I got my basis from this video all thanks to you man. I deeply appreciate.
@ideepakmathur 2 жыл бұрын
Started listening salenium today, and this video tutorial is gem 💎. This guy is really awesome at teaching. #ThanksMuch
@ilCvlto 2 жыл бұрын
Jim is always a great teacher ✌🏻✌🏻
@murtazaburhani4022 2 жыл бұрын
I loved his beautiful soup tutorial too 👌
@jatinvashisht4293 2 жыл бұрын
After watching the tutorial: This is one of the best tutorial for beginners, after this you will be able to scrap easily and make your own bots and can understand the documentation easily. Thank You Jim and FCC for this amazing course 🙏
@lbb2rfarangkiinok Жыл бұрын
It's "scrape/scraping". "Scrapping" means something else. :)
@switchi8663 2 жыл бұрын
You guys have been dropping bangers
@RobsonDev 2 жыл бұрын
Amazing course! Thank you for share
@AznBMan75 2 жыл бұрын
Hi! Can you use Expected Condition to wait after I go through login page, and then after the next page to perform web scraping on the 3rd page? If yes, then I would have to refer to the 3rd page and methods in the Expected Condition?
@dattran9665 11 ай бұрын
the first time I watch a excellent tutorial video. thank you very much from Vietnam. wish you all the best
@18kukki Жыл бұрын
9:30 I'm getting 'FileNotFoundError: [Errno 2] No such file or directory: 'chromedriver; if I use os.environ['PATH'] However if I use PATH="/Users/surajk/Pictures/Selenium/chromedriver", I'm not getting any error. Any suggestions.
@sootybuu2963 Жыл бұрын
Hey, I had a similar issue. After inspecting the path I found that each path string is seperated by a ; so changing it to += ';C:\\SeleniumDrivers' may work for you. I was running in a venv though, so I had to add it to my venv site-packages. Hope that helps.
@mikavandam8465 Жыл бұрын
@@sootybuu2963 Had the same issue, now it works! Thank you
@xinyuecao9130 Жыл бұрын
Help! the selenium easy links does not work! It said: "Page Not found"
@littleshort95 Жыл бұрын
Hi, the seleniumeasy demo site is no longer working
@Nandhis 2 жыл бұрын
Great tutorial! Thank you!💐🎂
@codesuki3005 2 жыл бұрын
Please upload more of this tutorial and also teach automated data collection. Your style of teaching is nice
@gurjot2318 2 жыл бұрын
brilliant... been waiting for selenium tutorial for ages
@crosshawk1944 Жыл бұрын
This is the best tutorial I have ever seen. Thank you very much!
@Superdupafool Жыл бұрын
how? it's literally outdated. there's no way you learned anything useful with this.
@satyamraj17 10 ай бұрын
@@Superdupafool wow
@HerozTech 2 жыл бұрын
I love this course ❣️
@vedantsohoni6278 2 жыл бұрын
The requested page "/test/jquery-download-progress-bar-demo.html" could not be found. Any solution to this?
@jasondubon8090 2 жыл бұрын
the page was moved here: demo.seleniumeasy.com/jquery-download-progress-bar-demo.html
@_mytube_ 2 жыл бұрын
Nicely explained man, thanks
@SkWebStudio 2 жыл бұрын
Thanks for the video!
@hmm7458 2 жыл бұрын
i needed this😭
@Brocollipy 2 жыл бұрын
Really helpful stuff, many thanks!
@saburiyusuf7451 2 жыл бұрын
This is a great Selenium Tutorial.
@tahagoren7011 2 жыл бұрын
Please keep continue this diamond tutorials thanks a lot
@maherhanna2432 2 жыл бұрын
This is an amazing tutorial thank you very much
@ahmedqureshi4831 2 жыл бұрын
Great Project! Loved the accent. However, there is one thing i would request for the next selenium tutorial, I am would like to learn more about multiple browser window handling and switching. Thank you for this amazing #CrashCourse
@Palmit_ 2 жыл бұрын
Almost all the tests links are dead AND the code is outdated. Selenium now uses find_elements() NOT find_by_id() or find_by_class_name() etc.
@Vyzinn Жыл бұрын
How does find_elements() work?
@reklamsz3m3t Жыл бұрын
@@Vyzinn class = .classname, id = #id etc.
@Palmit_ Жыл бұрын
@@reklamsz3m3t a belated but sincere thank you to you :)
@shivamgupta-hn2nf Жыл бұрын
Can you please tell me How to find this attribute name losser-id = "name-of-loser" in selinium
@sinbad2597 Жыл бұрын
@@shivamgupta-hn2nf self.find_element(By.CSS_SELECTOR,'tag[losser_id="name_of_loser]') Here for tag,you have to find which tag this stuff is located in . Like div or a or li. (if you aren't using classes , use driver instead of self )
@aben6717 2 жыл бұрын
Thanks for the great content. Just one thing : Regarding selenium links: the pages aren't found. Anyone could help? thanks!
@mohammedbadi6938 2 жыл бұрын
where was this 8 months ago Greatest channel ever
@Interesantes91 Жыл бұрын
the SeleniumEasy links no longer work
@judahnat6529 Жыл бұрын
Great tutorial and great and clear teaching
@alibulus4382 2 жыл бұрын
Hey, help me please ! I am trying to use default profile of Chrome (or Firefox) but i cant. I need to keep my login informations because i want keep my accounts on until i close them. But every single time when i run project it creates new profile (i think?) and my login informations disappears, my accounts closes. I couldn't find any useful documentation or advise, please help !
@et8175 2 жыл бұрын
Anyone else have an issue with the os.environ['PATH']? I had to add a semicolon for it to work: r";C:/SeleniumDrivers". Anyone know why it doesn't add it automatically? Thanks.
@et8175 2 жыл бұрын
Are the seleniumeasy links working? They seem to be removed from the website.
@Doom_C 2 жыл бұрын
Wow, posted 1 day ago, and fixed my exact problem. No idea why this works.
@emimartin7044 2 жыл бұрын
Thanks for sharing this, i didnt know i should add semicolons there.
@muhmmedarbee8014 2 жыл бұрын
THANK YOU FOR THIS!!!!
@Manikanta-ko7qy 2 жыл бұрын
@@et8175 thanks man..I am searching for this..
@grumpy653 2 жыл бұрын
Amazing tutorial!
@kamiln851 2 жыл бұрын
This tutorial is great!
@____kklw7148 2 жыл бұрын
Thanks for teaching. I have learnt a lots
@neillunavat 2 жыл бұрын
Finally a tutorial I will surely watch.
@neillunavat 2 жыл бұрын
I have an issue called 'procrastiantion'. I just don't end up watching most of the vids I wanna watch...
@sinbad2597 Жыл бұрын
You watched?
@neillunavat Жыл бұрын
@@sinbad2597 hell no 😂
@mcredifine 2 жыл бұрын
what I dont understand is what happens if we upload this project to a server. will it still work if that server is on linux for example and they dont have chrome installed on that server. is it using the user's chrome?
@evanmedi6144 2 жыл бұрын
how can i click each element with same class name in row, like click a tab copy some content text and then return to click the second otherwise how can i add an argument to go back to previous page using this exact code?? because when i use built in driver.back() it return an error that driver has not been defined
@KevinOgutu-cx8ix 10 ай бұрын
I really love the tutorial, I have been folloeing your tutorials always and thumbs up. I have an enquiry; suppose I want selenium to be refreshing a specific website at an interval of 2 seconds without stoppind, and if a certain link pop ups on that website, click on it. What is the procedure for that?
@Rohan-bg8ci 2 жыл бұрын
And as always thank you FCC
@sampasomnathvloger7781 2 жыл бұрын
Beautiful 🙏🏻
@yorukama 2 жыл бұрын
hey its this guy. Love this guy.
@muhmmedarbee8014 2 жыл бұрын
Hi Could you provide me with the version details, what version of python you used, what version of pycharm you used, etc etc
@manjunath7497 2 жыл бұрын
😫guys please.. I beg u, really u are making much efforts. Giving all these for free is not a small task.
@exspider64 Жыл бұрын
I needed to add ; before the driver path
@rAskRedditHighlights Жыл бұрын
Does anyone know if there is a way to use selenium but on an already existing tab (so i don't have to sign in on the thing it opens)
@fighting_777 10 ай бұрын
Hi, which link is the one you mentioned in your video 44:50, which explains python project structures.
@andreotako2020 Жыл бұрын
Hello If I wanna build an app on mobile as APK integrated with selenium to do my automation testing on mobile instead of computer could I do it with selenium ?
@user-cq5hr2zg1u 2 жыл бұрын
Videos on this channel are just free gold mines, can’t believe you can get so much knowledge for free.
@alejo5007 2 жыл бұрын
Hi, i got an error specifying the path to the web driver like it's done in the video. Instead, i managed to make it work by adding the path like this: driver = webdriver.Chrome("path_to_webdriver") Is there any problem doing it like that? Thanks in advance and thanks for this course.
@MyLoweLife 2 жыл бұрын
This method worked for me as well so thanks for this
@lc4355 2 жыл бұрын
This also worked for me on macOS, thanks!
@justme-ns1rm Жыл бұрын
Hi, I'm getting an error both ways. Has anyone else had this problem?
@lucasmate1582 Жыл бұрын
@@justme-ns1rm if you use windows try to use semicolon before the C:..... like ;C:/SeleniumDriver/
@FTCHLE Жыл бұрын
@@lucasmate1582 work thx!!!
@user-rk7sx2uu2l 11 ай бұрын
"Thank you sincerely for creating and sharing the video. I'm incredibly grateful for the wealth of knowledge I gained about coding through your content. The concepts I learned, particularly finding new locators and changing the codes, have provided me with valuable opportunities to put my skills to the test. I can't express enough how much I appreciate your efforts in helping me expand my coding abilities. Thank you wholeheartedly once again!"
@sunday-ucheawaji7966 2 жыл бұрын
This is great!
@dontbelasagna5968 2 жыл бұрын
it works until after search click.. filtration didnt work on me :( copied word by word but it says unable to locate element how do i fix this
@realbutters 2 жыл бұрын
Was getting 'chromedriver' executable needs to be in PATH despite following along with the recommendations and I figured out the issue for both Mac and Win. First, I added a print statement right after the os.environ path setting piece: print(os.environ['PATH']) I noticed it was missing a ; in Windows (: in Mac). So, I modified driver_path append step to this and it worked: os.environ['PATH'] += os.pathsep + self.driver_path os.pathsep adds your OS specific path separator character.
@realbutters 2 жыл бұрын
There are other ways to do fix this problem, but this is what I did in a pinch to see what I could do to make it work like the teacher was doing it.
@rejkee Жыл бұрын
Thank you, it worked for me.
@MiturGrunge Жыл бұрын
Great solution! Thanks! I figured out that simply adding a ; in front of the driver path works as well, so my line looks like os.environ['PATH'] += r";C:/selenium_drivers"
@bugrahanozcan8682 Жыл бұрын
@@MiturGrunge what is meaning of semi-colon?
@adrianford468 Жыл бұрын
Can you share you your code at this part for an example
@mainbotpy Жыл бұрын
It was a great one for sure
@engrkpt 10 ай бұрын
Hi all! The seleniumeasy links are not working anymore so can anyone suggest a URL like those for us just starting? Thanks!
@mayurasandakalumsellapperu6778 Жыл бұрын
Thank you 😍🔥🔥
@bhadreshdas1567 Жыл бұрын
How do i setup the chrome driver for MAC OS? I am using a MAC M1 system. Need help
@matthewtalbot-paine7977 2 жыл бұрын
is find elementS not better? If you find element and it doesn't find that element then it will throw an error where as if you use find elements it returns a list and it will be empty if it doesn't find anything but it won't throw an error and then you can check the list.
@irFrank-woo 2 ай бұрын
i really appreciate your work, and i have a wired question here, sometimes i run the run.py, it comes to the privacy stuff page, and it stop my code from continuing the flow. Anyway, thanks and respect
@davidwang9827 Жыл бұрын
It's helpful for me. thank you very much.
@sheillaprosper7126 16 күн бұрын
i'm a newbie. What IDE are you using? how did you get here? I already installed selenium on the terminal, got that. But which IDE is this
@ovp1774 Жыл бұрын
1:01:04 when i try to run the code, it gives me the error with Booking() as bot: AttributeError: __enter__
@sovietcat4825 2 жыл бұрын
this is awesome
@amanshaikh9900 2 жыл бұрын
Thank you so much guys
@rishabhkhandelwal4627 11 ай бұрын
options = webdriver.ChromeOptions() options.add_experimental_option("detach",True) driver = webdriver.Chrome(options=options) Use this code if your chrome crash.
@user-mc7is4kx3o 10 ай бұрын
куда это писать во второй части, когда мы открываем booking?
@joaovitordutra9365 2 жыл бұрын
Are the tests from seleniumeasy down? I've tried to enter to code along with the tutorial, with no success. Are there any alternative links? Thanks
@muhammadshaharyar1 2 жыл бұрын
It's also not working when I am selecting rating. So, I did it on my own. star_filteration_box = self.find_element_by_css_selector( 'div[data-filters-item="class:class=5"]' ) If you are stuck here, it will help you.
@KimKim-fu7zs 2 жыл бұрын
@@muhammadshaharyar1 star_filtration_box = self.driver.find_element_by_xpath( '//*[@id="searchboxInc"]/div[1]/div/div/div[1]/div[5]') I tried this way, it works.
@zhanhongkang5897 2 жыл бұрын
u may also find the Section 3 via the URL above as well. (since I'm not further to Section 3 yet)
@xiaozhenyuan2624 Жыл бұрын
Thank you, the example url of the download button is lost. And I found a similar website with download button without id, how can I do?
@ichigokurosaki7945 2 жыл бұрын
I am running code in virtual environment sum1.send_keys(keys.NUMPAD1, keys.NUMPAD5) not working should run code outside of virtual environment??
@laus-thecurious4120 2 жыл бұрын
is the pycharm free , or professional used in this vedio . which is it ?
@RaushanKumarRavi 2 жыл бұрын
is there any way to see the result in terminal without opening the chrome browser?
@thiagocruz1149 Жыл бұрын
Hi There! Thank you so very much for this tutorial. I just would like to fix something happening on my machine. I've passed thru all the initial steps to get chromedriver on my c:\SeleniumDrivers folder. However, When I try to run the command "navegadorChrome = webdriver.Chrome()", my Google Chrome browser it comes up for a few seconds then it shuts down itself. I am wondering if you could help solving this problem. Thanks in advance!
@RadomName3457 Жыл бұрын
Wonder if you do the context manager like in the tutorial, if so then don't do it. the chrome will automatically shut down (i am not sure why but i experienced this). Instead, create a separate function to quit the window. Hope this helps if you are still struggling with this
@vlads7774 2 жыл бұрын
none of the links actually work , it says the page can’t be found . can you please try to fix that ?
@dimazepam 8 ай бұрын
great course
@matthewtalbot-paine7977 2 жыл бұрын
I've just started using selenium and I've created a bot that collects from a bitcoin faucet using a vpn and I'm currently running 10 accounts at once although I need to investigate headless and I need to add some asynchronisty to it.
@trido3815 Жыл бұрын
Thanks, Jim