Paging @SQLBalls!

I tried to post earlier but it did not post, lets see if it does this time. I think it would depend on the scenario. In the past when I used Beutiful Soup to scrape data I was storing it in flat files in blob, then consuming them in a pipeline, then loading them to SQL, eventually I wrote a python pipeline to insert directly into SQL but it wasn't super efficient because it must use a cursor. Fabric really changes that. Where this was useful for me, was when asked to create a sample data set I opened up OneLake and copied it right over to my desktop and uploaded it to GitHub! I wanted to make sure I showed that both options were possible, but it would really depend on the scenario.

Hi @GuillaumeBerthier, paging @SQLBalls!

Hi @GuillaumeBerthier, Ok this could be a long post. Let me start with what I CAN do, I've uploaded an edited version of the notebook as well as a csv file with sample data. It is not legal to scrape data from a website where you do not have permission to, or if you intend to distribute the data or use it for profit. Unless the owner says it's ok. I need to make that very clear. I *cannot* give out the code to the scraper at this time, because while it is ok for me I cannot give that permission to the world at large. However, I will build a sample web page and next week I will make a video along the lines of "how to use Beautiful Soup to scrape data". Then I will repost the notebook using the code to scrape my website's data, which I can give people permission to do! I hope this helps!