HI Soumil, Perfect solution Soumil. So my case, I have some streaming tables same like your demo, and after landing on S3, how can I do join them for further real time analytics ? Can Flink do it by select data from Sink table and join each other for further analytics ?

nope doesn't work, don't waste your time

You will reach more heights soumil… will be there to watch ❤

Sir you are amazing. Thanks for the content...

You deserve it bro

Amazing, I have good experience in Python but no video gave me the right insight or interest to understand these patterns thank you Soumil because of you I have learnt these things otherwise I was running away here and there.....

gtihub link not working

essentially enjoyed

Hi Soumil, I am unable to access the pdf can you help me with that. Thanks

good

what a legend!

can you please put a video on iceberg DR?

very annoying to watch. frequent switching between windows very stressing

Do you bowl as fast as you talk?

Thanks for posting this ❤ it

Hi Soumil, Thank you for your detailed videos. Could you point to any resource that can help setup the IntelliJ for pyspark?

Hey bro I cloned a website and now I am opening that website code in vs code editor but after doing the necessary editing only text is changing not the images. Like I am putting my image URL on the place of website image URL but after saving it and opening it with live server the preview is showing me the images of cloned website not mine and in inspect element it is showing the image code of cloned website not mine why. I am trying from 6 hours and nothing is works for me. Will you plz tell me how can I change the images and edit it.

After installing, when I try to run elasticsearch.bat file it is showing error like \Java\jdk-21.0.1 was unexpected at this time. But my jdk and java bin folder paths setted correctly in environment variables

yes, please slow down yourself in every respect pls

Thanks for the video. It's a good one. Do you have any samples related to the scenario where we have to read the Avro data from a Kafka topic and upsert into the Hudi tables?

amazing work @soumil. Thanks

Hi Soumil, thanks for the video. Using openjdk 11 and Python 3.8. I can't see the table printed when run 'Creating Dataframe from List of Tuples'. Used Jupyter notebook as well as VS code Editor. Any idea.

Hello, I was looking at your video channel. We may be helping a company that uses secure images to increase supply chain security and help cloud native development. Would you be willing to help try their software, make a video, and help show devs how to use their tools? This is not an offer, but just to start a conversation about your willingness to take on sponsorship. Please provide me with your email if you are interested. You'd have a chance to look at their technology and decide if it's the type of software that you'd be interested in covering in your channel.

hello soumil data,schema assigned to a dataframe but when i used df.show() i am getting error..

Thanks for the great content man. Can you please re-upload the Lab link as repository is not accessible

aviter game hack video plese

cool video content

Hi Soumil , Thank you for the video. I know there are certain catalogs available now for iceberg . In this case we utilizing glue as catalog or one table as a catalog? Also to automatically or incrementally sync data into iceberg table we have to event based trigger process to run that Java command?

Where is the concept for CDC?

how to handle zero values in csv file and how to fill those values

@Soumil, How to run glue notebook ondocker container? Please refer any video?

How to handle if secret manager has rotation in python?

Thanks for the video, I noticed that the error logs were not marked as error by datadog, any idea on how to do that? I'm trying to send an artificial error to see if I can create a notification when something fails but datadog always mark them as INFO logs

Thanks for sharing! For the future video, same with UntyCatalog maybe?

Soumil thank you for the video! Is the same thing possible, but with the Iceberg format?

Hello im having trouble with creating the connection

Hi please provide the link to for the smart library

We need the same thing about Iceberg. Thx Soumil!

Could not create pixmap from (the path)

thank you so much

Why did u hashed? You can directly remove the duplicates by .dropDuplicates right?

Hi Soumil you have not given any configuration to do in airflow.cfg did the solution you give will work when we want to parallelise multiple task inside a dag and parallelise multiple dag ? Other people are giving solution like change the database to mysql or postgress and chnage executer to LocalExecutor what do you think about these solutions?

We dont really need to attach ticketId to user as user can exist without a ticket. Also for fetching all the tickets associated with a given user, we can use userId GSI on ticket model

awesome video! thanks for sharing. +1 sub

you are a god

thanks soumil

@soumilshah thanks for your informative video, but the link you have given in description for pdf files is not working. could you please update that with right url.

Sir, can i store and unzip these docs in other drives and not in c:?

Thanks Mr. Shah--- I used these commands to prepare a cheat sheet for data cleaning--- regards