Stack Overflow, ChatGPT, and Microsoft documentation couldn't easily explain this process to me. This KZfaq video is gold - thank you for the detailed and clear steps! This worked perfectly and helped me understand the nuances of the ADF pipeline.

It is Really helpful annu. I suppose to start like this. But dont have proper time. Anyways you done a great job. My self Anil. I’m a solution architect in data engineering

Thanks for a great step-by-step explanation of how to copy files by the last modified date.

very helpfull,but walk through bit slowly

I like the way of teaching and content

Awesome video and very helpful, thank you!

Excellent explanation, thank you!

HI, Thanks for the detailed steps .Its a very good content and its working also. Could you please share how to load latest file to snowflake table from blob storage ? Please note the file is excel one.

Very good explanation 🙂 many thanks ANU. i am following your each every video keep going....

This is Golden thanks a lot you just made my day.

Nicely explained

Nice explanation Annu 👌 👍 👏

Crystal clear 🔥

Clean and very simple explanation. Can you please let me know how I can send the latest file as a attachment in a email instead of copying it to another folder

Great video, I am having this doubt here in this video we are getting latest file in dataset from there can we use that lastest file in dataflow

Good explanation. Small advice for you to make it more useful this video to viewers is write the steps while implementing or before starting the implementation. Then learners can easily remember the steps and process. Don't think this is -ve comment. Just explored my experience after watching your video. I have seen lot of videos in other KZfaq channels, no one is explaining the implementation with writing steps. Beginners like get confusing with the steps. If possible to you then you can add the steps. This is my suggestion only

Annu garu ..good explanation

Hi Annu, I have two parquet files in my storage. After I run my pipeline why am I getting two set variable2 and two set variable1 as the pipeline result? It is supposed only to show one set variable1 and one set variable2 right as the result for the latest modified file?

Thaks a lot Annu..

Hi ,kudos! This is really an informative video, however- I have a small doubt- “what if all 5 filenames are same but only last modified date is different. Then how will it pick the latest file? Bcz you only set the - FileName variable to set variable output. In this condition- all files are same. I hope i am able to explain my question. Pls help with a solution. Thanks

hi annu...i am looking for the same opertaion to be performed using azcopy command in a shell script

Is it possible to apply same in SharePoint to extract the new/modified files..?

Would the set variable capture two files with same last modified data and two different file names?

Good explanation

Thank you !!!

great tutorial, thanks!

Hello your videos are awesome..in real time scenario what if we are moving the files after copying to archieve folder this way source folder will have only latest files..should we still consider using this approach??if we have more number of files it will take huge time and performance may impact?please advice

When I re-run same pipeline, Why I am getting different folder names as latest folder? Can you help.. I am trying this logic on folders.

Thanks very much Annu. it's been a very helpful and handy video. It helped me to get the latest file in a blob storage. I have a question however, if I have a mix of different file types (lets say csv and JSON), is it possible to specify to extract the name of only the CSV file? I created a blob>eliminated csv dataset, however it seems to select any file times in the directory. I would appreciate it if you could advise how to sort this issue out. Thanks

Hi Annu, if i want to compare current date and lastModified date then what would be the if condition expression? , If yes thn success els failure..

But this only work for single file ,like when i upload more than one file Ex.- 2 file ,then it only fetch latest uploaded file not both files , how can i handle this ?

Very good explanation. Thanks for that . I have followed your video and tried but i was getting multiple set variables in my second run and each one showing the files name and last modified . I need only one out put for my second run for latest file . please help me

How do I copy the contents of the last found file into a table in Snowflake?

Hello I need your help...In my scenario we have a CSV file in azure data lake with no header and some column data contained comma delimeter like a,b,c while doing copy data task ADF treat this data as a separate column but this data should be on one column... thanks

Hello, I have read that we can not use set variable activity inside For-Each when for-each is set to parallel . But here we are using it.Can you please explain In which case this limitation will create impact.

Great

Hi Annu I am Data Analyst can I start this playlist?

Imagine the level of debugging required incase of an issue. Cant you just put the list of files into a SQL table and then return file on the max modified date and just copy that file ?

Anu why you have used last modified value to referdate inside if condition..It should be after if condition. do you thing it is right approach

kzfaq.info/get/bejne/qb99ac6H1dTLYmw.html I did not undesrstend, you say that value of output similar as value of file in storage account, BUT time of output 2022-09-03T12:14:01Z, and time of the last modification file in container 9/3/2022, 5:44:02 PM How can it be the same time?