No video

32. Copy data from REST API which sends response in Pages using Azure data factory

  Рет қаралды 51,154

WafaStudies

WafaStudies

Күн бұрын

In this video, I discussed about how to make REST API call which supports paginations and store response in to files or table using Azure data factory.
Link to dummy REST API Used:
reqres.in/api/...
Link for Azure Synapse Analytics Playlist:
• 1. Introduction to Azu...
Link for Azure Databricks Play list:
• 1. Introduction to Az...
Link for Azure Functions Play list:
• 1. Introduction to Azu...
Link for Azure Basics Play list:
• 1. What is Azure and C...
Link for Azure Data factory Play list:
• 1. Introduction to Azu...
Link for Azure Data Factory Real time Scenarios
• 1. Handle Error Rows i...
Link for Azure LogicApps playlist
• 1. Introduction to Azu...
#Azure #AzureDatafactory #DataFactory

Пікірлер: 48
@ganubikash7838
@ganubikash7838 Жыл бұрын
your sentence - "what it will do NA" - it gives feel that we are sitting at class room in front of a teacher. Good job. Keep it up.
@HiYurd
@HiYurd Жыл бұрын
This is excellent video on how to loop through a REST API with pages. Thanks Maheer!
@WafaStudies
@WafaStudies Жыл бұрын
Thank you 😊
@MaheshReddyPeddaggari
@MaheshReddyPeddaggari 2 жыл бұрын
Valuble learning Thanks Maheer
@WafaStudies
@WafaStudies 2 жыл бұрын
Welcome 🤗
@suresh.suthar.24
@suresh.suthar.24 Жыл бұрын
you make adf so simple, thank you
@jayantbishnoi9617
@jayantbishnoi9617 Жыл бұрын
this is so useful! these tutorials are gold!
@annukumari9629
@annukumari9629 2 жыл бұрын
Very helpful 😊 Thanks
@WafaStudies
@WafaStudies 2 жыл бұрын
Thank you 😊
@sunilbailwal4994
@sunilbailwal4994 2 жыл бұрын
Thanks Maheer, there is pagination rule in copy activity which will write all the pages in one file. Can you please provide an example? Thanks
@bindrarawat6623
@bindrarawat6623 2 жыл бұрын
Thanks for the video. My question is how can we achieve the same result when continuation token is being generated and we have to use that token to get the whole data. In that scenario we will not get any total pages property in JSON output
@subhashkumar209
@subhashkumar209 2 жыл бұрын
Hi Maheer, you said that you will prepare a video in future and the same content we achieved here can be done using pagination in ADF. Could you please prepare a video on that?
@DeGuy567
@DeGuy567 Жыл бұрын
Did he make new video on pagination in ADF?
@sr_newchannel8635
@sr_newchannel8635 2 жыл бұрын
Thanks Maheer for sharing not only these but other tutorials, its very useful !! At the end of the video you mentioned about pagination, Were you able to create any video on this scenario of API returning __next?
@AHMEDALDAFAAE1
@AHMEDALDAFAAE1 11 ай бұрын
Thank you so much! However, Pagination rule can help to make it easier without setting parameters
@Lifestyle-mf4fh
@Lifestyle-mf4fh 8 ай бұрын
I created this but after successful debug its showing only one record copied. Two files created in Sink side but only one one record in each files.
@franksomto3211
@franksomto3211 2 жыл бұрын
Thanks Maheer, really insightful. Any update on the next video you promised to make?
@anamariabahudu5536
@anamariabahudu5536 2 жыл бұрын
Can you please make the video on how to create the pagination for APIs that have next links? Thank you!
@sanjeev8406
@sanjeev8406 4 ай бұрын
Awesome Bro...
@pbhaskar5197
@pbhaskar5197 Жыл бұрын
Hi Maheer .. your videos are very helpful to us.. Could you please post the video like how to ingest csv file data from datalake to oracle cloud fusion using ADF and also target side is rest API post method
@WafaStudies
@WafaStudies Жыл бұрын
Thank you. Oracle fusion environment i can't get it. But i try something posting data to api in sink. That may useful
@pbhaskar5197
@pbhaskar5197 Жыл бұрын
@@WafaStudies Thank you so much..
@pbhaskar5197
@pbhaskar5197 Жыл бұрын
and also is there any way to convert CSV file to base64 format using ADF?
@kcsvenkat
@kcsvenkat Жыл бұрын
Hi, Im facing one issue, Client give one API (SAP data), every API call got only 1000 rows we get data every 15 mins 10000+ data we have 2 API, one is get the total row count and other one is actual data I need to assign 1st API count to one parameter please help me how to will do it. based on this parameter need to set number of loops.
@errajinoura3942
@errajinoura3942 11 ай бұрын
hello and thank you for all this explanations, pleease can you answer me about how can I Copy data from multiples REST API which sends response in a sql server table using Azure data factory ?
@carpediem5666
@carpediem5666 2 жыл бұрын
Where is the linke to the next video of using Copy ACtivity using Pagination Rules ?
@liyaspasha3820
@liyaspasha3820 Жыл бұрын
HI Maheer, Can you please create a Video on configure Multiple Rest APIS along with incremental load.
@nindersingh
@nindersingh 11 ай бұрын
Helped me a lot Bro. thanks a ton
@NawazShareefMohammed
@NawazShareefMohammed Жыл бұрын
hello, can we connect dataroma to adlsgen2 and import data from adls to dataroma?
@azurelearner4055
@azurelearner4055 Жыл бұрын
Hello Can we copy the data from powerbi to azure storage please advise
@smotocelll
@smotocelll Жыл бұрын
Can you do the same example for a data flow with pagination? It is diffetent than for copy activity
@WafaStudies
@WafaStudies Жыл бұрын
Sure dan. I will try plan that soon.
@smotocelll
@smotocelll Жыл бұрын
@@WafaStudies Thanks! There is not much info on the internet
@epuriraghavendra3847
@epuriraghavendra3847 7 ай бұрын
can we put sink as rest api and post the data to api ?
@narendras5086
@narendras5086 Жыл бұрын
Hi Maheer I have one doubt can u copy the file from SharePoint to Adls what is the process can u please tel me
@vaibhavshendre1
@vaibhavshendre1 3 ай бұрын
Hey @WafaStudies , Whats the difference between API and Rest API ?
@dilipmevada
@dilipmevada Жыл бұрын
Hi Maheer, How can we make post API request with dynamic field for Paging?
@conaxlearn8566
@conaxlearn8566 Жыл бұрын
Did you end up creating a video with pagination?
@WafaStudies
@WafaStudies Жыл бұрын
Yes kindly check part 48 of same playlist
@mustafakamal5945
@mustafakamal5945 2 жыл бұрын
Thank you very much for creating this, this is really valuable and helpful!! However I was wondering how can we achieve the opposite scenario- posting json data from blob storage to rest api ? Please advise..
@vikaskumarbrahmbhatt3452
@vikaskumarbrahmbhatt3452 2 жыл бұрын
Hello, I am using ADF - Copy Activity - Rest API call (with hapikey) to HubSpot to fetch records... the response provided me the next records URL but it does not have hapikey. Can you kindly provide step by step guide on how to get this working? I have been trying options but none of them seems to work. Also, HubSpot API does not give me how many pages I need to iterate so I really can not use For Loop as you have shown in the above example. I know similar topics are discussed in many forums but are not able to get an exact step-by-step guide on how to achieve them. I really appreciate your videos and explanations as they are easy to follow and do clarify the majority of doubts. Thank you in advance.
@larnelleground9159
@larnelleground9159 9 ай бұрын
@vikaskumarbrahmbhatt3452 Did you manage to figure this out ?
@omkaromble5032
@omkaromble5032 2 жыл бұрын
Hi, please suggest how can I do reverse of this? i.e from ADF Azure SQL table, I want to push the data to on prem SQL Server DB for which I have the REST API link given to this on prem SQL DB , through POST call ? TIA
@mustafakamal5945
@mustafakamal5945 2 жыл бұрын
Hi, Did you get a solution for this scenario?
@krish6338
@krish6338 2 жыл бұрын
try to post how to call coupa API from ADF
@Acidlocuaz
@Acidlocuaz 2 жыл бұрын
Excelente!
@WafaStudies
@WafaStudies 2 жыл бұрын
Thank you 😊
40. Working with Token based REST API in Azure Data Factory
19:39
КТО ЛЮБИТ ГРИБЫ?? #shorts
00:24
Паша Осадчий
Рет қаралды 1,2 МЛН
My Cheetos🍕PIZZA #cooking #shorts
00:43
BANKII
Рет қаралды 27 МЛН
How to - Unzipping your download on Windows
6:14
The Crafty Witches
Рет қаралды 4
Azure Data Factory Mapping Data Flows Tutorial | Build ETL visual way!
26:25
Adam Marczak - Azure for Everyone
Рет қаралды 225 М.
23. Send Email alert when Pipeline fails in Azure Data Factory
21:59
Azure Data Factory | Copy multiple tables in Bulk with Lookup & ForEach
23:16
Adam Marczak - Azure for Everyone
Рет қаралды 188 М.
Working with OAuth 2 0 APIs in Azure Data Factory
54:42
Hybrid Virtual Group
Рет қаралды 10 М.
DataFactory: Extract data from SharePoint Online
10:29
javiBI
Рет қаралды 67 М.