Tutorial: How to upload files into a S3 Bucket using a REST API via API Gateway

  Рет қаралды 32,180

Let's Find Solutions

Let's Find Solutions

Жыл бұрын

A quick tutorial on how to use API Gateway to create a REST API to upload files into a S3 Bucket
Topics Covered:
1) S3 Bucket Creation
2) IAM Role Creation
3) IAM Policy Creation
4) API Gateway Creation
5) REST API Creation

Пікірлер: 105
@rascode
@rascode Ай бұрын
This was incredible. It would have taken approximately 1 million years to figure this out by reading the docs alone. You're a hero.
@letsfindsolutions
@letsfindsolutions Ай бұрын
Glad you found it helpful
@letsfindsolutions
@letsfindsolutions Ай бұрын
Glad you found it helpful
@dieselKopter
@dieselKopter 12 күн бұрын
one of the smoothest tutorials. i'm shaking your hand virtually!
@letsfindsolutions
@letsfindsolutions 3 күн бұрын
Happy to hear that!
@JavierHernandez-xo5nb
@JavierHernandez-xo5nb Жыл бұрын
Thank you so much for sharing your knowledge. I also want to add that the way you present the process was very satisfactory. I look forward to seeing more videos on the subject.
@letsfindsolutions
@letsfindsolutions Жыл бұрын
I have been busy lately, but I plan on doing another video in the next couple of weeks. Thanks for the encouraging comment.
@satheeshbabu
@satheeshbabu 7 күн бұрын
Exactly what i was looking for.. Thanks a lot
@letsfindsolutions
@letsfindsolutions 3 күн бұрын
Glad it helped
@jameswild7558
@jameswild7558 5 ай бұрын
So insightful! Thank you.
@letsfindsolutions
@letsfindsolutions 5 ай бұрын
Thanks
@pa7235
@pa7235 4 ай бұрын
nice, no nonsense, no silly accent, 10/10 thank you sir
@letsfindsolutions
@letsfindsolutions 4 ай бұрын
Thank you kindly
@RebornLegacy
@RebornLegacy 4 ай бұрын
Just curious, what's a silly accent?
@zishanchaudhary221
@zishanchaudhary221 2 ай бұрын
@@RebornLegacy indian accent
@RebornLegacy
@RebornLegacy 2 ай бұрын
@@zishanchaudhary221 Ohhh, so you guys are neanderthals complaining about free instructional material. Got it
@azoz1433alamodi
@azoz1433alamodi 10 ай бұрын
sir , you are a life saver , thank you
@letsfindsolutions
@letsfindsolutions 10 ай бұрын
Glad you found it helpful.
@anthonysosa1900
@anthonysosa1900 2 ай бұрын
genioooooo!!!, gracias loco
@letsfindsolutions
@letsfindsolutions 2 ай бұрын
Gracias
@waytobhushan
@waytobhushan 5 ай бұрын
Hi @letsfindsolutions Your video is like, follow me. Crisp and up to the point. It took hardly 20min to reproduce what you have explained. Wish to see more video from you.
@letsfindsolutions
@letsfindsolutions 5 ай бұрын
Will eventually add more when I get time.
@rghp6128
@rghp6128 Жыл бұрын
good one, thank you
@letsfindsolutions
@letsfindsolutions 10 ай бұрын
Glad you liked it!
@yesimscheidj
@yesimscheidj Жыл бұрын
Im really grateful for this tutorial that you created. Also may I ask if how can I use it or the api so I can use it in php and upload files from the local storage into s3 bucket instead of the server side?
@letsfindsolutions
@letsfindsolutions Жыл бұрын
You can have your php grab files from local storage and call the api...similar to having a web page with a file selector and doing ajax call to the api to upload the file.
@coder040
@coder040 Жыл бұрын
Hey Thank You for the wonderful explanation, just had one question, is it possible to pass "chacterset=utf-8" while uploading file
@letsfindsolutions
@letsfindsolutions Жыл бұрын
I am glad you like the video. As for your question I have never really tried using utf8. I think as long as you just include all the media types it can handle any file encoding you want to handle.
@lucasvargasquintana8396
@lucasvargasquintana8396 Жыл бұрын
Wow, this a really straightforward solution! I was able to follow your tutorial easily and upload a file to AWS S3 without any trouble. Thank you for sharing your knowledge! I have one question though. Is it possible to upload a file to a specific folder within a bucket on AWS S3? If so, could you please explain the process of achieving this? I would greatly appreciate your guidance. Keep up the amazing work
@letsfindsolutions
@letsfindsolutions Жыл бұрын
Glad it helped! As for your question, I have not tried out specific folders, but looking into some threads I see that S3 is pretty much a flat storage structure. So in this thread, they mentioned including the full path as the file name...I would give this a try. (stackoverflow.com/questions/51214518/upload-files-to-a-specific-folder-in-the-bucket-with-aws-sdk). Essentially when specifying the filename, try passing the FULLPATH and see if that works.
@sayedimran5972
@sayedimran5972 10 ай бұрын
@@letsfindsolutions @lucasvargasquintana8396 I was able to find the solution to this. Having the same configuration as in this video, you can provide the path with %2f. Eg: Suppose you want to upload to path bucket/folder1/folder2/test.txt, so you will be doing: bucket%2ffolder1%2ffolder/test.txt
@letsfindsolutions
@letsfindsolutions 10 ай бұрын
@@sayedimran5972 Thanks for sharing
@azertyytreza846
@azertyytreza846 6 ай бұрын
@@sayedimran5972 thank you! easy and working well
@mateotorrecilla5019
@mateotorrecilla5019 5 ай бұрын
That was really helpfull, thanks for sharing!@@sayedimran5972
@muhammadhsajjaadali855
@muhammadhsajjaadali855 Жыл бұрын
Thank you. This was very informative. Will this video help me in uploading a Json file into aws?
@letsfindsolutions
@letsfindsolutions Жыл бұрын
Yes, you can upload any file type into the s3 bucket.
@user-qj1jx6wi4l
@user-qj1jx6wi4l 3 ай бұрын
i want to do OTA update by giving this the url of the uploaded firmware file. is this method ok for that? what happened if I just upload the binary file?
@letsfindsolutions
@letsfindsolutions 3 ай бұрын
This methid requires a valid jwt token...so as long as you can authenticate and pass along a token this will work.
@rishabhraj1772
@rishabhraj1772 4 ай бұрын
Thank you man Actually i am trying this its works on postman ,but i want to use API gateway endpoint in my JS code can you please suggest me .
@letsfindsolutions
@letsfindsolutions 4 ай бұрын
You could just use a library like axios or just use jquery ajax calls to the api. Keep in mind this expose your keys, so really its best to have it call your own back end layer which then calls your api to keep the keys hidden.
@user-gr7no2pn3l
@user-gr7no2pn3l 5 ай бұрын
thanks for the great video, I'm starting a project where I'll need to use S3, why use the API Gateway and not S3's own REST API?
@letsfindsolutions
@letsfindsolutions 4 ай бұрын
I had a use case to use a custom authorizer, so api gateway allows for this type of configuration.
@AWSFan
@AWSFan 4 ай бұрын
Absolutely the content am looking for.. I also would like to know how to write custom response body in postman when status code is "200 OK"
@letsfindsolutions
@letsfindsolutions 4 ай бұрын
You would need to write a custom lambda for that.
@leandroosterne
@leandroosterne Жыл бұрын
How would it be via POST, to send file via base64 string?
@letsfindsolutions
@letsfindsolutions Жыл бұрын
I think you can change the method to a POST and just add in additional media types to handle the base64 encoded data being passed.
@govindanpotti5005
@govindanpotti5005 4 ай бұрын
Thanks a lot for this video. I've a question about authentication. Here, the REST API call from postman worked as we have given the IAM role with putobj policy. How about if I have to restrict the upload to a specific user? Say, the upload happens from a custom web application, the user uploads the file, is there a way to include IAM user authentication here? Could you please give any pointers to this?
@letsfindsolutions
@letsfindsolutions 4 ай бұрын
You could use a lambda authorizer (formerly known as custom authorizer). Here you have full control of the auth logic and can validate your keys against specific groups or users.
@govindanpotti5005
@govindanpotti5005 3 ай бұрын
@@letsfindsolutionsThank you!
@proan_1992
@proan_1992 Ай бұрын
was there a change in AWS UI, I am having a problem creating api resource as path parameter using {}?
@letsfindsolutions
@letsfindsolutions Ай бұрын
Yes they changed the ui a while back
@mandagore4931
@mandagore4931 28 күн бұрын
Why do you use the path override {bucket}/{filename} for your child resource {filename} if you already gave your endpoint these same structure /{bucket}/{filename}. An action name in your method of your {filename} would have been good enough right?
@letsfindsolutions
@letsfindsolutions 14 күн бұрын
I guess just for demo purposes...but good point.
@user-iy7gp8zk3u
@user-iy7gp8zk3u 3 ай бұрын
It is uploading the file into but there are some issues m facing: 1) even if am specifying the content type in Binary-media-type (audio/mp3), still m able to upload any file 2) the file gets corrupted when it's uploaded I tried form postman and even tried using flutter
@letsfindsolutions
@letsfindsolutions 3 ай бұрын
Possibly the encoding is messing up somehow...aside from that i. Not really sure what else it could be.
@kienvan1006
@kienvan1006 9 ай бұрын
How can I get the content of the file in the bucket using the api? I mean GET method
@letsfindsolutions
@letsfindsolutions 9 ай бұрын
You would need to write a new get method to read and serve back the content.
@user-ti5zm2wy1l
@user-ti5zm2wy1l 10 ай бұрын
what are configuration for your stage of dev? And my problem is that file successfully upload but size is 0kb so any solution?
@letsfindsolutions
@letsfindsolutions 10 ай бұрын
What are you using to test the upload? In the video I use postman and i believe all its doing is converting the data to base64 encoded binary data. Similar to what a web browser would do.
@user-ti5zm2wy1l
@user-ti5zm2wy1l 10 ай бұрын
Thanks @@letsfindsolutions
@user-uj4fo6oo5d
@user-uj4fo6oo5d 9 ай бұрын
Thanks it works for me, now problem i am facing is, once file upload to s3 bucket i can see file, but when i use to open that file with object url than my file is not opening, its giving me error like access denied so let me know how to give public writes to view that file simultaneously.
@letsfindsolutions
@letsfindsolutions 9 ай бұрын
I believe you will need to update the permissions of the s3 bucket to allow for public read.
@user-uj4fo6oo5d
@user-uj4fo6oo5d 9 ай бұрын
@@letsfindsolutions Thanks i done yesterday only by myself by creating policy
@P.Shivakrishna
@P.Shivakrishna 10 ай бұрын
While Edit integration request to add URL path parameters when click save it showing Network Failure( Anyone help Currently I am Using ap-south-1 Region ) i don't know what is the problem ??
@letsfindsolutions
@letsfindsolutions 10 ай бұрын
Not sure im understanding the iasue...but maybe try switching regions. I usually use us east.
@P.Shivakrishna
@P.Shivakrishna 10 ай бұрын
@@letsfindsolutions I tried us east also but it's not working in api gateway
@gurpreetkaur-cw8wo
@gurpreetkaur-cw8wo 2 ай бұрын
i am also facing same problem while creating resource in API gateway can't add brackets with bucket in resource path, same with filename , my resource path is not taking brackets or I can say anything . error invalid mapping expression parameter specified
@gurpreetkaur-cw8wo
@gurpreetkaur-cw8wo 2 ай бұрын
I resolve my problem put path in single inverted comma '' that will work as i am not getting option to put curly brackets
@sandipkuila
@sandipkuila 11 ай бұрын
Thanks for this solution. I have implemented this for MS excel file upload but the size got change every time. Can you please help me out with the solution for that. Facing the issue with format change and file size got increased during upload.
@letsfindsolutions
@letsfindsolutions 11 ай бұрын
Did you follow the video...what media types have you defined?
@sandipkuila
@sandipkuila 11 ай бұрын
@letsfindsolutions I have allowed all media types. But only for Excel, it got format change happens
@letsfindsolutions
@letsfindsolutions 11 ай бұрын
Maybe the encoding ia somehow getting changed on upload...excel would be similar to other binary files.
@TPDene
@TPDene Ай бұрын
I don't have the "Create policy" button from 2:37 .. any ideas why?
@letsfindsolutions
@letsfindsolutions Ай бұрын
They changed the ui a while back...it might look different now.
@ujjwalm29
@ujjwalm29 8 ай бұрын
In this method, can anybody in the world upload to my bucket? What if i wanted to restrict it to just myself? Using an access token or api key?
@letsfindsolutions
@letsfindsolutions 8 ай бұрын
With api gateway you can enable an Authorizer and have it validate a token i the header. You can setup cognito to work with the authorizer to validate the tokens.
@letsfindsolutions
@letsfindsolutions 8 ай бұрын
Look at my other video about jwt authorizer....you would use the same approach there.
@MayankGaurm
@MayankGaurm 11 ай бұрын
This is really great video. I have few questions 1. Did you pass any token in header for validation. 2. I can give this url to upload file to anyone, whether he is a AWS user or not
@letsfindsolutions
@letsfindsolutions 11 ай бұрын
1. There is no security around it so no token is needed. You can add cognito security with an authorizer to validate a token if needed. 2. Yes the url is public and open. It can be called like any other api publicly. They do not need to be anaws user.
@MayankGaurm
@MayankGaurm 11 ай бұрын
@@letsfindsolutions many thanks.
@letsfindsolutions
@letsfindsolutions 11 ай бұрын
I am glad you found the video helpful...I really do plan to make more...just haven't found the time to lately.
@sudeepshiv
@sudeepshiv 8 ай бұрын
great video and thanks for sharing. I would like to upload via browser instead of postman, is there any way to api via browser ?
@letsfindsolutions
@letsfindsolutions 8 ай бұрын
Yes, you would need to build a front end ui...like a react ui web page that can make calls to the api similar to postman.
@PreetkanwalSingh
@PreetkanwalSingh 27 күн бұрын
@@letsfindsolutions can you tell how to do that in Vue.js , i tried a lot, didnt get any solution
@damarisscott7643
@damarisscott7643 2 ай бұрын
Hi all, I'm pretty new. I followed all these steps, made sure that my S3 and API endpoint are in the same region and yet, I still keep getting this error: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint I'm trying to upload a JSON file. Have any of you had this issue?
@letsfindsolutions
@letsfindsolutions 2 ай бұрын
I have never encountered this issue. In readibg your error, this looks to be endpoint uri related or something wrong with the endpoint permissions.
@damarisscott7643
@damarisscott7643 Ай бұрын
@@letsfindsolutions Hi, it was something with the permissions :) Thanks for your reply, I was able to resolve it
@TheRealKosalaHandlersMail
@TheRealKosalaHandlersMail 20 күн бұрын
Does anyone know how to do the same with Terraform?
@letsfindsolutions
@letsfindsolutions 14 күн бұрын
I sure there are some yaml samples out there. Unfortunately I don't have any.
@Vstosky
@Vstosky 10 ай бұрын
It should return the link for access the file
@letsfindsolutions
@letsfindsolutions 10 ай бұрын
That can be customized if needed
@ManishKumar-rz9ub
@ManishKumar-rz9ub 7 күн бұрын
i am getting "Internal server error"
@letsfindsolutions
@letsfindsolutions 3 күн бұрын
I know the ui looks different, but my guess is that this is most likely configuration related
@sachinbaghel6225
@sachinbaghel6225 5 ай бұрын
When I give a path resource, it does not get it
@letsfindsolutions
@letsfindsolutions 5 ай бұрын
Maybe try recreating it. Sometimes that helps.
@sachinbaghel6225
@sachinbaghel6225 5 ай бұрын
@@letsfindsolutions by the way thak its help
@gurpreetkaur-cw8wo
@gurpreetkaur-cw8wo 2 ай бұрын
i am also facing same problem while creating resource in API gateway can't add brackets with bucket in resource path, same with filename , my resource path is not taking brackets or I can say anything . error invalid mapping expression parameter specified can you help me
@sachinbaghel6225
@sachinbaghel6225 2 ай бұрын
@@gurpreetkaur-cw8woWhen creating a resource, simply write the name within {} brackets, and it will work.
@user-pe3bd3sq7s
@user-pe3bd3sq7s 10 ай бұрын
no it did not work time wasted
@letsfindsolutions
@letsfindsolutions 10 ай бұрын
Sorry to hear it did not work for you.
@shantanusahu9168
@shantanusahu9168 8 ай бұрын
i am getting internal server error while calling the api. is it something to do with WAF or ListWebACLs?
@letsfindsolutions
@letsfindsolutions 8 ай бұрын
Internal server error leads me to believe maybe code related...did you modify the sample code?
@shantanusahu9168
@shantanusahu9168 8 ай бұрын
@@letsfindsolutions there's no code..i followed your steps.. m getting a WCL error on the screen tho,,,could it be the issue? or it seems like s3 access isue
@letsfindsolutions
@letsfindsolutions 8 ай бұрын
Ya could be s3 permission issue
NodeJs How to upload Files + uploading to AWS S3 using Express, Multer
1:16:31
Sanjeev Thiyagarajan
Рет қаралды 46 М.
Upload files to S3 using API Gateway - Step by Step Tutorial
10:53
Enrico Portolan
Рет қаралды 58 М.
НЫСАНА КОНЦЕРТ 2024
2:26:34
Нысана театры
Рет қаралды 1,1 МЛН
Double Stacked Pizza @Lionfield @ChefRush
00:33
albert_cancook
Рет қаралды 98 МЛН
Summer shower by Secret Vlog
00:17
Secret Vlog
Рет қаралды 12 МЛН
Examining Billing data with BigQuery
12:43
cloudsdevspace
Рет қаралды 22
.Net 6 WebAPI - Upload files to AWS S3  - Step by Step
1:03:50
Mohamad Lawand
Рет қаралды 14 М.
AWS S3 APIs Walkthrough (The most important ones!)
21:57
Be A Better Dev
Рет қаралды 14 М.
AWS API Gateway to Lambda Tutorial in Python | Build a REST API
11:17
Be A Better Dev
Рет қаралды 337 М.
НЫСАНА КОНЦЕРТ 2024
2:26:34
Нысана театры
Рет қаралды 1,1 МЛН