No video

How to Build Python Data Engineering Pipelines with Snowpark

  Рет қаралды 19,918

Snowflake Developers

Snowflake Developers

Күн бұрын

Join Jeremiah Hansen, Principal Data Platform Architect at Snowflake, for a virtual hands-on lab that walks you step-by-step through the process of building Python data engineering pipelines on Snowflake using Snowpark Python. The video includes a brief overview of Snowpark and the benefits it offers by allowing users to work with Python, Scala, and Java natively in Snowflake.
To learn more about Snowflake, enroll in Snowflake's Coursera course: www.coursera.org/learn/snowfl...
00:00-Introduction
01:01-An Overview of Snowpark and Its Key Features
09:40-The Snowpark Python Quickstart Guide
10:33-An Overview of Topics to be Covered in the Hands-on Lab
14:11-Hands-on Lab Begins
15:01-Prerequisites for Participating in the Lab
17:16-Setting Up Snowflake
19:50-A Look at the Snowflake Extension
20:38-The Work Begins: Loading Data
26:41-Collecting Data from the Data Marketplace
30:44-Creating Views and Streams on Views
33:35-Creating a Python UDF
40:12-Building the First Pipeline
48:16-Creating a Second Stored Procedure to Populate Weather Data Table
51:38-Joining Data Together
53:53-The Deployment
55:35-Orchestrating the Stored Procedures
59:30-Running the Pipelines Incrementally
1:01:39-Viewing the Query History
1:06:25-Deploying Through a DevOps Pipeline using GitHub Actions
1:13:42-Reviewing What We’ve Done
To connect with Jeremiah Hansen of Snowflake:
→LinkedIn: / jeremiahhansen
❄ Join our KZfaq community ❄
→ bit.ly/3lzfeeB
Learn how to build your application on Snowflake:
→ developers.snowflake.com
Join the Snowflake Community:
→ community.snowflake.com

Пікірлер: 16
@avitabayansarma1011
@avitabayansarma1011 11 ай бұрын
This was very informative!
@snowflakedevelopers
@snowflakedevelopers 11 ай бұрын
Glad it was helpful!
@jaysonp9426
@jaysonp9426 6 ай бұрын
This was so good, thank you!
@kishorekumarkuruguntla8167
@kishorekumarkuruguntla8167 10 ай бұрын
Great Content
@snowflakedevelopers
@snowflakedevelopers 10 ай бұрын
Thank you!
@luvlifk
@luvlifk 5 ай бұрын
Amazing!!
@sathwikemmadi4428
@sathwikemmadi4428 3 ай бұрын
I think this video is helpful but appreciate if you can send other videos for detailed explanation or online documentation.I am asking about the architecture it starts from 07:19
@joerozario4406
@joerozario4406 10 ай бұрын
If the raw files in csv format, is there anyway to infer the schema without explicitly defining the schema StructType.
@treyhannam3806
@treyhannam3806 8 ай бұрын
Is there documentation on the toml file?
@snowflakedevelopers
@snowflakedevelopers 8 ай бұрын
Hi @treyhannam3806, check this out: docs.snowflake.com/en/developer-guide/python-connector/python-connector-connect#connecting-using-the-connections-toml-file
@bharathraj4545
@bharathraj4545 4 ай бұрын
Do we need an external cloud storage account to implement this project
@snowflakedevelopers
@snowflakedevelopers 4 ай бұрын
No not at all! Try out the lab today and let us know what you think!
@rogierhelmus3481
@rogierhelmus3481 11 ай бұрын
I would appreciate a video on how to set up the environment , since I get stuck on trying to run the python script for loading the raw data. Edit: I found this video very usefull: kzfaq.info/get/bejne/bJ55YNKpyrCYeHk.html&ab_channel=NeuralNine
@snowflakedevelopers
@snowflakedevelopers 11 ай бұрын
While not quite a video, you may find Steps 1, 2, and 3 of the associated Quickstart helpful: quickstarts.snowflake.com/guide/data_engineering_pipelines_with_snowpark_python/#0
@snowflakedevelopers
@snowflakedevelopers 10 ай бұрын
Just a follow up to this. The associated GitHub repo for this demo now uses GitHub Codespaces to automatically provision and set up the environment for you. Check out the updated instructions in Step 2 here: quickstarts.snowflake.com/guide/data_engineering_pipelines_with_snowpark_python/#1
Query Snowflake From Within VS Code In Just 3 Minutes
3:10
Snowflake Developers
Рет қаралды 8 М.
Data Engineering With Python In Snowflake
37:09
Snowflake Developers
Рет қаралды 11 М.
World’s Largest Jello Pool
01:00
Mark Rober
Рет қаралды 124 МЛН
EVOLUTION OF ICE CREAM 😱 #shorts
00:11
Savage Vlogs
Рет қаралды 14 МЛН
No empty
00:35
Mamasoboliha
Рет қаралды 12 МЛН
Parenting hacks and gadgets against mosquitoes 🦟👶
00:21
Let's GLOW!
Рет қаралды 9 МЛН
#01 | What is Snowpark in Snowflake
21:36
Data Engineering Simplified
Рет қаралды 24 М.
Orchestrating Data Pipelines With Snowpark dbt Python Models And Airflow
17:12
Snowflake Developers
Рет қаралды 10 М.
What's next for AI agentic workflows ft. Andrew Ng of AI Fund
13:27
Data Pond Bangladesh
Рет қаралды 1,6 М.
Using dbt And Snowflake To Develop And Deploy Analytics Code  | LAB
1:17:39
Snowflake Developers
Рет қаралды 28 М.
Snowpark for Python | Snowflake Tutorial
28:18
Aimpoint Digital
Рет қаралды 29 М.
I've been using Redis wrong this whole time...
20:53
Dreams of Code
Рет қаралды 350 М.
Snowpark Python Worksheets // Overview Of Python Worksheets for Data Manipulation & Transformations
23:04
Using Snowpark For Python To Build ML Workflows: A Hands-On Lab
1:22:27
Snowflake Developers
Рет қаралды 9 М.
Top 5 FREE Resources to 10X Your Data Engineering Skills
11:49
Jash Radia
Рет қаралды 54 М.
World’s Largest Jello Pool
01:00
Mark Rober
Рет қаралды 124 МЛН