Connect Apache Airflow to Snowflake Data Warehouse

  Рет қаралды 5,693

Knowledge Amplifier

Knowledge Amplifier

Күн бұрын

Пікірлер: 10
@KnowledgeAmplifier1
@KnowledgeAmplifier1 Жыл бұрын
Code: ---------- import logging import airflow from airflow import DAG from airflow.operators.python_operator import PythonOperator from airflow.contrib.operators.snowflake_operator import SnowflakeOperator from airflow.contrib.hooks.snowflake_hook import SnowflakeHook from datetime import datetime, timedelta logging.basicConfig(level=logging.INFO) logger = logging.getLogger(__name__) args = {"owner": "Airflow", "start_date": airflow.utils.dates.days_ago(2)} dag = DAG( dag_id="snowflake_automation_dag", default_args=args, schedule_interval=None ) snowflake_query = [ """ create table if not exists source_table( emp_no int,emp_name text,salary int, hra int ,Dept text); """, """INSERT INTO source_table VALUES (100, 'A' ,2000, 100,'HR'), (101, 'B' ,5000, 300,'HR'), (102, 'C' ,6000, 400,'Sales'), (103, 'D' ,500, 50,'Sales'), (104, 'E' ,15000, 3000,'Tech'), (105, 'F' ,150000, 20050,'Tech'), (105, 'F' ,150000, 20060,'Tech'); """ ] with dag: create_table = SnowflakeOperator( task_id="create_table", sql=snowflake_query[0] , snowflake_conn_id="snowflake_conn" ) insert_data = SnowflakeOperator( task_id="insert_snowflake_data", sql=snowflake_query[1] , snowflake_conn_id="snowflake_conn" ) create_table >> insert_data
@kaushikgorrepati6015
@kaushikgorrepati6015 8 ай бұрын
Thank you for all your videos, they're really helpful and inspiring us to learn more.🙂
@KnowledgeAmplifier1
@KnowledgeAmplifier1 8 ай бұрын
Glad to hear this Kaushik Gorrepati! Happy Learning
@prasannakusugal4333
@prasannakusugal4333 Жыл бұрын
Thanks for this video and many other videos too, they are really helpful :)
@KnowledgeAmplifier1
@KnowledgeAmplifier1 11 ай бұрын
Glad it was helpful!
@roshankumargupta46
@roshankumargupta46 6 ай бұрын
Thanks for your videos. Can you please make a different playlist for snowflake?
@KnowledgeAmplifier1
@KnowledgeAmplifier1 6 ай бұрын
You are welcome @roshankumargupta46! Here is the complete snowflake course (all KZfaq videos are available in order) -- doc.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510 Happy Learning
@manikumar-vr3kp
@manikumar-vr3kp 4 ай бұрын
hey hii pls help me im following the steps excatly as u to creation snowflake connection in airflow but it keeps on fails
@GsureTechnologies
@GsureTechnologies 7 ай бұрын
From where to get the KeyPair file ? I am new to all this. So pls help. Thanks.
Automate the Monitoring Process of Data Recency
13:18
Knowledge Amplifier
Рет қаралды 499
Мы сделали гигантские сухарики!  #большаяеда
00:44
Glow Stick Secret Pt.4 😱 #shorts
00:35
Mr DegrEE
Рет қаралды 12 МЛН
Getting Started With Snowflake CLI
23:38
Snowflake Developers
Рет қаралды 1,2 М.
How to submit Spark jobs to EMR cluster from Airflow
14:38
StartDataEngineering
Рет қаралды 12 М.
Building Pipelines with Dynamic Tables
21:23
Snowflake Developers
Рет қаралды 14 М.
What is ETL | What is Data Warehouse | OLTP vs OLAP
8:07
codebasics
Рет қаралды 415 М.
Installing Airflow on Windows 10 without Docker
7:20
Airflow for Beginners - Run Spotify ETL Job in 15 minutes!
16:38
Karolina Sowinska
Рет қаралды 141 М.
Вот и лето прошло…
1:00
Тёма Пименов
Рет қаралды 250 М.
Data recovery from MicroSD using PC3000 Flash & Spider Board 😎
1:01
Apple удивляет! Даже на Андроид такого нет!
0:42
ÉЖИ АКСЁНОВ
Рет қаралды 589 М.
САМЫЙ ОПАСНЫЙ iPHONE В МИРЕ 🤯 #iphone
1:01
ТЕХНОБЛОГ АЛИША
Рет қаралды 310 М.
Самый дорогой телефон 2000х
0:54
МАДНЕСС
Рет қаралды 2,6 МЛН