No video

Connect Apache Airflow to Snowflake Data Warehouse

  Рет қаралды 5,693

Knowledge Amplifier

Knowledge Amplifier

Күн бұрын

Пікірлер: 10
@KnowledgeAmplifier1
@KnowledgeAmplifier1 Жыл бұрын
Code: ---------- import logging import airflow from airflow import DAG from airflow.operators.python_operator import PythonOperator from airflow.contrib.operators.snowflake_operator import SnowflakeOperator from airflow.contrib.hooks.snowflake_hook import SnowflakeHook from datetime import datetime, timedelta logging.basicConfig(level=logging.INFO) logger = logging.getLogger(__name__) args = {"owner": "Airflow", "start_date": airflow.utils.dates.days_ago(2)} dag = DAG( dag_id="snowflake_automation_dag", default_args=args, schedule_interval=None ) snowflake_query = [ """ create table if not exists source_table( emp_no int,emp_name text,salary int, hra int ,Dept text); """, """INSERT INTO source_table VALUES (100, 'A' ,2000, 100,'HR'), (101, 'B' ,5000, 300,'HR'), (102, 'C' ,6000, 400,'Sales'), (103, 'D' ,500, 50,'Sales'), (104, 'E' ,15000, 3000,'Tech'), (105, 'F' ,150000, 20050,'Tech'), (105, 'F' ,150000, 20060,'Tech'); """ ] with dag: create_table = SnowflakeOperator( task_id="create_table", sql=snowflake_query[0] , snowflake_conn_id="snowflake_conn" ) insert_data = SnowflakeOperator( task_id="insert_snowflake_data", sql=snowflake_query[1] , snowflake_conn_id="snowflake_conn" ) create_table >> insert_data
@kaushikgorrepati6015
@kaushikgorrepati6015 8 ай бұрын
Thank you for all your videos, they're really helpful and inspiring us to learn more.🙂
@KnowledgeAmplifier1
@KnowledgeAmplifier1 8 ай бұрын
Glad to hear this Kaushik Gorrepati! Happy Learning
@prasannakusugal4333
@prasannakusugal4333 Жыл бұрын
Thanks for this video and many other videos too, they are really helpful :)
@KnowledgeAmplifier1
@KnowledgeAmplifier1 11 ай бұрын
Glad it was helpful!
@roshankumargupta46
@roshankumargupta46 6 ай бұрын
Thanks for your videos. Can you please make a different playlist for snowflake?
@KnowledgeAmplifier1
@KnowledgeAmplifier1 6 ай бұрын
You are welcome @roshankumargupta46! Here is the complete snowflake course (all KZfaq videos are available in order) -- doc.clickup.com/37466271/d/h/13qc4z-104/d4346819bd8d510 Happy Learning
@manikumar-vr3kp
@manikumar-vr3kp 4 ай бұрын
hey hii pls help me im following the steps excatly as u to creation snowflake connection in airflow but it keeps on fails
@GsureTechnologies
@GsureTechnologies 7 ай бұрын
From where to get the KeyPair file ? I am new to all this. So pls help. Thanks.
Automate the Monitoring Process of Data Recency
13:18
Knowledge Amplifier
Рет қаралды 499
🩷🩵VS👿
00:38
ISSEI / いっせい
Рет қаралды 28 МЛН
Whoa
01:00
Justin Flom
Рет қаралды 50 МЛН
Kind Waiter's Gesture to Homeless Boy #shorts
00:32
I migliori trucchetti di Fabiosa
Рет қаралды 16 МЛН
拉了好大一坨#斗罗大陆#唐三小舞#小丑
00:11
超凡蜘蛛
Рет қаралды 14 МЛН
Getting Started With Snowflake CLI
23:38
Snowflake Developers
Рет қаралды 1,2 М.
How to submit Spark jobs to EMR cluster from Airflow
14:38
StartDataEngineering
Рет қаралды 12 М.
Orchestrating Data Pipelines With Snowpark dbt Python Models And Airflow
17:12
Snowflake Developers
Рет қаралды 10 М.
Airflow for Beginners - Run Spotify ETL Job in 15 minutes!
16:38
Karolina Sowinska
Рет қаралды 141 М.
Airflow DAG: Coding your first DAG for Beginners
20:31
Data with Marc
Рет қаралды 222 М.
Airflow with DBT tutorial - The best way!
17:54
Data with Marc
Рет қаралды 43 М.
🩷🩵VS👿
00:38
ISSEI / いっせい
Рет қаралды 28 МЛН