PySpark Tutorial for Beginners
48:12
pandas filter by conditions
0:57
Жыл бұрын
pandas select columns
0:27
Жыл бұрын
pandas select rows
0:31
Жыл бұрын
SQL Tutorial | MySQL in 1 Hour!
1:01:18
pandas rename column
0:30
Жыл бұрын
Пікірлер
@fayezchamp9365
@fayezchamp9365 Күн бұрын
Noice 👌
@LiangBowen-x8c
@LiangBowen-x8c Күн бұрын
Best Airflow tutorial ever👍
@coder2j
@coder2j Күн бұрын
Glad you think so!
@pruthvirajbhosale6109
@pruthvirajbhosale6109 4 күн бұрын
please chenage the voice setting you are sounding like a robot
@engil2534
@engil2534 5 күн бұрын
Almost downvoted as soon as I heard the AI voice and assumed a low-effort BS tutorial, but this is actually very helpful.
@coder2j
@coder2j 4 күн бұрын
Do you feel the AI voice is a distraction? Would you be more engaged with my own voice over?
@nitishroy6898
@nitishroy6898 5 күн бұрын
Hey, are you using any AI for voice modulation or content creation.? nice content though
@coder2j
@coder2j 4 күн бұрын
I am using the text to speech for the audio. Do you enjoy it or find it annoying?
@nitishroy6898
@nitishroy6898 4 күн бұрын
@@coder2j content is very nice,tts sounds robotic
@coder2j
@coder2j 3 күн бұрын
Thanks for your honest feedback. Would you enjoy it more if I use my own voice?
@nitishroy6898
@nitishroy6898 3 күн бұрын
Absolutely! 💯
@nicpolyakoff
@nicpolyakoff 5 күн бұрын
Thank you for the video! Exactly what I was looking for
@simonr-i8i
@simonr-i8i 8 күн бұрын
Nice content thanks a lot
@AakashYadav-v1b
@AakashYadav-v1b 9 күн бұрын
This is best video for Airflow.
@Mani-w4c
@Mani-w4c 11 күн бұрын
installation is succesfull, but when i launch jupyterlab by giving 'jupyter-lab' cmd is throwing error as it is not recognized as an internal or external command. any help about it?
@coder2j
@coder2j 11 күн бұрын
It seems like you don't have Jupiter lab packages installed in your python virtual environment.
@Mani-w4c
@Mani-w4c 11 күн бұрын
@@coder2j thank you, can you tell me how to set it up right
@berke237
@berke237 13 күн бұрын
how did u stop in 8.20
@mohamedbessar672
@mohamedbessar672 13 күн бұрын
You are the best
@vohoang6693
@vohoang6693 20 күн бұрын
Thank you for sharing this free. I have watch and practise your airflow course . It's very easy for everyone to understand
@coder2j
@coder2j 17 күн бұрын
Thank you! 😊
@vohoang6693
@vohoang6693 23 күн бұрын
Thank you for sharing free. This is the best airflow course I have leant. Everything is expressed fluently and easy to understand and practise quickly. This courage still working at airflow 2.9.2 - latest version at the time of this command except aws s3 connection (I must watch another video of you to update). I have a question about docker-compose file. What's different type of airflow executors at 13:30 and the purpose of the services you deleted in docker-compose file after that. That good be greet if you have some video to explain them.
@KarthikReddyJannapureddy
@KarthikReddyJannapureddy 29 күн бұрын
Hello @coder2j , I am getting error while running "airflow db init" command. Error - airflow.exceptions.AirflowConfigException: Cannot use relative path: `sqlite:///./airflow.db` to connect to sqlite. Please use absolute path such as `sqlite:////tmp/airflow.db`.
@baigalaa1769
@baigalaa1769 Ай бұрын
You are the GOAT!!! Thank you so much!!!❤
@nataliaxarenicuevas2486
@nataliaxarenicuevas2486 Ай бұрын
Incredibly useful
@russel2506
@russel2506 Ай бұрын
as newbie i found using docker its more easy to understand than all the other ways
@anmoldhuwalia167
@anmoldhuwalia167 Ай бұрын
finally!!
@aem870
@aem870 Ай бұрын
Can we create a file containing database content on a website? This video saves it on a local folder, I'd like to do it on a website's folder.
@askplace
@askplace Ай бұрын
If you don't want to build an image every time you can use the following commands docker exec -it <container_name> bash pip install <package> exit I think it is the airflow-scheduler container you should execute
@gera4798
@gera4798 Ай бұрын
Thank you)
@namansinghal3685
@namansinghal3685 Ай бұрын
Haven't seen a worse video teaching something technical. Video explanation not beginner friendly
@0523036
@0523036 Ай бұрын
Hi, This tutorial is good .. I would love to have more indepth knowledge in Dagster as i am thinking this would be the feature of Orchestration as well as ingestion tool.
@cyhunakar
@cyhunakar Ай бұрын
this videos are awesome, and probably the most helpful information about usage of airflow. very brief and well prepared. Thanks Coder2j!
@coder2j
@coder2j Ай бұрын
Thanks for your nice words! 🤗
@haowen2799
@haowen2799 Ай бұрын
Great video! Thanks for sharing! Just a question about using Jupyternote book. Are there any benefits of using a Jupyternotebook in Jupyterlab over using that in VS Code?
@coder2j
@coder2j Ай бұрын
I prefer the Jupiter lab layout. 😊
@haowen-qg3np
@haowen-qg3np Ай бұрын
HELP NEEDED. Anyone using windows had issue running 'airflow db init' on windows? I got error msg saying something like 'cannot use relative path, please use absolute path'. I tried 'set AIRFLOW_HOME=~/airflow', but no luck.
@coder2j
@coder2j Ай бұрын
You need to use an absolute path. Open file explorer and find your airflow project, copy the full directory then paste it as the AIRFLOW_HOME
@Gowdaashray
@Gowdaashray Ай бұрын
Great Tutorial.
@writerconnect1596
@writerconnect1596 Ай бұрын
BEST
@talhaanwar2911
@talhaanwar2911 Ай бұрын
thanks a lot, if i have pandas dataframe, how can i pass it from one task to another
@coder2j
@coder2j Ай бұрын
You can save it locally as a text file then share the file name or you can use cloud storage like S3.
@HungCao-fk2hj
@HungCao-fk2hj Ай бұрын
why my code take so much time to run on vscode example scripts spark = SparkSession.builder\ .appName("NYSE Count")\ .master("local")\ .getOrCreate()
@kor5312
@kor5312 Ай бұрын
Hello, I have a question. How can I fix this error? ---Cannot relative path--- --After airflow db init-- move In the 07:40
@wah866sky7
@wah866sky7 2 ай бұрын
Very great tutorial, I hope you can launch the tutorials of Spark ML and Streaming asap. Thanks
@coder2j
@coder2j 2 ай бұрын
Thanks. Stay tuned!
@anketambekar3902
@anketambekar3902 2 ай бұрын
How to configure AWS Elastic cache in airflow
@jessicafb5398
@jessicafb5398 2 ай бұрын
This was super helpful, thanks!! It answered all of my questions & provided me with everything I needed to know!
@randomhandle307
@randomhandle307 2 ай бұрын
This tutorial was pretty solid. Thanks.
@hamzasabri5610
@hamzasabri5610 2 ай бұрын
exporting the airflow on my end I had to export it this way for it to work globally export AIRFLOW_HOME=~/airflow
@abdiyerzhan
@abdiyerzhan 2 ай бұрын
can you explain bout trigger?
@TanushreeNagar-tt1pq
@TanushreeNagar-tt1pq 2 ай бұрын
unable to install docker
@piggyshuks
@piggyshuks 2 ай бұрын
this was super helpful! thank you for making this video!
@coder2j
@coder2j 2 ай бұрын
You are welcome 🤗
@joshuadanielmendoza6124
@joshuadanielmendoza6124 2 ай бұрын
why is that airflow.cfg is not in my current directory?
@coder2j
@coder2j 2 ай бұрын
Because you are not setting the AIRFLOW_HOME environment variable right. You need to set it to your current absolute directory.
@randomerickintheinternet
@randomerickintheinternet 2 ай бұрын
I dont normally comment in KZfaq videos but your dagster videos is awesome! learned alot as its the direct and concise.
@coder2j
@coder2j 2 ай бұрын
Thanks you.
@SrinivasaParankusham
@SrinivasaParankusham 2 ай бұрын
Please also cover the dagster sensors session
@coder2j
@coder2j 2 ай бұрын
Thanks for the feedback, will consider it!
@user-nh3uw8lr3e
@user-nh3uw8lr3e 2 ай бұрын
Please make a video explaining the components in the docker-compose.yaml like the x-ariflow common
@user-nh3uw8lr3e
@user-nh3uw8lr3e 2 ай бұрын
can someone help im getting local file error in windows *** Reading local file: /opt/airflow/logs/example_branch_dop_operator_v3/condition/2024-05-02T00:00:00+00:00/1.log
@airank3861
@airank3861 2 ай бұрын
9:51
@dimitrigbo823
@dimitrigbo823 2 ай бұрын
Hi coder2j... Ive been battling with this error for the past 12 hours. Would really appreciate your help and that of anyone else. I'm running vscode on windows and have tried to set AIRFLOW_HOME=. When I try to initiate the airflow database, I get the error below. I have scoured the internet for a solution and have come stuck. Plz help1 AirflowConfigException( airflow.exceptions.AirflowConfigException: Cannot use relative path: `sqlite:///C:\Users\Dimitri/airflow/airflow.db` to connect to sqlite. Please use absolute path such as `sqlite:////tmp/airflow.db`
@peteru2872
@peteru2872 2 ай бұрын
i had to comment nothing worked for me so far, i mainly dont have issue with the example dags. it just could not read the dag folder no matter what i tried i am using Ubuntu wsl 2, i use 'code .' to connect with vscode doesnt matter if i use C:\Users\user\Documents\\airflow\dags or /mnt/c/Users/user/Documents//airflow/dags as the path on airflow.cfg still my project does not load on the airflow UI. load_examples = False dont stop the examples from loading too. used 3 days trying to set airflow but now my project files are currently on example_dags folder and its working fine i just deleted the .py files i dont need there
@coder2j
@coder2j 2 ай бұрын
Please check if you have set the AIRFLOW_HOME environment variable right. By default it is in the path ~/airflow. If you want to set it to your project directory, use the absolute path.
@SwamyNaiduLenka-tr3tg
@SwamyNaiduLenka-tr3tg 2 ай бұрын
getting below error:: airflow.exceptions.AirflowConfigException: Cannot use relative path: `sqlite:///C:\Users\ADMIN/airflow/airflow.db` to connect to sqlite. Please use absolute path such as `sqlite:////tmp/airflow.db`. --- I am unable to get the airflow.cfg file(basically it is not getting created)
@ajaxjson3320
@ajaxjson3320 2 ай бұрын
Thank you brother, this is the best Spark introduction.
@coder2j
@coder2j 2 ай бұрын
Thank you!
@cea2016
@cea2016 2 ай бұрын
thanks! this tutorial is super helpful and I've learned a lot!
@coder2j
@coder2j 2 ай бұрын
Thank you!