Your youtube playlist is an example how one should build a youtube playlist. Every video is sequenced in such a way that you need to go through the previous videos in order to understand the current video. Excellent work Maheer Sir. Thank you for all the hardwork ._/\_...

Hi bhaii, I can't explain how much it is useful for me. working on same as Data engineer onAzure Databricks . Each and every topic from this playlist I am using... so helpful. please continue.... Thanks a ton.

Thank you Anna for pyspark playlist Please add more pyspark related classes

Good One .. Keep Creating Such Videos .

Thanks Maheer! Videos are very useful

Thank you so so much man, this is very helpful.

Thanks Maheer Good vedio. Very helpful

Very helpful series. Thank you for your efforts. You earned another subscriber!😄 I had one quick question which I faced during an interview, please make a short video if you get the time: if there are multiple tabs present in an excel file (.xlsx) how to load the data present in any one of the tabs in that file to a dataframe?

Great videos, excellent, thank you

Very well explanation for beginners

Nice explanation bro 👍 👌 👏

Big Thank you Brother ! 🤗

God bless you brother 🙏🏿🙏🏿

Thanks for the content

Very well explanation....Can you please share the code of all your videos ? it will help us to do practice on databricks

2 csv files in 2 paths like data, data1, but columns and schema is different ,then how can read this ,this can be possible to use in list these files to read

Thanks Maheer :)

hi sir..one doubt ..which one i should learn at first? databricks or pyspark?

How to find the file is available in the path or not?

Is there anyone who is not able to create a folder inside 'Data' ? Any hack to do it?

I am ongoing with playlist as explanation is crystal clear. Thanks a lot for the list. Can you please help sharing the csv file as resource somewhere in description? Or else, pin your comment with csv files?

hai sir..this play list have full content of pyspark

can you please some data file which you are showing in this video .

it would be helpful if you make ur worked notebook, and respective dataset into one repo and share here!

hii,when I created data folder in filestore it 's not shown.so where I uploaded employee data plz guide me.when I created dta folder in file store its created but not shown

Completed

You can use also "recursiveFileLookup" to read all files (inside multiple folders) in one go.

Hi Maheer , Can we get those files

why don't you guys put the date field in your sample data..???

i am not able to create folder it is showing create bt it is not creating any folder

schema=StructType().add('ID', IntegerType())\ .add('NAME', StringType())\ .add('GENDER', StringType())\ .add('SALARY', StringType()) schema1=StructType([StructField('ID',IntegerType()), StructField('NAME',StringType()), StructField('GENDER',StringType()), StructField('SALARY',StringType())]) Which method is good schema vs schema1

Make same with different files

Nice Explanation. Hello everyone I am planning to move to data engineer role and looking for real time support who can guide me in right direction. Kindly let me know. Thanks

Hi , Could you please create a video to combine below 3 csv data files into one data frame dynamically File name: Class_01.csv StudentID Student Name Gender Subject B Subject C Subject D 1 Balbinder Male 91 56 65 2 Sushma Female 90 60 70 3 Simon Male 75 67 89 4 Banita Female 52 65 73 5 Anita Female 78 92 57 File name: Class_02.csv StudentID Student Name Gender Subject A Subject B Subject C Subject E 1 Richard Male 50 55 64 66 2 Sam Male 44 67 84 72 3 Rohan Male 67 54 75 96 4 Reshma Female 64 83 46 78 5 Kamal Male 78 89 91 90 File name: Class_03.csv StudentID Student Name Gender Subject A Subject D Subject E 1 Mohan Male 70 39 45 2 Sohan Male 56 73 80 3 shyam Male 60 50 55 4 Radha Female 75 80 72 5 Kirthi Female 60 50 55

Hi @wafastudies, when i pass header in load function its working df = spark.read.format('csv').option(key='header',value=True).load(path='emp1.csv') display(df) df.printSchema() df.show() output: DataFrame[EMPLOYEE_ID: string, FIRST_NAME: string, LAST_NAME: string, SALARY: string, DEPARTMENT_ID: string, LOCATION_ID: string, HIRE_DATE: string] root |-- EMPLOYEE_ID: string (nullable = true) |-- FIRST_NAME: string (nullable = true) |-- LAST_NAME: string (nullable = true) |-- SALARY: string (nullable = true) |-- DEPARTMENT_ID: string (nullable = true) |-- LOCATION_ID: string (nullable = true) |-- HIRE_DATE: string (nullable = true) +-----------+----------+---------+------+-------------+-----------+---------+ |EMPLOYEE_ID|FIRST_NAME|LAST_NAME|SALARY|DEPARTMENT_ID|LOCATION_ID|HIRE_DATE| +-----------+----------+---------+------+-------------+-----------+---------+ | 101| Donald| null| 2600| 10| 1701|21-Jun-07| | 102| Douglas| Grant| 2600| 20| 1702|13-Jan-08| | 103| Jennifer| Whalen| 4400| 30| 1703|17-Sep-03| +-----------+----------+---------+------+-------------+-----------+---------+