Data Warehousing With BigQuery: Best Practices (Cloud Next '19)

Рет қаралды 145,009

Күн бұрын

Take an in-depth look at modern data warehousing using BigQuery and how to operate your data warehouse in the cloud. During this session, we'll give lessons learned and best practices from prior implementations to give you the playbook for implementing your own modern data warehouse.
"BigQuery ML → bit.ly/2Khkqoq
BigQuery for data warehouse practioners → bit.ly/2TWh6P9"
Watch more:
Next '19 Data Analytics Sessions here → bit.ly/Next19DataAnalytics
Next ‘19 All Sessions playlist → bit.ly/Next19AllSessions
Subscribe to the GCP Channel → bit.ly/GCloudPlatform
Speaker(s): Ryan McDowell, Alban Perillat-Merceroz
Session ID: DA307
product:BigQuery; fullname:Ryan McDowell; event: Google Cloud Next 2019; re_ty: Publish; product: Cloud - Data Analytics - BigQuery; fullname: Ryan McDowell;

Пікірлер: 21

@kunfang2457 3 жыл бұрын

the best 'best practice' sharing session i ve ever had

@anandakumarsanthinathan4740 2 жыл бұрын

Beautifully explained. Many thanks. Now, why wouldn't anybody want to move to the sunny side of France and land a job at Teads !! Both of the presenters did an excellent job.

@3rdinnings326 Жыл бұрын

Excellent session!

@TheMitali_ 4 жыл бұрын

How to perform sum of columns value in big query. And no of columns are notfixed , at runtime we need to decide no of columns need to consider for sum. Depends on user inputs

@jennwng 9 ай бұрын

At 13:36, why do we need to load into GCS first for batch loads? Can't we use Dafaflow/BigQuery BatchLoad / Data Fusion directly from GoldenGate?

@ThisIsAli_Off Жыл бұрын

Very useful!

@krishnabg1350 4 жыл бұрын

Can you share slides?

@LuckyHongTJ 8 ай бұрын

At 13:35, what is the benefit of loading data from Oracle into GCS first and then BigQuery (i.e. won't directly loading into BigQuery from Oracle be faster)? I know GCS can serve as a staging area and we get another copy of the data for fault tolerance, but is there any other benefit? Thanks :)

@jennwng 9 ай бұрын

At 11:58 why DataFlow / Data Fusion can speed up pipelines? Like, the essence is either to do the transformation in BigQuery or in DataFusion. Even if the query is complex, won't it still be faster to do the transformation directly in BigQuery, rather than connecting to DataFusion and transforming data there?

@vishnureddys4801 4 ай бұрын

It is because they can bill you in both Services, for using BigQuery and also dataflow.

@amieewright4417 3 жыл бұрын

H thanks u so much love you amiee Wright and Bella daws Xxxxx

@Vinch157 5 жыл бұрын

Hello, will you also share the slides? Thank you

@vishnureddys4801 4 ай бұрын

Has he yet?

@mzamanmintu3694 Жыл бұрын

Congratulations

@Thiago280690 Жыл бұрын

6:30 do nada aparece uma nota de dez conto hahaha adorei

@LouisChiaki 3 жыл бұрын

How is this different from Spark?

@jean4j_ 2 жыл бұрын

Well it's auto-managed and much simpler to work with. That's just SQL. You don't need to fine-tune the configuration like you would for a Spark job I think.

@JOPINC 3 жыл бұрын

Dedicado a JMB!!!! :)

@sunilpipara 2 жыл бұрын

Kafka to S3 and S3 to Cassandra via Spark is wrong approach.

@anandakumarsanthinathan4740 2 жыл бұрын

Yes, @Sunil Jain. I feel the same too. The source application would directly write it out to S3 for batch processing. Kafka would be required only for the streaming jobs.