Engenharia de Dados [Cast]

Simplify Data Engineering Projects in Your Lakehouse with Delta Lake Framework with Matthew Powers & Denny Lee, Developer Advocates at Databricks

May 23, 2023 Luan Moreno M. Maciel Season 3 Episode 12
Engenharia de Dados [Cast]
Simplify Data Engineering Projects in Your Lakehouse with Delta Lake Framework with Matthew Powers & Denny Lee, Developer Advocates at Databricks
Show Notes Chapter Markers

No episódio de hoje, Luan Moreno e Mateus Oliveira entrevistaram Denny Lee & Mathew Powers, atualmente Developer Advocates na Databricks.

Delta Lake é um produto open-source, que nos permite aplicar o famoso Data Lakehouse {Data Lake + Data Warehouse}, desenvolvido pela empresa dos criadores do Apache Spark. Delta Lake resolve o problema do Apache Spark, armazenamento, processamento de dados no Data Lake de forma otimizada.

Com Delta Lake, você tem os seguintes benefícios:

  • Formato de arquivo como se fosse uma tabela;
  • Time Travel;
  • ACID;
  • Batch e Streaming Unificados.


Falamos também nesse bate-papo sobre os seguintes temas:

  • Estado da arte dos dados;
  • Delta Lake.


Aprenda mais sobre Delta Lake, como utilizar uma tecnologia para Data LakeHouse, junto com o time da databricks que mais impulsiona a comunidade com conteúdos, releases e eventos para ajudar este produto open-source.

Denny Lee - Linkedin
Mathew Powers - Linkedin

https://delta.io/



Luan Moreno =
https://www.linkedin.com/in/luanmoreno/


Guests Introduction - Denny Lee & Mathew Powers
Which is the most complicated part of working with a data engineering project?
In your opinion, what is the main role of a Data Engineer nowadays?.
What are the main challenges in dealing with Data Lake?
What is Data Lakehouse, and what are the main benefits of adopting it in your analytics workloads?
The importance of Data Modelling is circling back again; new strategies lot of traction. What are your thoughts about it?
Delta 1.0 was a big breakthrough, and now we are at version 2.3.0 for those who are listing to us. What is Delta Lake in a nutshell?
What about the Python adoption?
What are the next steps for Delta Lake