Simplify Data Engineering Projects in Your Lakehouse with Delta Lake Framework with Matthew Powers & Denny Lee, Developer Advocates at Databricks

No episódio de hoje, Luan Moreno e Mateus Oliveira entrevistaram Denny Lee & Mathew Powers, atualmente Developer Advocates na Databricks.

Delta Lake é um produto open-source, que nos permite aplicar o famoso Data Lakehouse {Data Lake + Data Warehouse}, desenvolvido pela empresa dos criadores do Apache Spark. Delta Lake resolve o problema do Apache Spark, armazenamento, processamento de dados no Data Lake de forma otimizada.

Com Delta Lake, você tem os seguintes benefícios:

• Formato de arquivo como se fosse uma tabela;
• Time Travel;
• ACID;
• Batch e Streaming Unificados.

Falamos também nesse bate-papo sobre os seguintes temas:

• Estado da arte dos dados;
• Delta Lake.

Aprenda mais sobre Delta Lake, como utilizar uma tecnologia para Data LakeHouse, junto com o time da databricks que mais impulsiona a comunidade com conteúdos, releases e eventos para ajudar este produto open-source.

Denny Lee - Linkedin
Mathew Powers - Linkedin

https://delta.io/

Luan Moreno =
https://www.linkedin.com/in/luanmoreno/

Chapters

• 1:39 Guests Introduction - Denny Lee & Mathew Powers
• 3:07 Which is the most complicated part of working with a data engineering project?
• 13:06 In your opinion, what is the main role of a Data Engineer nowadays?.
• 23:07 What are the main challenges in dealing with Data Lake?
• 33:52 What is Data Lakehouse, and what are the main benefits of adopting it in your analytics workloads?
• 42:30 The importance of Data Modelling is circling back again; new strategies lot of traction. What are your thoughts about it?
• 42:30 Delta 1.0 was a big breakthrough, and now we are at version 2.3.0 for those who are listing to us. What is Delta Lake in a nutshell?
• 54:12 Matthew Powers Can you talk a little bit about the https://github.com/mrpowers? What is the most requested feature in the community?
• 58:13 What about the Python adoption?
• 1:11:12 What are the next steps for Delta Lake