NOV 2022 - Modernize your Data wrangling approach for Datalakes by Dan King (@SQLSWAGG)
Abstract
The evolution of technology and the sheer amount of data increases the use of modern-day architectures. Carefully using the term modern-day architecture because the data lake architecture isn’t just for data warehousing and prescriptive analytics. It spans multiple modern-day use cases and architectural designs. Data wrangling and choosing your tools effectively continue to change as well. Think about Azure data factory v1 and how it evolved into ADF V2. Look at some of the additional data movement components and options. Databricks and notebooks modernized data movement and ACID transactions with delta lake. We have witnessed the next shift with Power Query and data wrangling within your ADF pipelines. In this session, we will look at some of these services and open mic and discuss the pros and cons of each. I will demo different data movement techniques and provide a forum to discuss experiences.
Speaker Bio
Dan King is currently a Solutions Architect Competency lead at 3Cloud Solutions. His daily activities are not only working with customers but with a primary focus on repeatable processes and standardizing go-to-market analytics architectural design to fit the needs of different industries. From a Microsoft mindset, you can relate it to common data models with common ETL/ELT processes.
His experience spans over a decade that started from networking and telecommunication to being a SQL database administrator with a final landing zone in data analytics. Today is his core focus and goal. With expertise in analyzing data, data movement, ETL design, data modeling, and visualization, in his own words, he has been honored to see how organizations take data from various sources through ingestion as it flows to storage and eventually lands in the presentation layer.
Dan holds multiple Microsoft certifications and continues to progress with the changes in technology. He loves to work out, and do outdoor sports, and finds tremendous value in training his peers in areas in which he holds knowledge in. Family is a core value, and he loves family. He views the SQL community as his extended family.
Slide deck and code
TBD
-
15:52
JoBlo Horror Originals
23 hours agoThe Watchers Review: Shyamalans Sure Do Love Their Twists
4.15K1 -
31:49
MichaelBisping
1 day agoBISPING & ANIK: Is Conor McGregor INJURED?! | Who Could Replace Him? | What's next for Poirier?
5.84K5 -
16:00
Space Ice
18 hours ago'Resident Evil: Retribution' Is So Bad It's Like Looking Into A Mirror - Worst Movie Ever
6.88K13 -
23:59
Degenerate Plays
14 hours agoThe Denny's Dictator Impresses Women - Call Of Duty Modern Warfare Remastered : Part 5
7K -
1:13:51
Blue Siesta ASMR
23 hours agoASMR Sims 4 Gameplay | New Roommates Move in Tiny House | already dating? 😱 Soft Spoken
8.56K2 -
46:53
Chrissy Clark
16 hours agoAn Anti-Boomer, Anti-Cringey Podcast w/ Allie Cluff
12.8K32 -
9:07
shaneyyricch
13 hours agoAddressing my beef with Harry Sisson
25K28 -
20:40
The Nima Yamini Show
13 hours agoAlpha King Nima Exposes Anti-Masculinity Agenda
30.7K44 -
7:02
TENET Media
16 hours agoNYC Hotels TAKEN OVER By Migrants | Lauren Southern
44.4K93 -
47:35
Alexis Wilkins
17 hours agoBetween the Headlines with Alexis Wilkins: NYPD Officers, Mexicos Leader, Fauci Hearing, DDay Salute
50.1K29