data-engineering-zoomcamp

DataTalksClub/data-engineering-zoomcamp

Fork 0

mirror of https://github.com/DataTalksClub/data-engineering-zoomcamp.git synced 2026-03-27 00:28:13 +00:00

Commit Graph

Select branches

Hide Pull Requests

copilot/fix-5dffed7e-30f5-4da5-a10c-1027bb303614

copilot/remove-google-calendar-references

main

#1

#10

#100

#101

#102

#103

#104

#105

#106

#107

#108

#109

#11

#110

#111

#12

#13

#14

#15

#16

#17

#18

#19

#191

#192

#195

#196

#197

#198

#199

#2

#20

#201

#204

#205

#206

#209

#21

#211

#212

#213

#214

#215

#217

#218

#22

#220

#221

#225

#226

#227

#228

#229

#23

#230

#231

#232

#233

#234

#235

#236

#238

#239

#24

#240

#241

#242

#243

#244

#245

#246

#247

#248

#249

#25

#250

#251

#252

#253

#254

#255

#257

#258

#259

#26

#260

#261

#263

#265

#266

#267

#27

#275

#276

#277

#278

#279

#28

#280

#281

#282

#283

#284

#285

#286

#287

#288

#289

#29

#290

#291

#292

#294

#295

#296

#297

#298

#299

#3

#30

#300

#301

#302

#303

#304

#305

#306

#308

#309

#31

#310

#311

#312

#313

#314

#315

#316

#317

#318

#319

#32

#320

#321

#322

#323

#324

#325

#326

#327

#328

#329

#33

#330

#331

#332

#333

#334

#335

#336

#337

#338

#339

#34

#340

#341

#342

#343

#344

#345

#346

#347

#348

#349

#35

#350

#351

#352

#353

#354

#355

#356

#357

#358

#359

#36

#360

#361

#362

#363

#364

#365

#366

#367

#368

#369

#37

#371

#372

#373

#374

#375

#376

#377

#378

#379

#38

#380

#381

#382

#383

#387

#388

#389

#39

#390

#391

#392

#393

#394

#395

#396

#399

#4

#40

#400

#401

#402

#403

#404

#405

#407

#409

#41

#410

#411

#412

#413

#414

#415

#416

#417

#418

#419

#42

#420

#421

#423

#424

#425

#429

#43

#430

#432

#433

#434

#435

#436

#437

#438

#439

#44

#441

#442

#443

#444

#445

#446

#447

#448

#449

#45

#450

#451

#452

#453

#454

#455

#456

#457

#458

#459

#46

#460

#461

#463

#464

#465

#466

#467

#468

#469

#47

#470

#471

#472

#473

#474

#475

#476

#477

#478

#479

#48

#480

#481

#482

#484

#484

#485

#486

#487

#488

#489

#49

#490

#491

#492

#493

#494

#495

#496

#497

#498

#499

#5

#50

#500

#501

#502

#503

#504

#505

#506

#507

#508

#509

#51

#510

#511

#512

#513

#514

#515

#516

#517

#518

#519

#52

#520

#521

#522

#523

#524

#525

#526

#527

#528

#529

#53

#530

#531

#532

#533

#534

#535

#536

#537

#538

#539

#54

#540

#541

#542

#543

#544

#545

#546

#547

#548

#549

#55

#550

#551

#552

#553

#554

#556

#557

#559

#56

#560

#561

#562

#563

#564

#565

#566

#567

#568

#569

#571

#572

#573

#574

#576

#578

#579

#58

#580

#581

#582

#583

#584

#585

#586

#587

#588

#589

#59

#590

#592

#593

#594

#595

#596

#597

#598

#599

#6

#60

#600

#601

#602

#603

#604

#605

#606

#607

#607

#608

#609

#609

#610

#611

#612

#614

#615

#616

#617

#618

#619

#62

#620

#621

#622

#623

#624

#625

#626

#627

#628

#629

#63

#630

#631

#632

#633

#634

#636

#637

#638

#639

#64

#641

#641

#642

#643

#644

#645

#646

#647

#648

#649

#65

#650

#651

#652

#653

#654

#655

#656

#657

#658

#659

#66

#660

#661

#662

#663

#664

#665

#666

#667

#668

#669

#67

#670

#671

#672

#673

#674

#675

#676

#677

#678

#679

#68

#680

#681

#682

#683

#684

#685

#686

#687

#688

#689

#69

#690

#691

#692

#693

#694

#695

#696

#697

#698

#699

#70

#700

#701

#702

#703

#704

#705

#706

#707

#708

#709

#71

#710

#711

#712

#713

#714

#715

#716

#717

#718

#719

#72

#721

#722

#723

#723

#724

#725

#727

#728

#729

#73

#730

#731

#732

#733

#734

#735

#736

#738

#739

#74

#740

#741

#742

#743

#744

#745

#746

#747

#748

#749

#75

#750

#751

#752

#754

#755

#756

#757

#758

#759

#76

#760

#761

#762

#764

#765

#766

#767

#768

#769

#77

#770

#771

#772

#773

#774

#775

#776

#777

#778

#778

#779

#78

#780

#781

#782

#783

#784

#785

#786

#787

#789

#79

#790

#790

#791

#792

#793

#794

#795

#796

#797

#798

#799

#8

#80

#800

#800

#802

#803

#804

#805

#806

#807

#808

#809

#81

#810

#811

#812

#814

#815

#816

#816

#817

#818

#819

#82

#820

#821

#822

#823

#824

#825

#826

#827

#828

#829

#83

#830

#831

#832

#833

#834

#834

#835

#835

#836

#836

#84

#85

#86

#88

#89

#9

#90

#91

#92

#93

#94

#95

#96

#97

#98

#99

ef44b885b9 Document Bruin MCP integration steps for VS Code (#832) main motho17 2026-03-19 14:23:14 +00:00
f556028369 Add files via upload (#833) Khang Tran 2026-03-19 21:22:41 +07:00
16fdabf012 Update streaming homework with verified answers and setup hints Alexey Grigorev 2026-03-12 05:02:49 +01:00
1e086f8449 Add streaming homework for 2026 cohort Alexey Grigorev 2026-03-09 16:35:32 +01:00
219f054559 Update README.md Alexey Grigorev 2026-03-05 21:40:09 +01:00
ca33cdbef1 Fix PySpark installation link in homework.md Alexey Grigorev 2026-03-05 21:35:40 +01:00
bf77cc9774 Update README.md Alexey Grigorev 2026-03-04 13:50:32 +01:00
2beeaeff85 mod 6 - update taxi_lookup link (forbiden access in aws) (#831) Michael Garcia-Rollet 2026-03-04 13:49:17 +01:00
6b1a891e0c Add live workshop code, link from README Alexey Grigorev 2026-03-04 13:45:03 +01:00
a1a93a3672 Update video link in workshop README Alexey Grigorev 2026-03-04 11:59:11 +01:00
2d11fa3dd2 Rewrite watermark explanation, add realtime producer, rename jobs Alexey Grigorev 2026-03-04 11:50:41 +01:00
2eb5c31b0c Credit Irem for the Python Kafka examples Alexey Grigorev 2026-03-04 10:28:16 +01:00
e2c679dd1b Fix: 2026 workshop is by Alexey, not Zach Alexey Grigorev 2026-03-04 10:27:15 +01:00
72f4303fe4 Fix pyflink attribution: Irem's workshop predates Zach's 2025 stream Alexey Grigorev 2026-03-04 10:26:59 +01:00
41229b0e6e Add mkdir before docker compose to prevent root-owned src/ Alexey Grigorev 2026-03-04 10:26:27 +01:00
84ef79bebc Fix pyflink attribution (Irem vs Zach), mark extras as optional Alexey Grigorev 2026-03-04 09:58:36 +01:00
25acee09c0 Delete dataset.md Alexey Grigorev 2026-03-04 09:53:51 +01:00
69ac9074ca Add .duckdb to .gitignore Alexey Grigorev 2026-03-04 09:53:20 +01:00
1407c6e1bf Delete taxi_rides_ny.duckdb Alexey Grigorev 2026-03-04 09:53:01 +01:00
c4c6340908 Reorganize 07-streaming: theory/, extras/, concise main README Alexey Grigorev 2026-03-04 09:48:12 +01:00
bf6923b7e1 Expand watermark explanation: why the name, how the subtraction works Alexey Grigorev 2026-03-04 09:34:13 +01:00
f6f3f9570d Fix workshop: slots bug, missing models.py, incomplete aggregation job Alexey Grigorev 2026-03-03 18:58:04 +01:00
1932b714d4 Rewrite workshop: gradual structure, NYC taxi data, shared models, Q&A Alexey Grigorev 2026-03-03 10:32:41 +01:00
eb9a931072 Switch to kafka-python, add clean Flink config, fix JDK warnings Alexey Grigorev 2026-03-03 09:15:19 +01:00
71fab1b2d1 Clean up workshop: update deps, simplify Dockerfile and docker-compose Alexey Grigorev 2026-03-02 23:36:38 +01:00
0db3627ae1 Write full workshop README Alexey Grigorev 2026-03-02 18:43:19 +01:00
a6dc20bfe3 Bump requires-python to >=3.12 Alexey Grigorev 2026-03-02 18:41:03 +01:00
2f32d55ac7 Add PyFlink workshop with Flink 2.2.0, Java 17, and uv Alexey Grigorev 2026-03-02 18:38:45 +01:00
b6c12937fe Add submission form link to homework.md (#826) ajay katte 2026-03-03 02:47:34 +05:30
efe6854a12 Update video link in README.md Alexey Grigorev 2026-03-02 12:58:00 +01:00
6c5a1f60b1 added 2026 notes links (#828) Khanh Nguyen 2026-03-01 12:15:52 +01:00
00eb01bc9a Add 2026 note in week 1 (#829) Khanh Nguyen 2026-03-01 12:15:28 +01:00
586fa25e71 Update macOS setup guide for Spark 4.1.1 (#827) Andrew X. GAO 2026-02-26 20:49:18 +13:00
c27e49aadd Enhance variable descriptions in variables.tf (#825) Yashraj Muthyapwar 2026-02-25 05:52:49 -06:00
feb83c45db Change registration link to workshop video Alexey Grigorev 2026-02-23 13:01:30 +01:00
aaee26d162 fixing links Alexey Grigorev 2026-02-23 09:19:12 +01:00
b83536f9b9 Update Spark setup guides for PySpark 4.x with uv Alexey Grigorev 2026-02-20 21:43:39 +01:00
a6d58caede batch homework Alexey Grigorev 2026-02-20 21:04:07 +01:00
58feb23e1a Add Bruin Core Concepts videos and notes to Module 5 Alexey Grigorev 2026-02-20 17:51:27 +01:00
9a2c3fbf5b Merge branch 'main' of github.com:DataTalksClub/data-engineering-zoomcamp Alexey Grigorev 2026-02-20 17:28:37 +01:00
9c0d8b3b6e Update image format in README.md Valeriia Kuka 2026-02-20 11:50:41 +03:00
f55875d2a0 Add files via upload Valeriia Kuka 2026-02-20 11:50:18 +03:00
896f266c35 Delete images/architecture/arch_v5_workshops.jpg.png Valeriia Kuka 2026-02-20 11:49:53 +03:00
129fff1b96 Change architecture image to version 5 Valeriia Kuka 2026-02-20 11:48:23 +03:00
6e99e2840d Add files via upload Valeriia Kuka 2026-02-20 11:46:20 +03:00
004f5fd1b3 Merge pull request #824 from connie-carey/fusion-compatible Juan Manuel Perafan 2026-02-20 08:50:50 +01:00
9eb388a812 update files for fusion compatibility Connie Carey 2026-02-19 21:36:35 -05:00
a09b2000b5 Fix Question 2 wording per Bruin maintainer feedback Alexey Grigorev 2026-02-19 14:41:47 +01:00
0fa92d9542 Fix Question 5 quality check syntax to match Bruin docs Alexey Grigorev 2026-02-19 12:08:03 +01:00
bb27eb6e46 Merge pull request #821 from sharadgupta27/patch-1 Juan Manuel Perafan 2026-02-18 21:30:46 +01:00
1d3d81caea Merge pull request #823 from tahanawfal/patch-4 Juan Manuel Perafan 2026-02-18 21:30:11 +01:00
f43e2a0879 Add VS Code and Claude Code MCP setup instructions Alexey Grigorev 2026-02-18 15:09:39 +01:00
adf0f850ad Merge PR #819: Workshop/dlt 2026 Alexey Grigorev 2026-02-18 14:47:10 +01:00
d8ff6fa28f Move embedded image to images folder anair123 2026-02-16 16:03:10 +01:00
9978a15386 update normalize explanation anair123 2026-02-16 10:29:31 +01:00
8ef9a72571 modify homework.md anair123 2026-02-12 18:04:43 +01:00
00d4c51934 change visual anair123 2026-02-12 15:51:34 +01:00
5473ece7e4 add description to homework.md anair123 2026-02-12 14:00:47 +01:00
cb64545bd6 add mermaid diagram to notebook anair123 2026-02-12 13:27:58 +01:00
36abba7d5a add mermaid diagram to notebook anair123 2026-02-12 13:18:45 +01:00
271cab50fa add mermaid diagram to notebook anair123 2026-02-12 13:10:34 +01:00
5ac535dd53 tweak notebook anair123 2026-02-12 12:45:18 +01:00
91045be494 dlt_homework anair123 2026-02-12 12:06:41 +01:00
c8f0bc5be0 modify pyproject anair123 2026-02-12 11:04:20 +01:00
2ddb6412e3 add image to notebook anair123 2026-02-11 17:32:30 +01:00
c5c86927d8 add dlt Jupyter Notebook anair123 2026-02-11 17:03:56 +01:00
7211e4ed39 Add dlt workshop files anair123 2026-02-11 16:58:08 +01:00
0ad649880c Add Bruin MCP and Cloud videos and notes to Module 5 Alexey Grigorev 2026-02-18 14:38:48 +01:00
214024d2b0 Revise data loading instructions in homework.md Taha Nawfal 2026-02-18 13:41:13 +03:00
026065f206 Update README.md (#808) CaThY-988 2026-02-18 06:45:23 +00:00
69a856b19b feat(module 3): add type enforcement to web_to_gcs.py + .env var + add progress bar, and no reupload try if present (#789) Michael Garcia-Rollet 2026-02-18 07:44:56 +01:00
3bef4fb5e3 Add 2026 notes link for batch processing to README (#820) ajay katte 2026-02-18 12:11:36 +05:30
d431bcd210 Add link to 2026 Notes by Sharad K. Gupta Sharad Kumar Gupta 2026-02-18 00:34:35 +01:00
1efd9e90a7 Add workshop materials and homework links to dlt workshop page Alexey Grigorev 2026-02-17 14:09:20 +01:00
06cce1fd3f Add multiple-choice options, workshop links, and submission form to dlt homework Alexey Grigorev 2026-02-17 14:04:50 +01:00
0bf1e07d6f Add Bruin tutorial videos and notes to Module 5: Data Platforms Alexey Grigorev 2026-02-13 19:14:18 +01:00
3b7fed8a92 Workshop/dlt 2026 (#817) Aashish Nair 2026-02-13 18:33:13 +01:00
f45493ee59 docs: fix documentation to reflect DuckDB as the data warehouse option for dbt Core (#818) Andrew X. GAO 2026-02-14 06:32:38 +13:00
ce0d7131a8 Merge pull request #814 from DataTalksClub/fix/module4-cloud-setup-data-source Juan Manuel Perafan 2026-02-11 13:42:47 +01:00
934bcfdf6c Point cloud setup to DataTalksClub data repo instead of NYC TLC site jmperafan 2026-02-11 13:40:02 +01:00
54eb9738b6 Merge pull request #812 from DataTalksClub/fix/module4-cloud-setup-note Juan Manuel Perafan 2026-02-10 12:10:30 +01:00
09b2f88a9e Clarify that Module 4 cloud setup requires different data than Module 3 (GCP setup is the real prerequisite) jmperafan 2026-02-10 10:51:40 +01:00
1f9a4dbb04 Merge pull request #806 from DataTalksClub/docs/duckdb-oom-troubleshooting Juan Manuel Perafan 2026-02-08 20:38:43 +01:00
7b3f913660 Add tips about Codespaces jmperafan 2026-02-08 20:37:40 +01:00
5ac9d557fe Merge pull request #811 from DataTalksClub/docs/module4-cloud-setup-data-loading Juan Manuel Perafan 2026-02-08 16:15:59 +01:00
a253310fb4 Merge pull request #805 from DataTalksClub/fix/dbt-build-target-prod Juan Manuel Perafan 2026-02-08 15:59:23 +01:00
b93b9b7894 Merge pull request #809 from mushroomsandchai/main Juan Manuel Perafan 2026-02-07 16:41:58 +01:00
fbd1101415 Set quote to false to check non-string values. ajay katte 2026-02-07 19:56:37 +05:30
1d1a50f1c5 Set quote to false to check non-string values ajay katte 2026-02-07 19:54:04 +05:30
70fca6f7af Merge pull request #807 from mushroomsandchai/main Juan Manuel Perafan 2026-02-07 13:37:53 +01:00
6f56a48dde Make it explicit that they need 2 years of data, not just 6 months jmperafan 2026-02-07 13:19:30 +01:00
18d3641d31 Fix record count for stg_fhv_tripdata ajay katte 2026-02-07 11:22:02 +05:30
d5dbad3e72 Add DuckDB OOM troubleshooting guide for module 4 jmperafan 2026-02-06 22:30:51 +01:00
a0f2b977d0 Add --target prod to dbt build instruction in module 4 homework jmperafan 2026-02-06 22:10:52 +01:00
7d7d10ac93 Remove 404 videos transcript link (#804) ajay katte 2026-02-07 00:37:19 +05:30
14cb63ae71 Add Arsalan Noorafkan to contributors list Alexey Grigorev 2026-02-06 20:06:03 +01:00
6dcc191e0e Merge pull request #803 from dr-pauly/remove-dupes-pie Juan Manuel Perafan 2026-02-06 16:16:00 +01:00
ad967878b9 kestra: replace kv with secret Will Russell 2026-02-06 10:11:42 +00:00
2b4ad5fd5c Merge pull request #794 from dboyraz/main Will Russell 2026-02-06 09:18:11 +00:00
525ca64710 deleted non-working and duplicated link in wk 4 readme under Differences between dbt Core and dbt Cloud Paul Ehizuelen Jr. 2026-02-06 03:51:04 -05:00

1 2 3 4 5 ...