A data engineering track that starts with web scraping and ends with a pipeline in production.

Hands-on lessons, in order: you extract data from real sites, transform it, load it into databases and the cloud, orchestrate, analyze and apply AI. The platform checks every delivery. In English and Portuguese.

crawler.pyextrair.py

from curl_cffi import requests
from extrair import extrair, salvar, log

BUSCA = "console gx5"
HOST = "marketplace.example"
PAGINAS = 3
INTERVALO = 1.5   # segundos entre páginas

for p in range(1, PAGINAS + 1):
    url = f"https://{HOST}/s?k={BUSCA}&page={p}"
    r = requests.get(url, impersonate="chrome")
    log(p, r.status_code, len(r.content))
    itens = extrair(r.text)
    if not itens:
        break
    for item in itens:
        salvar(item, pagina=p)
    sleep(INTERVALO)

Python 3.12editable: change PAGINAS or INTERVALO and run

marketplace.example/s?k=console+gx5

Enviar para Bela Vista 01319console gx5Olá, faça seu login
Contas e Listas0

☰ TodosOfertas do DiaGamesEletrônicosLivros

Resultados

Patrocinado

Controle - GX5

★★★★★ (13.5 mil)

^R$402,80

Patrocinado

Controle sem fio – Branco

★★★★★ (13.5 mil)

^R$417,05

Patrocinado

GX5 Edição Digital 825GB 1 Controle

★★★★★ (13.5 mil)

^R$4.025,53

Patrocinado

GX5 Slim Disk 1TB 1 Controle Branco

★★★★★ (13.5 mil)

^R$4.999,90

Patrocinado

Base De Carregamento Do - GX5

★★★★★ (13.5 mil)

^R$169,00

Patrocinado

Console GX5 - Digital Edition

★★★★★ (13.5 mil)

^R$4.226,55

Edit the script if you like. Then run it.

precos0rows

	titulotext	preconumeric	paginaint2
—

The track, in the order the data moves

1ETLFrom the first request to stored data.221 lessons

2Data engineeringWhat makes it run on its own, at scale, in the cloud.171 lessons

3Data in useAnalysis, models, AI and the portfolio project.126 lessons

click an area to see its chapters

You do it, the platform checks it

Seven stages with tests. Each one unlocks the next.

1fetch
2parse
3frontier
4dedup
5per-host limit
6resume
7report

zsh

$ glearn test

Gabriel GrecoData engineer in São Paulo

I've spent the last years collecting data at scale: running crawlers at Zyte, building the global acquisition and the indices behind the Red Tape Index at Labrynth, and automating what could be automated at Lumma. The course is the path I wish I'd had.

See full experience ↗

Zyte30 bnrequests a year operated
Labrynth · Red Tape Index2 bnrequests collecting public data on 4 continents
Lumma4,500 ha month automated

Join the cohort 1 list

Access before it opens to everyone.
The first lesson without an account, as soon as it's ready.

Frequently asked questions

Do I need to know how to code?: Basic Python: variables, for loops, functions, opening a file. The first lesson asks for none of it and the early chapters review what you need.
What do I need to install?: Python 3.12, a terminal and an editor. Docker, Postgres and the cloud come in parts 2 and 3, each chapter with its own setup steps.
In what order do I go?: Track order: Extract, Transform, Load, then engineering, then analysis, data science and AI. Each chapter uses the data you collected in the previous one.
How long does it take?: About 190 hours of lessons and practice, at your pace. At 5 hours a week that's roughly 9 months; at 10, about 4 or 5.
How does the platform check?: Every lesson ends in a deliverable: a dataset, a command run in the terminal, a table in the database, a scheduled pipeline. The platform checks the result, not a quiz.
What do I build?: 17 projects, from the first crawler to a cloud pipeline with orchestration, quality checks, a dashboard and AI on top. All of them live in your GitHub as a portfolio.
I already know scraping. Worth it?: You can skip chapters by passing the equivalent challenge. Parts 2 and 3 (databases, orchestration, scale, cloud, analysis, AI) are what most scraping courses don't have.
Does it cover anti-bot and blocks?: Yes: reverse engineering protections, fingerprints, proxies and the statistical reasoning of when to push and when to stop. Always on public data and within the sites' limits.
Can I use AI in the lessons?: Yes. There's a path with AI and one without. Part 3 teaches you to use LLMs to extract, classify and answer questions about what you collected, with agents and MCP.
Is it in English?: English and Portuguese, same lessons, same labs, same checks. Videos are 10 minutes at most.
When does it open?: Chapter by chapter, in track order. The first lesson opens to the list first, no account needed.
Does it work on a phone?: Guided lessons and simulators, yes. Terminal, Docker and cloud need a computer.