Skip to content
RAWG Data Pipeline (Fetch & Clean)

RAWG Data Pipeline (Fetch & Clean) #2

Sign in to view logs

RAWG Data Pipeline (Fetch & Clean)

RAWG Data Pipeline (Fetch & Clean) #2

Workflow file for this run

.github/workflows/rawg_pipeline.yml at cf31cf8
name: RAWG Data Pipeline (Fetch & Clean)
on:
schedule:
- cron: '0 2 * * 1,3,5'
workflow_dispatch:
jobs:
run-etl-pipeline:
runs-on: ubuntu-latest
steps:
- name: Checkout repository
uses: actions/checkout@v4
- name: Install uv
uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- name: Set up Python
run: uv python install 3.13
- name: Install dependencies
run: uv sync
- name: Run Fetcher (Extract)
env:
RAWG_API_KEY: ${{ secrets.RAWG_API_KEY }}
run: |
uv run python -m src.etl.fetchers.rawg_fetcher_games
- name: Run Cleaner (Transform & Load to Supabase)
env:
SUPABASE_URL: ${{ secrets.SUPABASE_URL }}
SUPABASE_KEY: ${{ secrets.SUPABASE_KEY }}
run: |
uv run python -m src.etl.cleaners.rawg_cleaner_games