Skip to content

Latest commit

ย 

History

History
executable file
ยท
413 lines (305 loc) ยท 7.41 KB

README.md

File metadata and controls

executable file
ยท
413 lines (305 loc) ยท 7.41 KB

Current Learning Plan

๐Ÿ“š AI + Data Engineering Learning Roadmap

๐ŸŽฏ Objective

Build a strong foundation across:

  • Linear Algebra & Numerical Optimization
  • Machine Learning
  • Data Warehouse & BI Stack (SSIS, SSAS, SSRS, Power BI)
  • Agentic AI Systems

๐Ÿงญ Learning Phases Overview

Phase Focus Area Duration
0 Prerequisites 2โ€“3 weeks
1 Linear Algebra 3โ€“4 weeks
2 Numerical Optimization 2โ€“3 weeks
3 Machine Learning 4โ€“6 weeks
4 Data Warehouse & BI Parallel
5 Integration Project 3โ€“4 weeks
6 Agentic AI 2โ€“4 weeks

โš™๏ธ Phase 0 โ€” Prerequisites

Topics

  • Python (NumPy, Pandas)
  • SQL Server
  • Basic Statistics

Deliverable

  • Clean dataset using SQL + Python
  • Simple Power BI dashboard

๐Ÿงฎ Phase 1 โ€” Linear Algebra

Topics

  • Vectors & Matrices
  • Matrix Multiplication
  • Eigenvalues & Eigenvectors
  • Singular Value Decomposition (SVD)

Practical Work

  • Implement matrix operations using NumPy
  • Apply PCA on dataset

๐Ÿ“‰ Phase 2 โ€” Numerical Optimization

Topics

  • Gradient Descent
  • Cost Functions
  • Convex vs Non-Convex Optimization
  • Regularization (L1, L2)

Practical Work

  • Implement Gradient Descent from scratch
  • Optimize regression model

๐Ÿค– Phase 3 โ€” Machine Learning

Topics

Supervised Learning

  • Linear Regression
  • Logistic Regression
  • Decision Trees

Unsupervised Learning

  • K-Means Clustering

Evaluation

  • RMSE
  • Accuracy
  • Cross-validation

Tools

  • scikit-learn

Deliverable

End-to-end ML project:

  • Data cleaning
  • Feature engineering
  • Model training
  • Evaluation

๐Ÿ—๏ธ Phase 4 โ€” Data Warehouse & BI Stack

1. Data Warehouse Modeling

  • Star Schema
  • Fact & Dimension Tables

2. SSIS (ETL)

  • Data extraction
  • Data cleaning
  • Data loading

3. SSAS (Analysis)

  • Cube design
  • Measures & KPIs
  • Tabular vs Multidimensional

4. SSRS (Reporting)

  • Paginated reports

5. Power BI

  • Dashboards
  • DAX
  • Row-Level Security (RLS)

๐Ÿ” Phase 5 โ€” Integration Project

Project: Retail Intelligence System

Step 1 โ€” Database Design

  • Star schema
  • FactSales + Dimensions (Customer, Product, Time)

Step 2 โ€” ETL Pipeline

  • Use SSIS for data cleaning and loading

Step 3 โ€” Cube Design

  • Build SSAS cube
  • Define KPIs (Revenue, Profit)

Step 4 โ€” Machine Learning

  • Sales forecasting
  • Customer churn prediction

Step 5 โ€” BI Dashboard

  • Executive dashboard using Power BI
  • Apply RLS

๐Ÿค– Phase 6 โ€” Agentic AI

Topics

  • AI Agents
  • Tool usage (APIs, databases)
  • Memory systems
  • Multi-step reasoning

Tools

  • LangChain
  • AutoGen
  • Python

Project: AI Data Analyst Agent

Features

  • Query SQL database automatically
  • Generate insights
  • Build dashboards suggestions
  • Answer business questions

๐Ÿš€ Final Outcome

By completing this roadmap, you will be able to:

โœ… Design Data Warehouses
โœ… Build ETL Pipelines
โœ… Create Analytical Cubes
โœ… Develop Machine Learning Models
โœ… Build Professional BI Dashboards
โœ… Create AI Agents for Data Analysis


๐Ÿง  Pro Tip

Focus on projects over theory. Each phase should produce something tangible.


๐Ÿ“Œ Suggested Weekly Schedule

  • Study: 2โ€“3 hours/day
  • Practice: 2 hours/day
  • Project work: weekends

๐Ÿ“‚ Repository Structure

/project-root
โ”‚
โ”œโ”€โ”€ data/
โ”œโ”€โ”€ notebooks/
โ”œโ”€โ”€ sql/
โ”œโ”€โ”€ etl/
โ”œโ”€โ”€ ssas/
โ”œโ”€โ”€ reports/
โ”œโ”€โ”€ powerbi/
โ”œโ”€โ”€ ml/
โ””โ”€โ”€ agentic-ai/

๐Ÿ“š AI + Data Engineering Roadmap (Execution Plan)

๐Ÿ“… Start Date

01 August 2026

โณ Estimated Duration

~20โ€“24 Weeks


๐Ÿ“Š Progress Tracking

โœ… Completion Formula

Progress % = (Completed Tasks / Total Tasks) * 100

๐Ÿ“Œ How to Use

  • Mark tasks with [x] when done
  • GitHub will visually track progress in the checklist
  • You can also connect this to Power BI later for a visual dashboard

๐Ÿงญ MASTER TASK LIST

โš™๏ธ Phase 0 โ€” Prerequisites (Week 1โ€“2)

๐Ÿ“… 01 Aug โ†’ 14 Aug

  • Install Python + environment
  • Learn NumPy basics
  • Learn Pandas basics
  • SQL basics (SELECT, JOIN, GROUP BY)
  • Load dataset into SQL Server
  • Clean dataset using SQL
  • Export to Power BI
  • Build simple dashboard

๐Ÿงฎ Phase 1 โ€” Linear Algebra (Week 3โ€“6)

๐Ÿ“… 15 Aug โ†’ 11 Sep

  • Understand vectors
  • Matrix operations (add, multiply)
  • Matrix multiplication implementation (NumPy)
  • Learn eigenvalues & eigenvectors
  • Understand SVD concept
  • Apply PCA on dataset

๐Ÿ“‰ Phase 2 โ€” Numerical Optimization (Week 7โ€“9)

๐Ÿ“… 12 Sep โ†’ 02 Oct

  • Understand cost functions
  • Learn Gradient Descent theory
  • Implement Gradient Descent (Python)
  • Learn learning rate tuning
  • Understand overfitting vs underfitting
  • Learn L1 / L2 Regularization

๐Ÿค– Phase 3 โ€” Machine Learning (Week 10โ€“15)

๐Ÿ“… 03 Oct โ†’ 13 Nov

  • Linear Regression model
  • Logistic Regression model
  • Decision Trees
  • K-Means clustering
  • Model evaluation (RMSE, Accuracy)
  • Feature engineering
  • Build ML pipeline
  • Final ML mini-project

๐Ÿ—๏ธ Phase 4 โ€” Data Warehouse & BI (Parallel Track)

๐Ÿ“… 01 Aug โ†’ 30 Nov

Data Warehouse

  • Design Star Schema
  • Create Fact & Dimension tables

SSIS

  • Build ETL pipeline
  • Data cleaning flow
  • Schedule ETL job

SSAS

  • Create cube
  • Define measures & KPIs

SSRS

  • Build paginated report

Power BI

  • Data modeling
  • DAX measures
  • Dashboard design
  • Implement RLS

๐Ÿ” Phase 5 โ€” Integration Project (Week 16โ€“18)

๐Ÿ“… 14 Nov โ†’ 04 Dec

Retail Intelligence System

  • Build full database schema
  • Implement ETL (SSIS)
  • Build SSAS cube
  • Train ML model (forecasting)
  • Build Power BI dashboard
  • Apply RLS
  • End-to-end testing

๐Ÿค– Phase 6 โ€” Agentic AI (Week 19โ€“22)

๐Ÿ“… 05 Dec โ†’ 31 Dec

  • Learn AI agents concept
  • Setup LangChain / AutoGen
  • Build SQL query agent
  • Add memory system
  • Connect agent to dataset
  • Generate automated insights

๐Ÿš€ FINAL CAPSTONE

๐Ÿ“… End of December 2026

Autonomous BI System

  • Agent monitors database
  • Detect anomalies
  • Trigger ML predictions
  • Update dashboards
  • Generate reports automatically

๐Ÿ“… Weekly Tracking Template

Week X
Planned:
Completed:
Blockers:
Notes:

๐Ÿ“Š Optional: Advanced Tracking (Power BI)

You can export tasks into a table:

Task Status Phase Completion Date
Learn NumPy Done Phase 0 02 Aug

Then:

  • Build a Power BI dashboard
  • Track progress %
  • Visualize the timeline

๐Ÿง  Rules for Success

  • ๐Ÿšซ No skipping phases
  • โœ… Build something every week
  • ๐ŸŽฏ Focus on integration, not isolated learning

๐Ÿ“‚ Project Structure

/project-root
โ”‚
โ”œโ”€โ”€ data/
โ”œโ”€โ”€ notebooks/
โ”œโ”€โ”€ sql/
โ”œโ”€โ”€ etl/
โ”œโ”€โ”€ ssas/
โ”œโ”€โ”€ reports/
โ”œโ”€โ”€ powerbi/
โ”œโ”€โ”€ ml/
โ””โ”€โ”€ agentic-ai/

๐Ÿ Success Criteria

By the end, you will:

  • โœ… Build a Data Warehouse
  • โœ… Create ETL pipelines
  • โœ… Design analytical cubes
  • โœ… Train ML models
  • โœ… Build dashboards
  • โœ… Create AI agents

๐Ÿ“Ž License

This roadmap is for educational purposes.