Add data ingestion (ophir ingest) and trading-best-practices skill - #3
Open
The-Data-is-a-lie wants to merge 4 commits into
Open
Add data ingestion (ophir ingest) and trading-best-practices skill#3The-Data-is-a-lie wants to merge 4 commits into
The-Data-is-a-lie wants to merge 4 commits into
Conversation
- New ophir.agent package: ingest.py pulls daily OHLC from Yahoo Finance and writes a model-ready parquet (Hive layout), reusing ophir.ticker.extract_features / extract_model_data; feed.py reads it back and latest_window_tensors bridges the latest window to the model (S,13)/(S,3) input. - Add an "ophir ingest <SYMBOL> --days" Typer command (default ~2 years; no GPU). - Yahoo data is fetched split/dividend-adjusted (auto_adjust=True); ophir's split back-adjustment is skipped on this path to avoid double-adjustment. - Add tests/test_ingest.py (5 network-mocked tests) and document the new modules in the API and CLI references.
- New .claude/skills/trading-best-practices/SKILL.md distilling trading-system best practices (paper-first / dry-run defaults, a pre-trade risk gate plus drawdown kill-switch, look-ahead/survivorship-bias-free backtests, and LLM-in-the-loop safety rails) to guide future trading-agent work. - Repo tooling; no runtime impact on the ophir package.
- New ophir.agent modules: config (pydantic-settings; paper/dry_run/allow_live + risk defaults with a live-mode guard), audit (structlog append-only JSON audit log), and predict (a Forecast dataclass with predict_ticker/predict_many/rank). feed.forecast_window_tensors builds a forward-looking window (real history + zeroed future) for genuine forecasts. - New CLI: ophir predict <SYMBOL>, ophir rank <SYMBOLS> [--top-k], and ophir train (full-US-market trainer, <2024 train / >=2024 val, fine-tune or from-scratch via --finetune-from/--max-steps). fetch_base_trainer gains a max_steps argument. - Add pydantic-settings and structlog; source torch from the CUDA 13.0 index and pin torch<2.11 (flex-attention compilation regresses on 2.11+); refresh uv.lock. - Network-mocked tests for config/audit/predict; API/CLI/README docs.
- OHLCMulitClassPredictor.forward now zeros the response-region rows of feature_input before the feature MLP. r_close/upside/downside are both input features and targets, so the self-attending response tokens could copy the answer instead of forecasting; at inference (future rows zeroed) the model collapsed to an identical-per-ticker constant. Shape-preserving (existing checkpoints load); the model must be retrained to benefit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First slice of a trading-agent build on top of the ophir model — roadmap step 1 (data ingestion) — plus a best-practices skill that guides the broader effort.
ophir ingest <SYMBOL> [--days N]: pulls daily OHLC from Yahoo Finance and writes a model-ready parquet, reusingophir.ticker.extract_features/extract_model_dataso the existingStockHanlder/StockStreamerpipeline reads it unchanged.ophir.agentpackage:ingest.py(fetch → normalize → quality checks → persist) andfeed.py(load_daily_ohlcv;latest_window_tensorsbridges the latest 365-day window to the model(S,13)/(S,3)input tensors).auto_adjust=True); ophir's split back-adjustment is skipped on this path to avoid double-adjustment.trading-best-practicesskill (.claude/skills/): paper-first defaults, a pre-trade risk gate + drawdown kill-switch, look-ahead/survivorship-bias-free backtesting rules, and LLM-in-the-loop safety rails.Why
Turns a ticker into a model-ready dataset end-to-end with no GPU — the first step toward forecasting and decisioning. Defaults to ~2 years of history because the model consumes 365-day windows (a 180-day pull isn't enough to fill one inference window).
Notes for reviewers
ophir.agent(not anagent/data/subpackage) because the repo's.gitignoreblanket-ignores anydata/directory — anagent/data/package would have been silently untracked.docs/api/index.rstanddocs/cli.rstdocument the new modules/command.0.2.0(feature) and0.2.1(skill), each with CHANGELOG + version bumps.Verification
--strict(11 files), 78 pytest (5 new, network-mocked), and the sphinx-Wdocs build — all green.ophir ingest AAPL-> 501 daily bars ->latest_window_tensorsyields(365,13)/(365,3)/(365,).