PPO Trading Research Sandbox

한국어

A retrospective reconstruction of a May–June 2025 reinforcement-learning trading prototype. The project explores a long-or-cash PPO agent, a custom trading environment, chronological evaluation, and a Streamlit interface.

This is an educational research sandbox—not a live trading system, a verified profitable strategy, or investment advice. No reliable out-of-sample performance record was preserved in the original repository.

What this version demonstrates

• a custom Gymnasium environment compatible with Stable-Baselines3;
• nine backward-looking, scale-reduced OHLCV features plus the current position;
• three actions: move to cash, keep the current position, or move long;
• transaction costs charged whenever the position changes;
• strictly non-overlapping training and test periods;
• deterministic evaluation against Buy & Hold with return, volatility, Sharpe-proxy, drawdown, and turnover diagnostics;
• command-line data, training, and evaluation workflows; and
• an interactive Streamlit dashboard using the same core modules.

Provenance and review

The original GitHub state is preserved on original-submission at commit 32d7234. The current default branch contains the portfolio reconstruction.

Reviewing the repository, E:\AIFinanceProject, its archive, and VS Code local history established the following:

1. GitHub referenced env.trading_env, but that module was not committed.
2. The missing source was stored inside a directory that also contained an entire virtual environment.
3. The legacy environment used the older Gym API while the retained Stable-Baselines3 installation expected Gymnasium-compatible environments.
4. The retained BTC CSV had no date column; this version refuses to invent dates unless the caller supplies an explicit daily-start-date assumption.
5. A more advanced dashboard draft was recoverable from editor history, but its default training and test dates overlapped.
6. Historical charts show that the prototype ran, but they do not establish out-of-sample profitability. The exact model configuration and clean evaluation record were not preserved.

The missing legacy files are retained unchanged under archive/e-drive-recovered/. They are audit artifacts, not the recommended implementation. See the reconstruction notes.

Verified smoke run

The full CLI was exercised against the retained 1,607-row BTC file using its original download start (2021-01-01) as an explicit daily-date assumption. A deliberately short 256-timestep PPO run trained through 2023-12-31 and was evaluated from 2024-01-01 onward. It changed position once and then stayed long: the strategy returned 148.58% versus 148.83% for Buy & Hold after the modeled entry cost. This is essentially a Buy & Hold replication, not evidence of learned alpha. See validation details.

Historical artifacts

The dashboard screenshot and PPO chart below were recovered from E:. They are included as evidence of the prototype's UI and experimentation—not as performance claims.

Quick start

python -m venv .venv
python -m pip install -r requirements.txt

Download adjusted daily prices. The yfinance end date is exclusive.

python main.py --symbol BTC-USD --start 2021-01-01 --end 2025-01-01

Train only on the earlier period:

python scripts/train.py \
  --csv data/prices.csv \
  --train-end 2023-12-31 \
  --timesteps 10000 \
  --model-path artifacts/ppo_trading_agent

Evaluate on a later period:

python scripts/evaluate.py \
  --csv data/prices.csv \
  --model-path artifacts/ppo_trading_agent \
  --train-end 2023-12-31 \
  --test-start 2024-01-01 \
  --test-end 2024-12-31

Launch the dashboard:

streamlit run ui/dashboard.py

Run checks:

python -m unittest discover -s tests -v

Repository map

Path	Purpose
rl_trader/data.py	OHLCV validation, features, download, and time split
rl_trader/environment.py	Gymnasium long-or-cash environment
rl_trader/training.py	Stable-Baselines3 PPO training helper
rl_trader/evaluation.py	Deterministic evaluation and Buy & Hold comparison
scripts/	Reproducible training and evaluation commands
ui/dashboard.py	Streamlit research interface
tests/	Synthetic-data environment and evaluation checks
archive/e-drive-recovered/	Unmodified legacy source recovered from E:
docs/	Reconstruction record and historical images

Limitations

• PPO results can vary with the selected dates, seed, training budget, and market regime.
• The environment supports only one asset and binary long/cash exposure.
• The transaction-cost model is a fixed approximation; spread, slippage, taxes, liquidity, and market impact are omitted.
• A single chronological holdout is not enough to establish robustness. Repeated walk-forward tests remain future work.
• Yahoo Finance data is downloaded at runtime and is not redistributed here. Users are responsible for applicable data terms.
• Sharpe values are diagnostic proxies with a zero risk-free-rate assumption, not audited investment results.

License

Newly written project code is available under the MIT License. Recovered artifacts and third-party data remain subject to their original terms.