This repository contains utilities and resources specific to wrangling networks for the Metropolitan Transportation Commission networks.
Relevant repos:
- network-wrangler/network_wrangler and the BayAreaMetro fork of network_wrangler (which should only contain code to be merged into the network-wrangler org's repo)
- network-wrangler/projectcard - Infrastructure for project cards, which are network diffs to be applied to a network scenario.
Performance Note:
I tried to run this after updating my python environment in Feb 2026 and the
osmnx.simplification.consolidate_intersections() took days rather than hours.
I further had trouble trying to re-create the previous environment using conda so I tried
using uv.
The repository carries a curated uv spec at environments/uv/requirements.in plus the compiled lock at environments/uv/requirements.lock. They mirror the January 2026 Conda stack (Python 3.10) so that we can rebuild the same toolchain without relying on Conda.
environments/uv/requirements.lock was generated thusly:
uv pip compile environments\uv\requirements.in --python 3.10 -o environments\uv\requirements.lock
If you want the environment in a specific location, create and activate it first, then run the sync. For example, to keep it under .venv\mtc-wrangler-uv in this repo:
py -3.10 -m venv .venv\mtc-wrangler-uv
# Activate in PowerShell
.\.venv\mtc-wrangler-uv\Scripts\Activate.ps1
# OR Activate in a Command Prompt
.\.venv\mtc-wrangler-uv\Scripts\activate.batThen update the environment for the packages:
uv pip sync environments\uv\requirements.lock --python 3.10uv pip sync will now install into that activated environment. Run deactivate when finished.
These are the packages that worked successfully:
(venv_network_wrangler_310) PS E:\GitHub\tm2\mtc_wrangler\create_baseyear_network> uv pip list
Using Python 3.10.19 environment at: E:\GitHub\tm2\venv_network_wrangler_310
Package Version Editable project location
------------------------- ----------- ------------------------------
annotated-types 0.7.0
anyio 4.12.1
argon2-cffi 25.1.0
argon2-cffi-bindings 25.1.0
arrow 1.4.0
asttokens 3.0.1
async-lru 2.2.0
attrs 25.4.0
babel 2.18.0
beautifulsoup4 4.14.3
bleach 6.3.0
branca 0.8.2
cartopy 0.25.0
certifi 2026.2.25
cffi 2.0.0
charset-normalizer 3.4.4
click 8.3.1
click-plugins 1.1.1.2
cligj 0.7.2
colorama 0.4.6
comm 0.2.3
contourpy 1.3.2
cycler 0.12.1
debugpy 1.8.20
decorator 5.2.1
defusedxml 0.7.1
et-xmlfile 2.0.0
exceptiongroup 1.3.1
executing 2.2.1
fastjsonschema 2.21.2
fiona 1.10.1
folium 0.17.0
fonttools 4.61.1
fqdn 1.5.1
geographiclib 2.0
geojson 3.1.0
geopandas 1.0.1
h11 0.16.0
httpcore 1.0.9
httpx 0.28.1
idna 3.11
ijson 3.3.0
ipykernel 7.2.0
ipython 8.38.0
ipywidgets 8.1.8
isoduration 20.11.0
jedi 0.19.2
jellyfish 1.2.1
jinja2 3.1.6
joblib 1.5.3
json5 0.13.0
jsonpointer 3.0.0
jsonref 1.1.0
jsonschema 4.26.0
jsonschema-specifications 2025.9.1
jupyter 1.1.1
jupyter-client 8.8.0
jupyter-console 6.6.3
jupyter-core 5.9.1
jupyter-events 0.12.0
jupyter-lsp 2.3.0
jupyter-server 2.17.0
jupyter-server-terminals 0.5.4
jupyterlab 4.2.7
jupyterlab-pygments 0.3.0
jupyterlab-server 2.28.0
jupyterlab-widgets 3.0.16
kiwisolver 1.4.9
lark 1.3.1
markupsafe 3.0.3
matplotlib 3.10.8
matplotlib-inline 0.2.1
mistune 3.2.0
mypy-extensions 1.1.0
narwhals 2.17.0
nbclient 0.10.4
nbconvert 7.17.0
nbformat 5.10.4
nest-asyncio 1.6.0
network-wrangler 1.0b3 E:\GitHub\tm2\network_wrangler
networkx 3.4.2
notebook 7.2.2
notebook-shim 0.2.4
numpy 2.2.6
openpyxl 3.1.5
osmnx 2.0.7
overrides 7.7.0
packaging 26.0
pandas 2.2.3
pandera 0.24.0
pandocfilters 1.5.1
parso 0.8.6
patsy 1.0.2
pillow 12.1.1
platformdirs 4.9.2
plotly 6.5.2
projectcard 0.3.3
prometheus-client 0.24.1
prompt-toolkit 3.0.52
psutil 6.0.0
pure-eval 0.2.3
pyarrow 17.0.0
pycparser 3.0
pydantic 2.9.2
pydantic-core 2.23.4
pygments 2.19.2
pygris 0.2.1
pyogrio 0.9.0
pyparsing 3.3.2
pyproj 3.7.1
pyshp 3.0.3
python-dateutil 2.9.0.post0
python-json-logger 4.0.0
pytz 2025.2
pywinpty 3.0.3
pyyaml 6.0.2
pyzmq 27.1.0
referencing 0.37.0
requests 2.32.5
rfc3339-validator 0.1.4
rfc3986-validator 0.1.1
rfc3987-syntax 1.1.0
rpds-py 0.30.0
ruff 0.15.4
scikit-learn 1.7.2
scipy 1.15.3
seaborn 0.13.2
send2trash 2.1.0
setuptools 82.0.0
shapely 2.1.2
six 1.17.0
soupsieve 2.8.3
stack-data 0.6.3
statsmodels 0.14.6
tableauhyperapi 0.0.24457
tabulate 0.9.0
terminado 0.18.1
threadpoolctl 3.6.0
tinycss2 1.4.0
toml 0.10.2
tomli 2.4.0
tornado 6.5.4
tqdm 4.67.3
traitlets 5.14.3
typeguard 4.4.2
typing-extensions 4.12.2
typing-inspect 0.9.0
tzdata 2025.3
uri-template 1.3.0
urllib3 2.6.3
us 3.2.0
wcwidth 0.6.0
webcolors 25.10.0
webencodings 0.5.1
websocket-client 1.9.0
widgetsnbextension 4.0.15
xlsxwriter 3.2.9
xyzservices 2025.11.0
zstandard 0.25.0
Don't forget to install network_wrangler in editable mode:
```powershell
(venv_network_wrangler_310) PS E:\GitHub\tm2\mtc_wrangler\create_baseyear_network> uv pip show network_wrangler
Using Python 3.10.19 environment at: E:\GitHub\tm2\venv_network_wrangler_310
Name: network-wrangler
Version: 1.0b3
Location: E:\GitHub\tm2\venv_network_wrangler_310\Lib\site-packages
Editable project location: E:\GitHub\tm2\network_wrangler
Requires: fiona, geographiclib, geojson, geopandas, ijson, osmnx, pandas, pandera, projectcard, psutil, pyarrow, pydantic, pyogrio, pyyaml, typing-extensions
Required-by:
The network creation script downloads road network data from OpenStreetMap. Rather than querying the live Overpass API (which has rate limits and size caps that are routinely hit for Bay Area-scale downloads), the script reads from a local Geofabrik PBF extract.
One-time setup:
-
Download the Northern California extract (~300 MB) from Geofabrik: https://download.geofabrik.de/north-america/us/california/norcal.html
-
Place the file in a subdirectory of your output directory:
<output_dir>/osm_geofabrik_extracts/norcal-latest.osm.pbfThe directory must contain exactly one
.osm.pbffile; the script errors with a clear message if zero or more than one are found. -
Run the script normally — no extra flags are needed. The script logs the OSM data vintage timestamp (read from the PBF header) so you always know what snapshot the network was built from.
The script uses osmium-tool (a C++ binary) to clip the PBF to the county bounding box before loading it with osmnx. It also optionally uses pyosmium (Python bindings) to read the PBF timestamp.
Important: osmium-tool is not a Python package and cannot be installed
with pip or uv. It must be installed separately.
conda install -c conda-forge osmium-toolIf conda update conda is needed first, be prepared for a long solve. Consider
switching to miniforge for future
installs, which uses the much faster mamba solver by default.
After install, verify the binary is on your PATH:
osmium --versionIf it is not found (common when running inside a venv), locate the binary and add its folder to PATH:
conda run where osmium # find the path
$env:PATH += ";C:\Users\<you>\anaconda3\Library\bin" # adjust as needed- Download the latest
osmium-tool-x.x.x-win64.zipfrom: https://github.com/osmcode/osmium-tool/releases - Extract to e.g.
E:\tools\osmium-tool\ - Add to PATH (per-session or permanently via System Properties):
$env:PATH += ";E:\tools\osmium-tool" osmium --version
uv pip install osmiumThis is optional — if pyosmium is not installed the script falls back to the
osmium fileinfo CLI for the timestamp, and then to the file's modification
time. The extraction step always uses the osmium-tool CLI binary.