Conversation
Add demo and repo people can clone. Goal: - Bring in joint users across all three platforms (Weave, Models, OpenPipe) - Gauge interest in Copilot analysis - Provide a clone-able repo to get started, which includes tricks like resuming crashed runs and integrating Weave scorers into the training loop
📚 Mintlify Preview Links📝 Changed (1 total)📄 Pages (1)
🤖 Generated automatically when Mintlify deployment succeeds |
training.mdx
Outdated
|
|
||
| To get started, satisfy the [prerequisites](/training/prerequisites) to start using the service and then see [OpenPipe's Serverless RL quickstart](https://art.openpipe.ai/getting-started/quick-start) to learn how to post-train your models. | ||
|
|
||
| We also recommend checking out this public [demo workspace](https://wandb.ai/wandb/demo-project-qwen-email-agent-with-art-weave-models/workspace?nw=nwuserjuliarose) and [repo](https://github.com/julia-g-rose/openpipe-ruler-wandb-demo) that shows how to: |
There was a problem hiding this comment.
Probably should have one CTA - and sending to the workspace isn't that helpful
| We also recommend checking out this public [demo workspace](https://wandb.ai/wandb/demo-project-qwen-email-agent-with-art-weave-models/workspace?nw=nwuserjuliarose) and [repo](https://github.com/julia-g-rose/openpipe-ruler-wandb-demo) that shows how to: | |
| We also recommend checking out this [repo](https://github.com/julia-g-rose/openpipe-ruler-wandb-demo) that shows how to: |
There was a problem hiding this comment.
+1 on CTA. On sharing the workspace, I think a demo workspace helps people see what a full integration looks like in the platform, prior to investing time into a project. Cloning a repo, running training, etc. is a big time commitment that is easier to justify if you know what the final integration will look like
There was a problem hiding this comment.
@scottire just updated to include a single CTA, but used the workspace (instead of the repo) so users can see the benefit of using W&B. The workspace includes a link to the repo in the project summary at the top.
training.mdx
Outdated
| * Train a Qwen model with OpenPipe RULER and [Weave Scorers](https://docs.wandb.ai/weave/guides/evaluation/scorers#create-your-own-scorers) | ||
| * Track training progress and [create custom plots](https://docs.wandb.ai/models/app/features/custom-charts) with W&B Models | ||
| * Evaluate the final results on a [Weave leaderboard](https://docs.wandb.ai/weave/cookbooks/leaderboard_quickstart#leaderboard-quickstart) | ||
| * Use AI to analyze your traces, loss curves and experiment results |
There was a problem hiding this comment.
Where does it use AI to analyze the traces?
There was a problem hiding this comment.
The AI analysis was done ad hoc with Cursor and shared in the workspace demo, so it's not part of the code base. Included it to start seeing if there is interest in offering this sort of analysis within Workspaces out-of-the-box (instead of my current method of sharing traces and charts with Cursor via prompting). I recognize the statement is somewhat misleading, so can remove, but I think it will get us feedback around the direction
There was a problem hiding this comment.
Yeah I don't really like this, it seems a bit hand-wavey. I'd be OK with you including the prompt that you gave to Cursor to do the analysis, but as it is now, I'm not sure that the reader will be able to extrapolate that you are talkiung about a manual analysis.
|
|
||
| * Train a Qwen model with OpenPipe RULER and [Weave Scorers](https://docs.wandb.ai/weave/guides/evaluation/scorers#create-your-own-scorers) | ||
| * Track training progress and [create custom plots](https://docs.wandb.ai/models/app/features/custom-charts) with W&B Models | ||
| * Evaluate the final results on a [Weave leaderboard](https://docs.wandb.ai/weave/cookbooks/leaderboard_quickstart#leaderboard-quickstart) |
There was a problem hiding this comment.
| * Evaluate the final results on a [Weave leaderboard](https://docs.wandb.ai/weave/cookbooks/leaderboard_quickstart#leaderboard-quickstart) | |
| * Evaluate the final results on a [Weave leaderboard](https://docs.wandb.ai/weave/cookbooks/leaderboard_quickstart#leaderboard-quickstart). |
training.mdx
Outdated
| * Train a Qwen model with OpenPipe RULER and [Weave Scorers](https://docs.wandb.ai/weave/guides/evaluation/scorers#create-your-own-scorers) | ||
| * Track training progress and [create custom plots](https://docs.wandb.ai/models/app/features/custom-charts) with W&B Models | ||
| * Evaluate the final results on a [Weave leaderboard](https://docs.wandb.ai/weave/cookbooks/leaderboard_quickstart#leaderboard-quickstart) | ||
| * Use AI to analyze your traces, loss curves and experiment results |
There was a problem hiding this comment.
Yeah I don't really like this, it seems a bit hand-wavey. I'd be OK with you including the prompt that you gave to Cursor to do the analysis, but as it is now, I'm not sure that the reader will be able to extrapolate that you are talkiung about a manual analysis.
Co-authored-by: Matt Linville <matt.linville@wandb.com>
Co-authored-by: Matt Linville <matt.linville@wandb.com>
Co-authored-by: Matt Linville <matt.linville@wandb.com>
Co-authored-by: Matt Linville <matt.linville@wandb.com>
4 participants
Add demo and repo people can clone. Goal:
See Loom with project overview: https://www.loom.com/share/7ae454b4068f4a4390bfcf3d7ea82ffc