feat(skore): Introduce CLI for installing skills#2976
Conversation
Coverage Report for |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| File | Stmts | Miss | Branch | BrPart | Cover | Missing |
|---|---|---|---|---|---|---|
| skore/src/skore | ||||||
| __init__.py | 40 | 2 | 2 | 0 | 95% | 118–119 |
| _config.py | 58 | 3 | 12 | 1 | 94% | 70, 117–118 |
| exceptions.py | 4 | 4 | 0 | 0 | 0% | 4, 15, 19, 23 |
| skore/src/skore/_cli | ||||||
| __init__.py | 8 | 0 | 0 | 0 | 100% | |
| __main__.py | 1 | 1 | 0 | 0 | 0% | 3 |
| _style.py | 18 | 0 | 0 | 0 | 100% | |
| skore/src/skore/_cli/skills | ||||||
| __init__.py | 2 | 0 | 0 | 0 | 100% | |
| _agents.py | 25 | 0 | 4 | 0 | 100% | |
| _catalog.py | 33 | 0 | 0 | 0 | 100% | |
| _commands.py | 276 | 11 | 116 | 11 | 96% | 116, 169–170, 226, 297, 528–529, 532–535 |
| skore/src/skore/_cli/skills/app | ||||||
| __init__.py | 6 | 0 | 0 | 0 | 100% | |
| _find.py | 30 | 0 | 2 | 0 | 100% | |
| _install.py | 94 | 4 | 26 | 7 | 95% | 123, 128, 170–171 |
| _manage.py | 36 | 21 | 2 | 2 | 41% | 47–50, 53–57, 61–63, 66, 69–71, 75–77, 80–81 |
| _menu.py | 28 | 13 | 0 | 0 | 53% | 55–56, 59–62, 66, 73, 76, 79–80, 83–84 |
| _widgets.py | 74 | 2 | 22 | 3 | 97% | 33–34 |
| skore/src/skore/_plugins | ||||||
| __init__.py | 12 | 0 | 0 | 0 | 100% | |
| serde.py | 32 | 0 | 8 | 0 | 100% | |
| skore/src/skore/_plugins/hub | ||||||
| __init__.py | 9 | 2 | 0 | 0 | 77% | 15, 20 |
| exception.py | 2 | 0 | 0 | 0 | 100% | |
| json.py | 10 | 1 | 2 | 1 | 90% | 16 |
| skore/src/skore/_plugins/hub/artifact | ||||||
| __init__.py | 0 | 0 | 0 | 0 | 100% | |
| artifact.py | 23 | 0 | 4 | 0 | 100% | |
| serializer.py | 27 | 0 | 2 | 0 | 100% | |
| upload.py | 26 | 0 | 4 | 0 | 100% | |
| skore/src/skore/_plugins/hub/artifact/media | ||||||
| __init__.py | 5 | 0 | 0 | 0 | 100% | |
| data.py | 20 | 0 | 0 | 0 | 100% | |
| inspection.py | 72 | 10 | 20 | 8 | 86% | 46–49, 51, 53, 60, 62, 68, 107 |
| media.py | 10 | 0 | 0 | 0 | 100% | |
| model.py | 10 | 0 | 0 | 0 | 100% | |
| performance.py | 59 | 0 | 6 | 0 | 100% | |
| skore/src/skore/_plugins/hub/artifact/pickle | ||||||
| __init__.py | 2 | 0 | 0 | 0 | 100% | |
| pickle.py | 24 | 0 | 2 | 0 | 100% | |
| skore/src/skore/_plugins/hub/authentication | ||||||
| __init__.py | 0 | 0 | 0 | 0 | 100% | |
| apikey.py | 7 | 0 | 0 | 0 | 100% | |
| login.py | 28 | 4 | 4 | 2 | 85% | 37, 42–43, 52 |
| token.py | 80 | 0 | 8 | 0 | 100% | |
| uri.py | 6 | 0 | 0 | 0 | 100% | |
| skore/src/skore/_plugins/hub/client | ||||||
| __init__.py | 0 | 0 | 0 | 0 | 100% | |
| client.py | 88 | 10 | 18 | 3 | 88% | 140, 187–189, 191–192, 194, 196, 198, 230 |
| skore/src/skore/_plugins/hub/metric | ||||||
| __init__.py | 10 | 0 | 0 | 0 | 100% | |
| accuracy.py | 35 | 0 | 0 | 0 | 100% | |
| brier_score.py | 35 | 0 | 0 | 0 | 100% | |
| log_loss.py | 35 | 0 | 0 | 0 | 100% | |
| metric.py | 55 | 4 | 2 | 1 | 92% | 38, 77–78, 84 |
| precision.py | 57 | 0 | 0 | 0 | 100% | |
| r2.py | 35 | 0 | 0 | 0 | 100% | |
| recall.py | 59 | 0 | 0 | 0 | 100% | |
| rmse.py | 35 | 0 | 0 | 0 | 100% | |
| roc_auc.py | 35 | 0 | 0 | 0 | 100% | |
| timing.py | 76 | 4 | 0 | 0 | 94% | 45–46, 104–105 |
| skore/src/skore/_plugins/hub/project | ||||||
| __init__.py | 0 | 0 | 0 | 0 | 100% | |
| project.py | 138 | 6 | 26 | 5 | 95% | 84, 109, 124, 323, 403, 433 |
| skore/src/skore/_plugins/hub/report | ||||||
| __init__.py | 3 | 0 | 0 | 0 | 100% | |
| cross_validation_report.py | 121 | 2 | 28 | 3 | 98% | 208, 248 |
| estimator_report.py | 10 | 0 | 0 | 0 | 100% | |
| report.py | 60 | 0 | 6 | 0 | 100% | |
| skore/src/skore/_plugins/local | ||||||
| __init__.py | 2 | 0 | 0 | 0 | 100% | |
| metadata.py | 81 | 3 | 8 | 1 | 96% | 29, 141–142 |
| project.py | 93 | 1 | 30 | 1 | 98% | 238 |
| storage.py | 42 | 2 | 6 | 1 | 95% | 45, 189 |
| skore/src/skore/_plugins/mlflow | ||||||
| __init__.py | 5 | 0 | 0 | 0 | 100% | |
| project.py | 259 | 26 | 62 | 9 | 89% | 47, 49–51, 117–121, 226, 244, 262–263, 358, 466, 468, 484, 486–491, 493–495 |
| reports.py | 155 | 6 | 34 | 4 | 96% | 129, 170, 207–208, 270, 283 |
| skore/src/skore/_project | ||||||
| __init__.py | 0 | 0 | 0 | 0 | 100% | |
| _summary.py | 80 | 1 | 38 | 3 | 98% | 146 |
| _widget.py | 191 | 0 | 44 | 2 | 100% | |
| dependencies.py | 19 | 0 | 6 | 0 | 100% | |
| git.py | 25 | 0 | 4 | 0 | 100% | |
| login.py | 17 | 3 | 6 | 2 | 82% | 73, 82–83 |
| plugin.py | 9 | 0 | 0 | 0 | 100% | |
| project.py | 56 | 2 | 16 | 3 | 96% | 136, 145 |
| types.py | 3 | 0 | 0 | 0 | 100% | |
| skore/src/skore/_sklearn | ||||||
| __init__.py | 8 | 0 | 0 | 0 | 100% | |
| _base.py | 56 | 1 | 12 | 0 | 98% | 47 |
| compare.py | 5 | 0 | 0 | 0 | 100% | |
| evaluate.py | 45 | 0 | 26 | 0 | 100% | |
| feature_names.py | 26 | 0 | 12 | 0 | 100% | |
| find_ml_task.py | 61 | 0 | 46 | 2 | 100% | |
| metrics.py | 364 | 0 | 86 | 0 | 100% | |
| types.py | 19 | 1 | 0 | 0 | 94% | 31 |
| skore/src/skore/_sklearn/_checks | ||||||
| __init__.py | 3 | 0 | 0 | 0 | 100% | |
| _utils.py | 75 | 3 | 28 | 3 | 96% | 83, 217, 225 |
| accessor.py | 33 | 1 | 14 | 0 | 96% | 17 |
| base.py | 77 | 4 | 24 | 2 | 94% | 128–129, 266–267 |
| model_checks.py | 397 | 10 | 130 | 9 | 97% | 377, 385, 435, 439, 576, 653–654, 748, 767, 854 |
| tunable_hyperparameters.py | 5 | 0 | 0 | 0 | 100% | |
| skore/src/skore/_sklearn/_comparison | ||||||
| __init__.py | 9 | 0 | 0 | 0 | 100% | |
| inspection_accessor.py | 27 | 1 | 2 | 0 | 96% | 360 |
| metrics_accessor.py | 125 | 4 | 18 | 4 | 96% | 255–256, 325, 1181 |
| report.py | 160 | 5 | 68 | 0 | 96% | 576, 582, 640–642 |
| skore/src/skore/_sklearn/_cross_validation | ||||||
| __init__.py | 11 | 0 | 0 | 0 | 100% | |
| data_accessor.py | 36 | 2 | 12 | 2 | 94% | 48, 74 |
| inspection_accessor.py | 27 | 1 | 2 | 0 | 96% | 332 |
| metrics_accessor.py | 119 | 3 | 16 | 3 | 97% | 215–216, 1148 |
| report.py | 204 | 9 | 48 | 5 | 95% | 74, 79, 84, 340, 664, 670, 756–758 |
| skore/src/skore/_sklearn/_estimator | ||||||
| __init__.py | 11 | 0 | 0 | 0 | 100% | |
| data_accessor.py | 48 | 2 | 20 | 1 | 95% | 61, 178 |
| inspection_accessor.py | 44 | 1 | 8 | 2 | 97% | 297 |
| metrics_accessor.py | 124 | 0 | 22 | 0 | 100% | |
| report.py | 361 | 18 | 106 | 14 | 95% | 67, 81, 310, 383, 470, 725, 814, 935–937, 1019, 1021, 1023, 1025, 1028–1029, 1033, 1036 |
| skore/src/skore/_sklearn/_plot | ||||||
| __init__.py | 3 | 0 | 0 | 0 | 100% | |
| base.py | 61 | 2 | 14 | 1 | 96% | 61–62 |
| utils.py | 151 | 3 | 70 | 3 | 98% | 254–255, 428 |
| skore/src/skore/_sklearn/_plot/data | ||||||
| __init__.py | 2 | 0 | 0 | 0 | 100% | |
| table_report.py | 177 | 1 | 60 | 1 | 99% | 676 |
| skore/src/skore/_sklearn/_plot/inspection | ||||||
| __init__.py | 0 | 0 | 0 | 0 | 100% | |
| calibration_curve.py | 82 | 11 | 14 | 8 | 86% | 123–126, 128, 133–134, 240, 243–244, 274 |
| coefficients.py | 181 | 0 | 88 | 1 | 100% | |
| impurity_decrease.py | 103 | 2 | 34 | 3 | 98% | 441, 485 |
| permutation_importance.py | 198 | 1 | 90 | 1 | 99% | 650 |
| utils.py | 32 | 0 | 10 | 0 | 100% | |
| skore/src/skore/_sklearn/_plot/metrics | ||||||
| __init__.py | 6 | 0 | 0 | 0 | 100% | |
| confusion_matrix.py | 200 | 0 | 66 | 2 | 100% | |
| metrics_summary_display.py | 136 | 1 | 62 | 2 | 99% | 244 |
| precision_recall_curve.py | 117 | 0 | 32 | 1 | 100% | |
| prediction_error.py | 166 | 0 | 54 | 2 | 100% | |
| roc_curve.py | 121 | 0 | 34 | 2 | 100% | |
| skore/src/skore/_sklearn/train_test_split | ||||||
| __init__.py | 2 | 0 | 0 | 0 | 100% | |
| train_test_split.py | 71 | 0 | 34 | 2 | 100% | |
| skore/src/skore/_sklearn/train_test_split/warning | ||||||
| __init__.py | 8 | 0 | 0 | 0 | 100% | |
| high_class_imbalance_too_few_examples_warning.py | 19 | 0 | 6 | 0 | 100% | |
| high_class_imbalance_warning.py | 20 | 0 | 6 | 0 | 100% | |
| random_state_unset_warning.py | 10 | 0 | 2 | 0 | 100% | |
| shuffle_true_warning.py | 9 | 0 | 2 | 0 | 100% | |
| stratify_is_set_warning.py | 10 | 0 | 2 | 0 | 100% | |
| time_based_column_warning.py | 21 | 0 | 4 | 0 | 100% | |
| train_test_split_warning.py | 3 | 0 | 0 | 0 | 100% | |
| skore/src/skore/_utils | ||||||
| __init__.py | 6 | 2 | 0 | 0 | 66% | 8, 13 |
| _accessor.py | 106 | 20 | 30 | 11 | 81% | 13, 36, 61–65, 68, 70–71, 76, 81, 83, 85, 92–94, 164, 216, 236 |
| _cache.py | 37 | 0 | 2 | 1 | 100% | |
| _cache_key.py | 35 | 5 | 22 | 5 | 85% | 22, 24, 51, 59, 68 |
| _callable_name.py | 9 | 0 | 4 | 0 | 100% | |
| _dataframe.py | 43 | 4 | 18 | 4 | 90% | 27, 46, 48, 63 |
| _environment.py | 33 | 1 | 10 | 2 | 96% | 49 |
| _fixes.py | 8 | 0 | 2 | 0 | 100% | |
| _index.py | 5 | 0 | 2 | 0 | 100% | |
| _jupyter.py | 8 | 2 | 0 | 0 | 75% | 13–14 |
| _measure_time.py | 10 | 0 | 0 | 0 | 100% | |
| _parallel.py | 17 | 0 | 0 | 0 | 100% | |
| _patch.py | 21 | 12 | 8 | 8 | 42% | 30, 35–39, 42–43, 46–47, 58, 60 |
| _progress_bar.py | 42 | 4 | 4 | 0 | 90% | 53–54, 64–65 |
| _show_versions.py | 66 | 1 | 22 | 1 | 98% | 26 |
| _skrub.py | 37 | 0 | 4 | 0 | 100% | |
| _testing.py | 128 | 14 | 12 | 2 | 89% | 24, 33, 71–72, 94, 193, 202, 213–218, 220 |
| skore/src/skore/_utils/repr | ||||||
| __init__.py | 2 | 0 | 0 | 0 | 100% | |
| base.py | 54 | 0 | 4 | 0 | 100% | |
| data.py | 128 | 0 | 30 | 1 | 100% | |
| html_repr.py | 40 | 0 | 0 | 0 | 100% | |
| rich_repr.py | 80 | 0 | 22 | 3 | 100% | |
| utils.py | 11 | 0 | 2 | 0 | 100% | |
| TOTAL | 8436 | 300 | 2162 | 187 | 96% | |
| Tests | Skipped | Failures | Errors | Time |
|---|---|---|---|---|
| 2742 | 5 💤 | 0 ❌ | 0 🔥 | 9m 13s ⏱️ |
rouk1
left a comment
There was a problem hiding this comment.
I tested that a bit and I have a few feedbacks.
- launching the CLI is a bit slow
- launching with no parameter does not trigger an interactive option UX (so you need to relaunch the script and it is slow so it is a bit frustrating)
- when selection skills pressing tab is hard to discover (bold text is not readable enough to me)
- when selecting agent and scope you must press two keys to select what you want (using the arrow should select the highlighted choice no ?)
IMO all this could have been done with a multistep non fullscreen questionary like experience.
- chose action (list, find, ... plus exit)
- chose action first choice (if only one skip) (for install select among workflows + cancel)
- continue through list of questions or go back
- final step may have a progress bar
I actually try @rouk1 I think that I can correct all your point easily apart of the speed to launch. For me this one could be done in a later step with lazy import or doing a separate package with only the cli (not sure that it is a good idea). |
|
The following PR (#2988) will decrease the import time from 2.5s to 0.15s with the lazy loading mechanism. |
|
Discussed IRL with @glemaitre , and given its independence from skore, i also think it's a good idea to host the CLI in its own repository. By the way, great job 👏🏼 . |
I would say there is two points that bring me towards this complex proposal:
Iterating with several persons then I think that the subpackage is actually the way to go. It postpone the complexity of lazy loading and we have a way to keep the entry point depending on |
|
I expect that skills installation could be triggered by the agent itself in non interactive terminal. Maybe we can have simpler and faster CLI and only trigger TUI when it's directly needed? |
You can do that by passing the option directly. |
Yep I need to revisit the command that I added the TUI yesterday to be sure that we have the non-interactive one. |
|
Starting a new repo: https://github.com/probabl-ai/skore-cli |
|
OK so one remaining question that I would have is the name of the CLI where I don't know if I need to use If using skore skills install
uvx --from skore-cli skore skills install
pixi exec --spec skore-cli skore skills installor with skore-cli skills install
uvx skore-cli skills install
pixi exec skore-cli skills install |
|
I think it should be simple “skore skills install” |
Implement an alternative to
npx skills addwith Python as first citizen.Creating interactive app using
textualand making that the cli usesclick,richandrich-click.Interactive app
The following two commands are instantiating a textual app:
skore skills installskore skills findTo guide the user to find the right information.
Those commands can be used in non-interactive mode when passing flags that corresponds to the choices in the interactive app.
Other commands
skore skills removeskore skills updateskore skills listare three other command to manage the skills.
Things to consider
The commands are a bit slow and I assume that it is related to the import. I think that the lazy loading of module would be a good thing to try to improve the state.
Screen.Recording.2026-06-03.at.1.02.21.PM.mov