Skip to content

feat(skore): Introduce CLI for installing skills#2976

Draft
glemaitre wants to merge 19 commits into
probabl-ai:mainfrom
glemaitre:cli_skills
Draft

feat(skore): Introduce CLI for installing skills#2976
glemaitre wants to merge 19 commits into
probabl-ai:mainfrom
glemaitre:cli_skills

Conversation

@glemaitre

@glemaitre glemaitre commented Jun 3, 2026

Copy link
Copy Markdown
Member

Implement an alternative to npx skills add with Python as first citizen.

Creating interactive app using textual and making that the cli uses click, rich and rich-click.

Interactive app

The following two commands are instantiating a textual app:

  • skore skills install
  • skore skills find

To guide the user to find the right information.

Those commands can be used in non-interactive mode when passing flags that corresponds to the choices in the interactive app.

Other commands

  • skore skills remove
  • skore skills update
  • skore skills list

are three other command to manage the skills.

Things to consider

The commands are a bit slow and I assume that it is related to the import. I think that the lazy loading of module would be a good thing to try to improve the state.

Screen.Recording.2026-06-03.at.1.02.21.PM.mov

@glemaitre glemaitre marked this pull request as draft June 3, 2026 10:18
@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Documentation preview @ 2c44731

@glemaitre glemaitre marked this pull request as ready for review June 3, 2026 11:06
@github-actions

github-actions Bot commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Coverage

Coverage Report for skore/
FileStmtsMissBranchBrPartCoverMissing
skore/src/skore
   __init__.py4022095%118–119
   _config.py58312194%70, 117–118
   exceptions.py44000%4, 15, 19, 23
skore/src/skore/_cli
   __init__.py8000100% 
   __main__.py11000%3
   _style.py18000100% 
skore/src/skore/_cli/skills
   __init__.py2000100% 
   _agents.py25040100% 
   _catalog.py33000100% 
   _commands.py276111161196%116, 169–170, 226, 297, 528–529, 532–535
skore/src/skore/_cli/skills/app
   __init__.py6000100% 
   _find.py30020100% 
   _install.py94426795%123, 128, 170–171
   _manage.py36212241%47–50, 53–57, 61–63, 66, 69–71, 75–77, 80–81
   _menu.py28130053%55–56, 59–62, 66, 73, 76, 79–80, 83–84
   _widgets.py74222397%33–34
skore/src/skore/_plugins
   __init__.py12000100% 
   serde.py32080100% 
skore/src/skore/_plugins/hub
   __init__.py920077%15, 20
   exception.py2000100% 
   json.py1012190%16
skore/src/skore/_plugins/hub/artifact
   __init__.py0000100% 
   artifact.py23040100% 
   serializer.py27020100% 
   upload.py26040100% 
skore/src/skore/_plugins/hub/artifact/media
   __init__.py5000100% 
   data.py20000100% 
   inspection.py721020886%46–49, 51, 53, 60, 62, 68, 107
   media.py10000100% 
   model.py10000100% 
   performance.py59060100% 
skore/src/skore/_plugins/hub/artifact/pickle
   __init__.py2000100% 
   pickle.py24020100% 
skore/src/skore/_plugins/hub/authentication
   __init__.py0000100% 
   apikey.py7000100% 
   login.py2844285%37, 42–43, 52
   token.py80080100% 
   uri.py6000100% 
skore/src/skore/_plugins/hub/client
   __init__.py0000100% 
   client.py881018388%140, 187–189, 191–192, 194, 196, 198, 230
skore/src/skore/_plugins/hub/metric
   __init__.py10000100% 
   accuracy.py35000100% 
   brier_score.py35000100% 
   log_loss.py35000100% 
   metric.py5542192%38, 77–78, 84
   precision.py57000100% 
   r2.py35000100% 
   recall.py59000100% 
   rmse.py35000100% 
   roc_auc.py35000100% 
   timing.py7640094%45–46, 104–105
skore/src/skore/_plugins/hub/project
   __init__.py0000100% 
   project.py138626595%84, 109, 124, 323, 403, 433
skore/src/skore/_plugins/hub/report
   __init__.py3000100% 
   cross_validation_report.py121228398%208, 248
   estimator_report.py10000100% 
   report.py60060100% 
skore/src/skore/_plugins/local
   __init__.py2000100% 
   metadata.py8138196%29, 141–142
   project.py93130198%238
   storage.py4226195%45, 189
skore/src/skore/_plugins/mlflow
   __init__.py5000100% 
   project.py2592662989%47, 49–51, 117–121, 226, 244, 262–263, 358, 466, 468, 484, 486–491, 493–495
   reports.py155634496%129, 170, 207–208, 270, 283
skore/src/skore/_project
   __init__.py0000100% 
   _summary.py80138398%146
   _widget.py1910442100% 
   dependencies.py19060100% 
   git.py25040100% 
   login.py1736282%73, 82–83
   plugin.py9000100% 
   project.py56216396%136, 145
   types.py3000100% 
skore/src/skore/_sklearn
   __init__.py8000100% 
   _base.py56112098%47
   compare.py5000100% 
   evaluate.py450260100% 
   feature_names.py260120100% 
   find_ml_task.py610462100% 
   metrics.py3640860100% 
   types.py1910094%31
skore/src/skore/_sklearn/_checks
   __init__.py3000100% 
   _utils.py75328396%83, 217, 225
   accessor.py33114096%17
   base.py77424294%128–129, 266–267
   model_checks.py39710130997%377, 385, 435, 439, 576, 653–654, 748, 767, 854
   tunable_hyperparameters.py5000100% 
skore/src/skore/_sklearn/_comparison
   __init__.py9000100% 
   inspection_accessor.py2712096%360
   metrics_accessor.py125418496%255–256, 325, 1181
   report.py160568096%576, 582, 640–642
skore/src/skore/_sklearn/_cross_validation
   __init__.py11000100% 
   data_accessor.py36212294%48, 74
   inspection_accessor.py2712096%332
   metrics_accessor.py119316397%215–216, 1148
   report.py204948595%74, 79, 84, 340, 664, 670, 756–758
skore/src/skore/_sklearn/_estimator
   __init__.py11000100% 
   data_accessor.py48220195%61, 178
   inspection_accessor.py4418297%297
   metrics_accessor.py1240220100% 
   report.py361181061495%67, 81, 310, 383, 470, 725, 814, 935–937, 1019, 1021, 1023, 1025, 1028–1029, 1033, 1036
skore/src/skore/_sklearn/_plot
   __init__.py3000100% 
   base.py61214196%61–62
   utils.py151370398%254–255, 428
skore/src/skore/_sklearn/_plot/data
   __init__.py2000100% 
   table_report.py177160199%676
skore/src/skore/_sklearn/_plot/inspection
   __init__.py0000100% 
   calibration_curve.py821114886%123–126, 128, 133–134, 240, 243–244, 274
   coefficients.py1810881100% 
   impurity_decrease.py103234398%441, 485
   permutation_importance.py198190199%650
   utils.py320100100% 
skore/src/skore/_sklearn/_plot/metrics
   __init__.py6000100% 
   confusion_matrix.py2000662100% 
   metrics_summary_display.py136162299%244
   precision_recall_curve.py1170321100% 
   prediction_error.py1660542100% 
   roc_curve.py1210342100% 
skore/src/skore/_sklearn/train_test_split
   __init__.py2000100% 
   train_test_split.py710342100% 
skore/src/skore/_sklearn/train_test_split/warning
   __init__.py8000100% 
   high_class_imbalance_too_few_examples_warning.py19060100% 
   high_class_imbalance_warning.py20060100% 
   random_state_unset_warning.py10020100% 
   shuffle_true_warning.py9020100% 
   stratify_is_set_warning.py10020100% 
   time_based_column_warning.py21040100% 
   train_test_split_warning.py3000100% 
skore/src/skore/_utils
   __init__.py620066%8, 13
   _accessor.py10620301181%13, 36, 61–65, 68, 70–71, 76, 81, 83, 85, 92–94, 164, 216, 236
   _cache.py37021100% 
   _cache_key.py35522585%22, 24, 51, 59, 68
   _callable_name.py9040100% 
   _dataframe.py43418490%27, 46, 48, 63
   _environment.py33110296%49
   _fixes.py8020100% 
   _index.py5020100% 
   _jupyter.py820075%13–14
   _measure_time.py10000100% 
   _parallel.py17000100% 
   _patch.py21128842%30, 35–39, 42–43, 46–47, 58, 60
   _progress_bar.py4244090%53–54, 64–65
   _show_versions.py66122198%26
   _skrub.py37040100% 
   _testing.py1281412289%24, 33, 71–72, 94, 193, 202, 213–218, 220
skore/src/skore/_utils/repr
   __init__.py2000100% 
   base.py54040100% 
   data.py1280301100% 
   html_repr.py40000100% 
   rich_repr.py800223100% 
   utils.py11020100% 
TOTAL8436300216218796% 

Tests Skipped Failures Errors Time
2742 5 💤 0 ❌ 0 🔥 9m 13s ⏱️

@glemaitre glemaitre marked this pull request as draft June 3, 2026 12:33
@glemaitre glemaitre marked this pull request as ready for review June 3, 2026 14:32

@rouk1 rouk1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tested that a bit and I have a few feedbacks.

  • launching the CLI is a bit slow
  • launching with no parameter does not trigger an interactive option UX (so you need to relaunch the script and it is slow so it is a bit frustrating)
  • when selection skills pressing tab is hard to discover (bold text is not readable enough to me)
  • when selecting agent and scope you must press two keys to select what you want (using the arrow should select the highlighted choice no ?)

IMO all this could have been done with a multistep non fullscreen questionary like experience.

  1. chose action (list, find, ... plus exit)
  2. chose action first choice (if only one skip) (for install select among workflows + cancel)
  3. continue through list of questions or go back
  4. final step may have a progress bar

@glemaitre

Copy link
Copy Markdown
Member Author

IMO all this could have been done with a multistep non fullscreen questionary like experience.

I actually try questionary, beaupyand InquirierPy and I was not happy with some styling issues (selection or background) for each of them and I ending up doing the textual app because it was where I can control the flow.

@rouk1 I think that I can correct all your point easily apart of the speed to launch. For me this one could be done in a later step with lazy import or doing a separate package with only the cli (not sure that it is a good idea).

@glemaitre

Copy link
Copy Markdown
Member Author

The following PR (#2988) will decrease the import time from 2.5s to 0.15s with the lazy loading mechanism.

@jeromedockes

Copy link
Copy Markdown
Collaborator
  • the skills do not seem to take a huge amount of disk space, so is there a downside to just downloading everything / do we need a TUI vs just a command like probabl-skills path/to/project or something similar?
  • it seems quite independent from the rest of skore so should it be in the same package?
  • as the cli does not seem to import the rest of skore maybe the lazy-loading thing could be done only in the top-level init and not subpackages

@thomass-dev

thomass-dev commented Jun 5, 2026

Copy link
Copy Markdown
Collaborator
  • it seems quite independent from the rest of skore so should it be in the same package?

Discussed IRL with @glemaitre , and given its independence from skore, i also think it's a good idea to host the CLI in its own repository.

By the way, great job 👏🏼 .

@glemaitre

Copy link
Copy Markdown
Member Author

the skills do not seem to take a huge amount of disk space, so is there a downside to just downloading everything / do we need a TUI vs just a command like probabl-skills path/to/project or something similar?

I would say there is two points that bring me towards this complex proposal:

  • others are doing this UX
  • the location of the skills (and later whatever will come) is agent dependent sometimes

it seems quite independent from the rest of skore so should it be in the same package?
as the cli does not seem to import the rest of skore maybe the lazy-loading thing could be done only in the top-level init and not subpackages

Iterating with several persons then I think that the subpackage is actually the way to go. It postpone the complexity of lazy loading and we have a way to keep the entry point depending on skore to live in the skore package, and the others orthogonal thing to live in a separate package with the dependencies that are only required for the CLI.

@Aljutor

Aljutor commented Jun 5, 2026

Copy link
Copy Markdown

I expect that skills installation could be triggered by the agent itself in non interactive terminal.

Maybe we can have simpler and faster CLI and only trigger TUI when it's directly needed?
Also I haven't found a way to list skills to install without TUI.

@glemaitre

Copy link
Copy Markdown
Member Author

I expect that skills installation could be triggered by the agent itself in non interactive terminal.

You can do that by passing the option directly.

@glemaitre

Copy link
Copy Markdown
Member Author

Also I haven't found a way to list skills to install without TUI.

Yep I need to revisit the command that I added the TUI yesterday to be sure that we have the non-interactive one.

@glemaitre glemaitre marked this pull request as draft June 5, 2026 16:21
@glemaitre

Copy link
Copy Markdown
Member Author

Starting a new repo: https://github.com/probabl-ai/skore-cli

@glemaitre

Copy link
Copy Markdown
Member Author

OK so one remaining question that I would have is the name of the CLI where I don't know if I need to use skore or skore-cli.

If using skore, we will have:

skore skills install
uvx --from skore-cli skore skills install
pixi exec --spec skore-cli skore skills install

or with skore-cli:

skore-cli skills install
uvx skore-cli skills install
pixi exec skore-cli skills install

@Aljutor

Aljutor commented Jun 5, 2026

Copy link
Copy Markdown

I think it should be simple “skore skills install”

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants