ValueError: shapes not aligned

[DanieleMuraro]

Dear Vitalii,

Following private communication, I post this issue as a reminder of the following bug.

After training the regression model, I am getting the following error

ValueError: shapes (1000,20) and (21,12931) not aligned: 20 (dim 1) != 21 (dim 0)

when trying to plot the QC.

As you mentioned, the problem is that the QC is plotted for a random subset of cells and if not all cell types are represented (20/21) this error occurs.

The way to address this on user side is to increase the number of cells in the subset with the parameter use_n_obs (for example, if mod is the regression model, using mod.plot_QC(use_n_obs=5000)).

Thank you so much for your help.

All the best,

Daniele

[johnyaku]

This is still present in cell2location 0.1.5 / scvi-tools 1.3.3, and I think it's worth reopening — the use_n_obs workaround above is effective, but the default makes the failure the likely outcome rather than an edge case on references with many donors.

Root cause

RegressionBaseModule.compute_expected builds the covariate one-hot from the subsampled rows only (models/reference/_reference_module.py):

obs2extra_categoricals = np.concatenate(
    [
        pd.get_dummies(extra_categoricals.iloc[ind_x, i]).astype("float32")
        for i, n_cat in enumerate(self.n_extra_categoricals)
    ],
    axis=1,
)

Any level absent from ind_x silently loses a column, so the subsequent

mu = mu * np.dot(obs2extra_categoricals, samples["detection_tech_gene_tg"])

fails because detection_tech_gene_tg always has the full level count. Note that n_cat — exactly the number needed — is bound by the loop and never used.

Why the default is the problem, quantitatively

plot_QC(summary_name="means", use_n_obs=1000) selects observations with an unseeded np.random.choice. On a reference here with 101 donor_id levels + 2 assay levels (103 total) over 427,823 cells, where the rarest donor holds 516 cells (0.12%):

use_n_obs	E[levels missed]	P(plot_QC succeeds)
1000 (default)	1.17	28.5%
10000	~6e-6	~99.999%

So it fails roughly 71% of the time on this reference, and which annotation level fails changes between runs. Concretely, we saw ValueError: shapes (1000,102) and (103,17130) not aligned — the same signature as the original report, just with 103 levels instead of 21.

Because plot_QC is typically called right after export_posterior() and before mod.save(), that coin flip can discard a completed training run: for us, 2 h 37 m of V100 time lost to a diagnostic PDF.

Suggested fix

Pin the dummies to the known level count. The registry stores CAT_COVS_KEY as integer codes, so n_cat is directly usable:

pd.get_dummies(
    pd.Categorical(extra_categoricals.iloc[ind_x, i], categories=range(n_cat))
).astype("float32")

(I've only verified the use_n_obs workaround in practice, not this patch, so please treat the exact form as a suggestion.)

Two smaller things in the same block:

• obs2sample and obs2label use pd.get_dummies(...).values[ind_x, :] — one-hot over all rows, then index. Those are already correct, which is why only the extra-categoricals path breaks; worth making the three consistent.
• use_n_obs=None is documented as "use all observations" but leaves ind_x = None, which makes this block evaluate extra_categoricals.iloc[None, i] and .values[None, :]. So the obvious way to sidestep the bug doesn't work.

Happy to open a PR for the n_cat fix if that's useful.