docs: add API reference for CollectorRegistry and custom collector classes

[k1chik]

Closes #1163

Follows the same pattern as #1021 and #1162.

collector/custom.md: Collector protocol section (collect/describe), value vs labels mutual exclusivity note with correct per-type parameter names, full constructor and add_metric tables for GaugeMetricFamily, CounterMetricFamily, SummaryMetricFamily, HistogramMetricFamily, and InfoMetricFamily, plus a runnable real-world example.

collector/_index.md: constructor parameter tables for ProcessCollector, PlatformCollector, and GCCollector, with exported metrics listed for each.

registry/_index.md (new page): CollectorRegistry constructor and all public methods (register, unregister, collect, restricted_registry, get_sample_value, set_target_info, get_target_info), the global REGISTRY instance, and examples for isolated registry usage and registry=None.

All code examples verified by running them in Python.

cc @csmarchbanks

[csmarchbanks]

Thanks!

[calestyo]

Hey.

Thanks for improving on the documentation... that really helps a lot, compared to few years ago when I had to basically fiddle out most of these things from the code :-)

But one thing:

Why the yields in the collect()? I don’t quite get that.

Shouldn't collect() return sequences of *MetricFamily objects? You now return only a single one, and there even alternating ones like in:
a708480#diff-eadfec94068584be6fec5cce354e065ead34a754bc84ead564adf9e8c497ab10R237-R243

Or maybe you mean this as two independent examples?
What makes it confusing though is (which is however not from your commit):

client_python/docs/content/collector/custom.md

Lines 16 to 21 in 482656c

	def collect(self):
	yield GaugeMetricFamily('my_gauge', 'Help text', value=7)
	c = CounterMetricFamily('my_counter_total', 'Help text', labels=['foo'])
	c.add_metric(['bar'], 1.7)
	c.add_metric(['baz'], 3.8)
	yield c

There collect() alternates between returning (well yielding) a Gauge and Counter (also not as sequences)?

I generally don't quite understand why one would yield here at all.

Isn't the reason for custom collectors to start every time fresh (and get rid of all stale metrics, labels, etc.)?
With yield wouldn't you keep all that?

Thanks,
Chris.

[calestyo]

Shouldn't do stuff late night... yield of course makes it an iterator o.O

Still I wonder, is there any advantage over using that in the current form rather than return?

I mean is Python' GC smart enough that it can already clean the objects that have already been yielded and thus give an advantage by reducing e.g. memory use for very large metric sets?

In:

        yield GaugeMetricFamily('my_gauge', 'Help text', value=7)
        c = CounterMetricFamily('my_counter_total', 'Help text', labels=['foo'])
        c.add_metric(['bar'], 1.7)
        c.add_metric(['baz'], 3.8)
        yield c

I could imagine that it may release the GaugeMetricFamily object early... but c? It still has a reference?

In any case it would be nice to have in the docs why generators are used (if it's memory advantages) and how to use them so that this actually works out. :-)

[k1chik]

Hi @calestyo, thanks for the read! glad to see you've found the docs useful.

And yes, my bad, the code examples are missing the def collect(self): wrapper which makes it unclear what context they're meant to be in, i'll fix that asap.

Your GC read is right too, yield GaugeMetricFamily(...) inline like that can be reclaimed as soon as the caller moves on, but c keeps a reference in the generator frame until it's done. The main reason for yield here isn't memory though, it's just that the registry iterates over whatever collect() returns and a generator is cleaner than building a list. For most collectors you'd never notice the difference anyway.

I'll open a follow-up PR with both fixes and add a note explaining why generators are used. Thanks again for your feedback.

Shouldn't do stuff late night... - :-)

[calestyo]

Hmm, but if you say memory isn't the main reason, the real one should perhaps be given?

And if memory is at least part of the reason, then it might have made sense to explicitly mention that and also explain to people that ideally they should re-use the same identifier so that the GC can kick in.

Even perhaps if they have a case where it's reasonable to first collect a few metrics and only yield all of them afterwards, like in:

for loopOver in SomeJSONelements:
    metric[0].add_metric()
    metric[1].add_metric()
for m in metric:
    yield m

#... next metric again uses `metric` var

I.e. telling that it's still good to use only one identifier metric here, so that the GC can work

[k1chik]

Hi Chris, thanks for the response. Good point on both counts! I'll add a note that memory is at least part of the reason for using yield, and mention that reusing the same identifier helps the GC kick in sooner. PR coming shortly :)