repomatic.metrics.store module

Accumulate what forges say about a set of repositories, one reading at a time.

Every reading is one row of one table: which repository, which metric, on which date, what it said, and where the figure came from. A new metric is a Metric entry and one line in the forge reader, not a new file, a new schema or a new command.

Note

How long a reading is kept is a property of the metric, not of the caller.

A counter accrues: its whole point is the curve, so every dated reading is kept and charted. A star count is the one that motivated all this.

An attribute does not: the date of a project’s newest commit is a fact about today, nothing reads it chronologically, and a hundred subjects sampled weekly would pile up thousands of rows a year that no page ever opens. Only the newest reading is kept, dated when the value last moved, so a quiet week leaves the file untouched rather than restamping every row.

Retention is where that choice lives, and MetricStore.upsert() is the only code that has to know about it.

Note

The star history replaces the third-party charts a project used to embed. On 2026-06-30 GitHub restricted the REST stargazer endpoints to a repository’s own admins and collaborators, and closed the equivalent GraphQL field on 2026-07-17, which left every such embed on the web rendering an error card.

GitHub reopened the aggregate half on 2026-09-04, as a star-history endpoint reporting counts per day without naming a single account. It needs no token and answers for any public repository, so a curve no longer depends on who holds the credentials. Sampling still runs on a schedule, because a history that accrues in the repository cannot be revoked upstream.

Warning

A reconstruction and a sample do not measure the same thing, and the difference is deliberate rather than a defect.

GitHub builds the star history from the accounts that still have the repository starred, so a reconstruction attributes today’s surviving stars to the dates they were given: it understates every past date by the number of stars since withdrawn, converging on the true figure at the present day. Kept on purpose, since a curve that sags where a project shed followers carries a signal a monotonic one hides. Each row therefore names its Source, so a reader can always tell which question a point answers.

repomatic.metrics.store.METRIC_HEADERS = ('repo', 'metric', 'date', 'value', 'source')

Columns of the committed store, in file order.

The three key columns first, then the payload, so the file reads top to bottom as one repository at a time, one metric at a time, chronologically. That is also the sort order, which is what makes a scheduled commit an append per subject rather than a reshuffle.

repomatic.metrics.store.PREDECESSOR_SUFFIX = ':prior'

Marks a predecessor’s series key, appended to the subject it belongs to.

Keeps the configured subject list exactly the curves a chart plots, while still letting the collectors and the renderer address the extra one through the same code paths.

repomatic.metrics.store.RecordKey

Identity of one reading: its subject’s URL, its metric and its day.

alias of tuple[str, str, str]

repomatic.metrics.store.SAMPLE_HEADER_DEFS: tuple[ColumnSpec, ...] = (ColumnSpec(id='subject', label='Subject', description='', max_width=None, optional=False, sortable=True), ColumnSpec(id='phase', label='Phase', description='', max_width=None, optional=False, sortable=True), ColumnSpec(id='repository', label='Repository', description='', max_width=None, optional=False, sortable=True), ColumnSpec(id='stars', label='Stars', description='', max_width=None, optional=False, sortable=True), ColumnSpec(id='rows', label='Rows', description='', max_width=None, optional=False, sortable=True), ColumnSpec(id='note', label='Note', description='', max_width=None, optional=False, sortable=True))

Column definitions for the repomatic sample-metrics table.

Lives beside SampleOutcome.row(), so the columns and the fields they render cannot drift apart; the CLI derives its --sort-by choices from it.

repomatic.metrics.store.STAR_HISTORY_MAX_PAGES = 100

Pages of star history GitHub serves before refusing with a 422.

At thirty weeks a page this reaches back about fifty-seven years, longer than GitHub has existed, so no repository can outrun it and the ceiling is a guard against a walk that never terminates rather than a limit on coverage.

class repomatic.metrics.store.Source(*values)

Bases: StrEnum

Where a reading came from, recorded on every row.

A chart may mix methodologies it cannot reconcile, so it records which one each point came from rather than presenting a uniform curve it cannot honestly claim.

STAR_HISTORY and WAYBACK are retired: nothing writes them since GitHub reopened a public star history. They stay because a store populated before that still names them, and a reading already in the file is data, not a collector that has to keep existing.

CREATED = 'created'

Repository creation, the one date a star count is known to be 0.

Not a measurement but a fact. A repository whose earliest reading already shows a count, like one only ever sampled forward, has no knowable first star, so its curve would otherwise begin in mid-air. The row is also what a by-age chart aligns on, though a repository starred on its creation day keeps no anchor: that day’s measurement outranks it and takes its place, per rank.

GITHUB = 'github'

Reconstructed from GitHub’s star history, surviving stars only.

SAMPLE = 'sample'

Read from the forge’s own API, contemporaneous.

STAR_HISTORY = 'star-history'

Count at a date, imported from a star-history.com export. Retired.

WAYBACK = 'wayback'

Contemporaneous count mined from an archived GitHub page. Retired.

property rank: int

How authoritative this provenance is, to settle two readings of a day.

An exact reconstruction supersedes a mined or imported count; a contemporaneous sample supersedes those two, since it was taken by this collector against the live API. A weaker reading never overwrites a stronger one of the same day, so a collector run against an already-populated store degrades nothing.

CREATED ranks under everything, because it is the weakest claim in the vocabulary rather than the strongest: it asserts a count of zero from the fact that a repository cannot be starred before it exists, which stops being true on the creation day itself. A repository starred within hours of being published has a real reading for that day, and any measurement of it beats the assumption.

The store holds one row per day, so that measurement replaces the anchor instead of sitting beside it, and the anchor does not come back: every later sample re-asserts it and this same rule refuses it. The creation day survives, since the reading that displaced the anchor carries its date. The zero does not, and with it goes the marker saying that a series’ first point is day zero rather than merely its first measurement.

Todo

Carry the creation day as a non-accruing attribute beside commit and release, so no star reading can collide with it, and let repomatic.metrics.chart.render_chart() read a by-age origin from it. This is a store schema change: it adds a metric value that every consumer’s own store-shape test enumerates, so it lands with a note to downstream repositories rather than silently.

class repomatic.metrics.store.SamplePhase(*values)

Bases: StrEnum

The collector that produced a SampleOutcome.

The CLI reports one row per subject per collector, and the columns mean different things in each, so the row names the collector whose meaning it carries.

FORWARD = 'forward'

sample_subject(): today’s readings, from the subject’s own forge.

RECONSTRUCT = 'reconstruct'

reconstruct_from_github(): a whole star curve, rebuilt.

class repomatic.metrics.store.Retention(*values)

Bases: Enum

How long the store keeps a metric’s readings.

HISTORY = 1

Every dated reading, forever. For a counter, whose curve is the point.

LATEST = 2

Only the newest reading, dated when the value last moved.

For an attribute, which describes today rather than accruing. Nothing reads it chronologically, and keeping every sample would bury the file in rows restating what the previous one already said.

class repomatic.metrics.store.Metric(id, retention, label)

Bases: object

One thing a forge can be asked about a repository.

id: str

Value of the store’s metric column, and the name a chart selects on.

retention: Retention

Which of Retention governs this metric’s rows.

label: str

Human-readable name, for a rendered table or a chart axis.

property accrues: bool

Whether this metric’s past readings are kept and can be charted.

repomatic.metrics.store.METRICS: tuple[Metric, ...] = (Metric(id='commit', retention=<Retention.LATEST: 2>, label='Last commit'), Metric(id='release', retention=<Retention.LATEST: 2>, label='Last release'), Metric(id='release_source', retention=<Retention.LATEST: 2>, label='Release kind'), Metric(id='stars', retention=<Retention.HISTORY: 1>, label='Stars'))

Every metric the sampler collects, sorted by ID.

The extension point: a new counter is one entry here plus one yield in readings(). Nothing else changes, because the store, the retention rule and the chart all read this registry.

repomatic.metrics.store.METRICS_BY_ID: dict[str, Metric] = {'commit': Metric(id='commit', retention=<Retention.LATEST: 2>, label='Last commit'), 'release': Metric(id='release', retention=<Retention.LATEST: 2>, label='Last release'), 'release_source': Metric(id='release_source', retention=<Retention.LATEST: 2>, label='Release kind'), 'stars': Metric(id='stars', retention=<Retention.HISTORY: 1>, label='Stars')}

Index for O(1) metric lookup by ID.

repomatic.metrics.store.CHARTABLE_METRICS: tuple[str, ...] = ('stars',)

Metrics a chart can plot, since only an accruing one has a curve.

repomatic.metrics.store.chartable_metric(metric_id)

Look up a metric a chart can plot.

Parameters:

metric_id (str) – The metric a chart or a series names.

Return type:

Metric

Returns:

The registered metric.

Raises:

ValueError – When metric_id names no metric, or one that does not accrue and so has no curve to plot.

class repomatic.metrics.store.MetricRecord(repo, metric, day, value, source)

Bases: object

One reading: what a forge said about one repository on one date.

repo: str

Canonical https://host/owner/name URL of the subject.

metric: str

Which METRICS entry this reading is of.

day: str

The reading’s date, in YYYY-MM-DD form.

For an accruing metric, when the reading was taken. For an attribute, when its value last changed.

value: str

What the forge answered, as text.

CSV carries no types, so a consumer wanting a number coerces it. The store keeps the forge’s own answer rather than a parsed one, since a metric added later may not be numeric at all.

source: Source

Where the figure came from.

property key: tuple[str, str, str]

Deduplication identity: one reading per subject, metric and day.

property subject_key: tuple[str, str]

What an attribute keeps only one row of.

property count: int

The reading as an integer, for a counter metric.

Raises:

ValueError – When the value is not a number, which means a chart was pointed at an attribute.

as_row()

Flatten to one CSV row, in METRIC_HEADERS order.

Return type:

tuple[str, ...]

classmethod from_row(row)

Rebuild a record from one parsed CSV row.

Parameters:

row (Mapping[str, str]) – The row, keyed by column name.

Return type:

MetricRecord

Returns:

The corresponding record.

Raises:

• KeyError – When a column is missing.

• ValueError – When the source is not a Source.

class repomatic.metrics.store.MetricStore(records=())

Bases: object

Every reading of one store, keyed by subject, metric and day.

Indexes the readings by subject and metric once, when built, so recording a reading looks up the rows it may replace rather than scanning the whole store.

records: dict[tuple[str, str, str], MetricRecord]

Every reading, keyed by MetricRecord.key.

Read it freely. Change it through upsert() and discard() only, which keep the index in step.

classmethod load(path)

Read the committed store.

Parameters:

path (Path) – Path to the CSV store.

Return type:

MetricStore

Returns:

The store, empty when the file does not exist.

Raises:

ValueError – When the file exists but cannot be parsed. Loud on purpose: a corrupt store must never be silently clobbered by the next save().

save(path)

Write the store back, sorted by subject, metric and date.

The file becomes exactly what the store holds, so a row dropped in memory is dropped on disk. That is what lets a collector re-derive a whole curve: a reconstruction that no longer dates a reading to a given week has to be able to retire that week’s row, and a week left behind would state a total the rest of the curve contradicts.

Caution

A caller must therefore save a store it loaded from this same file. One assembled from nothing truncates every reading it did not collect, which is every collector’s own habit here and worth keeping.

Parameters:

path (Path) – Path to the CSV store.

Return type:

bool

Returns:

True when the file content changed.

held(repo, metric)

Every reading of one metric of one subject, oldest first.

Parameters:

• repo (str) – The subject’s canonical URL.

• metric (str) – The metric ID.

Return type:

list[MetricRecord]

discard(record)

Drop the reading stored under the key of record.

Return type:

None

upsert(record)

Record one reading, returning whether it changed anything.

Re-running on the same day overwrites rather than appends, which is what keeps the scheduled job idempotent. Beyond that the metric’s Retention decides:

• An accruing metric keeps every day, and a more authoritative source wins over a weaker one for the same day, per Source.rank.

• An attribute keeps one row. An unchanged value leaves the stored date alone, so a quiet week rewrites nothing; a moved value replaces the row and takes the new date, which is therefore when the value last changed rather than when it was last confirmed.

Parameters:

record (MetricRecord) – The reading to store.

Return type:

bool

Returns:

True when the store moved.

Raises:

KeyError – When the metric is not in METRICS_BY_ID.

class repomatic.metrics.store.SampleOutcome(subject, repo, phase, stars=None, rows=0, note='')

Bases: object

What one subject’s sample produced, for the CLI to report.

subject: str

Name the repository gives this subject.

repo: str

Canonical URL it read from.

phase: SamplePhase

The collector that produced this outcome.

stars: int | None = None

Its current star count, when the collector read one.

rows: int = 0

How many stored rows this collector added or moved.

note: str = ''

Why nothing was collected, empty when something was.

row()

Render the outcome as one row of SAMPLE_HEADER_DEFS.

Return type:

tuple[str, ...]

repomatic.metrics.store.predecessor_key(name)

Name the series of the forerunner of subject name.

Return type:

str

repomatic.metrics.store.is_predecessor(key)

Whether series key is a forerunner’s, rather than a subject’s own.

Return type:

bool

repomatic.metrics.store.subject_of(key)

Name the subject series key belongs to.

Return type:

str

Returns:

key itself for a subject’s own series, or the subject a forerunner precedes.

repomatic.metrics.store.collected_subjects(subjects, predecessors=None)

Every repository a collector touches, keyed by its subject name.

Parameters:

• subjects (Mapping[str, str]) – Tracked subjects, mapping each name to a slug or URL.

• predecessors (Mapping[str, str] | None) – Retired forerunners, mapping the name of the subject they belong to onto their own slug or URL.

Return type:

dict[str, str]

Returns:

The subjects, plus one entry per forerunner whose key carries PREDECESSOR_SUFFIX so a caller can tell the two apart. Every value is a canonical URL.

Raises:

ValueError – When a declared subject parses as neither a slug nor a URL.

repomatic.metrics.store.sample_subject(store, subject, repo, extra_forges=None, day=None)

Read every metric of one subject, through whichever forge hosts it.

The collector every run applies to every subject, and the only one that reads a repository outside GitHub.

Parameters:

• store (MetricStore) – The in-memory store, changed in place.

• subject (str) – Name the repository gives this subject.

• repo (str) – Its canonical URL.

• extra_forges (Mapping[str, str] | None) – Host-to-forge entries for self-hosted instances.

• day (str | None) – Reading date in YYYY-MM-DD form. Today (UTC) when None.

Return type:

SampleOutcome

Returns:

What the sample produced.

repomatic.metrics.store.reconstruct_from_github(store, subject, repo)

Rebuild one repository’s star curve from GitHub’s star history.

Reads the aggregate endpoint GitHub opened on 2026-09-04, which reports how many stars a repository gained on each day without naming who gave them. Anonymous and public, so this reaches every subject a project tracks rather than only the ones a token administers.

Note

The endpoint pages by week, not by star, so its cost follows a repository’s age and not its popularity: a ten-year repository costs the same eighteen requests whether it holds six hundred stars or thirty thousand.

One reading is kept per week that gained a star, dated on the last such day of that week. The endpoint resolves to the day, but the store deliberately does not: a cumulative curve counts the stars a repository still holds, so one withdrawal in 2019 lowers every later point, and at daily resolution a single unstar rewrites thousands of committed rows. A week is also the cadence the job samples at and the unit the charts plot over years.

Caution

GitHub buckets the days in its own timezone, America/Los_Angeles, and honours daylight saving. Measured against 612 exact star timestamps, that zone puts every one of them in the bucket the endpoint reported, where reading the days as UTC misplaces about a fifth of them by one day.

Honouring it needs no conversion: a week’s week field is midnight UTC on the Sunday labelling that Los Angeles week, so the calendar date of that instant plus a day’s offset is already the local day. Converting the timestamp into a zone would reintroduce the error.

Pagination is all-or-nothing on purpose, and more sharply than it looks. The walk runs newest first, so a run abandoned halfway holds only the recent weeks: totalling those would date a fraction of the stars as if it were the whole history, and every point of the resulting curve would look exactly as legitimate as the rest.

Parameters:

• store (MetricStore) – The in-memory store, changed in place once the whole walk succeeded.

• subject (str) – Name the repository gives this subject.

• repo (str) – Its canonical URL.

Return type:

SampleOutcome

Returns:

What the reconstruction produced.

repomatic.metrics.store.series(store, subjects, metric='stars', predecessors=None)

Group one metric’s readings into a chronological series per subject.

Parameters:

• store (MetricStore) – The store.

• subjects (Mapping[str, str]) – Tracked subjects, mapping each name to a slug or URL.

• metric (str) – Which accruing metric to plot.

• predecessors (Mapping[str, str] | None) – Retired forerunners, keyed by the subject they precede.

Return type:

dict[str, list[tuple[date, int]]]

Returns:

One sorted list of (day, value) per subject that has any reading, forerunners under their PREDECESSOR_SUFFIX key.

Raises:

ValueError – When metric does not accrue, so has no curve to plot.