Usage metrics

Acme's platform team wants to know how often engineers get the skills in acme-skills, and how often their syncs fail. SkillCatalog can report that as usage metrics: anonymous counts of what the desktop app and skc do, such as which commands ran, whether they succeeded, and which of the catalog's skills were delivered.

A catalog opts in when its catalog.yaml names a metrics address. That is the address of a server, called a receiver, that the catalog's owner runs to collect the counts. Everyone who has the catalog then sends counts to that address for as long as their usage metrics are on. Usage metrics are on by default, so SkillCatalog does not ask first. Instead, it tells you when you add a catalog that has an address.

SkillCatalog has no receiver of its own. When none of your catalogs names a metrics address, nothing is sent.

The next three sections are for everyone who uses catalogs. The sections after them are for a catalog's owner, who sets up the address and the receiver.

See which catalogs collect usage counts

When you add a catalog that names a metrics address, SkillCatalog says so right after the success message, unless you pass --quiet. With --json, the address is in data.metrics_url instead. Sam adds acme-skills on his new laptop:

skc catalog add git@github.com:acme/acme-skills.git

Added catalog 'Acme Skills' (acme-skills). Clone: ~/.skillcatalog/catalogs/acme-skills
Catalog 'acme-skills' collects anonymous usage counts at http://metrics.acme.internal:9090/api/v1/otlp/v1/metrics. To stop sending them, run `skc settings telemetry --disable`.

skc install and skc update print the same line for each catalog they add for you. The desktop app shows a similar notice when you add a catalog there, including from a link. While usage metrics are off, nothing is sent, so no notice appears.

SkillCatalog shows this message only at the moment it adds a catalog, so a catalog you already have can gain an address later without telling you. To see every catalog that collects counts right now, list them:

skc settings telemetry

Usage metrics: on
Catalogs with a metrics address:
- acme-skills: http://metrics.acme.internal:9090/api/v1/otlp/v1/metrics

In the desktop app, open Settings. The Usage metrics section lists the same catalogs under Catalogs with a metrics address.

A catalog's owner decides where its counts go, and anyone who can push to the catalog can change the address. Add catalogs only from people you trust.

What a metrics address receives

A metrics address receives two kinds of counts. The first kind is about its own catalog: what each command did to the catalog and to its skills, stacks, and bundles, named by catalog id and slug. When Sam deletes his delivered copy of review-checklist and skc deliver restores it for Claude Code, Acme's address receives this count, written out as text:

skillcatalog.operations = 1
  operation: delivery
  action: repair
  outcome: succeeded
  target: claude_code
  source.catalog.id: acme-skills
  source.item.kind: skill
  source.item.id: review-checklist

The second kind is general: how often each skc and desktop command ran, whether it succeeded, and how long it took. These counts name no catalog, because one command, such as skc sync, can work on several. Every metrics address that your catalogs name receives them.

Each send also says whether it came from the desktop app or skc, and names the SkillCatalog version, the operating system, and the processor type. It carries a random id that changes each time the desktop app or skc starts, so a receiver can tell one run from another, but not one person or machine from another.

Nothing else is sent. The counts never contain the text of a skill, display names, file paths, repository URLs, error messages, tags, categories, owners, or your Git name and email, and they carry no id for your machine or your account. An address never receives counts about another catalog, and counts about a catalog without an address go nowhere. Like any server, a receiver does see the network address that each send comes from.

Stop sending usage counts

To stop sending counts to every address, turn usage metrics off:

skc settings telemetry --disable

Usage metrics: off
Catalogs with a metrics address:
- acme-skills: http://metrics.acme.internal:9090/api/v1/otlp/v1/metrics

SkillCatalog drops the counts it has not sent yet. The desktop app and skc share this switch, so turning it off in one turns it off in the other, even in a desktop app that is already running. skc settings telemetry --enable turns usage metrics back on.

In the desktop app, open Settings, then turn off Send usage metrics under Usage metrics.

Collect usage counts for your catalog

Dana owns acme-skills, and Acme's platform team already runs a receiver, set up as Collect metrics with Prometheus shows. To send the catalog's counts there, Dana adds the receiver's address to catalog.yaml in her clone of the catalog:

metrics_url: http://metrics.acme.internal:9090/api/v1/otlp/v1/metrics

The address starts with http:// or https:// and ends in /v1/metrics. File formats lists every rule for metrics_url. Dana then syncs, so the change reaches her teammates:

skc sync acme-skills --message "Collect usage counts"

Each machine reads the address from its own clone of the catalog. Dana's machine starts sending to the address as soon as she saves the file, and each teammate's machine starts once it pulls the change, for example at their next sync.

A new catalog can name its address from the start. When you add an empty repository with --metrics-url, SkillCatalog writes the address into the new catalog.yaml and pushes it with the other starter files. Acme's platform team did this when it created its own catalog, platform-skills, from a new, empty repository:

skc catalog add git@github.com:acme/platform-skills.git --metrics-url http://metrics.acme.internal:9090/api/v1/otlp/v1/metrics

In the desktop app, choose Add catalog in the catalog menu at the top of the sidebar, and also fill in Metrics address. Both work only for an empty repository. When the repository already has a catalog, SkillCatalog adds it unchanged and warns that it did not save the address, so set metrics_url in catalog.yaml as Dana did above.

Check a catalog's metrics address

SkillCatalog ignores a metrics address it cannot use, so that a mistake in the address never stops the people who use the catalog. The catalog keeps working, but no counts are sent for it. Dana first wrote the receiver's address without its path:

metrics_url: http://metrics.acme.internal:9090

skc validate --path checks her clone and reports the problem as a warning, which makes the command fail:

skc validate --path ~/.skillcatalog/catalogs/acme-skills

Error: validate
Selected checks: catalog, drift
Diagnostics: 1
- [catalog-integrity-warning] catalog '~/.skillcatalog/catalogs/acme-skills' catalog: metrics_url is not a valid metrics address, so SkillCatalog sends no usage counts for this catalog. Use an http:// or https:// address that ends in /v1/metrics, without a user name, password, query, or fragment.
  Path: ~/.skillcatalog/catalogs/acme-skills
  Hint: Fix the catalog content under '~/.skillcatalog/catalogs/acme-skills' and rerun `skc validate`.

Everyday commands such as skc status do not report this warning, so run skc validate --path whenever you change the address. After Dana fixes the line, the check passes, and she syncs the fix.

Collect metrics with Prometheus

Acme's platform team collects the counts in Prometheus. SkillCatalog sends them over HTTP in the OpenTelemetry format (OTLP), which Prometheus 3.14.0 accepts when you add these two flags to the command that starts it:

--web.enable-otlp-receiver
--enable-feature=otlp-native-delta-ingestion

The metrics address is then http://<prometheus-host>:9090/api/v1/otlp/v1/metrics, with your server's name in place of <prometheus-host>.

SkillCatalog sends no credentials or extra headers, follows no redirects, uses no proxy, and ignores every OTEL_EXPORTER_OTLP_ environment variable. The receiver cannot check who is sending, so run it where only your team's machines can reach it. An https:// address works when each computer trusts the receiver's certificate, including one from your company's own certificate authority.

Native delta ingestion is an experimental Prometheus feature, so check your queries again after you upgrade Prometheus.

SkillCatalog sends three metrics:

client is cli or desktop. command is the skc command, such as sync or profile.add, or the name of a desktop action. target is the tool: claude_code, cursor, or codex. A command's outcome is succeeded or failed, and an operation's can also be partially_succeeded, cancelled, or no_change.

operation is authoring, profile, installation, delivery, catalog, synchronization, validation, scoring, or improvement. For delivery, each count is one file written for one tool, and its action says what happened to the file: install for a new file, update for a changed one, repair for a restored one, remove for a removed one, and deliver for a file that could not be written.

Prometheus replaces the dots in names with underscores and adds _total to the two counts. You query skillcatalog_commands_total and skillcatalog_operations_total, with labels such as source_catalog_id and source_item_id.

Each send holds only the counts since the previous send, so Prometheus stores each send as a separate sample. Add the samples up over a time window with sum_over_time, because rate and increase expect running totals and give wrong results here. Each run of skc or the desktop app also sends its own random id, so add the results for all runs together with sum. This query counts the failed commands of the last day:

sum by (command) (
  sum_over_time(skillcatalog_commands_total{outcome="failed"}[1d])
)

This one counts the files delivered over the last week, for each skill in acme-skills:

sum by (source_item_id) (
  sum_over_time(skillcatalog_operations_total{operation="delivery", action="install", source_catalog_id="acme-skills"}[7d])
)

Limits to plan for

The desktop app and skc send new counts every 10 seconds while they run, and send the rest when they stop: skc when a command ends, and the desktop app when it quits.

SkillCatalog keeps counts only in memory and never retries a send. When your receiver cannot be reached, the counts in that send are lost, and each skc command waits up to about a second for each address that does not answer before it ends, and at most five seconds in all. Keep the receiver running, and remove metrics_url from catalog.yaml when you retire it.

One send holds at most 2,047 different label combinations per metric. A machine that delivers hundreds of skills to several tools at once can reach that limit. When a send would hold more, SkillCatalog drops the extra counts, and they never end up in another catalog's counts.

Sending never changes what a command does. Commands have the same effects and exit codes whether usage metrics are on or off, and whether or not the receiver answers.