← Backlog

Music Catalog Intelligence

Date Published:

Discovery

Tags: musicmetadatallmarize-phoenixbifrost

The catalog already contains useful information. This project adds analysis and reviewed suggestions to make that information easier to use.

Arize Phoenix · traces · evals · experimentsArize Phoenix · traces · evals · experimentscallsownsstoressyncssubmits jobanalysesembedsdraftsasksCatalog appbrowse · review · curateCatalog APIartists · releases · tracksPostgreSQLcanonical · provenanceImageKitstorage · deliveryEnrichment APIjobs · validation ·provenanceSignal analysisEssentia · librosaAudio MLCLAP embeddingsSemantic AItags · descriptionBifrostmodel gateway · routing ·fallback
Diagram source
flowchart TB
  App["Catalog app<br/>browse · review · curate"]
  API["Catalog API<br/>artists · releases · tracks"]
  PG[("PostgreSQL<br/>canonical · provenance")]
  IK["ImageKit<br/>storage · delivery"]

  subgraph PHX["Arize Phoenix · traces · evals · experiments"]
    EA["Enrichment API<br/>jobs · validation · provenance"]
    SIG["Signal analysis<br/>Essentia · librosa"]
    ML["Audio ML<br/>CLAP embeddings"]
    LLM["Semantic AI<br/>tags · description"]
    BF["Bifrost<br/>model gateway · routing · fallback"]
  end

  App -->|calls| API
  API -->|owns| PG
  API -->|stores| IK
  API -->|syncs| IK
  API -->|submits job| EA
  EA -->|analyses| SIG
  EA -->|embeds| ML
  EA -->|drafts| LLM
  LLM -->|asks| BF

Fig. 1. Enrichment runs beside the catalog. ImageKit stores files, Postgres owns canonical data, and Phoenix traces the AI workflow.

The canonical catalog lives in the application's own database. ImageKit handles files, transformations, and delivery, and receives selected fields for asset discovery. The catalog remains the source of truth while changes sync in both directions.

Keep music-specific rules such as versioning, credits, and provenance out of ImageKit.

Measured facts stay out of the LLM

language model onlydeterministic · no modelfeaturesvectorspromptwritesSignal analysistempo · loudnessAudio MLembeddingsEvidencenumbers, not wordingSemantic AIvia BifrostDraft metadatamood · tags
Diagram source
flowchart LR
  subgraph DET["deterministic · no model"]
    SIG["Signal analysis<br/>tempo · loudness"]
    ML["Audio ML<br/>embeddings"]
  end

  EV["Evidence<br/>numbers, not wording"]

  subgraph LM["language model only"]
    SA["Semantic AI<br/>via Bifrost"]
    DM["Draft metadata<br/>mood · tags"]
  end

  SIG -->|features| EV
  ML -->|vectors| EV
  EV -->|prompt| SA
  SA -->|writes| DM

Fig. 2. Signal analysis and audio models produce evidence. The language model turns it into words.

Tempo is measured, so the model should use that value rather than guess. Mood requires judgment, so a library cannot measure it. Each layer has a defined job, output type, and owner.

Seed the prompt with measured facts and the nearest neighbours. The LLM can interpret them without inventing a BPM.

Start with one path

Intake · steps 1–31 Upload trackyou · app2 Register assetImageKit · file stored3 Read existing metadatadeterministic · no model
Diagram source
flowchart LR
  subgraph G1["Intake · steps 1–3"]
    S1["1 Upload track<br/>you · app"] --> S2["2 Register asset<br/>ImageKit · file stored"] --> S3["3 Read existing metadata<br/>deterministic · no model"]
  end
Enrichment · steps 4–64 Analyse audioEssentia · CLAP5 Generate suggestionsBifrost · one task per call6 Validate and recordprovenancedeterministic · schema andrules
Diagram source
flowchart LR
  subgraph G2["Enrichment · steps 4–6"]
    S4["4 Analyse audio<br/>Essentia · CLAP"] --> S5["5 Generate suggestions<br/>Bifrost · one task per call"] --> S6["6 Validate and record provenance<br/>deterministic · schema and rules"]
  end
Review and write · steps 7–97 Human reviewyou · accept, edit, reject8 Update canonicalmetadatacatalog · the only writer9 Sync selected fieldsImageKit · discovery only
Diagram source
flowchart LR
  subgraph G3["Review and write · steps 7–9"]
    S7["7 Human review<br/>you · accept, edit, reject"] --> S8["8 Update canonical metadata<br/>catalog · the only writer"] --> S9["9 Sync selected fields<br/>ImageKit · discovery only"]
  end

Fig. 3. Steps 1–6 run unattended. A person reviews the suggestions before canonical metadata changes.

Make this path reliable before building a large catalog interface. A polished interface cannot make an unreliable pipeline dependable.

Human review approves suggestions

importedproposedimportedqueuesapprovesrejectsretested bycompared withOriginal metadataingestionField recordgenreMachine suggestionBifrost · v1External sourceattributedHuman reviewaccept · edit · rejectCanonical metadataEvaluation exampleModel v2
Diagram source
flowchart TD
  O["Original metadata<br/>ingestion"] -->|imported| REC["Field record<br/>genre"]
  M["Machine suggestion<br/>Bifrost · v1"] -->|proposed| REC
  X["External source<br/>attributed"] -->|imported| REC

  REC -->|queues| HR["Human review<br/>accept · edit · reject"]
  HR -->|approves| CAN[("Canonical metadata")]
  HR -->|rejects| EVA["Evaluation example"]

  CAN -->|retested by| M2["Model v2"]
  M2 -.->|compared with| EVA

Fig. 4. Each value records who created it and when. An owner's edit becomes an evaluation case (illustrative sequence).

One field can have four values, but only one is canonical.

RepresentationWritten byCan overwrite canonical
Original metadataIngestion, existing tagsno
Machine suggestionEnrichment runno
Human-approved metadataOwner after reviewyes
Externally sourced metadataThird party, with attributionno

Record the task, field, input asset, model and version, prompt version, timestamp, output, validation result, and review decision. Treat model-reported confidence as a hint until it has been calibrated.

Route by task, then measure the route

TaskModel tierGuard
Basic classificationsmall, low costschema check
Mood interpretationstrongersampled human review
Editorial descriptionmid-tierschema and length
Metadata validationsmall, plus rulesdeterministic checks first
Low confidenceescalatehuman review

Bifrost is the only gateway to a model. It handles provider abstraction, routing, fallback, caching, and cost control.

Arize Phoenix traces that path (Fig. 1) and compares prompt versions across the catalog. A response parsing successfully does not show whether the prompt is better.