Paper year
2021
Paper dossier
On Evaluation of Inter- and Intra-Rater Agreement in Music Recommendation
Detail viewSimilarity handoff
Review source metadata, abstract, authors, topics, and local similarity context before moving into explanation and ranking views.
Citations
40
Authors
3
Topic labels
3
Source readout
Source and corpus status
Venue
Transactions of the International Society for Music Information Retrieval
Source slug
tismir
Corpus placement
Core corpus
Similarity rows
6
Ranking readout
Where this paper lands in the current run
Run shadow-generalization-product-candidate-ranking-v1Top 50 surfaced
This block uses the same resolved ranking run as Recommended. Ranks here are materialized paper_scores ranks; live Emerging may be reordered by the bounded ML scorer. Family rank is global within each family, but rank is only shown when this paper lands inside the surfaced top 50.
Families present
0
Top 50
0
Run label
shadow-generalization-product-candidate-ranking-v1
Snapshot
source-snapshot-shadow-generalization-v1-20260521
Scope: family global | run rank-83787b91ef
Emerging
No materialized row for this family in the resolved run
This paper did not surface into the current materialized family row set.
Bridge
No materialized row for this family in the resolved run
This paper did not surface into the current materialized family row set.
Under-cited
No materialized row for this family in the resolved run
This paper did not surface into the current materialized family row set.
Abstract
Our work is concerned with the subjective perception of music similarity in the context of music recommendation. We present two user studies to explore inter- and intra-rater agreement in quantification of general similarity between pieces of recommended music. Contrary to previous efforts, our test participants are of more uniform age and share a comparable musical background to lower variation within the participant group. The first study uses carefully curated song material from five distinct genres while the second uses songs from a single genre only, with almost all songs in both studies previously unknown to test participants. Repeating the listening tests with a two week lag shows that intra-rater agreement is higher than inter-rater agreement for both studies. Agreement for the single genre study is lower since genre of songs seems a major factor in judging similarity between songs. Mood of raters at test-time is found to have an influence on intra-rater agreement. We discuss the impacts of our results on evaluation of music recommenders and question the validity of experiments on general music similarity.
Authors
Neighborhood labels
Topics
3 labels
Topic labels are imported metadata and can be noisy; use them as coarse navigation hints, not authoritative classifications.
Music and Audio ProcessingNeuroscience and Music PerceptionSpeech and Audio Processing
Neighbor surface
Similar papers
6 total neighborsEmbedding v1-title-abstract-1536-cleantext-r3
Similar papers use a separately configured neighbor embedding; it may differ from the embedding version used by the current ranked run.
Investigating the Perceptual Validity of Evaluation Metrics for Automatic Piano Music Transcription
0.675Music and Audio ProcessingMusic Technology and Sound StudiesSpeech and Audio Processing- Music and Audio ProcessingMusic Technology and Sound StudiesDiverse Musicological Studies
- Music and Audio ProcessingMusic Technology and Sound StudiesNeuroscience and Music Perception
- Music and Audio ProcessingMusic Technology and Sound StudiesNeuroscience and Music Perception
- Music and Audio ProcessingMusic Technology and Sound StudiesDiverse Music Education Insights
- Music and Audio ProcessingMusic Technology and Sound StudiesAnimal Vocal Communication and Behavior
Next handoff
Best next moves from here
01
Check recommendation families
Use Recommended to see whether this paper behaves like an emerging or undercited signal in the current ranked feed, or how it appears on the bridge preview / diagnostics view.
02
Inspect nearby topics
Use Trends to understand whether its attached labels are heating up or cooling down inside the curated corpus.
03
Cross-check evaluation baselines
Use Evaluation to compare the dossier readout against citation and recency baselines for the same resolved family run.