Paper year
2026
Paper dossier
Multimodal Automatic Music Transcription Using Piano Audio and Hand-Skeleton Information
Detail viewSimilarity handoff
Review source metadata, abstract, authors, topics, and local similarity context before moving into explanation and ranking views.
Citations
0
Authors
0
Topic labels
0
Source readout
Source and corpus status
Venue
Unknown venue
Source slug
unknown
Corpus placement
Controlled edge slice
Similarity rows
Not available yet
Ranking readout
Where this paper lands in the current run
Run shadow-generalization-product-candidate-ranking-v1Top 50 surfaced
This block uses the same resolved ranking run as Recommended. Ranks here are materialized paper_scores ranks; live Emerging may be reordered by the bounded ML scorer. Family rank is global within each family, but rank is only shown when this paper lands inside the surfaced top 50.
Families present
2
Top 50
0
Run label
shadow-generalization-product-candidate-ranking-v1
Snapshot
source-snapshot-shadow-generalization-v1-20260521
Scope: family global | run rank-83787b91ef
Emerging
Present in run, outside top 50
Emerging: embedding slice fit vs included-corpus centroid (title+abstract), plus citation velocity and topic growth; not universal relevance. Bridge signal not used here.
Signals: semantic=0.8446, citation_velocity=0.0000, topic_growth=0.0000, diversity_penalty=0.0000
Why this surfaced | 3 used | 1 penalty | 1 not computed
Embedding slice fit (corpus centroid)used
Embedding slice fit (corpus centroid): high; used in final ranking (contribution to score: 0.1689)
Recent attentionused
Recent attention: low; used in final ranking (contribution to score: 0.0000)
Topic momentumused
Topic momentum: low; used in final ranking (contribution to score: 0.0000)
Cross-cluster signalnot computed
Cross-cluster signal: not computed for this run
Similarity penaltypenalty
Similarity penalty: reduces score when non-zero (contribution to score: 0.0000)
Bridge
Present in run, outside top 50
Multi-topic paper in active topics; no cluster_version on this run so bridge_score was not computed.
Signals: citation_velocity=0.0000, topic_growth=0.0000, diversity_penalty=1.0000
Why this surfaced | 2 used | 1 penalty | 2 not computed
Semantic matchnot computed
Semantic match: not computed for this run
Recent attentionused
Recent attention: low; used in final ranking (contribution to score: 0.0000)
Topic momentumused
Topic momentum: low; used in final ranking (contribution to score: 0.0000)
Cross-cluster signalnot computed
Cross-cluster signal: not computed for this run
Topic breadth penaltypenalty
Topic breadth penalty: reduces score when non-zero (contribution to score: -0.2000)
Under-cited
No materialized row for this family in the resolved run
This paper did not surface into the current materialized family row set.
Abstract
Automatic Music Transcription (AMT) for piano is difficult for audio-only systems due to dense polyphony, resonance, and reverberation, which lead to false positives and unstable onset decisions. We present a multimodal AMT framework that fuses Omnizart audio probability maps with visual cues from hand-skeleton tracking. A graph-based model called HandSkeletonNet estimates per-key onset probabilities from hand trajectories, and the two modalities are merged via a weighting-and-masking scheme or a compact CNN-based merger. Experiments show consistent improvements over the audio-only baseline on our self-compiled dataset, while evaluations with external datasets primarily improve frame-level sensitivity. The frame-level F1 score improved from 75.12% to 75.76% for the PianoYT dataset and from 54.68% to 57.57% for the PianoVAM dataset compared with the audio-only baseline. Our experiments also reveal limited onset-level gains under domain shift. Remaining errors are largely explained by timing/misalignment and note fragmentation in MIDI decoding, suggesting that robustness to missing hand detections and explicit temporal alignment are key directions.
Authors
No authors available.
Neighborhood labels
Topics
0 labels
Topic labels are imported metadata and can be noisy; use them as coarse navigation hints, not authoritative classifications.
Neighbor surface
Similar papers
Similar papers use a separately configured neighbor embedding; it may differ from the embedding version used by the current ranked run.
No embedding-backed neighbors available for this paper/version yet.
Next handoff
Best next moves from here
01
Check recommendation families
Use Recommended to see whether this paper behaves like an emerging or undercited signal in the current ranked feed, or how it appears on the bridge preview / diagnostics view.
02
Inspect nearby topics
Use Trends to understand whether its attached labels are heating up or cooling down inside the curated corpus.
03
Cross-check evaluation baselines
Use Evaluation to compare the dossier readout against citation and recency baselines for the same resolved family run.