Category Archives: publications

28/04/2016 · 09:42

Paper on melodic similarity in flamenco now online

Our paper on melodic similarity is finally online! The paper is titled

Melodic Contour and Mid-Level Global Features Applied to the Analysis of Flamenco Cantes

This work focuses on the topic of melodic characterization and similarity in a specific musical repertoire: a cappella flamenco singing, more specifically in debla and martinete styles. We propose the combination of manual and automatic description. First, we use a state-of-the-art automatic transcription method to account for general melodic similarity from music recordings. Second, we define a specific set of representative mid-level melodic features, which are manually labelled by flamenco experts. Both approaches are then contrasted and combined into a global similarity measure. This similarity measure is assessed by inspecting the clusters obtained through phylogenetic algorithms and by relating similarity to categorization in terms of style. Finally, we discuss the advantage of combining automatic and expert annotations as well as the need to include repertoire-specific descriptions for meaningful melodic characterization in traditional music collections.

This is the result of a joint work of the COFLA group, where I am contributing with tecnologies for the automatic transcription and melody description of music recordings.

This is an example on how we compare flamenco tonás using melodic similarity and phylogenetic trees:

nnmr_a_1174717_f0007_b

And this is a video example of the type of styles we analyze in this paper, done by Nadine Kroher based on her work at the MTG:

You can read the full paper online:

http://www.tandfonline.com/doi/full/10.1080/09298215.2016.1174717

Leave a comment

Filed under publications, research, Uncategorized

Tagged as , , , , , , , ,

05/04/2016 · 07:22

CANTE: Open Algorithm, Code & Data for the Automatic Transcription of Flamenco Singing

Over the last months, several journal publications related to our research on flamenco & technology are finally online.

One of them is a work with my former PhD student, Nadine Kroher (who now moved to Universidad de Sevilla), on the automatic transcription of flamenco singing. Flamenco singing is really challenging in terms of computational modelling, given its ornamented character and variety, and we have designed a system for its automatic transcription, focusing on polyphonic recordings.

flamencoTranscriptionKroherGomez

The proposed system outperforms state of the art singing transcription systems with respect to voicing accuracy, onset detection, and overall performance when evaluated on flamenco singing datasets. We hope it think will be a contribution not only to flamenco research but to other singing styles.

You can read about our algorithm at the paper we published at IEEE TASP, where we present the method, strategies for evaluation and comparison with state of the art approaches. You can not only read, but actually try it, as we published an open source software for the algorithm, plus a music dataset for its comparative evaluation, cante2midi (I will talk about flamenco corpus in another post). All of this to foster research reproducibility and motivate people to work on flamenco music.

¡Olé!

 

 

 

Leave a comment

Filed under publications, research, software

Tagged as , , , , ,

03/12/2015 · 10:51

CfP: Multimedia Technologies for Enriched Music Performance, Production, and Consumption

Publication: Jan. Mar. 2017
Submission deadline: Feb. 1st 2016

Screen Shot 2015-12-03 at 11.50.49

I am co-editing, together with my colleagues Cynthia Liem and George Tzanetakis,  a special issue on IEEE Multimedia related to music.

Internet access, mobile devices, social networks, and automated multimedia technologies enabling sophisticated information analysis and access have radically changed the ways in which people find entertainment, discover new interests, and generally express themselves online — seemingly without any physical or social barriers. Thanks to the increasing affordability of sensing, storage, and sharing, we note that information takes increasingly rich and hybrid multimedia forms, in which multimodal information streams co-occur in various social consumption settings.

This phenomenon also has enabled opportunities in the music domain. In music performance, novel opportunities for expression are found, exploiting (live) analysis and novel interaction mechanisms with musical data in multiple modalities. In music production, sophisticated multimedia data analysis techniques can both lead to more efficient and scalable workflows, as well as richer and better interfaces. In music consumption, the music data richness and its contextual and social embedding lead to novel consumer experiences stimulating music appreciation. Concerts turn into multimodal, multiperspective, and multilayer digital artifacts that can be easily explored, customized, personalized, (re)enjoyed and shared among various types of users; similar notions and opportunities hold for the consumption of general music recordings.

The goal of this special Issue is to gather state-of-the-art research on multimedia methods and technologies aimed at enriching music performance, production and consumption. We solicit novel, original work that is not published or under review elsewhere.

Topics of interest include, but are not limited to:

  • Processing of multimodal music data streams (e.g. audio, video, images, score, text, gesture…) for music performance, production and/or consumption
  • Multimedia content description and indexing for music performance, production and/or consumption
  • Multimedia information retrieval methods for music performance, production and/or consumption
  • Novel interaction mechanisms for music performance, production and/or consumption
  • Novel user interfaces for music performance, production and/or consumption
  • Novel user experience paradigms for music performance, production and/or consumption
  • Social networking and sharing for music performance, production and/or consumption
  • Digital mechanisms for remote music performers and audiences
  • Active listening, audience immersion, and inclusion of new music audiences
  • User-awareness, personalization and intent in music performance, production and/or consumption
  • Context-awareness and automatic context adaptation in music performance, production and/or consumption

Submission Guidelines

See www.computer.org/web/peer-review/magazines. Submissions should not exceed 6,500 words, with each table and figure counting for 200 words. Manuscripts should be submitted electronically (https://mc.manuscriptcentral.com/mm-cs), selecting this special issue option.

Guest Editors

Detailed call for papers

Leave a comment

Filed under CFP, publications, research, Uncategorized

21/08/2015 · 14:34

Correlation between musical descriptors and emotions recognized in Beethoven’s Eroica

Last Wednesday I presented a poster at the Ninth Triennial Conference of the European Society for the Cognitive Sciences of Music (ESCOM 2015), that took place at the Royal Northern College of Music, Manchester, UK. It was a very interesting conference, including a very nice symposium in understanding musical audiences and inspiring talks on music education, psychology and wellbeing. Really impressed by how music have influence to improve quality of live from early years to the end of our lives.

The work I presented was leaded by Erika Trent, a student from the MIT that spent last summer at my lab thanks to the MIT Spain program. It was a very productive stay!

In this study we analysed the emotions that listener perceive when listening to Beethoven Symphony No. 3, Eroica, PHENICX target piece, played by the Royal Concertgebouw Orchestra Amsterdam. We then quantify the correlation between listeners’ perceived emotions from music and 1) musical descriptors, and 2) listeners’ backgrounds (country of origin, musical knowledge, exposure to classical music and knowledge of Eroica).

One conclusion of this study is that tonal strength (i.e. key clarity) correlates significantly with listener ratings of peacefulness, joyful activation, tension and sadness. Other significant correlations between emotion ratings and musical descriptors agree with the literature. This agreed with our hypothesis, being different parts on the same musical piece.

But there are two other unexpected and interesting findings that we might need to continue researching on.

First, we found out that listeners of varying backgrounds agree most on their ratings of sadness, compared to other emotions. Would that be similar for other musical pieces?

Second, listeners of similarly unmusical backgrounds, and listeners of young ages, recognise similar emotions to same music. On the contrary, listeners with more musical experience recognise different emotions to the same music. Caused by personal biases?

Interesting results that might corroborate the need for personalisation in music recommendation engines!

You can read the whole paper and access the poster here. 

0168TrentGomezPoster

Leave a comment

Filed under publications

Tagged as , , , ,

26/09/2014 · 09:20

Music Information Retrieval: Recent Developments and Applications (152 pages, 311 references, amazing reviewers!)

After one year of hard work we finished our paper (152 pages, 311 references!) on Music Information Retrieval: Recent Developments and Applications. I collaborated with Markus Schedl and Julián Urbano in this amazing project at Foundations and Trends in Information Retrieval (h-index=15, Q1 Computer Science). We tried to cover all existing techniques, approaches and key references in MIR, and to reflect the interest of our community on combining audio description, context mining, user modelling and proper evaluation methodologies.

We hope it will be an interesting reference for our community, and we also hope this paper can serve to motivate and introduce people outside or our field.

I would really like to thank the anonymous reviewers and the editor, Mark Sanderson. I don’t know who the reviewers are but I think they deserve being in the author list!  Great suggestions, discussions, restructuring, editions for a great outcome.

Enjoy!

Abstract

Music Information Retrieval: Recent Developments and Applications surveys the young but established field of research that is Music Information Retrieval (MIR). In doing so, it pays particular attention to the latest developments in MIR, such as semantic auto-tagging and user-centric retrieval and recommendation approaches.

Music Information Retrieval: Recent Developments and Applications starts by reviewing the well-established and proven methods for feature extraction and music indexing, from both the audio signal and contextual data sources about music items, such as web pages or collaborative tags. These in turn enable a wide variety of music retrieval tasks, such as semantic music search or music identification (“query by example”). Subsequently, it elaborates on the current work on user analysis and modeling in the context of music recommendation and retrieval, addressing the recent trend towards user-centric and adaptive approaches and systems. A discussion follows about the important aspect of how various MIR approaches to different problems are evaluated and compared. It concludes with a discussion about the major open challenges facing MIR.

Music Information Retrieval: Recent Developments and Applications is an invaluable reference for researchers, students or practitioners working on, or with an interest in MIR.

Leave a comment

Filed under publications

Tagged as

02/06/2014 · 09:31

Paper & Matlab framework for hierarchical multi-scale set-class analysis

Journal on Mathematics and Music

As part of his recent PhD thesis, Agustín Martorell has studied the potential of multi-scale representations in music analysis. In particular, he focuses on the description of tonality from score representations and on the analysis of pitch-class sets.  We have recently published the results of this study  in Journal of Mathematics and Music: Mathematical and Computational Approaches to Music Theory, Analysis, Composition and Performance. The paper is now online!

Several analyses are discussed within the paper while addressing the problem of visualization. As a result of the work, there is also a MATLAB Toolbox that you are able to download from here.

Hierarchical multi-scale set-class analysis
Agustín Martorell & Emilia Gómez

Abstract

This work presents a systematic methodology for set-class surface analysis using temporal multi-scale techniques. The method extracts the set-class content of all the possible temporal segments, addressing the representational problems derived from the massive overlapping of segments. A time versus time-scale representation, named class-scape, provides a global hierarchical overview of the class content in the piece, and it serves as a visual index for interactive inspection. Additional data structures summarize the set-class inclusion relations over time and quantify the class and subclass content in pieces or collections, helping to decide about sets of analytical interest. Case studies include the comparative subclass characterization of diatonicism in Victoria’s masses (in Ionian mode) and Bach’s preludes and fugues (in major mode), as well as the structural analysis of Webern’s Variations for piano op. 27, under different class-equivalences.

 

Leave a comment

Filed under publications, research, software

Tagged as , , , , , , , ,

22/05/2014 · 11:04

Forum on transcription in the journal “Twentieth-Century Music”

I contributed by means of an enriching interview to the “Forum on Transcription”, authored by Jason Stanyek (University of Oxford) in the journal Twentieth-Century Music. As stated on the web site, this journal disseminates research on all aspects of music in the long twentieth century to a broad readership. Emphasis is placed upon the presentation of the full spectrum of scholarly insight, with the goal of fostering exchange and debate between disciplinary fields.

I share an interesting conversation about transcription with Parag Chordia. In this conversation with Jason we discussed about the challenges and potential of audio analysis tools for computer-assisted transcription and description of music recordings. I gave some examples on my work on the transcription of flamenco singing that is being carried out within the COFLA project. 

You can find the results of the forum and the rest of a very impressive special issue on transcription on the web.

 

Leave a comment

Filed under press, publications, research

Tagged as , , ,

Image

New IEEE/ACM TASP paper on multi-feature beat tracking

Multi-feature beat tracking Our article on multi-feature beat tracking for the IEEE/ACM Transactions on Audio, Speech and Signal Processing is now available online! This is a work carried leaded by Jose Ricardo Zapata for his PhD thesis in collaboration with Mathew Davies from the SMC group in Porto, based on the idea of combining different experts, represented by periodicity from different onset detection functions, for beat estimation. This is a simple and clever idea, already used to combine different beat tracking algorithms and evaluate the difficulty of the task, that has been integrated in a different method.

Zapata, J. R., Davies M. E. P., & Gómez E. (2014).  Multi-feature beat tracking. IEEE/ACM Transactions on Audio, Speech, and Language Processing. 22(4), 816 – 825. RTF, Tagged, XML, BibTex, Google Scholar

Abstract:

A recent trend in the field of beat tracking for musical audio signals has been to explore techniques for measuring the level of agreement and disagreement between a committee of beat tracking algorithms. By using beat tracking evaluation methods to compare all pairwise combinations of beat tracker outputs, it has been shown that selecting the beat tracker which most agrees with the remainder of the committee, on a song-by-song basis, leads to improved performance which surpasses the accuracy of any individual beat tracker used on its own. In this paper we extend this idea towards presenting a single, standalone beat tracking solution which can exploit the benefit of mutual agreement without the need to run multiple separate beat tracking algorithms. In contrast to existing work, we re-cast the problem as one of selecting between the beat outputs resulting from a single beat tracking model with multiple, diverse input features. Through extended evaluation on a large annotated database, we show that our multi-feature beat tracker can outperform the state of the art, and thereby demonstrate that there is sufficient diversity in input features for beat tracking, without the need for multiple tracking models.

Leave a comment

27/02/2014 · 15:53
17/02/2014 · 08:58

IEEE Signal Processing Magazine Melody Extraction Review published online

Our review article on melody extraction algorithms for the IEEE Signal Processing Magazine is finally available online! The printed edition will be coming out in March 2014.

I believe (not just as I am a co-author!) that it will become a key reference in the Music Information Retrieval area and beyond, as it provides a very nice overview of approaches, challenges and applications for melody extraction from polyphonic music signals. Justin Salamon has been the main author (congratulations, Justin!) and the paper has benefit from the contribution of two key experts: Gaël Richard, and Dan Ellis, with who I had the chance to collaborate on a previous comparative study on melody extraction published at IEEE TASP (128 citations according to google scholar).

Finally, I like very much this kind of tutorial papers providing a comprehensive introduction to a given topic and with a very attractive design. I hope you will enjoy it!

Image
J. Salamon, E. Gómez, D. P. W. Ellis and G. Richard, “Melody Extraction from Polyphonic Music Signals: Approaches, Applications and Challenges“, IEEE Signal Processing Magazine, 31(2):118-134, Mar. 2014.

Abstract—Melody extraction algorithms aim to produce a sequence of frequency values corresponding to the pitch of the dominant melody from a musical recording. Over the past decade melody extraction has emerged as an active research topic, comprising a large variety of proposed algorithms spanning a wide range of techniques. This article provides an overview of these techniques, the applications for which melody extraction is useful, and the challenges that remain. We start with a discussion of ‘melody’ from both musical and signal processing perspectives, and provide a case study which interprets the output of a melody extraction algorithm for specific excerpts. We then provide a comprehensive comparative analysis of melody extraction algorithms based on the results of an international evaluation campaign. We discuss issues of algorithm design, evaluation and applications which build upon melody extraction. Finally, we discuss some of the remaining challenges in melody extraction research in terms of algorithmic performance, development, and evaluation methodology.

For further information about this article please visit Justin Salamon’s research page.

Leave a comment

Filed under publications, research, teaching

Tagged as , , ,

05/09/2013 · 07:27

Special Issue on Computational Ethnomusicology

I have been editing, together with Perfecto Herrera and Paco Gómez, a Special Issue on Computational Ethnomusicology at the Journal of New Music Research.
The goal of this special issue is to gather relevant, high-quality research on computational methods and applications in ethnomusicology. The papers included here deal with different musical facets such as pitch, pulse and tempo, and voice timbre. They address different musical repertoires, from Central-African to Basque folk music. They also cover a broad area: tools, including data collections, methodology and Ethnomusicology core-problems. Althgouth it was a hard work, thanks to the authors and reviewers we managed to get a varied and interesting set of articles:
  • Computational Ethnomusicology: perspectives and challenges
  • Antipattern Discovery in Folk Tunes
  • Tarsos, a Modular Platform for Precise Pitch Analysis of Western and Non-Western Music
  • Evaluation and Recommendation of Pulse and Tempo Annotation in Ethnic Music
  • Breathy, Resonant, Pressed – Automatic Detection of Phonation Mode from Audio Recordings of Singing
  • A Location-Tracking Interface for Ethnomusicological Collections

The issue is now available online at JNMR web site. And our introduction is available here and here.

Leave a comment

Filed under publications, research

Tagged as , , , , ,