<?xml version="1.0" encoding="UTF-8"?>
<TEI xmlns="http://www.tei-c.org/ns/1.0">
    <teiHeader>
        <fileDesc>
            <titleStmt>
                <title>Visualizing the Aura of Musical Instruments</title>
                <author>
                    <persName>
                        <surname>Kusnick</surname>
                        <forename>Jakob</forename>
                    </persName>
                    <affiliation>University of Southern Denmark, Denmark</affiliation>
                    <email>kusnick@imada.sdu.dk</email>
                </author>
                <author>
                    <persName>
                        <surname>Focht</surname>
                        <forename>Josef</forename>
                    </persName>
                    <affiliation>Musical Instrument Museum of Leipzig University, Germany</affiliation>
                    <email>josef.focht@uni-leipzig.de</email>
                </author>
                <author>
                    <persName>
                        <surname>Jänicke</surname>
                        <forename>Stefan</forename>
                    </persName>
                    <affiliation>University of Southern Denmark, Denmark</affiliation>
                    <email>stjaenicke@imada.sdu.dk</email>
                </author>
            </titleStmt>
            <editionStmt>
                <edition>
                    <date>2021-09-22T08:19:00Z</date>
                </edition>
            </editionStmt>
            <publicationStmt>
                <publisher>Elisabeth Burr, University of Leipzig</publisher>
                <address>
                    <addrLine>Beethovenstr. 15</addrLine>
                    <addrLine>04107 Leipzig</addrLine>
                    <addrLine>Germany</addrLine>
                    <addrLine>Elisabeth Burr</addrLine>
                </address>
            </publicationStmt>
            <sourceDesc>
                <p>Converted from a Word document</p>
            </sourceDesc>
        </fileDesc>
        <encodingDesc>
            <appInfo>
                <application ident="DHCONVALIDATOR" version="1.22">
                    <label>DHConvalidator</label>
                </application>
            </appInfo>
        </encodingDesc>
        <profileDesc>
            <textClass>
                <keywords scheme="ConfTool" n="category">
                    <term>Paper</term>
                </keywords>
                <keywords scheme="ConfTool" n="subcategory">
                    <term>Long paper</term>
                </keywords>
                <keywords scheme="ConfTool" n="keywords">
                    <term>musical compositions</term>
                    <term>relations</term>
                    <term>tag cloud</term>
                    <term>visual design</term>
                    <term>timeline visualization</term>
                    <term>geovisualization</term>
                </keywords>
                <keywords scheme="ConfTool" n="topics">
                    <term>Discovering</term>
                    <term>Imaging</term>
                    <term>Visualization</term>
                    <term>Contextualizing</term>
                    <term>Communicating</term>
                    <term>Meta: Teaching/Learning</term>
                    <term>DigitalHumanities</term>
                    <term>Tools</term>
                    <term>Visualisation</term>
                    <term>Metadata</term>
                    <term>Link</term>
                    <term>not applicable</term>
                    <term>not applicable</term>
                    <term>not applicable</term>
                    <term>English</term>
                </keywords>
            </textClass>
        </profileDesc>
    </teiHeader>
    <text>
        <body>
            <div type="div1" rend="DH-Heading1">
                <head>Introduction</head>
                <p>Many historical musical instruments are resting in depots or are presented in
                    showcases in music museums. Digitization is one of the current key strategies to
                    gather data and to make those instruments virtually accessible through online
                    repositories. But not all of them are linked together, and so relations between
                    related objects remain hidden. Cooperating with musicologists, we unfold the
                    history of musical instruments by “auralizing” (Summers 2008) them through
                    matching compositions. The motivating research question is, in which particular
                    performances of compositions a musical instrument was used. Therefore we assign
                    musical pieces from RISM (1952), a digital collection of musical compositions,
                    to musical instruments from MIMO (2011a), a digital repository of meta- and
                    media-data on musical instruments in public collections. A multi-facetted
                    similarity measure generates matching results between the two repositories
                    automatically and interactive visualizations. We propose an extension of this
                    process by a tag cloud picturing the aura of instruments of interest. The system
                    is evaluated by musicologists who investigated different scenarios using the
                    tool.</p>
            </div>
            <div type="div1" rend="DH-Heading1">
                <head>Previous System</head>
                <p>In our previous work (Kusnick et al. 2020), we proposed a system for the calculation and visualization of hypothetical relations between musical instruments from MIMO and musical pieces from RISM.</p>
                <p>We take geospatial, temporal and descriptive meta-information into account to
                    assign musical instruments to musical pieces. These attributes are used to
                    define similarity measures which describe the likeness of a musical instrument
                    playing a part in a particular piece performance. The result of the matching
                    process is a list of musical instruments and the assigned musical piece
                    performances. Schlegel et al. (2011) created a static visual timeline for types
                    of lutes and lute-like instruments, enhanced with appropriate compositions. To
                    visualize such pairings interactively, we designed a new timeline metaphor,
                    where every instrument has its dedicated row. Distant and close reading research
                    questions can be answered through different semantic zoom levels, what has
                    previously been demonstrated for compositions (André et al. 2007). As described
                    by Khulusi et al. (2020), data on musicology include uncertainty aspects, which
                    are visually encoded in our approach. The timeline enables the explorative
                    analysis of hypothesized matches and a map as an additional view helps the user
                    to validate or falsify the automatically generated suggestions. So the
                    instruments’ careers are visualized through their temporal and geographical
                    located events in timeline and map. Now we propose an additional view to picture
                    the aura of one or a group of musical instruments of interest. </p>
            </div>
            <div type="div1" rend="DH-Heading1">
                <head>The Aura</head>
                <p>The timeline and map generate visual patterns, and potentially interesting findings stick out. The previous version of the system only allowed to select an individual musical instrument after filtering the results via search terms and clicking a particular instrument, getting to a single close reading view. The aura of an individual or multiple instruments was just perceivable by a follow-on, laborious clicking on each matched piece performance. The extended system now enables to select instruments via a rectangular selection box by click-and-drag in the timeline or map. This selection filters the result set and leads to the refreshed multi-view screen of timeline and map, including a new visualization depicting the aura of instruments.</p>
                <div type="div2" rend="DH-Heading2">
                    <head>Tag cloud for the Aura</head>
                    <p>The aura of instruments is pictured in the form of a tag cloud. It shows the resulting instruments, described by words aggregated from the matched musical pieces. The arrangement of words is automatically determined by a force directed graph drawing algorithm (Kobourov 2012), in which each item is a node and the links between them represent joint appearances in the same category. This leads to forming clusters, inspired by TagPies (Jänicke et al. 2018). In the cloud’s center, each instrument is symbolized by its image, or a placeholder image in a circle. The words, whose sizes are determined according to the number of occurrences, are taken from four different attributes of musical works, and, additionally, the titles of the instruments. So the words appear in five different colors to quickly depict the instrument’s aura visually. In most cases, the largest cluster (in purple) is built by the tokenized titles of the musical pieces found in the matching process. Due to the variety of possible languages, we did not apply further language processing like stemming or lemmatization. Nevertheless, the token sets are cleaned through stopword lists of the most frequently used languages like German, English, and Italian. </p>
                    <p>Furthermore, the names of the musical piece composers constitute the second cluster (in green) of the tag cloud. Due to the length of full names, they were cropped and only the surname is displayed. Thanks to the additional meta-information about the composers, the user can inspect the details by hovering the name tags and see the composer’s full name and lifetime. Additionally, genre names (in red) and languages (in blue) of the manuscripts are displayed in separate word clusters of the tag cloud. </p>
            </div>
                </div>
            <div type="div1" rend="DH-Heading1">
                <head>Use Cases</head>
                <div type="div2" rend="DH-Heading2">
                    <head>The Protestant Trombone</head>
                    <p>The visual exploration of a trombone (MIMO 2011b) originating Nuremberg,
                        Germany has been described in our previous paper (Kusnick et al. 2020). The
                        inspection of matched pieces on the map reveals that they were performed in
                        some cities close to Nuremberg. The tag cloud highlights that the most
                        frequently matching genre is the opera (closely related to “Singspiele”),
                        and German (“ger”) as the major language. Frequent terms in the titles are
                        “Zauberflöte” and “Giovanni”, in which trombones were used, as well as
                        “Mozart” as the corresponding composer. A closer look reveals many terms
                        related to religion. German cantatas, sacred songs and numerous names
                        including words like “Herr” (lord), “herrlich” (wonderful), but also
                        “Sünden” (sins) and “Opferfest” (sacrifice) from Peter von Winter and Georg
                        Philipp Telemann are found in the resulting list of musical piece tags
                        prominent in the word cloud. The musicologist working with this use case
                        commented that the denomination was very important around 1800.
                        Additionally, he took the predominant denomination of the piece
                        performances’ locations into account. All cities were predominantly
                        Protestant, except the Catholic Munich. Thus, the trombone from Nuremberg
                        may be better auralized through Protestant songs from one of the culturally
                        closer cities like Frankfurt am Main, even though it is geographically
                        farther away than Munich. Clicking “Cantatas” shows a list of all cantatas
                        in the musical piece matches, for example “Ertönet bald herrlich ihr letzten
                        Posaunen” (Telemann 1741) (also tagged as a sacred song) having even
                        “Posaune” (trombone) in its title. The song was performed in Frankfurt am
                        Main and was composed by Georg Philipp Telemann who is tagged as <hi
                            rend="italic">evangelic</hi> in the musiXplora (musiXplora 2020). </p>
                    <figure>
                        <graphic n="1001" width="13.253063888888889cm" height="10.997475cm" url="Pictures/3e834c59e2cf9ca67fd71eb83301db25.png" rend="inline"/>
                    </figure>
                    <p>Figure 1: The generated tag cloud for a trombone from Leipzig, Germany and its matched compositions within a range of 50 years and 200 kilometers.</p>
                </div>
                <div type="div2" rend="DH-Heading2">
                    <head>Lutes and Lute-like Instruments</head>
                    <p>The history of plucked instruments like lutes and lute-likes is pictured by a
                        second use case. Three dedicated epochs are visible in the timeline in
                        Figure <hi rend="bold">2</hi>, and the corresponding tag cloud is viewable
                        in Figure <hi rend="bold">3a</hi>. </p>
                    <p>The first epoch (1620–1760) refers to the classical lute, indicating a very sparse occupancy due to the low amount of pieces dedicated for lutes in RISM. However, a wide distribution of genres is depicted in the word cloud in Figure 
                        <hi rend="bold">3b</hi>. 
                    </p>
                    <p>The second epoch (1770–1810) is characterized by numerous precise datings (see Figure 
                        <hi rend="bold">2</hi>) for the production of the instruments. This might be contractual works commissioned by the courtly society because lutes were highly appreciated instruments at that time. Also, the word cloud gets more dense and manifold (Figure 
                        <hi rend="bold">3c</hi>), especially, the amount of mandolins increases.
                    </p>
                    <p>The third epoch (1870–1960) is marked by very uncertain production dates because of their more industrial production, visually encoded by more transparent and wider red ovals in the timeline. Also, this episode was the time where lutes and lute-like instruments only survived by being transformed into bass-lutes or guitars to play roles in operas because they were not part of uprising orchestras, as seen in Figure 
                        <hi rend="bold">3d</hi>.
                    </p>
                    <figure>
                        <graphic n="1002" width="16.002cm" height="4.517319444444444cm" url="Pictures/d409ce94d1aeba5e01ee54965eef9382.png" rend="inline"/>
                    </figure>
                    <p>Figure 2: All found lutes and lute-like instruments with their matching piece performances within a distance of 50 years and 100 kilometers around the instrument events. For details, see the original publication (Kusnick et al. 2020).</p>
                    <figure>
                        <graphic n="1003" width="16.002cm" height="10.668cm" url="Pictures/59f5da42bf01005e7758973bbd4fffbc.png" rend="inline"/>
                    </figure>
                    <p>Figure 3: Four different tag clouds for lutes and lute-like instruments summarizing all pairings of instruments and compositions for the time ranges (a) 1620–1960, (b) 1620–1760, (c) 1760–1870, and (d) 1870–1960.</p>
                </div>
            </div>
            <div type="div1" rend="DH-Heading1">
                <head>Conclusion</head>
                <p>While historical musical instruments are resting in showcases or depots of museums, digitization efforts serve us with intangible data about them. This data could be used to make the musical instruments and their history perceptible. We extended our visualization system by an interactive tag cloud to picture the career and aura of an instrument or group of instruments of interest in a manifold way. The created aura is composed of word clusters for five different attributes of the matched composition or instrument giving an overview of the selected data subset. This visual depiction could be presented to visitors of museums next to showcases to contextualize instruments through their music and to bring their particular careers to life.</p>
            </div>
        </body>
        <back>
            <div type="bibliogr">
                <listBibl>
                    <head>Bibliography</head>
                    <bibl>
                        <hi rend="bold">André, Paul</hi> / <hi rend="bold">Wilson, Max</hi> / <hi
                            rend="bold">Russell, Alistair</hi> / <hi rend="bold">Smith, Daniel</hi>
                        / <hi rend="bold">Owens, Alistair</hi> / <hi rend="bold">Schraefel,
                            Monica</hi> (2007): „Continuum: Designing Timelines for Hierarchies,
                        Relationships and Scale“, in: <hi rend="italic">Proceedings of the 20th
                            Annual ACM Symposium on User Interface Software and Technology</hi>. </bibl>
                    <bibl>
                        <hi rend="bold">Jänicke, Stefan</hi> / <hi rend="bold">Blumenstein,
                            Judith</hi> / <hi rend="bold">Rücker, Michaela</hi> / <hi rend="bold"
                            >Zeckzer, Dirk</hi> / <hi rend="bold">Scheuermann, Gerik</hi> (2018):
                        „TagPies: Comparative Visualization of Textual Data“, in: <hi rend="italic"
                            >Proceedings of the 9th International Conference on Information
                            Visualization Theory and Applications</hi>. </bibl>
                    <bibl>
                        <hi rend="bold">Jänicke, Stefan</hi> / <hi rend="bold">Focht, Josef</hi> /
                            <hi rend="bold">Scheuermann, Gerik (2016):</hi> „Interactive visual
                        profiling of musicians“, in: <hi rend="italic">IEEE transactions on
                            visualization and computer graphics</hi> 22. </bibl>
                    <bibl>
                        <hi rend="bold">Khulusi, Richard</hi> / <hi rend="bold">Kusnick, Jakob</hi>
                        / <hi rend="bold">Meinecke, Christofer</hi> / <hi rend="bold">Gillmann,
                            Christina</hi> / <hi rend="bold">Focht, Josef</hi> / <hi rend="bold"
                            >Jänicke, Stefan</hi> (2020): „A Survey on Visualizations for Musical
                        Data“, in: <hi rend="italic">Computer Graphics Forum</hi>. </bibl>
                    <bibl>
                        <hi rend="bold">Kobourov, Stephen</hi> (2012): „Spring embedders and force
                        directed graph drawing algorithms“, in: <hi rend="italic">arXiv</hi>. </bibl>
                    <bibl>
                        <hi rend="bold">Kusnick, Jakob</hi> / <hi rend="bold">Khulusi Richard</hi> /
                            <hi rend="bold">Focht, Josef</hi> / <hi rend="bold">Jänicke, Stefan</hi>
                        (2020): „A Timeline Metaphor for Analyzing the Relationships between Musical
                        Instruments and Musical Piece“, in: <hi rend="italic">Proceedings of the
                            11th International Conference on Information Visualization Theory and
                            Applications</hi>. </bibl>
                    <bibl>
                        <hi rend="bold">MIMO </hi>(2011a): <hi rend="italic">Musical Instrument
                            Museums Online</hi> &lt;<ref
                            target="https://mimo-international.com/MIMO"
                            >https://mimo-international.com/MIMO</ref>&gt; [ 15.06.2021]. </bibl>
                    <bibl>
                        <hi rend="bold">MIMO </hi>(2011b): „Tenorposaune in B“, in: <hi
                            rend="italic">Musical Instrument Museums Online</hi> &lt;<ref
                            target="https://mimo-international.com/MIMO/doc/IFD/OAI_ULEI_M0001892"
                            >https://mimo-international.com/MIMO/doc/IFD/OAI_ULEI_M0001892</ref>&gt;
                        [15.06.2021]. </bibl>
                    <bibl>
                        <hi rend="bold">musiXplora</hi> (2020): “Georg Philipp Telemann”, in: <hi
                            rend="italic">musiXplora</hi> &lt;<ref
                            target="https://home.uni-leipzig.de/mim/mxp/t0045"
                            >https://home.uni-leipzig.de/mim/mxp/t0045</ref>&gt; [15.06.2021]. </bibl>
                    <bibl>
                        <hi rend="bold">RISM </hi>(1952): <hi rend="italic">Répertoire International
                            des Sources Musicales</hi> &lt;<ref target="https://rism.info"
                            >https://rism.info</ref>&gt; [15.06.2021]. </bibl>
                    <bibl>
                        <hi rend="bold">Schlegel, Andreas</hi> / <hi rend="bold">Lüdtke,
                            Joachim</hi> / <hi rend="bold">Cabral, Pedro Caldeira</hi> (2011): „Die
                        Laute in Europa 2: Lauten, Gitarren, Mandolinen und Cister“, in: <hi
                            rend="italic">Lute Corner</hi>. </bibl>
                    <bibl>
                        <hi rend="bold">Summers, Jason</hi> (2008): „What exactly is meant by the
                        term “auralization?”, in: <hi rend="italic">The Journal of the Acoustical
                            Society of America</hi> 124.
                    </bibl>
                    <bibl>
                        <hi rend="bold">Telemann, Georg Philipp</hi> (1741): „Ertönet bald herrlich
                        ihr letzten Posaunen“, in: <hi rend="italic">Répertoire International des
                            Sources Musicales</hi> &lt;<ref
                            target="https://opac.rism.info/search?id=450004254&amp;View=rism"
                            >https://opac.rism.info/search?id=450004254&amp;View=rism</ref>&gt;
                        [15.06.2021]. </bibl>
                </listBibl>
            </div>
        </back>
    </text>
</TEI>
