BiblioScrape & BiblioMatch: Scraping and Matching Bibliographical Metadata

Who published on the Old English poem Beowulf during the long nineteenth century (1786-1914)? In what languages did scholars publish about the linguistic history of the English language of the early Middle Ages? Where did they publish and in what format? As part of our ERC-project project ‘Early Medieval English in Nineteenth-Century Europe’ [https://www.universiteitleiden.nl/en/research/research-projects/humanities/emergence], we are building a bibliographical database to find out answers to these questions. Rather than starting from scratch, we want to reuse bibliographical metadata that is available in other bibliographical resources. This is where we, researchers of the magical Middle Ages, need the help of modern-day tech-wizards; we hope you can help us out! Reusing publication metadata (title; author; etc.) is challenging because different bibliographical resources use different metadata standards (MARC21; BIBFRAME; DublinCore). In effect, each bibliographical resource requires its own ‘pipeline’ to scrape relevant metadata for potential reuse. For this project, we are looking for programmers who can build a number of these pipelines and create (or configure) software to match scraped bibliographical datasets with other bibliographical datasets. The specific aim for the BiblioScrape & BiblioMatch project is to help construct our OE-BaRD (Old English Bibliographical and Relational Database). However, the issues BiblioScrape & BiblioMatch address are generally applicable and ensure that this project will be of use to anyone working with bibliographical data.

See below for implementation of project.

Minimum-viable version of project: 1) BiblioScrape: Create API > XML > CSV pipelines to extract metadata about relevant publications in bibliographical databases (specifically: RI OPAC: Die Literaturdatenbank zum Mittelalter; Internet Archive) 2) BiblioMatch: Created or configure software that automatically detects bibliographical matches between two CSV-files with bibliographical data in different formats, using a select number of variables.

Fully completed: 3) BiblioFrontend: Create or configure simple web interface for bibliographical data, with basic filters and search functions (and advise on metadata format that is most suitable for this).