User talk:John Cummings/Archive 5

Annotation is nothing new. The glossators of medieval Europe annotated between the lines, or in the margins of legal manuscripts of texts going back to Roman times, and created a new discipline. In the form of web annotation, the idea is back, with texts being marked up inline, or with a stand-off system. Where could it lead?

ContentMine operates in the field of text and data mining (TDM), where annotation, simply put, can add value to mined text. It now sees annotation as a possible advance in semi-automation, the use of human judgement assisted by bot editing, which now plays a large part in Wikidata tools. While a human judgement call of yes/no, on the addition of a statement to Wikidata, is usually taken as decisive, it need not be. The human assent may be passed into an annotation system, and stored: this idea is standard on Wikisource, for example, where text is considered "validated" only when two different accounts have stated that the proof-reading is correct. A typical application would be to require more than one person to agree that what is said in the reference translates correctly into the formal Wikidata statement. Rejections are also potentially useful to record, for machine learning.

As a contribution to data integrity on Wikidata, annotation has much to offer. Some "hard cases" on importing data are much more difficult than average. There are for example biographical puzzles: whether person A in one context is really identical with person B, of the same name, in another context. In science, clinical medicine require special attention to sourcing (WP:MEDRS), and is challenging in terms of connecting findings with the methodology employed. Currently decisions in areas such as these, on Wikipedia and Wikidata, are often made ad hoc. In particular there may be no audit trail for those who want to check what is decided.

Annotations are subject to a World Wide Web Consortium standard, and behind the terminology constitute a simple JSON data structure. What WikiFactMine proposes to do with them is to implement the MEDRS guideline, as a formal algorithm, on bibliographical and methodological data. The structure will integrate with those inputs the human decisions on the interpretation of scientific papers that underlie claims on Wikidata. What is added to Wikidata will therefore be supported by a transparent and rigorous system that documents decisions.

An example of the possible future scope of annotation, for medical content, is in the first link below. That sort of detailed abstract of a publication can be a target for TDM, adds great value, and could be presented in machine-readable form. You are invited to discuss the detailed proposal on Wikidata, via its talk page.

• Jon Udell, blogpost Annotating to extract findings from scientific papers, 15 December 2015
• TDM and Libraries, Virginia Tech report
• Magnus Manske, The Whelming: Scaling up Wikidata editing
• OCLC and Internet Archive collaborate to expand library access to digital collections, metadata and linking exchange
• GLOW week in November: Wikidata workshops on politician info

Contest details: create biographical articles for women of any country or occupation in the world: November 2017 WiR Contest

Read more about how Women in Red is overcoming the gender gap: WikiProject Women in Red

(To subscribe: Women in Red/English language mailing list and Women in Red/international list. Unsubscribe: Women in Red/Opt-out list)

Under the heading rerum causas cognescere, the first ever Wikidata conference got under way in the Tagesspiegel building with two keynotes, One was on YAGO, about how a knowledge base conceived ten years ago if you assume automatic compilation from Wikipedia. The other was from manager Lydia Pintscher, on the "state of the data". Interesting rumours flourished: the mix'n'match tool and its 600+ datasets, mostly in digital humanities, to be taken off the hands of its author Magnus Manske by the WMF; a Wikibase incubator site is on its way. Announcements came in talks: structured data on Wikimedia Commons is scheduled to make substantive progress by 2019. The lexeme development on Wikidata is now not expected to make the Wiktionary sites redundant, but may facilitate automated compilation of dictionaries.

And so it went, with five strands of talks and workshops, through to 11 pm on Saturday. Wikidata applies to GLAM work via metadata. It may be used in education, raises issues such as author disambiguation, and lends itself to different types of graphical display and reuse. Many millions of SPARQL queries are run on the site every day. Over the summer a large open science bibliography has come into existence there.

Wikidata's fifth birthday party on the Sunday brought matters to a close. See a dozen and more reports by other hands.

• Wikidata statistics
• I4OC progress in its first year, with 47% of scientific citation data now open (announced two days ago)
• The flowering ORCID, Magnus Manske blogpost on identifying authors of scientific papers
• @querybook, a Twitter feed devoted to SPARQL queries
• Massive progress on Wikidata coverage of the UK parliament
• Reminder: WikiFactMine pages on Wikidata are at WD:WFM

At the beginning of December, Wikidata items on individual scientific articles passed the 10 million mark. This figure contrasts with the state of play in early summer, when there were around half a million. In the big picture, Wikidata is now documenting the scientific literature at a rate that is about eight times as fast as papers are published. As 2017 ends, progress is quite evident.

Behind this achievement are a technical advance (fatameh), and bots that do the lifting. Much more than dry migration of metadata is potentially involved, however. If paper A cites paper B, both papers having an item, a link can be created on Wikidata, and the information presented to both human readers, and machines. This cross-linking is one of the most significant aspects of the scientific literature, and now a long-sought open version is rapidly being built up.

The effort for the lifting of copyright restrictions on citation data of this kind has had real momentum behind it during 2017. WikiCite and the I4OC have been pushing hard, with the result that on CrossRef over 50% of the citation data is open. Now the holdout publishers are being lobbied to release rights on citations.

But all that is just the beginning. Topics of papers are identified, authors disambiguated, with significant progress on the use of the four million ORCID IDs for researchers, and proposals formulated to identify methodology in a machine-readable way. P4510 on Wikidata has been introduced so that methodology can sit comfortably on items about papers.

More is on the way. OABot applies the unpaywall principle to Wikipedia referencing. It has been proposed that Wikidata could assist WorldCat in compiling the global history of book translation. Watch this space.

And make promoting #1lib1ref one of your New Year's resolutions. Happy holidays, all!

• WikidataCon: Giving more people more access to more knowledge, report by Peter Kraker of Open Knowledge Maps
• This is a story of my knowledge adventure in New Zealand moths via Wikicommons, Wikipedia and Wikidata, @SiobhanLeachman
• Wikidata and Arabic dialects, research paper, DOI: 10.1109/AICCSA.2017.115
• c:Commons:British Library/Mechanical Curator collection/georeferencing status, Mechanical Curator project on Commons hits 50K maps milestone
• Historical dataset on the provenance of Wikipedia text: Who wrote this?, by Tilman Bayer, WMF blogpost
• "Anyone can edit", not everyone does: Wikipedia and the gender gap (PDF), journal paper, Heather Ford and Judy Wajcman
• Alpha Zero’s "Alien" Chess Shows the Power, and the Peculiarity, of AI, MIT Technology Review, by Will Knight, December 8, 2017

From the days of hard-copy liner notes on music albums, metadata have stood outside a piece or file, while adding to understanding of where it comes from, and some of what needs to be appreciated about its content. In the GLAM sector, the accumulation of accurate metadata for objects is key to the mission of an institution, and its presentation in cataloguing.

Today Wikipedia turns 17, with worlds still to conquer. Zooming out from the individual GLAM object to the ontology in which it is set, one such world becomes apparent: GLAMs use custom ontologies, and those introduce massive incompatibilities. From a recent article by sadads, we quote the observation that "vocabularies needed for many collections, topics and intellectual spaces defy the expectations of the larger professional communities." A job for the encyclopedist, certainly. But the data-minded Wikimedian has the advantages of Wikidata, starting with its multilingual data, and facility with aliases. The controlled vocabulary — sometimes referred to as a "thesaurus" as term of art — simplifies search: if a "spade" must be called that, rather than "shovel", it is easier to find all spade references. That control comes at a cost.

Case studies in that article show what can lie ahead. The schema crosswalk, in jargon, is a potential answer to the GLAM Babel of proliferating and expanding vocabularies. Even if you have no interest in Wikidata as such, simply vocabularies V and W, if both V and W are matched to Wikidata, then a "crosswalk" arises from term v in V to w in W, whenever v and w both match to the same item d in Wikidata.

For metadata mobility, match to Wikidata. It's apparently that simple: infrastructure requirements have turned out, so far, to be challenges that can be met.

• 1lib1ref campaign starts today, see m:The Wikipedia Library/1Lib1Ref: also #1lib1ref introductory video by Felix Nartey
• Funders should mandate open citations, article 9 January 2018 in Nature by David Shotton
• From snowflake to avalanche: Possibilities of using free citation data in libraries, translation from the German original of Annette Klein, Mannheim University Library
• outreach:GLAM/Newsletter/December 2017/Contents/WMF GLAM report
• Why Mickey Mouse’s 1998 copyright extension probably won't happen again: Copyrights from the 1920s will start expiring next year if Congress doesn't act, Timothy B. Lee, 8 January 2018, Arstechnica