User:John Cummings/wikidataimport guide

Importing data into Wikidata requires many skills, however the process can be broken down into individual steps. This means that the Wikidata community can work together to import data. The prerequisite skills to get started importing data are:

• Creating and editing wiki pages including interacting with Wikidata community members.
• Moving information into a spreadsheet and duplicating sheets within a spreadsheet including, additional helpful skills include:
  • Splitting cells before and after a certain character into separate columns
  • Find and replace text
  • Splitting column with different prefixes into separate columns (by using an A-Z filter)
  • Extracting URL links from hyperlinked text, alternatively you can create a copy of this spreadsheet.

The process of uploading data to Wikidata can be broken down into the following steps which can be broken down further into the following stages:

Preparing the data requires minimal technical skills and importing data into Wikidata can be done by either requesting the data be imported by bot (highly recommended) or by importing it yourself (only for experts and not yet documented).

Data imported into Wikidata must be:

1. Reliable.
2. Publicly available and preferably online.

If in doubt please ask about the dataset on the Partnerships and Data Imports discussion page.

Part A: Go to the Data Import Hub and follow instructions to create a new request.

Part B: Add the table and subheading as outlined in the Instructions for data importers section of the Data Import Hub. Please do this even if you are going to import the data yourself, it allow others to help you and understand what you have done for future updates to the data.

Complete the Description of dataset section of the table. This will mostly be a repetition of the data import request but please provide any other useful information on the dataset:

• Name: Name of the dataset, if a formal name does not exist please create a descriptive title.
  Source: The source of the data e.g the organisation who produced it.
  Link: A public link to the data or a document that structured data will be created from.
  Description: A description of the dataset including any information that is useful to know about the dataset.

Part A: Import the data into a spreadsheet, it is strongly recommended you create an online spreadsheet to work from as it will allow others to understand the issues and collaborate on importing a dataset. It is very helpful to include ID numbers if they exist for the uploading process.

Part B: Complete the following fields within the Create and import data into spreadsheet section in the table on the Data Import Hub

• Link: Add a link to the spreadsheet you have created
• Done: Add information about what tasks you have done within the spreadsheet, if you have not imported all the data add tasks to be completed in the To do field. Once all tasks have been completed simply write 'all' in the field.
• To do: Any tasks that need to be completed to import the data into the spreadsheet.
• Notes: Any extra information that are useful to know e.g if any changes were made to the original dataset, if there were spelling mistakes etc.

This step is often the most difficult, however there are many knowledgeable people within the Wikidata community that will be able to work with you to accomplish this step on the Wikidata:Partnerships and data imports page.

Part A: Look at the Wikidata glossary to understand the terms used in the following steps.

Part B: Look at examples of potentially similar data within Wikidata to understand what structure is already used for items.

• Showcase items provide examples of items with very rich levels of data within Wikidata
• Use the search function to search for items which may hold similar information stored in a way that could be copied for this data set.

Part C: Outline the structure within Wikidata in the table on the dataset import page. The dataset will need to be broken down into which parts of the data will be items, properties and values and if any qualifiers are needed. Also any issues or notes about the data e.g if the data is complete or if the data is related to any other datasets. Add what work has been done to the and any work still to do e.g propose properties. If you need help with defining the structure of the dataset ask on the data imports talk page.

• Items: Which part of the data will become items or use existing items
• Descriptions: the descriptions used for the items.
• Properties: What property or properties will be used. You can search current properties on the Property list page. If any new properties need to be created you can propose them on the Property Proposal page.
• Qualifiers: If any qualifiers are needed
• Values: Which parts of the data will be used as values.
• References: This can be one reference for the entire dataset or many.

Part D: Create one or more example items with the data structured in the way described, these practical examples will show how the data will be structured within an item and surface any issues in implementing the proposed data structure.

Part A: Duplicate the Original dataset sheet within your spreadsheet and rename the copy Structured for Wikidata.

Part B: Reorder your spreadsheet to use the following structure to make it easier for the people importing the data into Wikidata. A downloadable version of this format is available here.

Step A: Request data be imported into Wikidata on the Wikidata bot request page. To make a request click on the Add a new request button and then link to your request on the Data Hub page.

Step B: Check the Manual work needed section of the table once the import has started to see what work has to be done manually.

Before you can import data about any list of items, you will need to know the corresponding Wikidata Id numbers for each item in the list (essentially each row of your spreadsheet). You will also sometimes need to find Wikidata Ids for people, places, concepts or other things that are used to describe your main list of items.

a) Matching 'main' items

Finding a Wikidata item number for each 'row' in the data (e.g. the item number for each painting if the dataset was a collection of paintings). The options you have for this step are:

• Mix'n'match - Use this tool when a significant amount will need to be matched manually by community members.

OR

• Other semi-auto matching - When the data has some unique IDs, URLs or other data in common with Wikidata that allow you to generate a list of Q numbers (e.g. by running a query and matching common data in your import spreadsheet).

b) Matching items used as values

Finding Q numbers for any additional data to be added. E.g. a painting which has "country of origin" = "Italy", needs to become P495 = Q38 (Note: The properties should have been found already in Step 5 above).

The method you use for for this will depend on the specific case, but the Wikipedia and Wikidata Tools addon for Google Sheets would probably be the best starting point (e.g. for turning a column of country names in to corresponding Wikidata items).

With any method used, you may end up with some manual work at the end to finish it off.

• QuickStatements - For when there are statements, labels, descriptions etc that vary from item to item.

or

• PetScan - For when you need to add the same statement(s) to a list of items.

or

• Bot import - For when the job is too complex to use QuickStatements / petScan. You can request a bot import here.

Once the data has been imported into Wikidata request a query to ensure your data has been imported correctly at Wikidata:Request a query.This step ensures the data has been imported correctly and highlights any issues that may come about from importing data. A list of useful queries to check the data has been imported properly will be added here soon.

Part A: Add information the Date import complete and notes section of the table on the dataset import page adding a date that the data import into Wikidata is complete and any notes about issues or anything else people should know to improve or maintain the data e.g when an updated version of the dataset will be released.

Part B: Move the dataset from the Current datasets being imported section to the Completed datasets section.