Metadata and describing data
Metadata is documentation that describes data.
Properly describing and documenting data allows users (yourself included) to understand and track important details of the work. In addition to describing data, having metadata about the data also facilitates search and retrieval of the data when deposited in a data repository.
In a lab setting, much of the content used to describe data is initially collected in a notebook; metadata is a more formal, sharable expression of this information. It can include content such as contact information, geographic locations, details about units of measure, abbreviations or codes used in the dataset, instrument and protocol information, survey tool details, provenance and version information and much more. Where no appropriate, formal metadata standard exists, for internal use, writing “readme” style metadata is an appropriate strategy.
Metadata formats and standards
Metadata can take many different forms, from free text (make this a link to my read-me) to standardized, structured, machine-readable, extensible content. Specific disciplines, repositories or data centers may guide or even dictate the content and format of metadata, possibly using a formal standard. Because creation of standardized metadata can be difficult and time consuming, another consideration when selecting a standard is the availability of tools that can help generate the metadata (e.g. Morpho allows for easy creation of EML, Nesstar for DDI data, etc.).
The Digital Curation Center provides a catalog of common metadata standards, organized by discipline: http://www.dcc.ac.uk/resources/metadata-standards.
Some specific examples of metadata standards, both general and domain specific are:
- Dublin Core - domain agnostic, basic and widely used metadata standard
- DDI (Data Documentation Initiative) - common standard for social, behavioral and economic sciences, including survey data
- EML (Ecological Metadata Language) - specific for ecology disciplines
- ISO 19115 and FGDC-CSDGM (Federal Geographic Data Committee's Content Standard for Digital Geospatial Metadata) - for describing geospatial information
- MINSEQE (MINimal information about high throughput SEQeuencing Experiments) - Genomics standard
- FITS (Flexible Image Transport System) - Astronomy digital file standard that includes structured, embedded metadata
- MIBBI - Minimum Information for Biological and Biomedical Investigations
Best Practices in Creating Metadata. ICPSR. http://www.icpsr.umich.edu/icpsrweb/content/deposit/guide/chapter3docs.html. Part of the ICPSR's Guide to Social Science Data Preparation and Archiving.
Marine Metadata Interoperability Project. https://marinemetadata.org/. Useful vocabularies, standards, and metadata tools.
Metadata Best Practices. DataONE. http://www.dataone.org/best-practices/metadata
Metadata Services. Cornell Research Data Management Service Group. http://data.research.cornell.edu/services#Metadata
Minimum Information for Biological and Biomedical Investigations. MIBBI Project. https://biosharing.org/standards/?selected_facets=isMIBBI:true&view=table. Minimum Information guidelines from diverse bioscience communities.