Skip to main content
x
Call for Community Review of the Hierarchical Event Descriptors (HED) - Endorsed on 2024-11-27

The purpose of this document is to solicit community feedback on the Hierarchical Event Descriptors (HED) that was submitted to INCF for endorsement as a standard. The document contains the INCF standards and best practices committee's review of SDS, and the criteria in which it was evaluated (open, FAIR, testing and implementation, governance, adoption and use, stability and support, extensibility and comparison to similar standards). For the next 60 days, we are seeking community feedback on SDS.           

About HED:           
Hierarchical Event Descriptors (aka HED) is an open standard for describing experimental events, conditions, and experiment organization in a format that is both human- and machine-readable to enable analysis, re-analysis, and meta/mega-analysis. Its goal is to annotate participant experience and behavior as well as the environmental influence during an experiment. Thus, HED is relevant to all neuroimaging modalities as well as behavioral experiments and MOBI experiments (motion capture, eye-tracking etc.). 

Events in context—The HED framework for the study of brain, experience and behavior. Front. Neuroinform. 23 May 2024

Summary of Discussion:           
Overall, the members of the INCF Standards and Best Practices Committee could see the potential of HED to meet the criteria for INCF endorsement. It clearly supports open and FAIR science. For EEG data, HED is a good way to describe common events; it provides a specification language for events and stimuli. HED is open, supports FAIR, and has appropriate testing and implementations; it also has appropriate documentation and a decent size portfolio of open tools implementing it, as well as a pipeline of open tools under development. HED benefits from a clear governance framework as well. While the committee felt that HED is broadly applicable and was clearly designed with becoming a community standard in mind, it is concerned about its adoption and use. The evidence provided by the submitters did not indicate that there are a massive number of users; but from the committee’s perspective there could be; therefore, feedback from the community on the benefits of HED are crucial.

HED application document

Response to INCF reviewers

Recommendation:         
INCF Standards and Best Practices Committee voted to put HED forward for community review.  

Authors and Affiliations

Standards and Best Practices Committee, International Neuroinformatics Coordinating Facility, June 2023

Competing Interests

No competing interests were disclosed

Keywords
EEG
Standardized vocabularies
Python

Comments

19 Comments
#1

Tyler Collins

Thu, 08/08/2024 - 18:31

SHARCNET
The strength of HED tags has personally enabled me to work with multiple researchers and studies that would otherwise have been impossible to signal what individual events mean in a timely manner.

I believe that wider adoption of HED would enable researchers to share their studies on a more global scale. It removes the ambiguity of what constitutes a target stimulus, to a distractor, to an attentional cue.

Not only does this schema explain what events are, it also allows for researchers to succinctly report on data quality properties.

I have personally adopted HED in all of my studies and will encourage all researchers I work with to do the same.
#2

Dora Hermes

Thu, 08/08/2024 - 19:00

Mayo Clinic
I am using HED to annotate stimuli for data in BIDS. We also developed a new schema library for annotations of electrophysiology data (HED-SCORE) that is being used by several large EEG platforms.
Competing Interests
I am a member of the HED working group.
#3

Stefan Appelhoff

Thu, 08/08/2024 - 20:49

Max Planck Institute
HED tags are an indispensable tool for consistently annotating large time series databases with rich metadata. The comprehensiveness and internal consistency of the HED vocabulary make it applicable in a large diversity of studies and other use cases.

In my opinion, if we could get more and more researchers and dataset curators to apply this standard to their data, all present and future researchers would benefit immensely (especially for efficient dataset re-use, re-analysis, and for meta and mega analyses, and all other endeavors to combine data from different sources). A potential endorsement by INCF and the linked publicity and trust would be a great step into that direction.
Competing Interests
As a BIDS maintainer, I have collaborated with core members of the HED working group, and I am co-author one of their papers (https://doi.org/10.1016/j.neuroimage.2021.118766).
#4

Sandor Beniczky

Fri, 08/09/2024 - 17:10

Danish Epilepsy Centre
We have been using the SCORE system for reporting clinical EEGs for more than 11 years. This lead to improved interrater reliability, and established a large clinical database - the bases for numerous publications and development projects.
#5

Jorge Bosch-Bayard

Tue, 08/13/2024 - 22:48

Universidad Autonoma de Madrid, Spain
I have participated in the HED-SCORE project as a member of the Global Brain Consortium and the EEGNet. We are promoting crowdsourcing annotations of EEG artifacts standardized in the EEG-BIDS format. This project is key for developing open science programs, education, reproducible research, international collaborations, and training AI algorithms for automatic annotations of different graph elements present in the EEG signal.
#6

Jonathan Dan

Fri, 08/23/2024 - 09:41

EPFL
BIDS, HED and HED-SCORE are the main standards used for the development of SzCORE, a framework for the validation of automated seizure detection algorithms.

Based on these standards, we developed a toolbox (epilepsy2bids) to convert existing epileptic EEG datasets to a common format which can be interpreted and analysed by all.

epilepsy2bids: https://github.com/esl-epfl/epilepsy2bids
SzCORE: https://arxiv.org/pdf/2402.13005
#7

Christian O'Reilly

Wed, 09/18/2024 - 19:49

University of South Carolina
HED tags are essential in the effort to automate the process of EEG. This proposal is very timely with the rise of machine learning and its application to neuroimaging. It is also essential for efforts related to open science and data sharing.
Competing Interests
None.
#8

Arnaud Delorme

Mon, 09/30/2024 - 21:03

UCSD
The Hierarchical Event Descriptors (HED) initiative represents a powerful step forward in enabling standardized, open, and FAIR science within the neuroimaging and experimental research community. The HED framework’s potential to enhance the description and analysis of experimental events across a wide range of modalities—from EEG to behavioral experiments—underscores its versatility and broad applicability. Its commitment to openness, robust testing, and solid governance, combined with a growing portfolio of tools, shows that HED is well on its way to becoming an essential standard for researchers. Community feedback during this period will be key to addressing concerns about adoption, and it offers an exciting opportunity for researchers to contribute to the future of this transformative tool.
Competing Interests
I sometimes attend the publically held HED working group meetings.
#9

Bernhard Pöll

Tue, 10/01/2024 - 14:59

Paris Lodron Universität Salzburg, Austria
With HED's built-in library extension system, I collaborated on developing a library extension tailored to language cognition research. The HED framework simplifies the addition of specialized vocabularies, making domain specific terminology more accessible and usable for neuroscientists.
#10

James Desjardins

Tue, 10/01/2024 - 17:15

SHARCNet, Brock University, Digital Research Alliance of Canada,
As an analysts and HPC support consultant working with several research groups attempting large scale and open EEG projects, HED is a practical solution to a key requirement of achieving the potential benefits of large scale open neuroscience. Limitations in understanding how experimental events and data property annotations relate to each other across projects in the absence of a platform for standardization like HED is the achilles heel of the substantial potential for discovery associated with large scale open neuroscience.

As a developer of procedures for the automation of standardized signal quality control HED is invaluable, providing the framework allowing for a common shared reference structure describing aspects of complex signals.

Finally, the future potential of AI/ML in neuroscience depends on a platform for standardized tagging of data property annotations like that provided by HED.
#11

Stefon van Noordt

Wed, 10/02/2024 - 17:32

Mount Saint Vincent University
The HED infrastructure is essential for scalable neuroinformatics, particularly for multi-site/multi-modal open science platforms for sharing, processing, and analyzing data. The long term life cycle and significance of data, for both fundamental and clinical research, is directly linked to the design and implementation of tools such as the HED framework. Continued development and support of such initiatives is essential for researchers.
#12

Yaroslav Halchenko

Sun, 10/06/2024 - 19:57

Dartmouth College
Data standards are the languages of science. Without them we cannot communicate efficiently, we cannot be sure we understand what data means. HED gives us language to describe what in our world resulted in the data we collected and analyzed. It is great to see HED going strong and getting wider adoption and better integration with other standards (BIDS, NWB) and archives (OpenNeuro, DANDI, etc). Only by learning to speak common language(s), can we make the most of our data.
Competing Interests
Involved with BIDS, NWB, DANDI, OpenNeuro.
#13

Monique Denissen

Tue, 10/08/2024 - 14:27

Paris Lodron Universität Salzburg, Austria
I have been working with the Austrian NeuroCloud to build a FAIR repository for neuroimaging data with active curation: https://anc.plus.ac.at/. In this work, I encountered HED as an effective way to organize task-based fMRI data for analysis across multiple datasets. Recognizing the potential of HED, I joined the HED working group, where I have been directly involved in developing tools that enable the use of HED annotations in fMRI analysis, particularly within the BIDS framework.

In my own research, I’ve successfully applied HED alongside BIDSapps like BIDSpm and FitLins to automate and streamline reproducible fMRI analysis pipelines. These tools have made it easier to handle combined datasets, enhancing consistency across different studies. I have also worked on a new library schema for HED, HED LANG, to enable the annotation linguistic events.

Moreover, I have used HED to annotate both existing and previously shared neuroimaging datasets. These annotations not only make the datasets more discoverable but also enable automated analysis across multiple datasets. HED’s ability to standardize event-related metadata is crucial for improving the reusability and interoperability of shared datasets, making it a vital tool for advancing data sharing and collaboration.
Competing Interests
I am a member of the HED working group
#14

Florian Hutzler

Wed, 10/09/2024 - 14:45

Austrian NeuroCloud at the University of Salzburg
At the Austrian NeuroCloud, we’ve built a repository for neuroimaging data where active curation ensures that our datasets meet the latest neuroinformatics standards and adhere to FAIR principles. Our repository supports a wide range of neurocognitive data, including fMRI, MEG, and EEG. We view the addition of Hierarchical Event Descriptors (HED) as a key part of data curation, and actively support it so that datasets are not only FAIR-compliant but also optimized for reuse and interoperability.

We offer hands-on support to our users, assisting them with HED annotations to make their data more discoverable and interoperable. Additionally, we’re developing a query interface that will allow users to search for datasets based on HED tags, significantly enhancing the findability of our data.

By adopting HED as an official INCF standard, we believe the neuroimaging community will benefit from easier data sharing, enhanced dataset discovery, and more streamlined analysis across platforms. At the Austrian NeuroCloud, our commitment to best practices in data sharing is reflected in our re3data entry (https://www.re3data.org/repository/r3d100014355), and we are confident that formalizing HED as a standard will play a pivotal role in advancing open science and fostering data reuse across the field.
#15

Rémi Gau

Thu, 10/10/2024 - 17:22

INRIA, Paris-Saclay, FRANCE
HED provides a rich standardized framework for researcher to annotate their experimental paradigm. Without such standard comparing or pooling results across datasets becomes if not impossible at least much more difficult.

The room for improvement for annotations in experimental paradigm in some fields is huge: one only needs to browse openneuro datasets for a few minutes to find fMRI datasets where the events file just mention 'stimulus' and 'response' as only descriptors of the what happened during the experiment with no link to even a published article with a method section describing what the task was.
One can create automated pipelines to fully reproducibly preprocess and analyze such datasets, but without proper description of the experimental context like the one HED can provide, those beautiful activation blobs on brain images are almost completely uninterpretable.

Sure one may worry that HED has not been widely adopted, but this reflects to me more a lack of structural incentive rather a lack of effort and tools on the part of the HED team to make their work usable.

Referring to the 'open science' pyramid of cultural change (https://www.cos.io/blog/strategy-for-culture-change), HED sits in the "make it possible" / "make it easy" layers. So at least when the incentive changes, the tooling will be there.
Competing Interests
BIDS maintainers (BIDS officially supports HED)
#16

Morgan Montoya

Thu, 10/10/2024 - 20:05

Mayo Clinic
The HED tags have been invaluable for our research. We have used them to label stimuli in an intuitive way that is both human readable and standardizable. By using HED along with the BIDS framework, it enables our data to be more accessible for both current and future users.
#17

Sid Segalowitz

Thu, 10/10/2024 - 20:18

Brock University
The future of EEG research suddenly opened up to immense possibilities with the open neuroscience movement. However, because this technology is relatively inexpensive and not overly burdensome to develop software for, there are too many "standard" protocols making true an open EEG neuroscience feasible - the BIDS formatting has helped a great deal, but what is left is to have a standardized method for annotating the research structures. With this, clinical (as well as experimental) EEG studies will become much more routine, pulling together the vast number of datasets available, tremendously increasing the statistical power of our research field. Proposing HED for the community makes eminent sense. Although my career is winding down (after 50 year in the field), I am very pleased to see this new concern for making EEG data more fully and easily sharable on the horizon. The future looks great!
Competing Interests
none
#18

Jochem Rieger

Sat, 10/12/2024 - 19:56

University of Oldenburg
Standardized metadata description provided by HED is essential for transparency of experiments, and helps to make shared data efficiently reusable. Without such standardized descriptions even openly shared data can be useless when it is incomprehensible to others. In order to prevent this situation, annotation with HED should become a standard practice not only for openly shared data but also for lab internal documentation and reuse.
#19

Saskia de Vries

Fri, 10/25/2024 - 18:17

Allen Institute for Neural Dynamics
Having standardized language to annotate and describe behavioral paradigms of experiments is crucial for studying behavior, especially when integrating data across teams and behaviors. HED Tags provides a much needed annotation language.
Expanding the tags to incorporate more behavioral concepts used in animal research is needed, and I think the HED framework will be able to support this expansion.
Competing Interests
member of NWB TAB