Data & Hands-on: Where to start and how to get to L0

What this card changes in practice

On this page, a support-state addition now stays at family-split augmentation evidence unless the dataset card logs the fields above and then compares neuron-first baseline, strongest single added row, and full bundle under the same split, availability slice, and abstention rule. If the bundle mixes living-human proxy classes, use the Verification: Human Proxy Composition Card alongside the Verification: Fusion Card instead of treating co-acquisition as sufficient.

Reproducibility depends on the full execution chain

OpenNeuro and PhysioNet are entry points, but they do not guarantee reproducibility by themselves. First fix the snapshot / version, then align it with BIDS / EEG-BIDS, fix the reading and conversion path with tools such as MNE-BIDS, and finally define the comparison setting with a benchmark harness such as MOABB for within-session / cross-session / cross-subject. If you mix up repository, loader, and benchmark settings, the same dataset name will still yield incomparable results.

Why this distinction matters scientifically

Musall et al. (2019) showed that neural activity during tasks can be strongly dominated by uninstructed movements, and Egger et al. (2024) showed over a 10-hour EEG day that movement-related decoding changes enough to motivate adaptive decoders. But fast labels are still not the whole state story. de Quervain et al. (1998) and Oei et al. (2007) showed glucocorticoid-linked retrieval impairment and reduced human hippocampal / prefrontal retrieval activity, Barone et al. (2023) plus Birnie et al. (2023) showed circadian and corticosteroid-rhythm control of hippocampal plasticity, and Sherman et al. (2015) showed that memory-linked hippocampal activity varies with circadian-rhythm consistency. Finally, Wilson et al. (2025) showed that long-term BCI operation still requires recurrent recalibration. In other words, even for the same subject, short-term resolution, cross-day tolerance, and long-term operation are different barriers, and state annotation has to split fast labels from slow internal-milieu disclosure before temporal validity is read strongly.

2026-04-03 addendum: state annotation is not one free-text field

The remaining practical weakness on this page was subtler than simple split naming. It already separated cross-session, adaptation, and long-term use, but it still left state annotation too close to one free-text note about movement, arousal, or session ID. The current primary literature does not support that shortcut. Egger et al. (2024) showed over a 10-hour EEG day that movement-related decoding changes enough to motivate adaptive decoders, while de Quervain et al. (1998), Oei et al. (2007), Barone et al. (2023), Birnie et al. (2023), and Sherman et al. (2015) show that the same visible task can still run under different glucocorticoid, circadian, and broader slow internal-milieu regimes. At the same time, Wilson et al. (2025) and Wairagkar et al. (2025) show that recurrent recalibration burden and fast same-day throughput are different again. Therefore, on this site, a temporal claim now has to disclose fast labels such as movement / arousal / task mode separately from any relevant slow internal-milieu disclosure such as time-of-day / circadian phase, recent sleep-wake schedule, glucocorticoid or steroid exposure, and feeding / fasting or glucose-insulin regime, before fixed decoder interval, recalibration burden, or transfer ceiling are interpreted.

2026-03-18 addendum: fix the route behind the same score

Even when a within-session score is high, it can still be explained by eye-movement confounds shown by Mostert et al. (2018), the EMG route shown by McFarland et al. (2005), post-onset auditory feedback shown by Chen et al. (2024), identity confounding shown by Chaibub Neto et al. (2019), time-robust resting-state fingerprints shown by Wang et al. (2020) and Di et al. (2021), or subject-driven EEG variation summarized by Gibson et al. (2022). For that reason, this site now overlays the Verification: Specificity & Shortcut Card on dataset cards and baseline results, fixing plausible nuisance routes, auxiliary channels such as EOG / EMG / behavior / audio / metadata, nuisance-only baselines, fingerprint audit, nuisance-regime hold-outs, and the claim that must stop here.

2026-04-04 correction: acquisition distribution needs a recording-frame contract

This site previously stopped more clearly at subject / session fingerprint than at setup effects. That remained too weak for dataset cards. The official EEG-BIDS specification already separates electrodes, channels, coordinate system, and reference scheme. Hu et al. (2018) showed that the measured scalp potential itself changes with reference montage and electrode setup, Melnik et al. (2017) showed that EEG recordings vary not only by subject and session but also by recording system, Xu et al. (2020) showed that cross-dataset EEG decoding is degraded by environmental variability such as amplifier, cap, sampling rate, and filtering, Ceballos-Villegas et al. (2022) explicitly modeled multinational batch effects across studies and devices, and Dong et al. (2024) showed that cross-location comparison required an explicit REST-based offline transform rather than a generic claim that the datasets had already been harmonized. Therefore, this site now treats site / device / reference system / electrode layout / coordinate route / protocol distribution as a recording-frame contract rather than as harmless metadata. Inference from these sources: common-channel intersection, interpolation to a target montage, and REST-based transformation preserve different benchmark objects, so a dataset card now has to name the harmonization branch rather than only say that setup differences were handled.

2026-03-25 addendum: benchmark governance is part of the benchmark

The next practical weakness on this page was that split / leak / harmonization were visible, while benchmark governance could still be treated as administrative detail. The current primary and official sources do not support that shortcut. The official EEG Challenge (2025) homepage states that the original challenge preprint became outdated during execution and that the website plus starter kit should be treated as current. The official rules require disclosure of additional pretraining datasets, pretrained models / fine-tuning method, code submission at inference stage, and a single-GPU 20 GB inference budget. The official leaderboard then disclosed that Challenge 2 samples had not been randomized, allowing contiguous-trial same-subject structure to affect what the ranking meant and forcing separate awards. That warning is aligned with benchmark-side primary sources: Xiong et al. (2025) argued that inconsistent evaluation protocols make cross-model EEG-FM comparisons unreliable, and Liu et al. (2026) showed across 12 open-source foundation models and 13 datasets that ranking depends materially on transfer regime and benchmarking choices. Therefore, when this site reads a leaderboard, challenge result, or foundation-model benchmark, the card must also name benchmark version, split / randomization rule, hidden grouping structure, extra-data / pretrained-checkpoint policy, adaptation regime, inference-stage restrictions, and later organizer postmortems. If those fields are missing, we treat the result only as a qualified benchmark snapshot, not as a stable measure of portable EEG generalization.

2026-03-31 addendum: benchmark name is not yet the benchmark object

The next practical weakness was narrower. Even after benchmark governance became visible, a reader could still talk as if the benchmark name fixed the predicted object, independent prediction unit, grouped hold-out unit, adaptation regime, and operations budget. The current primary and official sources do not support that shortcut. The official EEG Challenge (2025) homepage separates Challenge 1 response-time regression from Challenge 2 subject-level externalizing prediction, the official rules and submission page add an inference-only code-submission workflow under a single-GPU 20 GB budget, Ma et al. (2022) use one five-session motor-imagery dataset to separate within-session, cross-session, and cross-session adaptation, Liu et al. (2026) separate leave-one-subject-out cross-subject evaluation from within-subject few-shot calibration, and Lahiri et al. (2026) show that six benchmark inconsistencies can reverse rankings on identical datasets by up to 24 percentage points. Therefore, on this site, a dataset or leaderboard name is still too coarse until the object / unit / budget matrix is disclosed explicitly.

2026-04-02 addendum: setup diversity is not yet physiology-equivalent transfer

The next weak point on this page was different from benchmark governance. A dataset or benchmark card could already expose site / device / reference / layout diversity and still leave a reader with the impression that a setup-agnostic foundation model had already solved physiology-preserving transfer. The current primary literature does not support that shortcut. Han et al. (2025) target channel-permutation equivariance, Chen et al. (2025) target coordinate-based adaptation across heterogeneous devices and more than 150 layouts, and El Ouahidi et al. (2025) push setup-agnostic pretraining to more than 60,000 hours from 92 datasets and 25,000 subjects. Those papers advance recording-frame compatibility. They still do not prove that different montages, coordinate routes, and reference families already preserve one shared physiology-side representation. Ma et al. (2026) then show that strong EEG foundation models can still generalize poorly when subject-level supervision is limited unless extra adaptation structure is added. Therefore, this page now treats setup diversity, coordinate route, reference family, omitted-channel policy, and label-limited adaptation burden as separate dataset / benchmark-card fields rather than one merged claim of portable generalization.

Site rule from this section

From this section onward, dataset cards and baseline results must report at least (1) evaluation family, (2) benchmark object plus independent prediction unit, (3) the independent hold-out unit, (4) raw-recording / window ancestry, (5) subject / session / site / device / reference-system / electrode-layout disjointness together with metadata-only baselines, (6) the channel-map / coordinate-route / reference-family / omitted-channel / sample-rate / filter harmonization log, including whether comparison used common-channel intersection, interpolation to a target montage, REST / another explicit transform, or no cross-setup harmonization, (7) whether target-session, target-subject, or target-site data were used, (8) recalibration amount and timing or extra label budget, (9) for leaderboard or challenge claims, benchmark provenance including version, split / randomization rule, hidden grouping, extra-data / checkpoint policy, inference-stage restrictions or operations budget, and postmortem disclosures, and (10) a stopping claim. If the claim spans more than one session or day, it must additionally disclose the site's Temporal Validity fields: state annotation split into fast labels and slow internal-milieu disclosure, fixed decoder interval, recalibration burden, and transfer ceiling. Scores without this context will be treated as limited L1 decode results, fingerprint-unresolved / acquisition-distribution-unresolved classifiers, or benchmark-object-unresolved / benchmark-governance-unresolved leaderboards rather than evidence of long-term stability or deployability.

2026-03-25 addendum: metric semantics are part of the benchmark

The next practical weakness on this page was that split / leak / harmonization and benchmark governance were visible, while metric semantics could still hide behind one headline number. The primary literature does not support that shortcut. Saito & Rehmsmeier (2015) showed why precision-recall views can be more informative than ROC summaries under strong class imbalance. In seizure tasks, Roy et al. (2021) and Scheuer et al. (2021) show that practical evaluation still turns on sensitivity, false alarms per hour or day, event-overlap logic, and latency rather than plain accuracy, while Segal et al. (2023) show that false-alarm control is itself a design target in seizure prediction rather than an afterthought. In sleep staging, Sun et al. (2017) used Cohen's kappa and showed that imbalance in stage proportions changes performance estimates, while Vallat & Walker (2021) show that pooled performance can still hide especially weak N1-stage agreement. Therefore, on this site, a dataset or benchmark card must now also disclose a task-matched metric bundle, not only a split and a score.

The last two columns in this section reflect this site's operating logic

The stopping claims and minimum operational rules in the table below are operational boundaries drawn from what is directly observed and annotated in the official dataset descriptions and primary literature. In other words, they are not claims explicitly made by the dataset providers; they are site rules derived from annotation provenance and time fidelity.

Why these stop-lines are evidence-backed rather than site style

The official EEG Motor Movement/Imagery dataset description itself fixes the ceiling: 109 volunteers, 64 channels, 14 cue-driven runs, 160 Hz, and T0/T1/T2 onset codes copied into both the annotation channel and .event files. That is enough to audit cue-locked decoding, preprocessing, and subject-split hygiene, but it is also why this site requires a separate audit of visual-cue, overt-movement, and myoelectric / ocular contributions before any stronger readout wording is allowed.

The official CHB-MIT description fixes a different ceiling: 22 pediatric subjects organized into 23 cases, case chb21 being the same subject as chb01, gaps between consecutively numbered EDF files, and seizure boundaries carried by .seizure files together with case summaries. Therefore file-level randomization overstates independence unless subject identity and case chronology remain explicit.

The official Sleep-EDF description likewise constrains the interpretation: the PSG uses only Fpz-Cz / Pz-Oz EEG together with EOG and chin EMG, while the event marker and some auxiliary channels are sampled at 1 Hz, and the hypnograms are manual Rechtschaffen & Kales scores. Rosenberg & Van Hout (2013) then showed that even modern sleep-stage scoring reaches only about 82.6% overall inter-scorer agreement, with weaker agreement for N1 and N3. That is why this site stops a Sleep-EDF result at staged-state practice unless label mapping, scoring regime, and time granularity are disclosed explicitly.

For TUH / TUSZ, Obeid & Picone (2016) explain that the clinical corpus pairs EDF recordings with clinician reports, while Shah et al. (2018) describe seizure-rich triage using report keyword search and automatic detectors. Later corpus-maintenance notes documented that an early Neureka 2020 release had non-exclusive subjects across train / dev / blind evaluation and high-frequency seizure annotation problems. Therefore report-usage flags, patient/session ancestry, and benchmark postmortems are treated on this site as part of the result rather than footnotes.

The most important site rule to add now

When introducing starter data, always include (1) label provenance, (2) time granularity, (3) clock domain plus stream-alignment rule, (4) timing-validation class, (5) event semantics, (6) independent split unit, (7) acquisition-distribution summary plus harmonization policy, and (8) stopping claim. A dataset card that does not include this will be considered insufficient as a practical guide for L0.

BIDS is a requirement, but not a ground truth

BIDS/EEG-BIDS is important, but it alone cannot prove the validity of source imaging or the comparability of cross-dataset decoding. The BIDS specification itself also requires EEGReference, SamplingFrequency, SoftwareFilters, and if *_electrodes.tsv is issued, *_coordsystem.json is also required. However, this is a condition that makes it possible for a third party to trace the incident, not a condition that allows the true source to be known or that automatically harmonizes reference mismatch, electrode-layout mismatch, and device / filter differences across cohorts.

If you want to claim ESI improvement, you need a different chain of evidence

Please provide at least the following four points.

• Individual anatomy: Individual MRI/CT or EEG-BIDS recordings with digitized electrode positions and *_electrodes.tsv / *_coordsystem.json
• Forward model audit: Head model and skull-conductivity sensitivity analysis
• External standards: Ground truth such as phantoms, simultaneous invasive recording, intracranial stimulation, or TMS-EEG
• Uncertainty: Report not only point estimates but also localization errors and interval estimates

Stage C is not one box

On this site, you should still write which C-stage validation class you used:

• stimulation ground truth: asks localization error against a known stimulation site and time (Mikulan et al., 2020; Unnwongse et al., 2023).
• simultaneous invasive recording: asks concordance with concurrent SEEG/ECoG under the same event regime (Hao et al., 2025).
• postsurgical outcome / clinical concordance: asks whether the source estimate points toward clinically relevant tissue, not whether the source was uniquely observed (Birot et al., 2014).

The most important thing now is the C stage public benchmark

Localize-MI by Mikulan et al. (2020) is a rare data resource that exposes intracerebral stimulation with 256ch scalp EEG and stereo-EEG, allowing source imaging to be directly audited against “known stimulation locations.” Hao et al. (2025) reported average localization errors of 14.07 mm for ictal ESI and 17.38 mm for interictal ESI in 29 simultaneous HD-EEG/SEEG cases, indicating that source power and source depth greatly affect accuracy. Jahromi et al. (2026) then added a 3D-printed pediatric deep-source phantom, showing that even phantom validation is not one universal board once deep epileptic source class and geometry are changed. Therefore, if you want to improve your source imaging, you need not only a C-level benchmark, but also a named C-stage validation class rather than an A-level starter dataset alone.

High density is no longer the only credible route, but it is still not a solver-free shortcut

One remaining simplification on this page was to make the entrance sound too close to HD-EEG is the serious route and low-density EEG is the weak route. The current primary literature is narrower than that. Horrillo-Maysonnial et al. (2023) showed that a targeted 33-36-electrode montage reached 54/58 sublobar concordance (93%) against an 83-electrode HD montage, but still showed larger peak-vertex distance for tangential generators. Rong et al. (2025) then showed that a DeepSIF-based approach stayed comparatively stable from 75 to 16 electrodes, with average spatial dispersions of 7.9/9.0 mm versus 21.9/28.1 mm for sLORETA and 20.0/28.9 mm for LCMV. But these papers do not erase the direct-validation ceiling. Unnwongse et al. (2023) showed that coverage geometry and conductivity assumptions still move localization error in direct validation, and Hao et al. (2025) showed that ictal and interictal ESI differed (14.07 ± 4.62 mm versus 17.38 ± 4.16 mm) and that source depth and spike power still matter. Therefore, this site no longer treats high density versus low density as the right first split. The safer split is named montage / coverage policy + inverse family + source regime + validation class.

Postoperative outcomes can be used, but should not be equated with ground truth

In a systematic review by Mouthaan et al. (2019), the summary sensitivity of electric source imaging in presurgical epilepsy was 82% and specificity was 53%. In other words, although postoperative outcomes and SOZ concordance are useful external criteria, source imaging itself cannot be fixed as the true value. Even at the C stage, what you can say now is ``How far has the error been reduced with this benchmark?'', not ``I was able to uniquely read the source in my brain.''

Practical reading

The first question when selecting data is not ``what is interesting?'' but what level of argument do you want to support this time? Level A is sufficient for practicing L0-L1. If you want to proceed with claiming improvements in source imaging, please put your claim on hold unless you audit the head model in stage B and take direct validation in stage C. If you do proceed to solver comparison, the next section fixes what has to stay the same across methods before any leaderboard is accepted.

Inverse family is not just a solver style label

This page now has to stop another shortcut explicitly. A posterior-support map, a debiased sparse interval, and an extended-source overlap estimate are not three visualizations of one identical hidden object. Luria et al. (2024) return posterior support for focal alternatives, Tong et al. (2025) return debiased estimation / inference for sparse spatial-temporal sources, and Feng et al. (2025) return uncertainty-aware extended-source reconstructions. Therefore, a benchmark on this site must name not only the board, but also which inverse family was used, what target object it aimed to recover, and what uncertainty object it actually returned.

Centre error and source extent are different benchmark objects

Another shortcut still had to be blocked here. A benchmark can be “directly validated” and still score only the geometric center of a source. Feng et al. (2025) explicitly target extended-source reconstruction rather than focal-center localization. Hao et al. (2025) likewise note that their ECD-based distance comparison neglects the spatial extent of the seizure-onset and irritative zones, and argue that future localization work should incorporate source extent in addition to geometric center. Therefore, a solver that wins a focal-site distance board is not automatically the best method for distributed, extended, or propagation-rich sources, and a public benchmark on this site must state whether it scores centre, extent, overlap, or propagation pattern.

Site rule from this section

A public inverse-problem comparison on this site must now disclose at least (1) validation class, (2) source regime and target object (focal-centre / sparse / extended / propagation-aware), (3) inverse family plus the uncertainty object it returns, (4) same-geometry controls including montage / coverage policy, (5) sensitivity sweep over conductivity and key hyperparameters, (6) inter-method disagreement summary, and (7) the claim that must stop here. Without these fields, a result will be treated as a method illustration or lab-specific pipeline note, not as a reusable benchmark.

Official benchmark postmortems are part of reproducibility, not footnotes

In practical work, a benchmark page, rules page, submission constraint, and final leaderboard can each fix a different part of what your score means. Brookshire et al. (2024) show that segment-based cross-validation in translational EEG can leak subject information between training and test sets and inflate headline performance. The later TUH/TUSZ maintenance record also documented a public-release case where subject exclusivity and annotation quality had to be repaired after downstream use had already begun. That is why this page now routes EEG foundation-model benchmarking not only through split / leakage hygiene but also through benchmark provenance. If the benchmark uses evolving challenge operations, continue directly to Wiki: EEG foundation models and pretraining and Verification: Pretraining Card before treating the ranking as portable generalization evidence.

2026-03-20 addendum: the public L0 pack is now synchronized

The weakness of this page was that the public checklist had become stricter than the wiki page readers actually use when assembling an L0 submission. That gap is now closed. If you want the submission shape itself, not only the route on this page, go directly to Wiki: Minimum artifact pack for L0. The synced pack now fixes dataset identity, event fidelity, label provenance, evaluation family + hold-out ancestry, acquisition-distribution summary, derivative lineage, and stopping claim as first-class deliverables.

Step 0: Freeze the version

The dataset name alone is not enough. OpenNeuro manages snapshots with semantic-version Git tags, and PhysioNet also displays and cites dataset versions for each project. Therefore, the first runbook should record snapshot / version / DOI / retrieval date, not just the dataset name.

Step 1: Create the BIDS skeleton first

Even if the contents are not aligned at first, just fixing the placement will reduce rework. If you create a file name and metadata template with the premise of passing it through a validator, subsequent QC and comparisons will become much easier.

Step 2: Eliminate standard violations first with Validator

Eliminate any problems found with the machine at an early stage. Passing the BIDS Validator is not a sufficient condition for research, but it is close to the minimum requirement for sharing.

Step 2.5: Separate and fix loader and benchmark

MNE-BIDS is a tool that helps with BIDSPath handling, data loading, and metadata extraction, while MOABB fixes the paradigm and evaluation family. There is a difference between being able to read data and being able to make fair comparisons. In particular, MNE-BIDS treats write-back of modified or preloaded data as an exception, so it is safer to treat preprocessed data as derivatives with explicit lineage.

Step 3: Leave QC logs as numerical values ​​instead of waveforms

With just the raw waveform, it is difficult for a third party to reconstruct what went wrong and what was left out. The main body of L0 is to record bad channels, bad segments, event synchronization, stimulus logs, and reaction logs with numerical values ​​and threshold values.

Step 4: Fix only one baseline

Rather than using SOTA, first place a comparison axis that is easy to reproduce. Having an initial baseline allows you to compare what has improved even after updating the preprocessing or updating the model.