Skip to content

feat(connectors): add EMIP programmer gaze dataset - #839

Draft
Flamour555 wants to merge 1 commit into
OpenTSLM:mainfrom
Flamour555:feature/emip-programmer-gaze
Draft

Flamour555 wants to merge 1 commit into
OpenTSLM:mainfrom
Flamour555:feature/emip-programmer-gaze

Conversation

@Flamour555

Copy link
Copy Markdown

What

A connector for EMIP, the Eye Movements in Programming dataset (Bednarik et al., 2020;
CC-BY-4.0; https://osf.io/53kts). As far as I can tell, it is the first eye-tracking / program-comprehension
dataset in TimeNet.

emip/programmer-gaze: 216 programmers each read two short programs (rectangle, vehicle)
in Java, Python or Scala on an SMI RED250 tracker at 250 Hz.

  • Records: one per code-reading trial (423). Each has four irregular series with one value per
    fixation: gaze x and y (px), fixation duration (ms), and mean pupil diameter (mm). Time
    offsets are µs since the stimulus appeared.
  • Tasks: one ClassificationTask per record. The target is the participant's self-reported
    expertise_programming: none / low / medium / high (11 / 45 / 120 / 40 participants).
  • Annotations: expertise in the experiment language, task, trial order, and whether the
    comprehension answer was correct. Record.metadata keeps participant_id, so users can make
    subject-disjoint splits. The release defines no split, so the connector imposes none.

How fixations are derived

The release ships raw gaze samples, not fixations. The connector runs a dispersion-threshold
(I-DT) pass over the right-eye samples:

  • dispersion is at most 25 px, the EMIP-Toolkit (Al Madi et al., 2021) value;
  • fixations last at least 80 ms.

A sample is invalid when the tracker flags it, when the point of regard is off-screen, or when the
pupil is ≤ 0. The tracker reports a lost eye as a "valid" sample at (0, 0) with pupil 0, so the last
two checks are needed. An invalid sample ends the running fixation. Trials with no fixation get no
record: 9 of the 432 trials, from very noisy tracking. For example, participant 2 has ~15 px
sample-to-sample jitter. No value is filled or repaired.

A trial runs from the stimulus image message to the next image message. Mouse-click messages
don't end a trial.

Files

packages/timenet-connectors/src/timenet_connectors/datasets/emip/
  __init__.py
  programmer_gaze/{__init__.py, connector.py, dataset.yaml, tests/test_connector.py}

The archive is pinned by sha256 (716c035b…9968). The connector has no extra requirements; it
uses only numpy and the standard library.

Verification

  • ruff format --check, ruff check (repo-wide) and ty check on the connector: clean.
  • Connector tests: 5 pass. They use synthetic SMI TSV fixtures and check:
    • trial segmentation;
    • a blink splitting fixations;
    • mouse clicks not ending a trial;
    • the minimum-duration rule;
    • an unknown label raising an error;
    • a gaze file / questionnaire mismatch raising an error.
  • timenet-build build emip/programmer-gaze --local-registry …: built 1.0.0. The record count
    matches the connector's direct conversion of the real archive (423 records and tasks), and the
    release reads back with TimeNet(...).open(...).
  • Ran on Windows. make was unavailable, so I ran the ruff/ty/pytest commands directly. One existing
    test, tests/test_build_env.py::test_uv_command_passes_the_requirements_file, fails on Windows
    only: it compares \fake\reqs.txt with /fake/reqs.txt. It is unrelated to this change.

Context

Built for GlanceNet, a hackathon project that classifies programmer expertise from fixation
sequences with OpenTSLM-SP. The connector stands on its own; nothing in it is specific to that model.

🤖 Generated with Claude Code

Add emip/programmer-gaze: eye fixations of 216 programmers reading two
short programs (EMIP, Bednarik et al. 2020, CC-BY-4.0). One record per
code-reading trial with four irregular series (gaze x/y in px, fixation
duration in ms, pupil diameter in mm) and one ClassificationTask whose
target is the self-reported programming expertise (none/low/medium/high).

Fixations come from an I-DT pass over the right-eye 250 Hz samples
(<= 25 px dispersion, >= 80 ms); samples where the tracker lost the eye
end the running fixation. The archive is pinned by sha256.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Flamour555

Copy link
Copy Markdown
Author

Forks can't set labels, so could a maintainer add `area:connectors`? Thanks!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants