Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
bb07f88
feat(connectors): add the OpenTSLM M4-Caption connector
karhu Nov 17, 2026
b589063
docs(connectors): describe the M4-Caption connector and a parent dataset
karhu Nov 17, 2026
6217eec
refactor(connectors): raise the CSV cell limit before M4-Caption reads
karhu Nov 17, 2026
c6f426d
refactor(connectors): use the renamed CSV cell limit method
karhu Nov 17, 2026
7bea0f0
refactor(connectors): freeze the M4-Caption split in splits.json.xz
karhu Nov 17, 2026
4b69228
fix(connectors): reject non-numeric M4-Caption values
karhu Nov 17, 2026
368899e
docs(connectors): call the M4-Caption source the Polybox release
karhu Nov 17, 2026
5616a3f
refactor(connectors): drop the parse check from generate_splits.py
karhu Nov 17, 2026
7e3be54
docs(connectors): shorten the M4-Caption card
karhu Nov 17, 2026
5531bb5
docs(connectors): shorten M4-Caption docstrings and card
karhu Nov 17, 2026
12ac217
refactor(connectors): drop the float32 check of M4-Caption values
karhu Nov 17, 2026
0a591e4
refactor(connectors): give each M4-Caption series its split
karhu Nov 17, 2026
c60d82f
docs(connectors): say why the M4-Caption task has no prompt
karhu Nov 17, 2026
b9f8804
refactor(connectors): lower the M4-Caption cell limit to 200,000 char…
karhu Nov 17, 2026
775a372
docs(connectors): tag M4-Caption with demographics
karhu Nov 18, 2026
3c2e707
refactor(connectors): let the CSV reader handle long M4-Caption cells
karhu Nov 18, 2026
dffd82e
feat(connectors): give M4-Caption tasks the prompt of OpenTSLM's loader
karhu Nov 18, 2026
4d928c8
feat(connectors): stratify the M4-Caption split by frequency
karhu Nov 19, 2026
dae9fb0
feat(connectors): leave out M4-Caption error captions, keep duplicate…
karhu Nov 20, 2026
8bcd9de
Merge remote-tracking branch 'origin/main' into HEAD
masquare Nov 20, 2026
1bdf0cf
refactor(connectors): move opentslm/m4_caption to the new connector API
masquare Nov 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# M4-Caption

M4-Caption pairs each of the 100,000 series of the M4 forecasting competition with one caption that
GPT-4o wrote. OpenTSLM publishes it as a zip archive of CSV files, one series file and one caption
file for each M4 frequency.

For 706 series, the release has an error message of the caption API instead of a caption. There are
three messages, each after the prefix "Error generating caption: ": "Connection error." (630
series), "Error code: 404" (75) and "Request timed out." (1). The connector leaves out these 253
daily, 57 monthly and 396 quarterly series, so the dataset has 99,294 series.

The connector converts each series into one Record and one `AnswerTask`. The Record holds the
series as one float32 Signal on an ordinal axis, the M4 id in its metadata, and the M4 frequency as
an annotation. The task target is the caption. The task prompt is the prompt of the OpenTSLM loader
(the post-prompt of `M4QADataset`), the same for every series.

## Split

The release archive stores no split. `generate_splits.py` makes an 80/10/10 split with
`StratifiedSplitter`, stratified by M4 frequency. Validation and test each hold a tenth of every
frequency, so the small hourly and weekly frequencies keep their share in each split. The result
depends on the version of scikit-learn, so the script ran once. It wrote the 9,929 validation and
9,929 test series, by M4 id, to `splits.json.xz`. The connector reads that file and puts every
other series in the train split.

M4 has 90 pairs of series whose histories are equal value for value: 19 monthly, 35 quarterly and
36 yearly pairs. The two series of a pair have different M4 ids and different M4 test horizons. All
pairs but one also have different captions. A model that trains on one series of a pair sees the
exact input of the other. So the script groups the series by their values with the `group_by`
option of `StratifiedSplitter`, and each pair stays in one split.

This split is not the split of the OpenTSLM paper. The OpenTSLM loader makes a random split of all
frequencies together, so its share of the small frequencies drifts. For example, its test split
holds 6.5% of the hourly series.

The file is JSON compressed with xz, about 19 KB. Python reads it with the `lzma` module of the
standard library. To read it yourself, run `xz -dc splits.json.xz`. The file also states the SHA-256
of the release archive. If it does not match the pinned archive, the build stops with an error.

## Relation to the M4 data

Each series of the release is the train part of the M4 series with the same id, from the
`Dataset/Train` files of the M4-methods repository. The release does not hold the test horizon or
the series information of `M4-info.csv`. Each value is the float32 rounding of the M4 value. The
largest relative difference is 6e-8.

## Future work: an M4 parent dataset

TimeNet can model this relation with a parent dataset. A connector for M4, for example
`mcompetitions/m4`, can convert the `Dataset/` folder of M4-methods: the train and test files and
`M4-info.csv`. M4-Caption can then declare that release under `parents` and import its Records.
The caption task then names the train part of each series with a `StepInterval` scope.

With a parent, the 24 million values exist once, and a reader gets the M4 category, horizon and
test part. The values become the float64 M4 values, not the float32 values that OpenTSLM plotted
for GPT-4o.

## Future work: change axis types

We currently use an ordinal axis for consistency, as some of the frequencies are only regular in
calendar terms, e.g. "monthly". If
[Proposal: calendar axes, value cells and civil-date origins](https://github.com/OpenTSLM/TimeNet/issues/555)
lands we can reconsider this choice.
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
"""M4-Caption: the M4 competition series, each with a caption that GPT-4o wrote."""

from timenet_connectors.datasets.opentslm.m4_caption.connector import (
CONNECTOR as CONNECTOR,
M4CaptionConnector as M4CaptionConnector,
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,323 @@
"""The M4-Caption connector: M4 competition series, each with a caption that GPT-4o wrote.

The OpenTSLM release holds the train part of all 100,000 M4 series, one CSV of series and one CSV of
captions for each of the six M4 frequencies. Each series becomes one Record and one AnswerTask. The
target is the caption. The connector leaves out the 706 series whose caption is an error message. The release archive stores no split. generate_splits.py made an 80/10/10 split,
stratified by frequency, and wrote it to splits.json.xz. The connector reads that file.
"""

from collections import Counter
from collections.abc import Iterator, Sequence
from dataclasses import dataclass
from enum import StrEnum
import lzma
from pathlib import Path
import re
from typing import Annotated, Self

import numpy as np
from pydantic import AllowInfNan, BaseModel, ConfigDict, Field, Json, StrictFloat, ValidationError, model_validator

from timenet.connectors import BaseConnector
from timenet.dataset import Record, Signal, Source, TimeFDataset
from timenet.dataset.axis import OrdinalAxis
from timenet.errors import TimeNetConnectorError
from timenet.types import Annotation, AnswerTask, Split, TimeSeriesSpec
from timenet_connectors.download import ensure_archive
from timenet_connectors.sources.readers import CsvDataReader, DataRow


M4_CAPTION_URL = "https://polybox.ethz.ch/index.php/s/MT3y9WdEebT8wfj/download/M4TimeSeriesCaptionDatasetV02.zip"
Comment thread
karhu marked this conversation as resolved.
Resolved
M4_CAPTION_SHA256 = "4acb748d94d7287fe58dbb5c3cf6314ea259010b7720f42506739269dec54edf"
RELEASE_FOLDER = "M4TimeSeriesCaptionDataset"
"""The folder in the archive that holds the CSV files."""
SPLITS_PATH = Path(__file__).with_name("splits.json.xz")
"""The stratified split of the release, as generate_splits.py wrote it."""

CAPTION_PROMPT = "Please generate a detailed caption for this time-series, describing it as accurately as possible."
"""The prompt of OpenTSLM's loader, the same for every series: the post-prompt of ``M4QADataset``."""

CAPTION_ERROR_PREFIX = "Error generating caption: "
"""The start of an error message of the caption API, which the release has instead of a caption for
706 series. The dataset leaves out these series."""

# The release states no unit, and the M4 series come from many domains.
M4_SERIES = TimeSeriesSpec(spec_type="m4_series", name="M4 series", unit_value=None, dtype="float32")

_SERIES_ID = re.compile(r"series-(?P<m4_id>(?P<letter>[A-Z])[1-9][0-9]*)")
"""A release id: the M4 id, such as M1, after a ``series-`` prefix that OpenTSLM added."""

_SeriesValue = Annotated[StrictFloat, AllowInfNan(False)]
"""A finite JSON number. Strict, so that a Boolean or a string in the list is an error, not a number."""


class M4Frequency(StrEnum):
Comment thread
karhu marked this conversation as resolved.
Resolved
"""An M4 frequency. The members are in the order that OpenTSLM's loader reads them."""

DAILY = "Daily"
HOURLY = "Hourly"
MONTHLY = "Monthly"
QUARTERLY = "Quarterly"
WEEKLY = "Weekly"
YEARLY = "Yearly"

@property
def id_letter(self) -> str:
"""The letter that starts the M4 id of each series of this frequency, such as M in M1."""
return _M4_ID_LETTERS[self]

def parse_m4_id(self, series_id: str) -> str:
"""Get the M4 id from a release id, and check that it belongs to this frequency.

Returns:
The M4 id, such as ``M1``.

Raises:
TimeNetConnectorError: If the id is not an M4 series id of this frequency.
"""
match = _SERIES_ID.fullmatch(series_id)
if match is None or match["letter"] != self.id_letter:
raise TimeNetConnectorError(f"{series_id!r} is not a {self} M4 series id")
return match["m4_id"]


_M4_ID_LETTERS = {
M4Frequency.DAILY: "D",
M4Frequency.HOURLY: "H",
M4Frequency.MONTHLY: "M",
M4Frequency.QUARTERLY: "Q",
M4Frequency.WEEKLY: "W",
M4Frequency.YEARLY: "Y",
}


class _SeriesRow(DataRow):
"""A row of a series file: the release id and the values as a JSON list."""

id: str
series: Json[Annotated[list[_SeriesValue], Field(min_length=1)]]


class _CaptionRow(DataRow):
"""A row of a caption file: the release id and the caption."""

id: str
caption: str


@dataclass(frozen=True)
class M4CaptionedSeries:
"""One M4 series of the release, with its frequency and caption, before the split is known."""

m4_id: str
frequency: M4Frequency
values: np.ndarray
caption: str


@dataclass(frozen=True)
class M4CaptionSeries:
"""One M4 series of the release, with its frequency, caption and split."""

m4_id: str
frequency: M4Frequency
values: np.ndarray
caption: str
split: Split

def create_record_and_caption_task(self) -> tuple[Record, AnswerTask]:
"""Create the Record of the series and the task that asks for its caption.

The Record holds the values, the M4 id and the frequency. The task names the Record as its
input. Add the Record to the dataset before the task.

Returns:
The Record and the task, not yet added to a dataset.
"""
signal = Signal.from_values(self.values, spec=M4_SERIES, name="value", time_axis=OrdinalAxis())
record = Record(
sources=[Source(name="M4", signals=[signal])],
annotations=[Annotation("frequency", value=self.frequency.value)],
metadata={"m4_id": self.m4_id},
)
task = AnswerTask(inputs=[record], prompt=CAPTION_PROMPT, targets=self.caption, split=self.split)
return record, task


class M4CaptionSplits(BaseModel):
"""The validation and test series of the split, by M4 id.

The release archive stores no split. generate_splits.py made an 80/10/10 split, stratified by
frequency, and wrote it to splits.json.xz. Every series that the file does not name is in the train
split.
"""

model_config = ConfigDict(extra="forbid", frozen=True)

source_sha256: str
"""SHA-256 digest of the release archive that the split was made from."""
validation: tuple[str, ...]
test: tuple[str, ...]

@model_validator(mode="after")
def _each_series_in_one_split(self) -> Self:
repeated = sorted(m4_id for m4_id, count in Counter(self.m4_ids).items() if count > 1)
if repeated:
raise ValueError(f"series {repeated} are named more than once")
return self

@property
def m4_ids(self) -> tuple[str, ...]:
"""The validation and test series together."""
return (*self.validation, *self.test)

@classmethod
def read_from_file(cls, path: Path) -> Self:
"""Read the split from an xz-compressed JSON file.

Returns:
The split.

Raises:
TimeNetConnectorError: If the file does not hold a valid split.
"""
try:
return cls.model_validate_json(lzma.decompress(path.read_bytes()))
except (lzma.LZMAError, ValidationError) as exc:
raise TimeNetConnectorError(f"{path.name} does not hold a valid split: {exc}") from exc

def assign_split_to_each_series(self, m4_ids: Sequence[str]) -> list[Split]:
"""Assign a split to each series of the pinned release.

Returns:
The split of each series, in the order of ``m4_ids``.

Raises:
TimeNetConnectorError: If the split was made from another release, or names a series that
the release does not have.
"""
if self.source_sha256 != M4_CAPTION_SHA256:
raise TimeNetConnectorError(
f"the split was made from the release {self.source_sha256}, not from the pinned {M4_CAPTION_SHA256}"
)
unknown = sorted(set(self.m4_ids) - set(m4_ids))
if unknown:
raise TimeNetConnectorError(f"the split names series {unknown} that the release does not have")
split_of = dict.fromkeys(self.validation, Split.VALIDATION) | dict.fromkeys(self.test, Split.TEST)
return [split_of.get(m4_id, Split.TRAIN) for m4_id in m4_ids]


@dataclass(frozen=True)
class M4CaptionReader:
"""Read the series, the captions and the split of the extracted release."""

directory: Path
splits_path: Path

def read_series(self) -> list[M4CaptionSeries]:
"""Read every series of the release with its caption and split, in the order of OpenTSLM's loader.

The loader reads the files of each frequency, and skips ``m4_series.csv`` and ``m4_captions.csv``.
A malformed file raises a :class:`~timenet.errors.TimeNetConnectorError` that names the file and
its row. The split comes from the split file, which the method checks against every M4 id.

Returns:
Each series with its caption and split.
"""
captioned = self.read_captioned_series()
split_file = M4CaptionSplits.read_from_file(self.splits_path)
splits = split_file.assign_split_to_each_series([series.m4_id for series in captioned])
return [
Comment thread
karhu marked this conversation as resolved.
Resolved
M4CaptionSeries(
m4_id=series.m4_id,
frequency=series.frequency,
values=series.values,
caption=series.caption,
split=split,
)
for series, split in zip(captioned, splits, strict=True)
]

def read_captioned_series(self) -> list[M4CaptionedSeries]:
"""Read every series of the release with its caption, in the order of :meth:`read_series`.

The method leaves out each series whose caption is an error message of the caption API.

Returns:
Each series with its caption, without the split.
"""
captioned: list[M4CaptionedSeries] = []
for frequency in M4Frequency:
captioned.extend(self._read_frequency(frequency))
return captioned

def _read_frequency(self, frequency: M4Frequency) -> Iterator[M4CaptionedSeries]:
"""Read the series of one frequency, paired with the caption in the same row of its caption file.

Yields:
Each series with its caption, except a series whose caption is an error message.

Raises:
TimeNetConnectorError: If the files do not pair one to one, or a series row is malformed.
"""
series_path = self.directory / f"m4_series_{frequency}.csv"
captions_path = self.directory / f"m4_captions_{frequency}.csv"
series_rows = list(CsvDataReader(series_path, _SeriesRow, expected_columns=("id", "series")).iter_rows())
caption_rows = list(CsvDataReader(captions_path, _CaptionRow, expected_columns=("id", "caption")).iter_rows())
if len(series_rows) != len(caption_rows):
raise TimeNetConnectorError(
f"{series_path.name} has {len(series_rows)} series rows but {len(caption_rows)} caption rows"
)
for number, (series_row, caption_row) in enumerate(zip(series_rows, caption_rows, strict=True), start=1):
try:
m4_id = frequency.parse_m4_id(series_row.id)
if series_row.id != caption_row.id:
raise TimeNetConnectorError(
f"series id {series_row.id!r} does not match caption id {caption_row.id!r}"
)
if caption_row.caption.startswith(CAPTION_ERROR_PREFIX):
continue
yield M4CaptionedSeries(
m4_id=m4_id,
frequency=frequency,
values=np.array(series_row.series, dtype=np.float32),
caption=caption_row.caption,
)
except TimeNetConnectorError as exc:
raise TimeNetConnectorError(f"{series_path.name} data row {number}: {exc}") from exc


class M4CaptionConnector(BaseConnector[list[M4CaptionReader]]):
"""Connector for the M4-Caption release of OpenTSLM."""

async def download_async(self, cache_dir: Path) -> list[M4CaptionReader]: # ruff: ignore[no-self-use]
"""Download and extract the release archive once.

Args:
cache_dir: The directory that holds the extracted files.

Returns:
A reader of the extracted release and of the split file of the connector.
"""
root = await ensure_archive(M4_CAPTION_URL, cache_dir, filename="m4_caption.zip", sha256=M4_CAPTION_SHA256)
return [M4CaptionReader(directory=root / RELEASE_FOLDER, splits_path=SPLITS_PATH)]

def convert(self, releases: list[M4CaptionReader]) -> TimeFDataset:
"""Create one Record and one AnswerTask for each series.

Args:
releases: The reader from :meth:`download_async`.

Returns:
The populated dataset.
"""
(release,) = releases
dataset = TimeFDataset(metadata=self.metadata())
for series in release.read_series():
record, task = series.create_record_and_caption_task()
dataset.add_record(record)
dataset.add_task(task)
return dataset


CONNECTOR = M4CaptionConnector
Loading