Repository navigation
Implement time-based cache invalidation for dataset description cache #1714
Description
### Problem
The function _get_dataset_description caches the dataset description (description.xml) to disk, but the cache is never invalidated.
Dataset descriptions contain metadata such as dataset status (active, deactivated, etc.), which may change over time on the OpenML server. Because the cached description file is reused indefinitely, users may receive stale metadata if the local cache is old.
There is also a TODO comment in the code suggesting that this cache should invalidate itself after some time.
Expected Behavior
Cached dataset descriptions should be refreshed periodically to ensure that metadata reflects the current state on the server.
Current Behavior
Once description.xml is downloaded and cached, it is reused indefinitely unless the cache directory is manually deleted or force_refresh_cache=True is used.
Activity
emphor11 commentedon Mar 17, 2026
/assign
PGijsbers commentedon Oct 13, 2026
Hi, thanks for making the time to open an issue for the TODO in the code and setting up a draft in #1715. I think the comment is a bit misleading, sorry about that. While it's true that we want cache invalidation for the description, I do not think that the implementation should be specific to that file. As part of the ongoing refactoring, we are introducing a generic caching layer that is also used by the HTTPClient. I think solutions for periodic cache invalidation should be based there (and be periodic as long as the server does not provide a useful TTL).
I realize it's been a while since your activity. If you're still interested in this, please let us know. Before you start though, please wait for confirmation on our side. We are currently working through the large number of open pull requests. On the one hand, we don't have time to process them in a timely fashion, and on the other hand there may still be important changes in the queue that touch on these systems that would affect how you develop the 'fix'.