Skip to content

feat(media): scan every folder level, save incrementally, TV tags, seed task progress - #3263

Open
PIKACHUIM wants to merge 5 commits into
mainfrom
feat/media-library-improvements
Open

PIKACHUIM wants to merge 5 commits into
mainfrom
feat/media-library-improvements

Conversation

@PIKACHUIM

@PIKACHUIM PIKACHUIM commented Oct 25, 2026 •

Copy link
Copy Markdown
Member

PR:OpenList-Backends(OpenList)

标题:feat(media): season-aware scanning, admin APIs and seed task progress
分支:feat/media-library-improvements → main(3 commits,21 files,+2700 / −384)

Summary / 摘要

补齐媒体库的扫描规则、刮削保护、后台管理接口与 Emby 兼容,并修复 site_url 带路径前缀时播放地址丢前缀的问题。前端配套改动见 OpenList-Frontend 仓库同名分支。

用户可感知的变化

  • 扫描:
    • 每个「真正装了媒体文件」的目录各自成条(不再整棵子树只出一条),边扫边落盘,大库能持续看到结果;
    • 合并模式支持固定层级 merge_depth:扫描根的直接子目录是第 1 层,/影视/群星/{第一季,第二季} 填 1 即聚合成一条「群星」;
    • 「第N季 / Season N / S01」目录即使被层级规则命中,也会上移并入剧名,不再一季一条、重复刮削;
    • 花絮 / 字幕 / 样片等附属目录不再单独成条;支持 .iso 原盘镜像。
  • 季号:季目录解析出 episode.season,播放器与 Emby 可按季分组;显式标「电影」的扫描路径不会被季目录反推成剧集。
  • 刮削:音乐条目的文件自带封面(内嵌 ID3/Vorbis、本地缩略图、站内地址)不再被在线封面覆盖,只补空。
  • 后台:
    • 条目可编辑字段扩充(选集、视频类型、季集、时长、外部 ID 等);
    • 封面上传;
    • 音乐按专辑聚合与批量编辑;
    • 导出改为带封面的 zip。
  • Emby:多季合集按每集季号拆成多个 Season;集 ID 带季号,避免跨季重号。
  • 种子:新增保存 / 生成任务与进度上报(任务页可见)。
  • 播放地址:site_url 配置为 /openlist 这类路径前缀时不再丢前缀(此前会 404)。

重要实现变化

  • internal/media/scanner.go:
    • 新增 walkMergeFolders 增量扫描、walkMergeDepth 固定层级、季目录识别与上移 promoteSeasonTarget;
    • 附属目录跳过、.iso 扩展名、buildEpisodesFromFolder 递归与相对路径;
    • video_type 判定链:显式类型标签 > 季目录证据 > 文件夹/单文件兜底。
  • internal/model/media.go:EpisodeInfo 增加 season;MergeDepth 注释与实现对齐。
  • internal/db/media.go:ListMediaItemsByAlbum、ListAlbums 固定排序(分页稳定)、重扫保留 video_type;新增 upsert 单测。
  • 新接口:
    • POST /api/admin/media/items/cover(media_cover_upload.go,落 <data>/<thumbnail_path>/cover-<id>-<ts>.<ext>,返回 thumb:/ 引用);
    • GET /api/admin/media/albums、GET /api/admin/media/album/items、POST /api/admin/media/album/update(media_album.go,专辑为聚合分组,批量写回该专辑所有曲目);
    • 导出 / 导入升级为 zip(media_export.go:media_export.json + covers/)。
  • Emby:episodesOf / seasonList / findEpisode 改用 episodeSeasonOf,集 ID 变为 ep-<itemID>-<season>-<num>(兼容旧 3 段解析)。
  • 种子任务:SeedSaveTask 进度任务 + 生成/保存进度上报;中转转存逐文件失败会聚合为任务失败(此前无条件标记成功)。
  • server/common/base.go:配置了 site_url 一律走 GetApiUrlFromRequest。

兼容性

  • 导出格式升到 v2(zip),导入同时兼容 v2 zip 与 v1 纯 JSON。

  • EpisodeInfo 增加可选 season 字段(JSON season,omitempty),旧数据无影响。

  • Emby 集 ID 格式变化:老客户端若缓存了旧 ID,需要重新拉取列表。

  • 新增后台路由均为增量接口,不改动既有公开 API。

  • This PR has breaking changes.
    / 此 PR 包含破坏性变更。(Emby 集 ID 格式变化,见上)

  • This PR changes public API, config, storage format, or migration behavior.
    / 此 PR 修改了公开 API、配置、存储格式或迁移行为。(导出格式 v2、EpisodeInfo.season、Emby 集 ID)

  • This PR requires corresponding changes in related repositories.
    / 此 PR 需要关联仓库同步修改。

Related repository PRs / 关联仓库 PR:

  • OpenList-Frontend: <前端 PR 链接>
  • OpenList-Docs:

Related Issues / 关联 Issue

Relates to #3297

Testing / 测试

  • go build ./...
  • go test ./internal/media ./internal/db ./server/handles ./server/common
  • 新增用例:
    • TestSeasonNumberOf、TestPromoteSeasonTarget、TestIsSeasonDir
    • TestWalkMergeDepthTargetsShowLevel / SeasonLevel / DeeperLevel、TestBuildEpisodesKeepsSubPath
    • TestMediaExportZipRoundTrip(zip 导出→导入逐字节比对封面)、TestCoverFileExtRejectsNonImage、TestIsFileOwnedCover
    • CreateOrUpdateMediaItem 重扫保留字段(internal/db/media_item_upsert_test.go)
    • GetPlaybackBase 路径前缀(server/common/base_test.go)
  • Manual test / 手动测试:
    • 真实库扫描:/影视资源/群星/{第一季,第二季} 产出单条「群星」,8 集分两季,Emby 侧两个 Season;
    • 播放:mkv 直链与转码 HLS 均可播,播放器返回按钮可用;
    • 导入导出:导出 zip 含 covers/,重新导入封面回填为 thumb:/...。

Checklist / 检查清单

  • I have read CONTRIBUTING.
    / 我已阅读 CONTRIBUTING。
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
    / 我确认此贡献符合仓库许可证、贡献规范和行为准则。
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
    / 我已按适用情况使用 gofmt、go fmt 或 prettier 格式化变更代码。
  • I have requested review from relevant maintainers or code owners where applicable.
    / 我已在适用情况下请求相关维护者或代码所有者审查。

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.
    / 此 PR 包含 AI 辅助内容。

Tools used / 使用工具:

  • ChatGPT
  • Codex
  • GitHub Copilot
  • Claude
  • Gemini
  • Other (please specify) / 其他(请注明): CodeBuddy

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Refactoring / 重构

  • Documentation / 文档

  • Tests / 测试

  • Translation / 翻译

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.
    / 我已审核并验证此 PR 中的所有 AI 辅助内容。

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.
    / 我已确保所有 AI 辅助提交都包含 Co-Authored-By 归属信息。(当前提交未附加 trailer,如需合规请补 trailer 后 force-push)

  • I can reproduce all AI-assisted content included in this PR without any AI tools.
    / 我可以在没有任何 AI 工具的情况下重现此 PR 中包含的所有 AI 辅助内容。

…ed task progress

Media library
- Merge mode now walks every folder level instead of only the first one.
  A folder that directly contains media files is one collection and is not
  split further; a folder without media is descended into. Loose files in
  the scan root stay individual items. Depth is capped at 16.
- Scanning no longer buffers the whole tree before writing: items are
  flushed to the database every 20 items or 3 seconds, for all four media
  types, so results appear while the scan is still running. Skipped and
  failed items still advance the progress counter.
- The scan path type tag now decides movie vs. series. Tags such as
  TV series / anime / variety / documentary mark every video under the
  path as "tv" (shown as series in Emby, also for single files); movie
  tags force "movie" even when merge mode is on. Unrecognised tags keep
  the old rule (folder items are series). Emby virtual folders use the
  same rule.
- Fix: a music folder containing a non-audio file was dropped entirely.

Playback URL
- When no site_url is configured and the request host is a loopback
  address (127.0.0.1 / localhost / ::1), stream_url now returns relative
  /d and /p paths instead of absolute loopback URLs. An explicit site_url
  or a forwarded host still produces absolute URLs.

Seed tasks
- Seed generation reports progress by bytes hashed plus a stage message
  and is exposed at /task/seed_generate.
- Seed save, rapid upload and transit transfer register a display-only
  task at /task/seed_save (the operation stays synchronous) so progress
  is visible in the task list.

Tests cover tag classification, index-length checks, level-by-level merge
scanning (stop at collection, list errors, depth limit) and loopback
detection.

Co-Authored-By: Claude <noreply@anthropic.com>
pikachuren

This comment was marked as outdated.

pikachuren added 2 commits October 27, 2026 17:47
site_url 允许配置成 /openlist 这样的路径前缀;原先只有以 http 开头才走绝对地址,前缀模式下播放地址会丢掉前缀导致 404。改为配置了 site_url 就统一用 GetApiUrlFromRequest 生成。
…p export

- 扫描:季目录(第N季/Season N/S01)解析季号写入 EpisodeInfo.season,Emby 与播放器可按季分组;显式电影标签不再被季目录反推成剧集;merge_depth 语义固定为「第 N 层=一条记录」,与模型注释、UI 一致
- 刮削:音乐条目的文件自带封面(data:/thumb:/站内地址)不再被在线封面覆盖
- 后台新增:封面 URL/上传(/admin/media/items/cover)、专辑聚合列表与批量编辑(/admin/media/albums、/album/items、/album/update)、条目可编辑字段扩充(选集/视频类型/季集/时长/外部ID)
- 导入导出升级为 zip(media_export.json + covers/),导入兼容旧 JSON
- Emby:多季合集按每集 season 拆分 Season,集 ID 带季号避免跨季重号
- fix(seed):中转转存部分文件失败时任务按失败收尾,不再一律标记成功
@PIKACHUIM
PIKACHUIM marked this pull request as ready for review October 27, 2026 09:53
…te, concurrent cover export

- UpdateMediaItemAdmin 改为 PATCH 语义(字段全部指针,只写出现过的键),修复只传 hidden 也会清空 cover/plot 的问题;cover 增加 12MB 上限
- 专辑批量更新走单事务(新增 db.UpdateMediaItems),失败整批回滚;旧封面文件在事务成功后才删除
- 导出封面改为并发读取(4 并发,单张 8MB / 总量 512MB / 5000 张上限),避免大库导出逐张串行
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] 媒体库:支持 ISO 扫描、单路径多 VFS、多季剧集聚合为同一部剧

2 participants