Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
64f990f
internal/pgparse: the parser is libpg_query 17-6.2.2 compiled to WebA…
kr9ly Sep 8, 2026
32639f3
no cgo: the node types are generated from libpg_query's pg_query.prot…
kr9ly Sep 8, 2026
11afacc
pgparse: the PostgreSQL 18 parser (libpg_query 18.0.0) is embedded be…
kr9ly Sep 8, 2026
40c1b59
CI: the generated protobuf node types (internal/pgparse/pg_query.pb.g…
kr9ly Sep 8, 2026
936d17d
schema.sql declares its PostgreSQL version — `-- sqlshape: postgres 1…
kr9ly Sep 8, 2026
5b138b1
CI: go test -count=1 for the coverage profile — a cached test result …
kr9ly Sep 9, 2026
bd69f65
one catalog, one oracle, one regress probe per PostgreSQL major versi…
kr9ly Sep 9, 2026
3f9dcb5
design: MySQL — the parser is generated from MySQL's own sql_yacc.yy …
kr9ly Sep 9, 2026
ef99e9c
design: MySQL — the lexer is extracted too, not rewritten: lex_one_to…
kr9ly Sep 9, 2026
83f1cd9
design: MySQL — the built-in function catalog is extracted too: item_…
kr9ly Sep 9, 2026
6f08f1a
design: MySQL support ships as a separate GPLv2 module, sqlshape-mysq…
kr9ly Sep 9, 2026
556ca57
PostgreSQL 18 DDL loads: named NOT NULL constraints (CONSTRAINT nn NO…
kr9ly Sep 9, 2026
89e9527
RETURNING old / new (PostgreSQL 18): the row before and after the wri…
kr9ly Sep 9, 2026
0cb48a0
PostgreSQL 18: an abbreviation the session time zone itself uses (LMT…
kr9ly Sep 9, 2026
19cb556
schema.sql must declare its PostgreSQL version: ReadSource refuses a …
kr9ly Sep 9, 2026
b01ebbd
PostgreSQL 18's new checks, judged as 18 does and as 17 did: a JSON_V…
kr9ly Sep 9, 2026
2162eea
gofmt
kr9ly Sep 9, 2026
90ea8f4
release: one job cross-compiles every target — linux, darwin and wind…
kr9ly Sep 9, 2026
29cc708
a temporal key (WITHOUT OVERLAPS, PostgreSQL 18) proves a single row …
kr9ly Sep 9, 2026
714aeda
diff / apply / verify-schema warn on stderr when the database runs an…
kr9ly Sep 9, 2026
e4e55e8
design: PostgreSQL version support is a decision, not an open item — …
kr9ly Sep 9, 2026
1f347fc
READMEs: the oracle runs one release per major version (StartVersion,…
kr9ly Sep 9, 2026
5941783
CHANGELOG: 1.2.0 — PostgreSQL 18 and the version declaration, the par…
kr9ly Sep 9, 2026
1639f24
README: install — Go 1.26+, the prebuilt binaries by name, what the f…
kr9ly Sep 9, 2026
444f472
README.ja: install — no cgo, the prebuilt binaries by name, what the …
kr9ly Sep 9, 2026
09fc6ec
README / README.ja: compatibility — 17 and 18, the parser is PostgreS…
kr9ly Sep 9, 2026
0e7ee90
checks.md / checks.ja.md: RETURNING old after an INSERT and new after…
kr9ly Sep 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
72 changes: 28 additions & 44 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -1,12 +1,11 @@
# Release: push a tag v* and this builds the sqlshape binary for each platform, packs it,
# and publishes a GitHub Release with checksums.
# Release: push a tag v* and this cross-compiles the sqlshape binary for every platform,
# packs it, and publishes a GitHub Release with checksums.
#
# git tag v0.1.0 && git push origin v0.1.0
#
# The binary links libpg_query through cgo, so every platform is built on its own runner
# rather than cross-compiled. Windows is not built here (libpg_query does not build reliably
# with the toolchains on the Windows runners); `go install` still works there with a C
# compiler installed.
# The binary is a plain Go build (libpg_query runs as WebAssembly, no cgo), so one runner
# builds every target; the smoke test runs the runner's own (linux/amd64) binary as a
# vettool over the examples, the only one it can execute.
#
# `go install github.com/kr9ly/sqlshape/cmd/sqlshape@<tag>` needs nothing from this
# workflow: it reads the tag straight from the module proxy.
Expand All @@ -20,14 +19,8 @@ permissions:
contents: write

jobs:
build:
strategy:
matrix:
include:
- { os: ubuntu-latest, goos: linux, goarch: amd64 }
- { os: ubuntu-24.04-arm, goos: linux, goarch: arm64 }
- { os: macos-14, goos: darwin, goarch: arm64 }
runs-on: ${{ matrix.os }}
release:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
Expand All @@ -36,41 +29,32 @@ jobs:

- name: Build
env:
CGO_ENABLED: "1"
GOOS: ${{ matrix.goos }}
GOARCH: ${{ matrix.goarch }}
CGO_ENABLED: "0"
run: |
go build -trimpath \
-ldflags "-s -w -X github.com/kr9ly/sqlshape/internal/cli.version=${GITHUB_REF_NAME}" \
-o sqlshape ./cmd/sqlshape
mkdir dist
for target in linux/amd64 linux/arm64 darwin/amd64 darwin/arm64 windows/amd64 windows/arm64; do
goos=${target%/*}; goarch=${target#*/}
bin=sqlshape; [ "$goos" = windows ] && bin=sqlshape.exe
GOOS=$goos GOARCH=$goarch go build -trimpath \
-ldflags "-s -w -X github.com/kr9ly/sqlshape/internal/cli.version=${GITHUB_REF_NAME}" \
-o "build/$goos-$goarch/$bin" ./cmd/sqlshape
name="sqlshape_${GITHUB_REF_NAME}_${goos}_${goarch}"
mkdir "$name"
cp "build/$goos-$goarch/$bin" LICENSE NOTICE README.md "$name"/
if [ "$goos" = windows ]; then
zip -qr "dist/$name.zip" "$name"
else
tar -czf "dist/$name.tar.gz" "$name"
fi
rm -r "$name"
done
ls -la dist

- name: Smoke test
run: .github/scripts/smoke.sh ./sqlshape "$GITHUB_REF_NAME"

- name: Pack
run: |
name="sqlshape_${GITHUB_REF_NAME}_${{ matrix.goos }}_${{ matrix.goarch }}"
mkdir "$name"
cp sqlshape LICENSE NOTICE README.md "$name"/
tar -czf "$name.tar.gz" "$name"

- uses: actions/upload-artifact@v4
with:
name: sqlshape_${{ matrix.goos }}_${{ matrix.goarch }}
path: sqlshape_*.tar.gz

release:
needs: build
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/download-artifact@v4
with:
path: dist
merge-multiple: true
run: .github/scripts/smoke.sh ./build/linux-amd64/sqlshape "$GITHUB_REF_NAME"

- name: Checksums
run: cd dist && sha256sum sqlshape_*.tar.gz > checksums.txt
run: cd dist && sha256sum sqlshape_* > checksums.txt

- name: Publish
env:
Expand Down
36 changes: 34 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,16 @@ jobs:
run: go build -o sqlshape ./cmd/sqlshape && .github/scripts/smoke.sh ./sqlshape

# coverage counts every package except the catalog generator (a manual tool run
# against a PostgreSQL) and the 5-statement main
# against a PostgreSQL) and the 5-statement main; the generated protobuf node types
# (internal/pgparse/pg_query.pb.go, 40k lines of accessors) are dropped from the profile.
# -count=1: a cached test result replays the profile it recorded, with block positions
# from the sources of that run; after an edit to a package the cached package does not
# depend on, the stale blocks no longer line up and the merged total drops by ten points
- name: Test
run: |
pkgs=$(go list ./... | grep -v -e /internal/catalog/gen -e /cmd/sqlshape | paste -sd,)
go test ./... -coverpkg="$pkgs" -coverprofile=cover.out -timeout 20m
go test ./... -count=1 -coverpkg="$pkgs" -coverprofile=cover.out -timeout 20m
grep -v '/pg_query\.pb\.go:' cover.out > cover.filtered && mv cover.filtered cover.out

# coverage badge: statement coverage of the whole module, committed as an SVG so the
# README can show it without a third-party service; --min makes it a regression gate
Expand All @@ -70,3 +75,30 @@ jobs:
git commit -m "CI: coverage badge [skip ci]"
git push
fi

# The analyzer against PostgreSQL 18: the embedded catalog loads, the oracle boots, and
# the regress probe replays 18's corpus (asserting its baseline once there is one).
pg18:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: go.mod

- name: Cache PostgreSQL 18 binaries and regress corpus
uses: actions/cache@v4
with:
path: ~/.cache/sqlshape
key: sqlshape-pg18-${{ hashFiles('internal/oracle/oracle.go', 'internal/analyze/testdata/tools/fetch-regress.sh') }}

- name: Fetch regress corpus
run: internal/analyze/testdata/tools/fetch-regress.sh REL_18_3

- name: Test
env:
SQLSHAPE_PG: "18"
run: |
go test ./internal/catalog ./internal/oracle -count=1
go test ./internal/analyze -run 'TestRegress$' -count=1 -v -timeout 25m 2>&1 | grep -E 'statements:|hits|PASS|FAIL|ok '
35 changes: 34 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,38 @@ release it is a candidate for.

## [Unreleased]

## [1.2.0] - 2026-09-09

### Added

- PostgreSQL 18. `schema.sql` declares its version with `-- sqlshape: postgres 18`, and the schema
and every statement are judged with 18's grammar and catalog: `RETURNING old` / `new` (with
`WITH (OLD AS ..., NEW AS ...)`), `WITHOUT OVERLAPS` keys and `PERIOD` foreign keys, `NOT ENFORCED`
constraints (which are no failure mode), named `NOT NULL` constraints, `VIRTUAL` generated columns,
and 18's own checks (a JSON_VALUE DEFAULT whose collation is not the RETURNING type's, a JSON path
that is not a jsonpath, stricter datetime and aclitem input, an outer-level aggregate over a nested
CTE). `pgtest` and the migration commands run the declared version's PostgreSQL; `diff`, `apply`
and `verify-schema` warn when the database runs another major version than the schema declares.
- `One` proves a single row through a temporal key: the scalar columns fixed by equality and the
range column equal to a known value or containing a known point (`valid_at @> {{.Day}}::date`).

### Fixed

- `SELECT DISTINCT ON` expressions are matched to the leading `ORDER BY` expressions by expression,
not by where each was written; the same expression in both no longer fails with 42P10.
- An UPDATE that assigns a generated column reports PostgreSQL's message (`column "b" can only be
updated to DEFAULT`) rather than INSERT's.

### Changed

- `schema.sql` must declare its PostgreSQL version (`-- sqlshape: postgres 17`); a schema without
the line is not read. Add the line to move from 1.1.
- The parser (libpg_query) is compiled to WebAssembly and run by wazero instead of being linked
through cgo. `go install` no longer needs a C compiler, and the binary is a plain Go build.
Parse trees, error messages and positions are unchanged.
- Prebuilt binaries for every platform, cross-compiled by one job: Linux, macOS and Windows, amd64
and arm64 each (Windows and Intel macOS are new).

## [1.1.0] - 2026-09-08

### Added
Expand Down Expand Up @@ -64,6 +96,7 @@ First release: `sqlshape.Query[R, P]` / `One[R, P]` templates checked by `go vet
test oracle), the runtime on pgx, and `sqlshape diff` / `apply` / `verify-schema` for migrations
from a declared schema.

[Unreleased]: https://github.com/kr9ly/sqlshape/compare/v1.1.0...HEAD
[Unreleased]: https://github.com/kr9ly/sqlshape/compare/v1.2.0...HEAD
[1.2.0]: https://github.com/kr9ly/sqlshape/compare/v1.1.0...v1.2.0
[1.1.0]: https://github.com/kr9ly/sqlshape/compare/v1.0.0...v1.1.0
[1.0.0]: https://github.com/kr9ly/sqlshape/releases/tag/v1.0.0
13 changes: 9 additions & 4 deletions README.ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,14 +22,16 @@ u, err := ByEmail.Get(ctx, db, struct{ Email string }{Email: email})

## インストール

検査器とマイグレーションコマンドは1つのバイナリになっている:
検査器とマイグレーションコマンドは1つのバイナリになっている。Go 1.26以上で:

```
$ go install github.com/kr9ly/sqlshape/cmd/sqlshape@latest
$ sqlshape version
```

PostgreSQLのパーサ(libpg_query)をcgoでリンクするので、`go install`にはCコンパイラ(gccかclang)が必要。ビルド済みのバイナリ(Linuxはamd64とarm64、macOSはApple Silicon)は[releasesページ](https://github.com/kr9ly/sqlshape/releases)にある。Intel Macは`go install`でビルドする。
ビルド済みのバイナリは[releasesページ](https://github.com/kr9ly/sqlshape/releases)にある。Linux・macOS・Windowsそれぞれamd64とarm64で、`sqlshape_<version>_<os>_<arch>.tar.gz`(Windowsは`.zip`)と`checksums.txt`。展開して`sqlshape`を`PATH`に置く。

検査に他の準備は要らない。PostgreSQLのパーサ(libpg_query、対応するメジャー版ごとに1つ)はWebAssemblyとして埋め込まれwazeroで動くので、Cコンパイラもリンクするライブラリも無い。初回だけモジュールのコンパイルに1秒ほどかかり、結果はユーザーのキャッシュディレクトリ(Linuxでは`~/.cache/sqlshape`)に置かれる。`pgtest`とマイグレーション系コマンドは本物のPostgreSQLでスキーマを動かすので、初回に宣言した版のサーババイナリを同じキャッシュにダウンロードする([migrations.ja.md](docs/migrations.ja.md)の要件を参照)。

ランタイムは通常のGoモジュール:

Expand All @@ -43,6 +45,7 @@ $ go get github.com/kr9ly/sqlshape

```sql
-- schema.sql
-- sqlshape: postgres 17
CREATE TABLE users (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
email text NOT NULL UNIQUE,
Expand All @@ -51,6 +54,8 @@ CREATE TABLE users (
);
```

`schema.sql`の最初の行が、どの版のPostgreSQL向けのスキーマかを名乗る。すべての文はその版の文法とカタログで判定される(17と18に対応)。

```go
// users.go
type User struct {
Expand Down Expand Up @@ -152,9 +157,9 @@ $ sqlshape verify-schema -db "$DSN" # ドリフト検出: データベ

## 互換性

PostgreSQL 17の構文はすべて扱える。SELECTとDML、MERGE、CTE、ウィンドウ関数、GROUPING SETS、SQL/JSON、範囲型、citextやhstoreなどの拡張、ビュー・関数(SQLとPL/pgSQLの本体まで)・トリガー・ポリシーを含むDDL。アナライザーはPostgreSQL自身のカタログから組み上げたpure Goの実装で、検査時にPostgreSQLへ接続することはない。
PostgreSQL 17と18に対応し、どちらかは`schema.sql`が宣言する。構文はPostgreSQL自身のものである。パーサはその版のlibpg_queryなので、サーバが読める文は検査器も同じように読む。SELECTとDML、MERGE、CTE、ウィンドウ関数、GROUPING SETS、SQL/JSON、範囲型、18の`RETURNING old` / `new`と時制キー、citextやhstoreなどの拡張、ビュー・関数(SQLとPL/pgSQLの本体まで)・トリガー・ポリシーを含むDDL。アナライザーはその版のカタログから組み上げたpure Goの実装で、検査時にPostgreSQLへ接続することはない。

「すべて」の裏付けはPostgreSQL自身の回帰テストである。`src/test/regress`の22,000文を本物のPostgreSQL 17と並走させ、パラメータの型・結果列・エラーの判定が一致することを確認している。一致しないのは19件で、行レベルセキュリティの再帰、権限、サーバー内部のエラーなど、静的解析では判定できないもの。この突き合わせは`go test ./...`の一部なので、新しい不一致が出ればテストが失敗する。
判定の裏付けはPostgreSQL自身の回帰テストである。`src/test/regress`の文を、アナライザーと同じ版の本物のPostgreSQLに並走させ、パラメータの型・結果列・エラーの判定が一致することを確認している。一致しないのは17で22,103文のうち19件、18で23,384文のうち31件。全件を`internal/analyze/testdata/regress_baseline_<version>.txt`に列挙してあり、どれも静的解析では判定できないもの(行レベルセキュリティの再帰、権限、サーバー内部のエラー)か、検査器が正しくサーバのDescribeには見えないもの(INSERTの`RETURNING old`がNULLであること)である。この突き合わせは`go test ./...`の一部なので、新しい不一致が出ればテストが失敗する。

## License

Expand Down
43 changes: 29 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,17 +31,23 @@ u, err := ByEmail.Get(ctx, db, struct{ Email string }{Email: email})

## Install

The checker and the migration commands are one binary:
The checker and the migration commands are one binary. With Go 1.26 or newer:

```
$ go install github.com/kr9ly/sqlshape/cmd/sqlshape@latest
$ sqlshape version
```

It links PostgreSQL's parser (libpg_query) through cgo, so `go install` needs a C compiler (gcc or
clang) on the machine. Prebuilt binaries for Linux (amd64, arm64) and macOS (Apple Silicon) are on
the [releases page](https://github.com/kr9ly/sqlshape/releases); an Intel Mac builds it with
`go install`.
Or take a prebuilt binary from the [releases page](https://github.com/kr9ly/sqlshape/releases):
Linux, macOS and Windows, amd64 and arm64 each, as `sqlshape_<version>_<os>_<arch>.tar.gz` (`.zip`
on Windows) with `checksums.txt` beside them. Unpack and put `sqlshape` on your `PATH`.

Nothing else is needed to check code. PostgreSQL's parser (libpg_query, one per supported major
version) is embedded as WebAssembly and runs on wazero, so there is no C compiler to install and no
library to link; the first run compiles the module (about a second) and caches the result under the
user cache directory (`~/.cache/sqlshape` on Linux). `pgtest` and the migration commands, which run
your schema on a real PostgreSQL, download the declared version's server binaries into the same
cache on first use (see [migrations.md](docs/migrations.md#requirements)).

The runtime is an ordinary Go module:

Expand All @@ -57,6 +63,7 @@ compatible; what the checker reports may grow with minor versions.

```sql
-- schema.sql
-- sqlshape: postgres 17
CREATE TABLE users (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
email text NOT NULL UNIQUE,
Expand All @@ -65,6 +72,9 @@ CREATE TABLE users (
);
```

The first line of `schema.sql` names the PostgreSQL version the schema is written for; every
statement is judged with that version's grammar and catalog (17 and 18 are supported).

```go
// users.go
type User struct {
Expand Down Expand Up @@ -195,15 +205,20 @@ rename or the removal of an enum label, are declared in `schema.sql` with `-- @m

## Compatibility

All of PostgreSQL 17's syntax is understood: SELECT and DML, MERGE, CTEs, window functions,
GROUPING SETS, SQL/JSON, ranges, extensions such as citext and hstore, and DDL including views,
functions (SQL and PL/pgSQL bodies), triggers and policies. The analyzer is a pure-Go implementation built from PostgreSQL's
own catalog; checking never connects to a PostgreSQL.

"All" is backed by PostgreSQL's own regression suite: the 22,000 statements of `src/test/regress`
are run through the analyzer and a real PostgreSQL 17 side by side, and parameter types, result
columns and errors must agree. They disagree on 19, all of them things static analysis cannot
decide (row-level security recursion, permissions, server internals). The comparison is part of
PostgreSQL 17 and 18; `schema.sql` declares which. The syntax is PostgreSQL's own: the parser is
libpg_query of that version, so everything the server parses, the checker parses the same way —
SELECT and DML, MERGE, CTEs, window functions, GROUPING SETS, SQL/JSON, ranges, `RETURNING old` /
`new` and temporal keys on 18, extensions such as citext and hstore, and DDL including views,
functions (SQL and PL/pgSQL bodies), triggers and policies. The analyzer is a pure-Go implementation
built from that version's catalog; checking never connects to a PostgreSQL.

The judgment is backed by PostgreSQL's own regression suite: the statements of `src/test/regress`
are run through the analyzer and a real PostgreSQL of the same version side by side, and parameter
types, result columns and errors must agree. On 17 they disagree on 19 of 22,103 statements, on 18
on 31 of 23,384; every one is listed (`internal/analyze/testdata/regress_baseline_<version>.txt`),
and each is either something static analysis cannot decide (row-level security recursion,
permissions, server internals) or a case where the checker is right and the server's Describe
cannot say (the NULLs of `RETURNING old` after an INSERT). The comparison is part of
`go test ./...`, so a new disagreement fails the build.

## License
Expand Down
Loading
Loading