Skip to content

Reject unknown type names inside containers - #590

Draft
Hakaou wants to merge 2 commits into
ClickHouse:masterfrom
Hakaou:unknown-type-names-are-rejected
Draft

Hakaou wants to merge 2 commits into
ClickHouse:masterfrom
Hakaou:unknown-type-names-are-rejected

Conversation

@Hakaou

@Hakaou Hakaou commented Oct 18, 2026 •

Copy link
Copy Markdown

Closes #589, the second point of the suggested fix in #571. Draft until #575 lands: this branch is stacked on it because it needs the Nested node, so the first commit here is #575's and only the last one belongs to this PR.

Summary

ValidateAST only looked at the root of a type name. Inside Array, Nullable, Tuple, Map, LowCardinality or Nested, a name the parser does not know became a Void terminal, the column was created as ColumnNothing, and the block was read one byte per row without any error. CreateColumnByType("Array(Int256)") returned a column typed Array(Void), and SELECT [CAST(1, 'Int256')] returned a row of wrong data.

Behavior changes

  • The elements of those six containers are validated. A client now gets unsupported column type: Array(Int256) instead of wrong data, the same error it already gets for Int256 at the root. Other names this applies to, all seen in toTypeName on 26.11: Variant, Dynamic, BFloat16, UInt256, the Interval* types, LineString, MultiPoint, QBit, AggregateFunction.
  • The arguments of every other type are left alone: the settings and typed paths of JSON(max_dynamic_paths=1, a Int256, SKIP b) keep decoding as before, and the function name of SimpleAggregateFunction(anyLast, LowCardinality(String)) is not a type.
  • Nothing, which the server sends for SELECT [] and SELECT NULL, is registered as the type it already was by accident, so those queries keep working.
  • Reject wrapper types built over unsupported nested types #578 covers the neighbouring case of a known name the factory cannot build inside a wrapper. The two change different functions and merge independently.

Tests

  • type_parser_ut: a parametrised test with a fresh AST per case, six unknown names inside containers rejected, Nothing, SimpleAggregateFunction, parameterised JSON, Enum8 and DateTime64 accepted.
  • CreateColumnByType_ut: Array(Int256) and Array(Nested(key String, value Dynamic)) give nullptr, Array(Nothing), Nullable(Nothing) and parameterised JSON give a column.
  • client_ut, both fixtures: SELECT [] AS empty, NULL AS null_value, [NULL] AS nulls decodes, SELECT [CAST(1, 'Int256')] throws UnimplementedError, and CAST('{"a":1}', 'JSON(max_dynamic_paths=1)') reads back {"a":1} with output_format_native_write_json_as_string = 1.

Full suite green locally (macOS arm64, Apple clang 21) against ClickHouse 26.11.1 master build, the new tests also against 25.12 and 26.3. Parser, factory, types and client suites pass under AddressSanitizer and UBSan.

Open question

Inside a container, should an unknown name make CreateColumnByType return nullptr, as it does at the root, or would you rather have an exception naming the inner type? The tests hold either way, only the end of ValidateAST would change.

The type parser had no entry for Nested, so a column typed
Array(Nested(key String, value String)) was created as Array(Void)
and the native block stream fell out of sync ("Can't cast from Void",
"some data was not read"). A top-level Nested column, which the server
sends with flatten_nested = 0, was rejected as unsupported.

The server defines Nested as a custom name over Array(Tuple(...))
(DataTypeNested.cpp). The parser now recognises Nested as its own
node, like Map and SimpleAggregateFunction, and the column factory
builds it as its Tuple wrapped in an Array.

Closes ClickHouse#571
ValidateAST only looked at the root of a type name. Inside Array,
Nullable, Tuple, Map, LowCardinality or Nested, a name the parser does
not know became a Void terminal, the column was created as
ColumnNothing, and the block stream was read one byte per row
without any error: SELECT [CAST(1, 'Int256')] returned a row of
garbage.

The elements of those containers are now validated too. The arguments
of every other type, such as the settings and typed paths of JSON or
the function name of SimpleAggregateFunction, are not types and are
left alone. Nothing, which the server sends for SELECT [] and
SELECT NULL, is registered as the type it already was by accident, so
those queries keep working. A client now gets "unsupported column
type: Array(Int256)" instead of wrong data.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unknown type names inside a container decode silently as wrong data

1 participant