Array(Nested(...)) columns silently decode as Array(Nothing) and desynchronize the native block stream #571

Description

The type parser has no entry for Nested, so a column whose server-reported type is Array(Nested(key String, value String)) is parsed as Array(<unknown terminal>) with Type::Void, and CreateTerminalColumn() maps Type::Void to ColumnNothing. The result is that CreateColumnByType() returns a perfectly valid-looking ColumnArray(ColumnNothing) — no error, no nullptr — and Client::Select() happily accepts it when reading the block header.

This matters because Nested inside Array(...) is not flattened by the server. Unlike a top-level Nested column (which the server splits into col.key Array(String), col.value Array(String) — the case discussed in #40), Array(Nested(...)) is reported over the native protocol with the literal type name:

$ clickhouse-client -q "SELECT name, type FROM system.columns WHERE table='some_data'"
id      UInt64
data    Array(Nested(key String, value String))

On the wire Array(Nested(k String, v String)) is serialized as Array(Array(Tuple(String, String))). But the client builds Array(Nothing), and ColumnNothing::LoadBody() (clickhouse/columns/nothing.h:61) just does input->Skip(rows) — 1 byte per element. So after reading the outer offsets the client skips N bytes where the server actually wrote 8*N inner-offset bytes plus all the string data. The input stream is desynchronized from that point on: the remaining block (and every block after it) is decoded as garbage, typically surfacing much later as an unrelated parse failure or a corrupted/empty result rather than a clear "unsupported type" error.

This is the C++ analogue of ClickHouse/clickhouse-java#3178, where the JDBC driver also failed on Array(Nested(...)) because its conversion layer did not account for Nested producing an extra list level.

Related but distinct from #40, which asks for a convenience API over flattened top-level Nested columns (those already work, since the server hands them over as plain Array(T)). Here the client receives a literal Nested(...) type name and silently produces wrong data.

ClickHouse server version

26.9.8.3 — used to confirm the type name the server reports for Array(Nested(...)) (shown above).

Code analysis only; the C++ repro below was written but not executed — binary execution was unavailable in the environment I investigated from. The static trace through TypeParser::Parse → CreateColumnFromAst → CreateTerminalColumn is given in full under "Suggested fix" below, and the first assertion (Array(Nothing)) is a pure parser/factory fact requiring no server.

Reproduction

Pure client-side, no server needed — this already demonstrates the root cause:

#include <clickhouse/columns/factory.h>
#include <gtest/gtest.h>

using namespace clickhouse;

TEST(CreateColumnByType, ArrayOfNested) {
    auto col = CreateColumnByType("Array(Nested(key String, value String))");
    ASSERT_NE(nullptr, col);
    // Expected: Array(Array(Tuple(String, String)))
    // Actual:   Array(Nothing)
    EXPECT_EQ("Array(Array(Tuple(String, String)))", col->Type()->GetName());
}

End-to-end, against a server:

CREATE TABLE some_data (
    `id`   UInt64,
    `data` Array(Nested(`key` String, `value` String))
) ENGINE = Memory;

INSERT INTO some_data VALUES
(1, [[('key1','test'), ('key2','another-test')], [('key1','more-data')]]);

#include <clickhouse/client.h>
#include <iostream>

using namespace clickhouse;

int main() {
    Client client(ClientOptions().SetHost("localhost").SetPort(9000));
    client.Select("SELECT id, data FROM some_data", [](const Block& block) {
        for (size_t c = 0; c < block.GetColumnCount(); ++c) {
            std::cout << block.GetColumnName(c) << " -> "
                      << block[c]->Type()->GetName() << "\n";
        }
    });
}

Expected: the data column comes back as Array(Array(Tuple(String, String))) with the two inner arrays intact (or, failing that, a clear UnimplementedError naming the unsupported type).

Actual: data is reported as Array(Nothing) carrying no values, and because ColumnNothing::LoadBody under-consumes the stream by the full size of the inner offsets and strings, decoding of the rest of the response is corrupted.

Suggested fix

Trace of the current behaviour:

• clickhouse/types/type_parser.cpp:116 — GetTypeMeta() has no Nested branch, so the Nested(...) node falls through to TypeAst::Terminal.
• clickhouse/types/type_parser.cpp:107 — GetTypeCode("Nested") misses kTypeCode and returns Type::Void.
• clickhouse/types/type_parser.cpp:152 — ValidateAST() does reject unknown Terminal + Void nodes, but it is only ever called on the root AST node (type_parser.cpp:238). The root here is Array, so the bad child is never validated.
• clickhouse/columns/factory.cpp:49 — CreateTerminalColumn() maps Type::Void to ColumnNothing, turning the unknown type into a silently-wrong column instead of a nullptr.

Two things worth doing, independently useful:

1. Support Nested. Nested(a T1, b T2) is exactly Array(Tuple(a T1, b T2)). Adding a Nested meta that desugars to that in CreateColumnFromAst would make Array(Nested(...)) decode correctly, and would also let named-tuple element names (already supported via TypeAst::element_name) carry through.

2. Fail loudly on unknown nested types. Apply ValidateAST recursively, or have CreateTerminalColumn return nullptr for Type::Void when ast.name is not literally Nothing/void. Right now any unrecognized type nested inside a container degrades to ColumnNothing and desynchronizes the stream rather than raising UnimplementedError. (Same silent-desync failure mode as ColumnArray::AppendAsColumn silently accepts a wrong-typed element column, writes zero data bytes and desynchronizes the native block stream #543.)

Link

Original client issue: ClickHouse/clickhouse-java#3178