You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
MySQL sources store information about the tables' columns in MySqlColumnDesc:
pubstructMySqlColumnDesc{/// The name of the column.pubname:String,/// The intended data type of this column within Materialize/// If this is None, the column is intended to be skipped within Materializepubcolumn_type:Option<ColumnType>,/// Optional metadata about the column that may be necessary for decodingpubmeta:Option<MySqlColumnMeta>,}
Here is the iimpl of mz_repr::relation::ColumnType:
pubstructColumnType{/// The underlying scalar type (e.g., Int32 or String) of this column.pubscalar_type:ScalarType,/// Whether this datum can be null.#[serde(default = "return_true")]pubnullable:bool,}
MySqlColumnDesc::column_type:
Is an Option because it is None when users specify IGNORE COLUMNS on this column.
Is capable of expressing all columns in terms of ScalarType because if the MySQL type is unsupported, we force users to specify TEXT COLUMNS.
This fails to separate concerns––e.g. the MySqlColumnDesc cannot describe the actual upstream table's schema because we apply apply configuration options to it.
If we fully seprated concerns, purification would be able to describe the tables we want to ingest, and processing the TEXT COLUMNS and IGNORE COLUMNS options could occur in planning. With the current design that is impossible.
The text was updated successfully, but these errors were encountered:
MySQL sources store information about the tables' columns in
MySqlColumnDesc
:Here is the iimpl of
mz_repr::relation::ColumnType
:MySqlColumnDesc::column_type
:Option
because it isNone
when users specifyIGNORE COLUMNS
on this column.ScalarType
because if the MySQL type is unsupported, we force users to specifyTEXT COLUMNS
.This fails to separate concerns––e.g. the
MySqlColumnDesc
cannot describe the actual upstream table's schema because we apply apply configuration options to it.If we fully seprated concerns, purification would be able to describe the tables we want to ingest, and processing the
TEXT COLUMNS
andIGNORE COLUMNS
options could occur in planning. With the current design that is impossible.The text was updated successfully, but these errors were encountered: