Get started with Propel's Serverless ClickHouse forever-free plan today. Propel is the only Serverless ClickHouse with a true pay-per-query pricing and instant auto-scaling.

ClickHouse version 24.10 introduces several major enhancements, new features, performance improvements, and some experimental updates that ClickHouse developers will find particularly useful. Below, we'll cover the key highlights of this new version.

Backward-incompatible changes

1. Settings clause behavior: ClickHouse now allows writing SETTINGS before the FORMAT in queries involving UNION with subqueries inside parentheses. Additionally, in cases where SETTINGS is used multiple times within a sequence, the closest SETTINGS clause takes precedence. In previous versions, the outermost SETTINGS clause would have overridden others.
2. Reordering of filter conditions: Reordering of filter conditions from [PRE]WHERE clauses is now allowed by default. This reordering can be disabled by setting allow_reorder_prewhere_conditions to false.

New features

1. Enhanced JSON support: ClickHouse now supports new settings for reading and writing JSON as binary strings in RowBinary format using input_format_binary_read_json_as_string and output_format_binary_write_json_as_string. It also allows serializing/deserializing JSON columns as single String columns in the Native format, controlled by output_format_native_write_json_as_string for output and using serialization version 1 for input.
2. Wildcard access grants: It is now possible to grant access based on wildcard prefixes. For example, you can use GRANT SELECT ON db.table_prefix_* TO user to easily manage permissions across multiple tables.
3. Progress table toggle during queries: By pressing the spacebar during a query runtime, the ClickHouse client now displays a real-time table with detailed metrics. You can enable this globally with the -progress-table option in clickhouse-client. Additionally, the -enable-progress-table-toggle allows toggling the rendering of the progress table by pressing the control key.
4. Caching object storage read files: Object storage table engines and data lakes now support caching read files using the hash from ETag and file path as cache keys. This feature enhances performance when reading frequently accessed data.
5. New system table: A new system table, system.query_metric_log, is introduced to maintain a history of memory and metric values from system.events for individual queries. This table provides a better way to track query metrics and performance over time.

New functions

1. arrayUnion: Adds support for the arrayUnion function, which returns an array with unique elements from the input arrays combined.
2. arrayElementOrNull: Introduces the arrayElementOrNull function, which returns NULL if the array index is out of range or if a map key is not found.
3. quantileExactWeightedInterpolated: A new aggregate function quantileExactWeightedInterpolated has been added, which provides an interpolated version based on quantileExactWeighted. This function offers improved accuracy compared to quantileExactInterpolatedWeighted.
4. RIPEMD160: Added the RIPEMD160 function for computing the RIPEMD-160 cryptographic hash of a string. Example: SELECT HEX(RIPEMD160('The crunchy taco zooms past the blazing hot sauce')) returns 53B9A442706901FF5F3C73399B3FA7B9F6AD2D3F.

Experimental features

1. JSON as a binary string: Added experimental support for reading/writing JSON as a binary string in RowBinary and Native formats. This is controlled through specific settings for input and output serialization, enhancing flexibility when working with JSON data.
2. Refreshable materialized views: Refreshable materialized views are now production-ready, including support for replicated databases, making it easier to maintain up-to-date data copies across nodes.
3. Parallel replicas moved to beta: Settings for controlling the behavior of parallel replicas have been reworked, and parallel replicas have now been moved from experimental to beta status. This includes settings like parallel_replicas_mode and the enable_parallel_replicas toggle.
4. Dynamic type support: Support for executing functions on Dynamic types has been added, allowing functions to be performed on internal types within a dynamic structure.
5. Aggressive merge selector for MergeTree: An experimental mode has been added for a more aggressive merge selector for MergeTree tables. This mode is useful for partitions that are nearing the limit by the number of parts.

Performance improvements

1. Object storage performance: The IDisk and IObjectStorage interfaces have been refactored for better performance, resulting in faster initialization of tables stored in plain or rewritable object storages.
2. Optimized Parquet reading: ClickHouse now supports parallel reading of Parquet row groups and prefetching row groups in single-threaded mode, leading to faster data ingestion from Parquet files.
3. Bloom filters for Parquet: Bloom filters are now used when reading Parquet files, improving the performance of searches on Parquet datasets.
4. Lock-free parts rename: The rename of parts in ReplicatedMergeTree tables is now lock-free, allowing INSERT operations to occur without affecting SELECT performance. This improves throughput significantly under high-load scenarios.
5. Optimized thread creation: Thread creation in the ThreadPool has been optimized to minimize lock contention, leading to improved responsiveness under high concurrent loads.

Bug fixes

1. Settings configuration update: Fixed issues where settings were not properly applied in the global context, which had caused configurations to be ignored.
2. JOIN optimization fix: Fixed incorrect optimization of JOIN ON conditions that involved IS NULL checks. This ensures accurate results when complex expressions are used in JOIN conditions.
3. Materialized view issues: Fixed crashes when creating materialized views with certain nested queries involving INTERSECT. Also fixed issues related to WITH and FILL clauses that could previously cause unexpected behavior.
4. Parallel replicas fixes: Resolved several bugs with the parallel replicas feature, ensuring more consistent and reliable behavior when using multiple replicas in parallel.

Summary

ClickHouse version 24.10 introduces significant improvements in query flexibility, security, and performance while bringing new features like wildcard access grants, enhanced progress metrics, and MongoDB integration. Experimental features like refreshable materialized views and aggressive merge selectors provide more options for tuning performance.

The version is packed with new possibilities and optimizations. If you are upgrading from previous versions, make sure to review the backward-incompatible changes and adjust any existing queries or configurations accordingly.

Let us know how these new features improve your setup, and happy querying!

Get started with Propel's Serverless ClickHouse forever-free plan today. Propel is the only Serverless ClickHouse with a true pay-per-query pricing and instant auto-scaling. Contact us to learn more about our volume-based discounts. Visit our pricing page for details.

‍