Skip to content

Instantly share code, notes, and snippets.

@llermaly

llermaly/esql-timeline.md

Last active June 25, 2025 02:56
Show Gist options
  • Save llermaly/18a6ea53553e04fd5217e21622b9beeb to your computer and use it in GitHub Desktop.
Save llermaly/18a6ea53553e04fd5217e21622b9beeb to your computer and use it in GitHub Desktop.
Download ZIP
ES|QL Timeline
Raw
esql-timeline.md

ES|QL Timeline 8.11 -> 9.0

Elasticsearch 8.11

New Features

  • Integrate Elasticsearch Query Language, ES|QL #98309
  • LEAST and GREATEST functions #98630
  • LEFT function #98942
  • LTRIM, RTRIM and fix unicode whitespace #98590
  • RIGHT function #98974
  • TopN sorting with min and max for multi-value fields #98337

Enhancements

  • Add CEIL function #98847
  • Add ability to perform date math #98870 (issue: #98402)
  • Add support for TEXT fields in comparison operators and SORT #98528 (issue: #98642)
  • Compact topn #99316
  • Date math for negatives #99711
  • Enable arithmetics for durations and periods #99432 (issue: #99293)
  • Enhance SHOW FUNCTIONS command #99736 (issue: #99507)
  • Improve log messages #99470
  • Log execution time consistently #99286
  • Log query and execution time #99058
  • Log start and end of queries #99746
  • Lower the implicit limit, if none is user-provided #99816 (issue: #99458)
  • Make settings dynamic #101516
  • Mark counter fields as unsupported #99054
  • Remove the swapped-args check for date_xxx() #101362 (issue: #99562)
  • Serialize the source in expressions #99956
  • Simple check if all blocks get released #100199
  • Support unsigned long in sqrt and log10 #98711
  • Use DEBUG log level to report execution steps #99303

Elasticsearch 8.12

Notable Changes

  • [ES|QL] pow function always returns double #102183 (issue: #99055)

Enhancements

  • ESQL: Add profile option #102713
  • ESQL: Alias duplicated aggregations in a stats #100642 (issue: #100544)
  • ESQL: Load more than one field at once #102192
  • ESQL: Load stored fields sequentially #102727
  • ESQL: Load text field from parent keyword field #102490 (issue: #102473)
  • ESQL: Make blocks ref counted #100408
  • ESQL: Make fieldcaps calls lighter #102510 (issues: #101763, #102393)
  • ESQL: More tracking in BlockHash impls #101488
  • ESQL: New telemetry commands #102937
  • ESQL: Share constant null Blocks #102673
  • ESQL: Short circuit loading empty doc values #102434
  • ESQL: Support the _source metadata field #102391
  • ESQL: Track blocks emitted from lucene #101396
  • ESQL: Track memory from values loaded from lucene #101383
  • Fast path for reading single doc with ordinals #102902
  • Introduce local block factory #102901
  • Load different way #101235
  • Track ESQL enrich memory #102184
  • Track blocks in AsyncOperator #102188
  • Track blocks of intermediate state of aggs #102562
  • Track blocks when hashing single multi-valued field #102612
  • Track pages in ESQL enrich request/response #102190

New Features

  • ESQL: emit warnings from single-value functions processing multi-values #102417 (issue: #98743)
  • GEO_POINT and CARTESIAN_POINT type support #102177

Elasticsearch 8.13

Breaking Changes

  • ESQL: Grammar - FROM METADATA no longer requires [] #105221
  • ES|QL: remove PROJECT keyword from the grammar #105064
  • [ESQL] Remove is_nan, is_finite, and is_infinite #104091

Enhancements

  • Add ES|QL async delete API #103628
  • Avoid humongous blocks #103340
  • ESQL: Add TO_UPPER and TO_LOWER functions #104309
  • ESQL: Add option to drop null fields #102428
  • ESQL: Add plan consistency verification after each optimizer #105371
  • ESQL: Check field exists before load from _source #103632
  • ESQL: Delay finding field load infrastructure #103821
  • ESQL: Expand shallow copy with vecs #103681 (issue: #100528)
  • ESQL: Extend STATS command to support aggregate expressions #104958
  • ESQL: Infer not null for aggregated fields #103673 (issue: #102787)
  • ESQL: Nested expressions inside stats command #104387 (issue: #99828)
  • ESQL: Pre-allocate rows in TopNOperator #104796
  • ESQL: Referencing expressions that contain backticks requires escaping those backticks. #100740(issue: #100312)
  • ESQL: Simpify IS NULL/IS NOT NULL evaluation #103099 (issue: #103097)
  • ESQL: Speed up reading many nulls #105088
  • ESQL: Support loading shapes from source into WKB blocks #104269
  • ESQL: Track the rest of DocVector #103727
  • ESQL: MV_FIRST and MV_LAST #103928
  • ESQL: add date_diff function #104118 (issue: #101942)
  • ESQL: push down "[text_field] is not null" #105593
  • ES|QL Async Query API #103398
  • Prepare enrich plan to support multi clusters #104355
  • Reading points from source to reduce precision loss #103698
  • Remove deprecated Block APIs #103592
  • Reserve bytes before serializing page #105269
  • Support ST_CENTROID over spatial points #104218 (issue: #104656)
  • Support cross clusters query in ESQL #101640
  • Support enrich ANY mode in cross clusters query #104840
  • Support enrich coordinator mode #104936
  • Support enrich remote mode #104993

New features

  • ESQL: Introduce mode setting for ENRICH #103949
  • ESQL: add =~ operator (case insensitive equality) #103656

8.13.3

Enhancements

  • ESQL: Introduce language versioning to REST API #106824

Elasticsearch 8.14(GA)

Enhancements

  • Add ES|QL Locate function #106899 (issue: #106818)
  • Add ES|QL signum function #106866
  • Add status for enrich operator #106036
  • Add two new OGC functions ST_X and ST_Y #105768
  • Adjust array resizing in block builder #106934
  • Bulk loading enrich fields in ESQL #106796
  • ENRICH support for TEXT fields #106435 (issue: #105384)
  • ESQL: Add timers to many status results #105421
  • ESQL: Allow grouping key inside stats expressions #106579
  • ESQL: Introduce expression validation phase #105477 (issue: #105425)
  • ESQL: Log queries at debug level #108257
  • ESQL: Regex improvements #106429
  • ESQL: Sum of constants #105454
  • ESQL: Support ST_DISJOINT #107007
  • ESQL: Support partially folding CASE #106094
  • ESQL: Use faster field caps #105067
  • ESQL: extend BUCKET with spans #107272
  • ESQL: perform a reduction on the data node #106516
  • Expand support for ENRICH to full set supported by ES ingest processors #106186 (issue: #106162)
  • Introduce ordinal bytesref block #106852 (issue: #106387)
  • Leverage ordinals in enrich lookup #107449
  • Serialize big array blocks #106373
  • Serialize big array vectors #106327
  • Specialize serialization for ArrayVectors #105893
  • Specialize serialization of array blocks #106102
  • Speed up serialization of BytesRefArray #106053
  • Support ST_CONTAINS and ST_WITHIN #106503
  • Support ST_INTERSECTS between geometry column and other geometry or string #104907 (issue: #104874)

New features

  • ESQL: Values aggregation function #106065 (issue: #103600)
  • ESQL: allow sorting by expressions and not only regular fields #107158
  • Support ES|QL requests through the NodeClient::execute #106244

Elasticsearch 8.15

New features

  • ESQL: Add ip_prefix function #109070 (issue: #99064)
  • ESQL: Introduce a casting operator, :: #107409
  • ESQL: top_list aggregation #109386 (issue: #109213)
  • ESQL: add Arrow dataframes output format #109873
  • Reapply "ESQL: Expose "_ignored" metadata field" #108871

Elasticsearch 8.16

Breaking Changes

  • Entirely remove META FUNCTIONS #113967

Enhancements

  • Add EXP ES|QL function #110879
  • Delay construction of warnings #114368
  • Add CircuitBreaker to TDigest, Step 3: Connect with ESQL CB #113387
  • Add CircuitBreaker to TDigest, Step 4: Take into account shallow classes size #113613 (issue: #113916)
  • Collect and display execution metadata for ES|QL cross cluster searches #112595 (issue: #112402)
  • Add support for multivalue fields in Arrow output #114774
  • BUCKET: allow numerical spans as whole numbers #111874 (issues: #104646, #109340, #105375)
  • Have BUCKET generate friendlier intervals #111879 (issue: #110916)
  • Profile more timing information #111855
  • Push down filters even in case of renames in Evals #114411
  • Speed up CASE for some parameters #112295
  • Speed up grouping by bytes #114021
  • Use less memory in listener #114358
  • Add support for cached strings in plan serialization #112929
  • Add Telemetry API and track top functions #111226
  • Enhance SORT push-down to Lucene to cover references to fields and ST_DISTANCE function #112938 (issue: #109973)
  • Siem ea 9521 improve test #111552
  • Support multi-valued fields in compute engine for ST_DISTANCE #114836 (issue: #112910)
  • Add SPACE function #112350
  • Add finish() elapsed time to aggregation profiling times #113172 (issue: #112950)
  • Make query wrapped by SingleValueQuery cacheable #110116
  • Add hypot function #114382
  • Cast mixed numeric types to a common numeric type for Coalesce and In at Analyzer #111917 (issue: #111486)
  • Combine Disjunctive CIDRMatch #111501 (issue: #105143)
  • Create Range in PushFiltersToSource for qualified pushable filters on the same field #111437
  • Name parameter with leading underscore #111950 (issue: #111821)
  • Named parameter for field names and field name patterns #112905
  • Validate index name in parser #112081
  • Add reverse function #113297
  • Explicit cast a string literal to date_period and time_duration in arithmetic operations #109193

New Features

  • Add match function #113374
  • Add MV_PSERIES_WEIGHTED_SUM for score calculations used by security solution #109017
  • Add async ID and is_running headers to ESQL async query #111840
  • Add boolean support to Max and Min aggs #110527
  • Add boolean support to TOP aggregation #110718
  • Added mv_percentile function #111749 (issue: #111591)
  • Introduce per agg filter #113735
  • Strings support for MAX and MIN aggregations #111544
  • Support IP fields in MAX and MIN aggregations #110921
  • TOP aggregation IP support #111105
  • TOP support for strings #113183 (issue: #109849)
  • mv_median_absolute_deviation function #112055 (issue: #111590)
  • Add MATCH operator #110971

Elasticsearch 8.17

Enhancements

  • Add ES|QL bit_length function #115792
  • ESQL: Honor skip_unavailable setting for nonmatching indices errors at planning time #116348 (issue: #114531)
  • ESQL: Remove parent from FieldAttribute #112881
  • ESQL: extract common filter from aggs #115678
  • ESQL: optimise aggregations filtered by false/null into evals #115858
  • ES|QL CCS uses skip_unavailable setting for handling disconnected remote clusters #115266(issue: #114531)
  • ES|QL: add metrics for functions #114620
  • Esql Enable Date Nanos (tech preview) #117080
  • [ES|QL] Implicit casting string literal to intervals #115814 (issue: #115352)

New features

  • Add support for BYTE_LENGTH scalar function #116591
  • Esql/lookup join grammar #116515
  • Remove snapshot build restriction for match and qstr functions #114482

Elasticsearch 8.18

Enhancements

  • Add ES|QL cross-cluster query telemetry collection #119474
  • Add a LicenseAware interface for licensed Nodes #118931 (issue: #117405)
  • Add a PostAnalysisAware, distribute verification #119798
  • Add a standard deviation aggregating function: STD_DEV #116531
  • Add cluster level reduction #117731
  • Add nulls support to Categorize #117655
  • Async search responses have CCS metadata while searches are running #117265
  • Backport Term query for ES|QL to 8.x #118135
  • Backport scoring support in ES|QL to 8.x branch #117747
  • Check for early termination in Driver #118188
  • Do not serialize EsIndex in plan #119580
  • ES|QL - Remove restrictions for disjunctions in full text functions #118544
  • ES|QL - enabling scoring with METADATA _score #113120
  • ES|QL Add ES|QL hash function #117989
  • ES|QL Support IN operator for Date nanos #119772 (issue: #118578)
  • ES|QL: CATEGORIZE as a BlockHash #114317
  • ES|QL: Enterprise license enforcement for CCS #118102
  • ES|QL: Partial result on demand for async queries #118122
  • Enable KQL function as a tech preview #119730
  • Enable LOOKUP JOIN in non-snapshot builds #121193 (issue: #121185)
  • Enable node-level reduction by default #119621
  • Enable physical plan verification #118114
  • ES|QL - Support date nanos in date extract function #120727 (issue: #110000)
  • ES|QL - support date nanos in date format function #120143 (issue: #109994)
  • ES|QL Support date nanos on date diff function #120645 (issue: #109999)
  • ES|QL bucket function for date nanos #118474 (issue: #118031)
  • ES|QL compare nanos and millis #118027 (issue: #116281)
  • ES|QL implicit casting for date nanos #118697 (issue: #118476)
  • Extend TranslationAware to all pushable expressions #120192
  • Hash functions #118938
  • Implement a MetricsAware interface #121074
  • LOOKUP JOIN using field-caps for field mapping #117246
  • Lookup join on multiple join fields not yet supported #118858
  • Move scoring in ES|QL out of snapshot #120354
  • Optimize ST_EXTENT_AGG for geo_shape and cartesian_shape #119889
  • Push down StartsWith and EndsWith functions to Lucene #123381 (issue: #123067)
  • Push down filter passed lookup join #118410
  • Resume Driver on cancelled or early finished #120020
  • Reuse child outputSet inside the plan where possible #124611
  • Rewrite TO_UPPER/TO_LOWER comparisons #118870 (issue: #118304)
  • ST_EXTENT_AGG optimize envelope extraction from doc-values for cartesian_shape #118802
  • Smarter field caps with subscribable listener #116755
  • Support some stats on aggregate_metric_double #120343 (issue: #110649)
  • Take named parameters for identifier and pattern out of snapshot #121850
  • Term query for ES|QL #117359
  • Update grammar to rely on indexPattern instead of identifier in join target #120494
  • _score should not be a reserved attribute in ES|QL #118435 (issue: #118460)

New Features

  • ES|QL - Add Match function options #120360
  • ES|QL - Allow full text functions disjunctions for non-full text functions #120291
  • ES|QL: Enable async get to support formatting #111104 (issue: #110926)
  • Expand type compatibility for match function and operator #117555
  • ST_EXTENT aggregation #117451 (issue: #104659)
  • Support ST_ENVELOPE and related (ST_XMIN, ST_XMAX, ST_YMIN, ST_YMAX) functions #116964(issue: #104875)

Elasticsearch 9.0

Highlights

ES|QL LOOKUP JOIN is now available in technical preview

LOOKUP JOIN is now available in technical preview. LOOKUP JOIN combines data from your ES|QL queries with matching records from a lookup index, enabling you to:

  • Enrich your search results with reference data
  • Speed up root-cause analysis and security investigations
  • Join data across indices without complex queries
  • Reduce operational overhead when correlating events

Enhancements

  • Add ES|QL cross-cluster query telemetry collection #119474
  • Add a LicenseAware interface for licensed Nodes #118931 (issue: #117405)
  • Add a PostAnalysisAware, distribute verification #119798
  • Add a standard deviation aggregating function: STD_DEV #116531
  • Add cluster level reduction #117731
  • Add nulls support to Categorize #117655
  • Allow skip shards with _tier and _index in ES|QL #123728
  • Async search responses have CCS metadata while searches are running #117265
  • Check for early termination in Driver #118188
  • Do not serialize EsIndex in plan #119580
  • ESQL - Add Match function options #120360
  • ESQL - Allow full text functions disjunctions for non-full text functions #120291
  • ESQL - Remove restrictions for disjunctions in full text functions #118544
  • ESQL - enabling scoring with METADATA _score #113120
  • ESQL Add esql hash function #117989
  • ESQL Support IN operator for Date nanos #119772 (issue: #118578)
  • ESQL: Align RENAME behavior with EVAL for sequential processing #122250 (issue: #121739)
  • ESQL: CATEGORIZE as a BlockHash #114317
  • ESQL: Enable async get to support formatting #111104 (issue: #110926)
  • ESQL: Enterprise license enforcement for CCS #118102
  • ES|QL - Add scoring for full text functions disjunctions #121793
  • ES|QL: Partial result on demand for async queries #118122
  • Enable KQL function as a tech preview #119730
  • Enable LOOKUP JOIN in non-snapshot builds #121193 (issue: #121185)
  • Enable node-level reduction by default #119621
  • Enable physical plan verification #118114
  • Ensure cluster string could be quoted #120355
  • Esql - Support date nanos in date extract function #120727 (issue: #110000)
  • Esql - support date nanos in date format function #120143 (issue: #109994)
  • Esql Support date nanos on date diff function #120645 (issue: #109999)
  • Esql bucket function for date nanos #118474 (issue: #118031)
  • Esql compare nanos and millis #118027 (issue: #116281)
  • Esql implicit casting for date nanos #118697 (issue: #118476)
  • Expand type compatibility for match function and operator #117555
  • Extend TranslationAware to all pushable expressions #120192
  • Fix Driver status iterations and cpuTime #123290 (issue: #122967)
  • Hash functions #118938
  • Implement a MetricsAware interface #121074
  • Initial support for unmapped fields #119886
  • LOOKUP JOIN using field-caps for field mapping #117246
  • Lookup join on multiple join fields not yet supported #118858
  • Move scoring in ES|QL out of snapshot #120354
  • Optimize ST_EXTENT_AGG for geo_shape and cartesian_shape #119889
  • Push down StartsWith and EndsWith functions to Lucene #123381 (issue: #123067)
  • Push down filter passed lookup join #118410
  • Resume Driver on cancelled or early finished #120020
  • Reuse child outputSet inside the plan where possible #124611
  • Rewrite TO_UPPER/TO_LOWER comparisons #118870 (issue: #118304)
  • ST_EXTENT aggregation #117451 (issue: #104659)
  • ST_EXTENT_AGG optimize envelope extraction from doc-values for cartesian_shape #118802
  • Smarter field caps with subscribable listener #116755
  • Support ST_ENVELOPE and related (ST_XMIN, ST_XMAX, ST_YMIN, ST_YMAX) functions #116964(issue: #104875)
  • Support partial sort fields in TopN pushdown #116043 (issue: #114515)
  • Support some stats on aggregate_metric_double #120343 (issue: #110649)
  • Take named parameters for identifier and pattern out of snapshot #121850
  • Term query for ES|QL #117359
  • Update grammar to rely on indexPattern instead of identifier in join target #120494
  • _score should not be a reserved attribute in ES|QL #118435 (issue: #118460)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment