AST & Custom Generators

FlyQL’s parser produces an Abstract Syntax Tree (AST) — a structured representation of the query that you can traverse to generate output for any target. The built-in SQL generators (ClickHouse, PostgreSQL, StarRocks) are all implemented by walking this same AST. You can build your own generator for Elasticsearch queries, MongoDB filters, Prometheus selectors, log grep commands, or any other target.

AST Node Types

The AST consists of three types: Node (tree structure), Expression (leaf conditions), and Key (field references).

Node

A Node is either a branch (combining two subtrees with a boolean operator) or a leaf (holding a single expression). It is the fundamental building block of the AST.

Field	Type	Description
boolOperator	string	and or or — how left and right are combined. Empty on leaf nodes.
expression	Expression or null	The condition, if this is a leaf node. null on branch nodes.
left	Node or null	Left child subtree. null on leaf nodes.
right	Node or null	Right child subtree. null on leaf nodes.
negated	boolean	true if this node is wrapped in not.

A node is either a leaf (expression is set, left/right are null) or a branch (left/right are set, expression is null). It is never both.

Expression

An Expression represents a single condition — a key, an operator, and a value.

Field	Type	Description
key	Key	The field being tested (e.g. status, user.metadata.role).
operator	string	One of: =, !=, ~, !~, >, <, >=, <=, truthy, in, not in, has, not has.
value	any	The comparison value. A string, number, boolean, null, FunctionCall, or Parameter for scalar operators. Empty for in and not in.
valueType	string or null	The explicit type of value: "int", "bigint", "float", "string", "bool", "null", "function", or "parameter". null for in/not in.
values	array or null	List of values for in and not in operators. Can be heterogeneous and may contain Parameter entries. null for other operators.
valuesTypes	string[] or null	Per-element type array parallel to values. Each element is one of the type constants (including "parameter"). null when values is not used.

Key

A Key represents the field path being queried.

Field	Type	Description
segments	string[]	Path segments. ["status"] for a simple key, ["user", "metadata", "role"] for nested.
raw	string	The original key string as written in the query.
isSegmented	boolean	true if the key has more than one segment (i.e. uses dot notation).

Parameter

A Parameter represents an unresolved placeholder ($name or $1) that will be substituted by bindParams() before SQL generation or matching.

Field	Type	Description
name	string	The placeholder identifier without the $ prefix. Named: "code". Positional: "1".
positional	boolean	true if the parameter is positional (digit-only name).

A Parameter can appear as:

• expression.value with valueType === "parameter" (scalar position)
• An entry inside expression.values (IN-list position)
• An entry inside FunctionCall.parameterArgs (temporal function argument position)

After bindParams() resolves them, no Parameter instances remain in the AST.

FunctionCall

A FunctionCall represents a temporal function value (ago(...), now(), today(...), startOf(...)).

Field	Type	Description
name	string	One of "ago", "now", "today", "startOf".
durationArgs	Duration[]	Parsed duration arguments (e.g. [{value: 5, unit: "m"}]).
unit	string	The unit argument for startOf ("day", "week", or "month").
timezone	string	The timezone argument for today and startOf.
parameterArgs	Parameter[]	Parameter placeholders that will fill in durationArgs/unit/timezone at bind time.

Operators

All operators are available as constants you can import:

// JavaScript
import { Operator, BoolOperator } from 'flyql'

Operator.EQUALS       // "="
Operator.NOT_EQUALS   // "!="
Operator.REGEX        // "~"
Operator.NOT_REGEX    // "!~"
Operator.GREATER_THAN // ">"
Operator.LOWER_THAN   // "<"
Operator.GREATER_OR_EQUALS_THAN // ">="
Operator.LOWER_OR_EQUALS_THAN   // "<="
Operator.TRUTHY       // "truthy"
Operator.IN           // "in"
Operator.NOT_IN       // "not in"
Operator.HAS          // "has"
Operator.NOT_HAS      // "not has"
Operator.LIKE         // "like"
Operator.NOT_LIKE     // "not like"
Operator.ILIKE        // "ilike"
Operator.NOT_ILIKE    // "not ilike"

BoolOperator.AND      // "and"
BoolOperator.OR       // "or"

// Go
import flyql "github.com/iamtelescope/flyql/golang"

flyql.OpEquals          // "="
flyql.OpNotEquals       // "!="
flyql.OpRegex           // "~"
flyql.OpNotRegex        // "!~"
flyql.OpGreater         // ">"
flyql.OpLess            // "<"
flyql.OpGreaterOrEquals // ">="
flyql.OpLessOrEquals    // "<="
flyql.OpTruthy          // "truthy"
flyql.OpIn              // "in"
flyql.OpNotIn           // "not in"
flyql.OpHas             // "has"
flyql.OpNotHas          // "not has"
flyql.OpLike            // "like"
flyql.OpNotLike         // "not like"
flyql.OpILike           // "ilike"
flyql.OpNotILike        // "not ilike"

flyql.BoolOpAnd      // "and"
flyql.BoolOpOr       // "or"

# Python
from flyql.core.constants import Operator, BoolOperator

Operator.EQUALS       # "="
Operator.NOT_EQUALS   # "!="
Operator.REGEX        # "~"
Operator.NOT_REGEX    # "!~"
Operator.GREATER_THAN # ">"
Operator.LOWER_THAN   # "<"
Operator.GREATER_OR_EQUALS_THAN # ">="
Operator.LOWER_OR_EQUALS_THAN   # "<="
Operator.TRUTHY       # "truthy"
Operator.IN           # "in"
Operator.NOT_IN       # "not in"
Operator.HAS          # "has"
Operator.NOT_HAS      # "not has"
Operator.LIKE         # "like"
Operator.NOT_LIKE     # "not like"
Operator.ILIKE        # "ilike"
Operator.NOT_ILIKE    # "not ilike"

BoolOperator.AND      # "and"
BoolOperator.OR       # "or"

AST Structure Examples

Simple condition

Query: status = 200

Node (leaf)
  expression:
    key: { segments: ["status"], raw: "status" }
    operator: "="
    value: 200

Boolean combination

Query: status = 200 and method = "GET"

Node (branch, boolOperator: "and")
  left: Node (leaf)
    expression:
      key: { segments: ["status"], raw: "status" }
      operator: "="
      value: 200
  right: Node (leaf)
    expression:
      key: { segments: ["method"], raw: "method" }
      operator: "="
      value: "GET"

Negation

Query: not status = 500

Node (leaf, negated: true)
  expression:
    key: { segments: ["status"], raw: "status" }
    operator: "="
    value: 500

Nested boolean with parentheses

Query: env='prod' and (status=500 or level = 'error')

Node (branch, boolOperator: "and")
  left: Node (leaf)
    expression:
      key: { segments: ["env"], raw: "env" }
      operator: "="
      value: "prod"
  right: Node (branch, boolOperator: "or")
    left: Node (leaf)
      expression:
        key: { segments: ["status"], raw: "status" }
        operator: "="
        value: 500
    right: Node (leaf)
      expression:
        key: { segments: ["level"], raw: "level" }
        operator: "="
        value: "error"

List membership

Query: status in [200, 201, 204]

Node (leaf)
  expression:
    key: { segments: ["status"], raw: "status" }
    operator: "in"
    value: ""
    values: [200, 201, 204]
    valuesType: "number"

Truthy check

Query: active

Node (leaf)
  expression:
    key: { segments: ["active"], raw: "active" }
    operator: "truthy"
    value: ""

Parameter placeholder

Query: status = $code

Node (leaf)
  expression:
    key: { segments: ["status"], raw: "status" }
    operator: "="
    value: { name: "code", positional: false }
    valueType: "parameter"

After bindParams(node, { code: 200 }):

Node (leaf)
  expression:
    key: { segments: ["status"], raw: "status" }
    operator: "="
    value: 200
    valueType: "int"

Parameter inside an IN-list

Query: status in [$x, 'ok']

Node (leaf)
  expression:
    key: { segments: ["status"], raw: "status" }
    operator: "in"
    values: [{ name: "x", positional: false }, "ok"]
    valuesTypes: ["parameter", "string"]

Parameter inside a temporal function

Query: created > ago($d)

Node (leaf)
  expression:
    key: { segments: ["created"], raw: "created" }
    operator: ">"
    value:
      name: "ago"
      durationArgs: []
      parameterArgs: [{ name: "d", positional: false }]
    valueType: "function"

Nested key

Query: user.metadata.role = "admin"

Node (leaf)
  expression:
    key: { segments: ["user", "metadata", "role"], raw: "user.metadata.role", isSegmented: true }
    operator: "="
    value: "admin"

Traversing the AST

The AST is a binary tree. To process it, use recursive traversal:

1. If the node has an expression (leaf node) — handle the condition.
2. If the node has left/right (branch node) — recurse into children and combine with boolOperator.
3. If negated is true — wrap the result in negation.

Building a Custom Generator

Here is a complete example of a custom generator that converts a FlyQL AST into an Elasticsearch Query DSL object.

JavaScript

import { parse } from 'flyql'
import { Operator } from 'flyql'

function generateElasticsearch(node) {
  if (!node) return { match_all: {} }

  let result

  if (node.expression) {
    // Leaf node — convert the expression
    result = expressionToES(node.expression)
  } else {
    // Branch node — recurse and combine
    const left = generateElasticsearch(node.left)
    const right = generateElasticsearch(node.right)

    if (node.boolOperator === 'and') {
      result = { bool: { must: [left, right] } }
    } else {
      result = { bool: { should: [left, right], minimum_should_match: 1 } }
    }
  }

  // Handle negation
  if (node.negated) {
    result = { bool: { must_not: [result] } }
  }

  return result
}

function expressionToES(expr) {
  const field = expr.key.raw

  switch (expr.operator) {
    case Operator.EQUALS:
      return { term: { [field]: expr.value } }

    case Operator.NOT_EQUALS:
      return { bool: { must_not: [{ term: { [field]: expr.value } }] } }

    case Operator.REGEX:
      return { regexp: { [field]: String(expr.value) } }

    case Operator.NOT_REGEX:
      return { bool: { must_not: [{ regexp: { [field]: String(expr.value) } }] } }

    case Operator.GREATER_THAN:
      return { range: { [field]: { gt: expr.value } } }

    case Operator.GREATER_OR_EQUALS_THAN:
      return { range: { [field]: { gte: expr.value } } }

    case Operator.LOWER_THAN:
      return { range: { [field]: { lt: expr.value } } }

    case Operator.LOWER_OR_EQUALS_THAN:
      return { range: { [field]: { lte: expr.value } } }

    case Operator.IN:
      return { terms: { [field]: expr.values || [] } }

    case Operator.NOT_IN:
      return { bool: { must_not: [{ terms: { [field]: expr.values || [] } }] } }

    case Operator.TRUTHY:
      return { exists: { field } }

    case Operator.HAS:
      return { wildcard: { [field]: `*${expr.value}*` } }

    case Operator.NOT_HAS:
      return { bool: { must_not: [{ wildcard: { [field]: `*${expr.value}*` } }] } }

    default:
      throw new Error(`unsupported operator: ${expr.operator}`)
  }
}

// Usage
const result = parse('status = 200 and message ~ "error.*" and env in ["prod", "staging"]')
const esQuery = generateElasticsearch(result.root)
console.log(JSON.stringify(esQuery, null, 2))

Go

package main

import (
  "encoding/json"
  "fmt"
  flyql "github.com/iamtelescope/flyql/golang"
)

func generateElasticsearch(node *flyql.Node) map[string]any {
  if node == nil {
    return map[string]any{"match_all": map[string]any{}}
  }

  var result map[string]any

  if node.Expression != nil {
    // Leaf node — convert the expression
    result = expressionToES(node.Expression)
  } else {
    // Branch node — recurse and combine
    left := generateElasticsearch(node.Left)
    right := generateElasticsearch(node.Right)

    if node.BoolOperator == flyql.BoolOpAnd {
      result = map[string]any{
        "bool": map[string]any{"must": []any{left, right}},
      }
    } else {
      result = map[string]any{
        "bool": map[string]any{"should": []any{left, right}, "minimum_should_match": 1},
      }
    }
  }

  // Handle negation
  if node.Negated {
    result = map[string]any{
      "bool": map[string]any{"must_not": []any{result}},
    }
  }

  return result
}

func expressionToES(expr *flyql.Expression) map[string]any {
  field := expr.Key.Raw

  switch expr.Operator {
  case flyql.OpEquals:
    return map[string]any{"term": map[string]any{field: expr.Value}}
  case flyql.OpNotEquals:
    return map[string]any{"bool": map[string]any{"must_not": []any{map[string]any{"term": map[string]any{field: expr.Value}}}}}
  case flyql.OpGreater:
    return map[string]any{"range": map[string]any{field: map[string]any{"gt": expr.Value}}}
  case flyql.OpLess:
    return map[string]any{"range": map[string]any{field: map[string]any{"lt": expr.Value}}}
  case flyql.OpIn:
    return map[string]any{"terms": map[string]any{field: expr.Values}}
  case flyql.OpNotIn:
    return map[string]any{"bool": map[string]any{"must_not": []any{map[string]any{"terms": map[string]any{field: expr.Values}}}}}
  case flyql.OpTruthy:
    return map[string]any{"exists": map[string]any{"field": field}}
  case flyql.OpRegex:
    return map[string]any{"regexp": map[string]any{field: fmt.Sprintf("%v", expr.Value)}}
  default:
    panic(fmt.Sprintf("unsupported operator: %s", expr.Operator))
  }
}

func main() {
  result, err := flyql.Parse(`status = 200 and env in ["prod", "staging"]`)
  if err != nil {
    panic(err)
  }

  esQuery := generateElasticsearch(result.Root)
  data, _ := json.MarshalIndent(esQuery, "", "  ")
  fmt.Println(string(data))
}

Python

from flyql import parse
from flyql.core.constants import Operator, BoolOperator

def generate_elasticsearch(node):
    if node is None:
        return {"match_all": {}}

    if node.expression is not None:
        # Leaf node — convert the expression
        result = expression_to_es(node.expression)
    else:
        # Branch node — recurse and combine
        left = generate_elasticsearch(node.left)
        right = generate_elasticsearch(node.right)

        if node.bool_operator == BoolOperator.AND.value:
            result = {"bool": {"must": [left, right]}}
        else:
            result = {"bool": {"should": [left, right], "minimum_should_match": 1}}

    # Handle negation
    if node.negated:
        result = {"bool": {"must_not": [result]}}

    return result

def expression_to_es(expr):
    field = expr.key.raw
    op = expr.operator  # operator is stored as a string value, e.g. "=", "!=", "~"

    if op == Operator.EQUALS.value:
        return {"term": {field: expr.value}}
    elif op == Operator.NOT_EQUALS.value:
        return {"bool": {"must_not": [{"term": {field: expr.value}}]}}
    elif op == Operator.REGEX.value:
        return {"regexp": {field: str(expr.value)}}
    elif op == Operator.NOT_REGEX.value:
        return {"bool": {"must_not": [{"regexp": {field: str(expr.value)}}]}}
    elif op == Operator.GREATER_THAN.value:
        return {"range": {field: {"gt": expr.value}}}
    elif op == Operator.LOWER_THAN.value:
        return {"range": {field: {"lt": expr.value}}}
    elif op == Operator.IN.value:
        return {"terms": {field: expr.values or []}}
    elif op == Operator.NOT_IN.value:
        return {"bool": {"must_not": [{"terms": {field: expr.values or []}}]}}
    elif op == Operator.TRUTHY.value:
        return {"exists": {"field": field}}
    elif op == Operator.HAS.value:
        return {"wildcard": {field: f"*{expr.value}*"}}
    elif op == Operator.NOT_HAS.value:
        return {"bool": {"must_not": [{"wildcard": {field: f"*{expr.value}*"}}]}}
    else:
        raise ValueError(f"unsupported operator: {op}")


# Usage
result = parse('status = 200 and message ~ "error.*" and env in ["prod", "staging"]')
es_query = generate_elasticsearch(result.root)
print(es_query)

How Built-In Generators Work

The built-in SQL generators follow the same recursive pattern shown above. Each generator:

1. Receives the root Node and a column schema map.
2. Recurses through the tree: branch nodes combine left OP right, with parentheses added only when SQL precedence requires them (for example, an OR subtree under an AND parent stays wrapped as a AND (b OR c), while same-precedence chains like a AND b AND c render flat). NOT (...) always wraps its operand per SQL convention. Leaf nodes become SQL conditions.
3. Handles each operator by switching on expression.operator and emitting dialect-specific SQL.
4. Wraps negated nodes in NOT (...).

The key difference from a custom generator is that SQL generators also validate expressions against a column schema (checking types, allowed values, and column existence). A custom generator can skip this step or implement its own validation logic.

Exported Types Reference

JavaScript

import {
  parse,               // Parse a query string into an AST
  bindParams,          // Resolve parameter placeholders: bindParams(node, params)
  Parser,              // Parser class (advanced usage)
  Node,                // AST branch/leaf node
  Expression,          // Leaf condition
  Parameter,           // Parameter placeholder ($name / $1)
  FunctionCall,        // Temporal function value (ago, now, today, startOf)
  Key,                 // Field path
  Range,               // Source position range [start, end)
  Column,              // Core column type for validator
  ColumnSchema,        // Column schema container with nested resolution
  Operator,            // Comparison operator constants
  BoolOperator,        // Boolean operator constants ("and", "or")
  FlyqlError,          // Base error class
  ParserError,         // Parse error (extends FlyqlError, includes errno)
  diagnose,            // AST validator: diagnose(ast, schema, registry?) → Diagnostic[]
  Diagnostic,          // Positioned diagnostic record
} from 'flyql'

Go

import flyql "github.com/iamtelescope/flyql/golang"

// flyql.Parse(query string) (*ParseResult, error)
// flyql.BindParams(node *Node, params map[string]any) error — resolve parameter placeholders
// flyql.Node         — AST node struct
// flyql.Expression   — leaf condition struct
// flyql.Parameter    — parameter placeholder struct ($name / $1)
// flyql.FunctionCall — temporal function value struct
// flyql.Key          — field path struct
// flyql.Range        — source position range [Start, End)
// flyql.Column       — core column type for validator
// flyql.ColumnSchema — column schema container with nested resolution
// flyql.FromColumns(cols) *ColumnSchema — build schema from flat column slice
// flyql.Diagnose(ast, schema, registry) []Diagnostic — AST validator
// flyql.Diagnostic   — positioned diagnostic record
//
// Operator constants: flyql.OpEquals, flyql.OpIn, flyql.OpTruthy, etc.
// Bool operator constants: flyql.BoolOpAnd, flyql.BoolOpOr

Python

from flyql import parse, bind_params, Parser, Parameter, FunctionCall
from flyql.core.tree import Node
from flyql.core.expression import Expression
from flyql.core.key import Key
from flyql.core.range import Range
from flyql.core.column import Column, ColumnSchema
from flyql.core.validator import diagnose, Diagnostic
from flyql.core.constants import Operator, BoolOperator
from flyql.core.exceptions import FlyqlError, ParserError