Skip to content

Added Support for Quoted Keys and JobProperties in CREATE TRIGGER#200

Merged
srnand merged 4 commits intomainfrom
sthakkar/mar25_26/enhanceCreateTrigger
Mar 26, 2026
Merged

Added Support for Quoted Keys and JobProperties in CREATE TRIGGER#200
srnand merged 4 commits intomainfrom
sthakkar/mar25_26/enhanceCreateTrigger

Conversation

@srnand
Copy link
Copy Markdown
Collaborator

@srnand srnand commented Mar 25, 2026

Problem Statement

  • Added support for dotted and quoted keys in WITH (...) clause for CREATE TRIGGER.
  • Added logic to extract job.properties.* options from triggers and store them as jobProperties on the TableTrigger K8s spec.
  • Followed codegen/README.md to generate java compiler code.

Example

With this job template applied,

apiVersion: hoptimator.linkedin.com/v1alpha1
kind: JobTemplate
metadata:
  name: retl-job-template
  namespace: default
spec:
  yaml: |
    apiVersion: batch/v1
    kind: Job
    metadata:
      name: {{name}}-job
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: alpine/k8s:1.33.0
            command: ["bash",  "-c", "echo {{name}}-trigger fired at `date`"]
          restartPolicy: Never
      backoffLimit: 4

This query,

  CREATE TRIGGER my_trigger ON ads.ad_clicks
    AS 'retl-job-template' IN 'my-mp'
    WITH (
      'offline.table.name' 'my-namespace.my-table',
      'job.properties.online.table.name' 'MyOnlineTable'
    );

produces a TableTrigger with

apiVersion: hoptimator.linkedin.com/v1alpha1
kind: TableTrigger
metadata:
  creationTimestamp: "2026-03-25T23:09:03Z"
  generation: 1
  labels:
    view: ads-adclicks
  name: mytrigger
  namespace: default
  resourceVersion: "3285"
  uid: e010e73a-9f4a-411b-844a-ed6dd0083e80
spec:
  jobProperties:
    online.table.name: MyOnlineTable
  schema: ADS
  table: AD_CLICKS
  yaml: |
    apiVersion: batch/v1
    kind: Job
    metadata:
      name: mytrigger-retl-job-template-job
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: alpine/k8s:1.33.0
            command: ["bash",  "-c", "echo mytrigger-retl-job-template-trigger fired at `date`"]
          restartPolicy: Never
      backoffLimit: 4

Testing

  • Added integration test.

@srnand srnand requested a review from ryannedolan March 25, 2026 23:12
@srnand srnand force-pushed the sthakkar/mar25_26/enhanceCreateTrigger branch from 4972467 to d473fea Compare March 25, 2026 23:13
Copy link
Copy Markdown
Collaborator

@ryannedolan ryannedolan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

neat!

@srnand srnand merged commit ac4bbb9 into main Mar 26, 2026
1 check passed
@srnand srnand deleted the sthakkar/mar25_26/enhanceCreateTrigger branch March 26, 2026 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants