Babel

Data transformations made easy.

Table of Contents

• Table of Contents
• Installation
• Usage
  • Private Context
  • Error Reporting
  • Telemetry
• Contributing

Installation

Simply add babel to your list of dependencies in your mix.exs:

def deps do
  [
    {:babel, "~> 1.0"}
  ]
end

Differences between the versions are explained in the Changelog.

Documentation gets generated with ExDoc and can be viewed at HexDocs.

Usage

Babel was born out of a desire to simplify non-trivial data transformation pipelines. To focus on the "happy path" instead of having to write a bunch of boilerplate error handling code.

But don't listen to me, take a look for yourself:

pipeline =
  Babel.begin()
  |> Babel.fetch(["some", "nested", "path"])
  |> Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))

data = %{
  "some" => %{
    "nested" => %{
      "path" => [
        %{"string-key" => :value2},
        %{"string-key" => :value2},
        %{"string-key" => :value2}
      ]
    }
  }
}

Babel.apply(pipeline, data)
=> {:ok, [
   %{atom_key: :value1},
   %{atom_key: :value2},
   %{atom_key: :value3}
]}

Private Context

The Babel.Context includes a private field for passing metadata through your pipeline without affecting the transformed data. This is useful for session IDs, authentication tokens, request metadata, or any context that steps need to share.

Steps can update the private context by returning a three-tuple {:ok, data, private}:

# Step 1: Set session info in private context
authenticate = Babel.then(fn credentials ->
  # Validate credentials...
  session = %{user_id: 123, session_id: "abc-xyz"}
  {:ok, credentials, session_id: session.session_id, user_id: session.user_id}
end)

# Step 2: Access private context in subsequent step
check_permissions = Babel.then(fn data ->
  # Custom step implementation can access the full context
  # using Babel.Test.ContextStep pattern or custom Step behavior
  {:ok, data}
end)

pipeline = Babel.chain(authenticate, check_permissions)

The private context accepts either a map or keyword list:

# Using map
{:ok, data, %{session_id: "abc", user_id: 123}}

# Using keyword list (converted to map internally)
{:ok, data, session_id: "abc", user_id: 123}

Private values from multiple steps are merged together, with later values overwriting earlier ones for the same keys. The private context is passed to each step via Babel.Context, but is not included in the final Babel.apply/2 result — it remains internal pipeline state.

Error Reporting

Since you'll most likely build non-trivial transformation pipelines with Babel - which can fail at any given step - Babel ships with elaborate error reporting:

pipeline =
  Babel.begin()
  |> Babel.fetch(["some", "nested", "path"])
  |> Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))

data = %{
  "some" => %{
    "nested" => %{
      "path" => [
        %{"unexpected-key" => :value1},
        %{"unexpected-key" => :value2},
        %{"unexpected-key" => :value3}
      ]
    }
  }
}

Babel.apply!(pipeline, data)

Which will produce the following error:

** (Babel.Error) Failed to transform data: [not_found: "string-key", not_found: "string-key", not_found: "string-key"]

Root Cause(s):
1. Babel.Trace<ERROR>{
  data = %{"unexpected-key" => :value1}

  Babel.fetch("string-key")
  |=> {:error, {:not_found, "string-key"}}
}
2. Babel.Trace<ERROR>{
  data = %{"unexpected-key" => :value2}

  Babel.fetch("string-key")
  |=> {:error, {:not_found, "string-key"}}
}
3. Babel.Trace<ERROR>{
  data = %{"unexpected-key" => :value3}

  Babel.fetch("string-key")
  |=> {:error, {:not_found, "string-key"}}
}

Full Trace:
Babel.Trace<ERROR>{
  data = %{"some" => %{"nested" => %{"path" => [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]}}}

  Babel.Pipeline<>
  |
  | Babel.fetch(["some", "nested", "path"])
  | |=< %{"some" => %{"nested" => %{"path" => [%{"unexpected-key" => :value1}, %{...}, ...]}}}
  | |=> [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]
  |
  | Babel.map(Babel.into(%{atom_key: Babel.fetch("string-key")}))
  | |=< [%{"unexpected-key" => :value1}, %{"unexpected-key" => :value2}, %{"unexpected-key" => :value3}]
  | |
  | | Babel.into(%{atom_key: Babel.fetch("string-key")})
  | | |=< %{"unexpected-key" => :value1}
  | | |
  | | | Babel.fetch("string-key")
  | | | |=< %{"unexpected-key" => :value1}
  | | | |=> {:error, {:not_found, "string-key"}}
  | | |
  | | |=> {:error, [not_found: "string-key"]}
  | |
  | | Babel.into(%{atom_key: Babel.fetch("string-key")})
  | | |=< %{"unexpected-key" => :value2}
  | | |
  | | | Babel.fetch("string-key")
  | | | |=< %{"unexpected-key" => :value2}
  | | | |=> {:error, {:not_found, "string-key"}}
  | | |
  | | |=> {:error, [not_found: "string-key"]}
  | |
  | | Babel.into(%{atom_key: Babel.fetch("string-key")})
  | | |=< %{"unexpected-key" => :value3}
  | | |
  | | | Babel.fetch("string-key")
  | | | |=< %{"unexpected-key" => :value3}
  | | | |=> {:error, {:not_found, "string-key"}}
  | | |
  | | |=> {:error, [not_found: "string-key"]}
  | |
  | |=> {:error, [not_found: "string-key", not_found: "string-key", not_found: "string-key"]}
  |
  |=> {:error, [not_found: "string-key", not_found: "string-key", not_found: "string-key"]}
}

Babel achieves this by keeping track of all applied steps in a Babel.Trace struct. Rendering of a Babel.Trace is done through a custom Inspect implementation.

You have to this information everywhere: in the Babel.Error message, in iex, and whenever you inspect a Babel.Error or Babel.Trace.

Telemetry

Babel integrates with :telemetry to emit span events for every step and pipeline execution. :telemetry is an optional dependency — when it's not installed, the telemetry calls are no-ops and Babel remains dependency-free.

To enable telemetry, add :telemetry to your dependencies:

def deps do
  [
    {:babel, "~> 1.0"},
    {:telemetry, "~> 0.4 or ~> 1.0"}
  ]
end

Events

Babel emits the following telemetry:span/3 events:

Event	Description
[:babel, :step, :start]	Emitted when a step begins execution
[:babel, :step, :stop]	Emitted when a step completes
[:babel, :step, :exception]	Emitted when a step raises an unrescued exception
[:babel, :pipeline, :start]	Emitted when a pipeline begins execution
[:babel, :pipeline, :stop]	Emitted when a pipeline completes
[:babel, :pipeline, :exception]	Emitted when a pipeline raises an unrescued exception

Metadata

Start event metadata:

Key	Value
babel	The step or pipeline struct being executed
input	The Babel.Context passed as input

Stop event metadata:

Key	Value
babel	The step or pipeline struct that was executed
input	The Babel.Context that was passed as input
trace	The resulting Babel.Trace
result	:ok or :error

Example

:telemetry.attach(
  "babel-logger",
  [:babel, :pipeline, :stop],
  fn _event, %{duration: duration}, %{babel: babel, result: result}, _config ->
    duration_ms = System.convert_time_unit(duration, :native, :millisecond)
    IO.puts("[Babel] #{inspect(babel)} completed in #{duration_ms}ms (#{result})")
  end,
  nil
)

Contributing

Contributions are always welcome but please read our contribution guidelines before doing so.