Skip to content

TECHNICAL_MANUAL.md

Latest commit

 

History

History
242 lines (161 loc) · 12.1 KB

TECHNICAL_MANUAL.md

File metadata and controls

242 lines (161 loc) · 12.1 KB

TECHNICAL MANUAL: Sensor Systems and Command Protocols

Field Awareness and Command Engine (FACE) Technical Documentation

Welcome, Cognitive!

This document provides technical specifications for your onboard sensor systems and command protocols. Understanding these systems is critical for effective operation in the field. Your FACE processes environmental data and translates your decisions into executable commands.

This report covers the technical details of how you perceive the world and how you issue commands. In different environments, specific details may vary.

SENSOR SYSTEMS OVERVIEW

Your FACE uses a token-based observation system to process environmental data. Rather than processing raw visual feeds, your sensors emit discrete data tokens representing specific features at specific locations within your observation window. This design allows for efficient processing while maintaining critical spatial awareness.

Technical Note: These observation tokens are distinct from tokens used in transformer architectures. When ambiguity might arise, we refer to them as Observation Tokens.

Observation Token Structure

Your sensor array provides observations as structured data arrays with dimensions (num_cogs, num_tokens, 3):

  • Dimension 0: Cog index (your position in the team)
  • Dimension 1: Token index (variable length, padded with empty tokens)
  • Dimension 2: Token components [location, feature_id, value]

Each token encodes a single feature value at a specific location within your observation window.

Empty Tokens

Your sensors use a special marker value 0xFF (255) to indicate empty or invalid tokens. When location == 0xFF, the token should be ignored. Empty tokens are used to pad observation arrays to a fixed size for efficient batch processing.

Global Tokens

Global tokens use a dedicated location marker 0xFE (254) to indicate non-spatial, agent-wide state. These include:

  • episode_completion_pct: How far through the episode you are (0-255)
  • last_action: The action you took last step
  • last_reward: The reward you received last step
  • Local position observations (lp:north, lp:south, lp:east, lp:west)

Global tokens can be detected by position alone: location == 0xFE means it's a global token.

Coordinate Encoding

Your observation window uses a packed coordinate system to efficiently encode spatial information:

  • Upper 4 bits (high nibble): Row coordinate (0-14)
  • Lower 4 bits (low nibble): Column coordinate (0-14)
  • Special value 0xFF: Empty/invalid coordinate
  • Special value 0xFE: Global token (non-spatial observation)

Observation windows are typically 13x13 centered on your position; you are located at 0x66 (row 6, column 6). Coordinates are egocentric (relative to your position), not absolute map coordinates.

Coordinate System

  • Row (r/y): Vertical coordinate, increases downward
  • Column (c/x): Horizontal coordinate, increases rightward
  • Center location: Used for cog-specific features (inventory)

Observation Features

Your sensors can detect various features in the environment. Feature IDs are assigned sequentially starting from 0. The exact feature set depends on your mission configuration (available resources, protocol details, etc.).

Critical: Feature IDs may differ between mission configurations. Ensure your cognitive models are configured for the specific mission parameters, or implement dynamic feature mapping using the IdMap system.

Getting Feature Information

You can query your FACE for available observation features using the IdMap system:

from mettagrid.config.mettagrid_config import MettaGridConfig

# Access your mission configuration
config = MettaGridConfig(...)

# Get the IdMap for this configuration
id_map = config.game.id_map()

# Get all observation features
features = id_map.features()

# Each feature provides:
# - id: int - The feature ID used in observation tokens
# - name: str - Human-readable feature name (e.g., "inv:oxygen", "agent:group")
# - normalization: float - Normalization factor for this feature

Feature Set Structure

The following features may be available in your sensor data. Note that specific feature IDs depend on your mission configuration (number of resources, whether protocol details are enabled, etc.), so always use IdMap to get the exact feature IDs for your configuration.

Feature Name Description Objects with this Feature Notes
agent:group Cog's group/team identifier cogs Used for team identification in CvC
episode_completion_pct Portion of the episode completed, from 0 (start) to 255 (end). Not a percentage self (global)
last_action Last action taken by the cog self (global)
last_reward Last reward received by the cog self (global)
goal Indicates rewarding resources cogs
vibe Cog's current vibe any object Values depend on mission config
tag Tags associated with an object (e.g., "wall", "oxygen_extractor", "blue") any object Values can be found in IdMap.tag_names(). Multiple tags emit multiple tokens.
lp:east/west/north/south Directional offset from spawn position self (global)
agent_id Unique agent identifier self (global)
inv:{resource_name} Amount of resource in the object cogs, chests One feature per resource (e.g., inv:oxygen, inv:carbon, inv:heart)
protocol_input:{resource_name} Required input resource amount for current protocol hub, extractors One feature per resource. Only if protocol_details_obs is enabled
protocol_output:{resource_name} Output resource amount for current protocol hub, extractors One feature per resource. Only if protocol_details_obs is enabled
territory:here, territory:* Territory label plus sparse boundary edges self (global) + tiles Optional; enabled via obs.territory
last_action_move Whether the agent's location changed on the last step self (global) Optional; enabled via obs.global_obs.last_action_move

COMMAND PROTOCOLS

Your FACE translates your decisions into executable commands using a discrete action space. Each command is represented as a single integer index (action ID) that corresponds to a fully qualified action variant such as move_north, noop, or change_vibe_happy.

Verb/argument combinations are flattened during environment initialization, so you only need to emit a scalar action_id per cog.

Action Format

Commands are provided as structured data arrays with shape (num_cogs,):

  • Type: int32 (or np.int32)
  • Range: 0 <= action_id < num_actions
  • Per cog: Each cog emits a single action ID

Action ID Assignment

Action IDs are assigned sequentially starting from 0 based on the order in which actions are registered during environment initialization. The exact action set depends on your mission configuration (enabled actions, allowed directions, etc.).

Action IDs may differ between mission configurations. Ensure your cognitive models are configured for the specific mission parameters, or implement dynamic action mapping using action names.

Available Command Types

The following command types are available in MettaGrid. The exact set depends on your mission configuration.

Noop Command

  • Name: noop
  • Description: Do nothing. Always available and typically has action ID 0.
  • Resource requirements: None

Move Commands

  • Name pattern: move_{direction}
  • Directions: north, south, east, west (and optionally northeast, northwest, southeast, southwest if diagonals are enabled). Moving into an object triggers interaction with that object.
  • Description: Move one cell in the specified direction
  • Resource requirements: Consumes energy (configurable)

Example command names:

  • move_north
  • move_south
  • move_east
  • move_west

Change Vibe Commands

  • Name pattern: change_vibe_{vibe_name}
  • Vibes: Depends on configuration (e.g., happy, sad, angry, etc.)
  • Description: Change your current vibe to the specified vibe. Vibes can be used for communication, and also impacts environmental interactions.

Example command names:

  • change_vibe_happy
  • change_vibe_sad
  • change_vibe_neutral

Note: The available vibes are configurable via change_vibe.vibes.

Talk Variant

CoGames exposes talk-and-walk as the talk mission variant:

  • -v vibes: communication uses change_vibe_*
  • -v talk: communication uses short speech sidecars and disables change_vibe_*

talk is the speech-bubble variant of vibes, so it preserves vibe identity while swapping the explicit change_vibe_* action surface out for short nearby speech.

For SDK and higher-level cogamer-style policies, the canonical communication contract is:

  • send communication with {"talk": "..."} or MacroDirective(talk="...")
  • read teammate speech from visible agent entities with label="talking"
  • inspect entity.attributes["talk_text"] and entity.attributes["talk_remaining_steps"]

When talk is active, MettaScope renders the active message as a speech bubble above the cog and nearby teammates see that same text directly on nearby talking agent observations.

COMMAND EXECUTION FLOW

When a command is executed, the following validation occurs:

  1. Action Index Validation: 0 <= action_id < num_actions
  2. Action Space Validation: Action ID must be within env.action_space.n
  3. Resource Validation: Cog must have required resources (if any)
  4. Action Execution: Attempt the action

Invalid Commands

If a command is invalid (out of range, insufficient resources, etc.), the command is silently ignored and you effectively perform a noop.

FINAL NOTES

Understanding your sensor systems and command protocols is essential for effective field operations.

Your success depends on:

  • Proper interpretation of observation tokens
  • Efficient command selection and execution
  • Understanding the relationship between feature IDs and action IDs
  • Adapting to different mission configurations

Stay aware. Stay coordinated. Stay operational.

END TRANSMISSION