Connector Ecosystem Roadmap: tracking 120+ connector targets across databases, cloud, IoT, and observability
Overview
Apache Iggy's connector ecosystem is growing. This issue tracks the full landscape of connectors -- what's done, what's in progress, what's planned, and what's proposed next -- to help contributors find high-impact work and avoid duplication.
Current state: 5 native Rust connectors (sinks + sources) + 2 Java connectors + 2 codecs in progress.
Competitive context: Kafka Connect has 200+ connectors, Redpanda Connect has 200+ (via Benthos), Pulsar IO has 40+ built-in. Iggy's Rust plugin model (cdylib + C FFI) offers performance and resource advantages but needs breadth to compete.
Status: Done
| Connector |
Type |
Issue |
Notes |
| PostgreSQL |
Sink + Source (CDC) |
#1849 |
Native Rust. WAL-based CDC for source. ~2,400 LOC |
| Elasticsearch |
Sink + Source |
#1851 |
Native Rust. Bulk API sink, scroll/search source. ~1,500 LOC |
| Apache Iceberg |
Sink |
#1850 |
Native Rust. 7k GitHub stars, used by Airbnb, Apple, LinkedIn |
| Quickwit |
Sink |
-- |
Native Rust. Cloud-native search engine. ~260 LOC |
| Stdout |
Sink |
-- |
Reference implementation for connector developers. ~155 LOC |
| Apache Pinot |
Sink |
-- |
Java connector via Kafka compatibility layer |
Connector runtime infrastructure (shipped):
Status: In Progress (Open Issues)
E2E tests in progress: Pinot (#2598), Elasticsearch source (#2593)
Runtime bug: Connector loading loop on Linux (#2712, @hubcio)
Status: Proposed -- Comprehensive Connector Target List
Organized by category. For each target:
- Priority: P0 (critical) / P1 (high) / P2 (medium) / P3 (low) / P4 (niche)
- Comp: How many of {Kafka, Pulsar, Redpanda, NATS} have this connector
- Rust Crate: The crate a contributor would use to build this connector
1. Relational Databases
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| MySQL / MariaDB |
Sink+Source |
P1 |
4/4 |
Most popular RDBMS globally. Covered by planned JDBC connector #2500. Every web application's default database. |
sqlx (54.9M downloads) or mysql_async |
| Microsoft SQL Server |
Sink+Source |
P2 |
3/4 |
Enterprise RDBMS. Required for Fortune 500 adoption. Debezium CDC in Kafka ecosystem. |
tiberius (native TDS protocol) |
| Oracle |
Sink+Source |
P3 |
3/4 |
Complex licensing (GoldenGate CDC). Enterprise-heavy. Required for legacy enterprise migration but licensing costs limit adoption. |
No mature Rust crate; JDBC bridge or C OCI FFI |
| SQLite |
Sink |
P4 |
2/4 |
Embedded database. Niche use case for local/edge analytics sinks. |
rusqlite |
| CockroachDB |
Sink+Source |
-- |
2/4 |
PostgreSQL wire protocol compatible. Works via existing Postgres connector automatically. |
Existing PG connector |
| IBM Db2 |
Sink+Source |
P4 |
1/4 |
Enterprise/mainframe only. Very niche. |
JDBC bridge |
| Generic JDBC |
Sink+Source |
P1 |
3/4 |
Planned as #2500. Covers MySQL, MariaDB, and dozens of other databases through a single connector. |
sqlx |
2. NoSQL / Document Databases
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| MongoDB |
Sink+Source |
P0 |
4/4 |
Planned #2739. 28k GitHub stars, 50k customers. Most popular NoSQL DB. Change Streams enable real-time CDC without WAL complexity. Essential for IoT device metadata and telemetry storage. |
mongodb (official, 9.5M downloads) |
| Cassandra |
Sink |
P2 |
3/4 |
9.6k stars. Wide-column store optimized for high-write workloads. Bloomberg serves 20B+ requests/day on 1,700+ nodes. Used by Netflix, Apple. Essential for time-series and IoT at massive scale. |
scylla (Cassandra-compatible, pure Rust) |
| DynamoDB |
Sink+Source |
P2 |
3/4 |
AWS-native NoSQL. Handles 10T+ requests/day. DynamoDB Streams for CDC source. Required for AWS-centric architectures. Used by Lyft, Airbnb, Samsung. |
aws-sdk-dynamodb (official AWS SDK) |
| Couchbase |
Sink+Source |
P3 |
2/4 |
Multi-model NoSQL (document + KV + search). Used in mobile/edge sync. Less common than MongoDB. |
No mature Rust crate; REST API via reqwest |
| HBase |
Sink |
P3 |
2/4 |
Hadoop ecosystem wide-column store. Declining as Cassandra and cloud-native alternatives grow. |
No Rust crate; Thrift/REST API |
| Azure Cosmos DB |
Sink+Source |
P3 |
1/4 |
Azure-native multi-model database. MongoDB API compatible, so MongoDB connector partially covers it. |
azure_data_cosmos |
| ScyllaDB |
Sink |
P3 |
1/4 |
Cassandra-compatible but written in C++ for 10x performance. Growing adoption for low-latency workloads. Covered by Cassandra connector. |
scylla (official Rust driver) |
| Apache CouchDB |
Sink+Source |
P4 |
1/4 |
Document DB with built-in replication. Niche. |
reqwest (CouchDB REST API) |
3. Search & Analytics Engines
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| OpenSearch |
Sink |
P1 |
2/4 |
12.4k GitHub stars. Elasticsearch fork, AWS-aligned. Linux Foundation project. May work as thin adapter over existing ES connector with compatibility layer. |
opensearch (official Rust client) |
| Apache Solr |
Sink |
P3 |
2/4 |
Legacy search engine. Declining as Elasticsearch/OpenSearch dominate. Only for existing Solr shops. |
reqwest (Solr REST API) |
| Splunk |
Sink |
P2 |
3/4 |
15k customers. 89 of Fortune 100. Acquired by Cisco for $28B. HTTP Event Collector (HEC) API makes integration straightforward. Enterprise observability standard. |
reqwest (HEC is HTTP POST) |
| Meilisearch |
Sink |
P4 |
0/4 |
48k GitHub stars. Lightweight search engine with great DX. Growing alternative to Elasticsearch for smaller deployments. |
meilisearch-sdk |
| Typesense |
Sink |
P4 |
0/4 |
22k GitHub stars. Another lightweight search alternative. REST API. |
reqwest (Typesense REST API) |
4. Cloud Object Storage
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| Amazon S3 |
Sink+Source |
P0 |
4/4 |
Universal data lake landing zone. 350T+ objects stored, 100M+ req/sec. 90%+ Fortune 100 via AWS. Enables Snowflake/Redshift S3 staging. Build with abstract storage interface so GCS/Azure/MinIO become thin adapters. |
aws-sdk-s3 (official AWS SDK, v1.123) |
| Google Cloud Storage |
Sink+Source |
P1 |
3/4 |
GCP object storage. Thin adapter over S3 connector architecture. GCP holds ~13% cloud market. |
cloud-storage or google-cloud-storage |
| Azure Blob Storage |
Sink+Source |
P1 |
3/4 |
Azure object storage. Thin adapter over S3 connector architecture. Azure holds ~20% cloud market. |
azure_storage_blobs |
| MinIO |
Sink+Source |
P1 |
1/4 |
S3-compatible, self-hosted. 50k+ GitHub stars. Works via S3 connector automatically since it implements S3 API. |
aws-sdk-s3 (S3-compatible) |
| DigitalOcean Spaces |
Sink |
P4 |
0/4 |
S3-compatible. Covered by S3 connector. |
aws-sdk-s3 (S3-compatible) |
| Backblaze B2 |
Sink |
P4 |
0/4 |
S3-compatible. Covered by S3 connector. |
aws-sdk-s3 (S3-compatible) |
5. Data Lakehouse Formats
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| Delta Lake |
Sink+Source |
P2 |
2/4 |
Planned #1852. 8.6k GitHub stars. Databricks/Linux Foundation. ACID transactions on data lakes. Growing fast with Databricks adoption. |
deltalake (Rust-native, delta-rs project) |
| Apache Hudi |
Sink |
P3 |
1/4 |
Uber's lakehouse format. 5.5k stars. Strong in incremental processing and CDC use cases. Less popular than Iceberg/Delta. |
hudi (Rust-native, apache/hudi-rs) |
| Apache Paimon |
Sink |
P4 |
0/4 |
Flink-native lakehouse format. Emerging. Formerly Flink Table Store. Optimized for streaming writes. |
No Rust crate; Java only |
6. Data Warehouses
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| Snowflake |
Sink |
P1 |
3/4 |
Leading cloud DW. 12k+ customers, $1.09B product revenue (Q2 FY26). Snowpipe Streaming API for real-time ingestion. 3/4 competitors have it. |
reqwest (Snowpipe REST API); or S3 staging |
| ClickHouse |
Sink |
P1 |
2/4 |
Planned #2539. 45k GitHub stars. Real-time OLAP. $400M Series D (Jan 2026). Native HTTP interface. Popular in Rust community. |
clickhouse; klickhouse (native protocol) |
| Amazon Redshift |
Sink |
P2 |
2/4 |
Planned #2540. AWS cloud DW. S3 staging + COPY pattern for efficient bulk loading. |
aws-sdk-s3 + sqlx (PG wire protocol) |
| Google BigQuery |
Sink |
P2 |
3/4 |
GCP flagship analytics warehouse. 13.6% DW market share. 13k+ companies. Storage Write API for streaming. |
gcp-bigquery-client |
| Databricks |
Sink |
P2 |
2/4 |
Via Delta Lake connector or SQL API. Databricks SQL Connector available. Spark ecosystem integration. |
deltalake crate or REST API |
| Azure Synapse |
Sink |
P3 |
1/4 |
Microsoft cloud DW. Competes with Snowflake on Azure. |
tiberius (TDS/SQL Server protocol) |
| StarRocks |
Sink |
P3 |
1/4 |
Real-time OLAP. Fast-growing fork of Apache Doris. Strong benchmark performance for complex JOINs. |
reqwest (Stream Load HTTP API) |
| Apache Doris |
Sink |
P3 |
1/4 |
StarRocks parent project. 13k+ GitHub stars. Strong in China/Asia-Pacific. |
reqwest (Stream Load HTTP API) |
| Firebolt |
Sink |
P4 |
0/4 |
Niche cloud DW for extreme query speed. Startup. |
REST API via reqwest |
| DuckDB |
Sink |
P3 |
0/4 |
28k+ GitHub stars. Embedded OLAP. Huge Rust/data community adoption. "SQLite for analytics." |
duckdb (official Rust bindings) |
7. Time-Series Databases
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| InfluxDB |
Sink+Source |
P0 |
2/4 |
Planned #2700. 31k GitHub stars. #1 time-series DB by community. IoT standard. Line protocol write API is universal. InfluxDB 3 is built in Rust. |
reqwest (HTTP API; no mature Rust client) |
| TimescaleDB |
Sink+Source |
-- |
2/4 |
PostgreSQL extension. Works via existing Postgres connector automatically. No additional work needed. |
Existing PG connector |
| Prometheus |
Sink |
P2 |
1/4 |
63k GitHub stars. CNCF graduated. Industry-standard monitoring. remote_write API (protobuf/snappy over HTTP) for pushing metrics from Iggy. |
prost + reqwest |
| QuestDB |
Sink |
P3 |
1/4 |
15k GitHub stars. Fast time-series DB. InfluxDB Line Protocol compatible, so InfluxDB connector could cover it. |
InfluxDB Line Protocol or PG wire protocol |
| TDengine |
Sink |
P3 |
0/4 |
IoT-focused TSDB. Popular in Asia-Pacific. REST API available. |
reqwest (REST API) |
| VictoriaMetrics |
Sink |
P3 |
0/4 |
13k+ GitHub stars. Prometheus-compatible. Covered by Prometheus remote_write connector. |
Same as Prometheus (remote_write) |
| CrateDB |
Sink |
P4 |
0/4 |
SQL time-series DB. PostgreSQL wire protocol compatible. Works via PG connector. |
Existing PG connector or sqlx |
8. Graph Databases
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| Neo4j |
Sink |
P3 |
3/4 |
$200M+ ARR. 84% Fortune 100. 44% graph DB market. Stream events into graph for relationship analysis, fraud detection, recommendation engines. |
neo4rs (async Bolt protocol client) |
| ArangoDB |
Sink |
P4 |
0/4 |
Multi-model (graph + document + KV). 14k GitHub stars. Niche but versatile. |
arangors |
| Amazon Neptune |
Sink |
P4 |
0/4 |
AWS-managed graph DB. Gremlin and SPARQL APIs. |
reqwest (Neptune REST API) |
| JanusGraph |
Sink |
P4 |
0/4 |
Open-source, Hadoop-backed graph DB. Declining adoption. |
Gremlin protocol via reqwest |
9. Message Queues & Streaming Bridges
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| MQTT |
Source+Sink |
P0 |
3/4 |
THE IoT protocol. De facto standard with 50%+ enterprise IoT adoption. Billions of connected devices. Iggy's "runs on Raspberry Pi" story is incomplete without MQTT. The single most important missing connector. |
rumqttc (237K downloads/month, async, MQTT v5) |
| RabbitMQ / AMQP |
Source+Sink |
P1 |
3/4 |
13k GitHub stars. Most popular traditional message broker. Migration path from legacy MQ to Iggy. AMQP 0.9.1 protocol. |
lapin (async AMQP 0.9.1) |
| Apache Kafka |
Source+Sink |
P2 |
n/a |
32k GitHub stars. 80%+ Fortune 100. Bridge connector for gradual Kafka-to-Iggy migration. Enables hybrid architectures during migration. |
rdkafka (18.6M sys downloads) |
| Amazon Kinesis |
Source+Sink |
P2 |
3/4 |
AWS streaming service. Cross-cloud bridge for AWS-centric organizations moving to Iggy. |
aws-sdk-kinesis (official AWS SDK) |
| Amazon SQS |
Source+Sink |
P3 |
2/4 |
AWS queue service. Simple point-to-point messaging. Bridge for SQS-to-Iggy migration. |
aws-sdk-sqs |
| Google Pub/Sub |
Source+Sink |
P2 |
2/4 |
GCP streaming/messaging service. Cross-cloud bridge for GCP organizations. |
google-cloud-pubsub |
| Apache Pulsar |
Source+Sink |
P3 |
1/4 |
Alternative streaming platform. Bridge connector for Pulsar-to-Iggy migration. |
pulsar (async Rust client) |
| NATS / JetStream |
Source+Sink |
P3 |
1/4 |
Lightweight messaging. Growing in cloud-native/K8s. Bridge for NATS-to-Iggy migration. |
async-nats (official async client) |
| NSQ |
Source+Sink |
P4 |
2/4 |
Simple distributed queue by Bitly. Declining. |
No mature Rust crate |
| IBM MQ |
Source+Sink |
P4 |
1/4 |
Enterprise mainframe messaging. Only for large financial institutions with IBM infrastructure. |
No native Rust crate; JMS bridge |
| Azure Service Bus |
Source+Sink |
P3 |
1/4 |
Azure messaging service. AMQP 1.0 protocol. |
azure_messaging_servicebus; or fe2o3-amqp |
| Azure Event Hubs |
Source |
P3 |
1/4 |
Azure Kafka-compatible event streaming. Since it supports Kafka wire protocol, the Kafka bridge connector covers it. |
rdkafka (Kafka-compatible protocol) |
10. Caches & Key-Value Stores
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| Redis |
Sink+Source |
P0 |
3/4 |
67k+ GitHub stars. Most popular KV store. Cache warming from streams, real-time state management, pub/sub bridge. 4B+ Docker pulls. 10k enterprise customers. |
redis (official); fred (high-level async) |
| DragonflyDB |
Sink+Source |
P3 |
0/4 |
29.4k GitHub stars. Redis-compatible with 25x throughput. Covered by Redis connector automatically. |
redis crate (Redis-compatible API) |
| Memcached |
Sink |
P4 |
1/4 |
Legacy cache. Declining in favor of Redis. Still used by Facebook, Wikipedia. |
memcache |
| Aerospike |
Sink |
P4 |
2/4 |
High-performance KV for adtech, fraud detection. $1B+ valuation. Niche. |
aerospike |
| Hazelcast |
Source |
P4 |
2/4 |
In-memory data grid. Java-centric. Low priority. |
REST API via reqwest |
| etcd |
Sink+Source |
P4 |
0/4 |
48k+ GitHub stars. CNCF graduated. Kubernetes config store. Streaming config changes for K8s auditing. |
etcd-client (async gRPC) |
11. File Systems & Formats
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| Local Filesystem |
Sink+Source |
P3 |
2/4 |
File tail source (like Filebeat) and file write sink. Basic but essential for log ingestion and data export. |
tokio::fs; notify (file watcher) |
| FTP / SFTP |
Sink+Source |
P3 |
3/4 |
Legacy file transfer. Still required in regulated industries (healthcare, banking). |
suppaftp; russh-sftp |
| HDFS |
Sink |
P3 |
3/4 |
Declining but still in legacy Hadoop shops. Being replaced by S3/GCS. |
hdfs-native |
| Parquet (format) |
Output fmt |
P1 |
3/4 |
De facto columnar storage format. Essential for S3/GCS/HDFS sinks. Without Parquet, streaming data to object storage is inefficient. |
parquet (Apache Arrow, very mature) |
| Avro (codec) |
Codec |
P0 |
4/4 |
Planned #1846. Kafka ecosystem standard. Without Avro, interop with Kafka-migrating users is blocked. Enables schema evolution. |
apache-avro (official) |
| BSON (codec) |
Codec |
P2 |
0/4 |
Planned #1847. MongoDB native format. Enables zero-transformation pipelines between Iggy and MongoDB. |
bson (official, 14.2M downloads) |
| ORC (format) |
Output fmt |
P4 |
1/4 |
Hadoop-specific columnar format. Declining as Parquet dominates. |
No mature Rust crate |
| CSV (format) |
Input/Output |
P3 |
2/4 |
Universal data exchange. Human-readable. Batch imports/exports. |
csv (BurntSushi's, very popular) |
| MessagePack (codec) |
Codec |
P4 |
1/4 |
Compact binary. More efficient than JSON. Useful for bandwidth-constrained IoT. |
rmp-serde |
12. HTTP & API Protocols
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| HTTP / Webhook |
Source+Sink |
P0 |
4/4 |
Universal integration. HTTP source receives webhooks from any service; HTTP sink pushes to any API. The "connect to anything" fallback. Every web service speaks HTTP. |
reqwest (client); axum (server) |
| gRPC |
Source+Sink |
P1 |
1/4 |
44k GitHub stars. High-performance RPC for microservices. Pairs with OTLP (which uses gRPC). Essential for microservice architectures. |
tonic (gRPC); prost (protobuf) |
| GraphQL |
Sink |
P4 |
0/4 |
Niche for streaming. GraphQL subscriptions could make Iggy a data source for GraphQL APIs. Specialized use case. |
graphql_client |
| WebSocket |
Source+Sink |
P3 |
1/4 |
Real-time bidirectional. Ingest trading data, chat, live updates. Push events to browser dashboards. |
tokio-tungstenite |
13. Observability & Monitoring
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| OpenTelemetry (OTLP) |
Source+Sink |
P0 |
1/4 |
2nd largest CNCF project. 10k contributors. 1,200 companies. OTLP becoming universal observability standard. Makes Iggy an observability pipeline backbone for logs, traces, and metrics. |
opentelemetry; opentelemetry-otlp |
| Splunk (HEC) |
Sink |
P2 |
3/4 |
46.8% SIEM market. 18k+ companies. Cisco-owned ($28B acquisition). HTTP Event Collector is simple HTTP POST. |
reqwest |
| Datadog |
Sink |
P3 |
2/4 |
$827M Q2 2025 revenue. 47k+ customers. SaaS observability leader. HTTP API for metrics, logs, traces. |
reqwest |
| Prometheus remote_write |
Sink |
P2 |
1/4 |
63k GitHub stars. De facto monitoring standard. One connector covers the entire Prometheus ecosystem (VictoriaMetrics, Cortex, Thanos, Mimir). |
prost + reqwest (protobuf/snappy over HTTP) |
| Grafana Loki |
Sink |
P2 |
1/4 |
28k GitHub stars. 100k+ active clusters. 1B+ Docker pulls. Log aggregation that pairs with Grafana. HTTP push API. |
reqwest |
| Jaeger |
Sink |
P3 |
0/4 |
22k+ GitHub stars. CNCF graduated. Being replaced by OTLP as ingestion standard, so OTLP connector covers it. |
opentelemetry-otlp (Jaeger now prefers OTLP) |
| New Relic |
Sink |
P4 |
1/4 |
552k+ companies. Supports OTLP, so the OTLP connector covers it. |
reqwest or OTLP connector |
14. AI/ML & Vector Databases
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| Qdrant |
Sink |
P3 |
0/4 |
27k+ GitHub stars. Rust-native vector DB. Natural ecosystem fit for Iggy. Stream embeddings for real-time semantic search, RAG, recommendations. |
qdrant-client (official gRPC client) |
| Weaviate |
Sink |
P3 |
0/4 |
11k GitHub stars. 1M+ pulls/month. Best-in-class hybrid search. REST API. |
reqwest (REST API) |
| Pinecone |
Sink |
P3 |
0/4 |
Leading SaaS vector DB. Simplest integration (REST API). Used by Gong, Vanguard. |
pinecone-sdk or reqwest |
| Milvus |
Sink |
P3 |
0/4 |
40k+ GitHub stars. Most popular OSS vector DB. Used by NVIDIA, Salesforce, eBay. 72% memory reduction in v2.6. |
milvus-sdk; or gRPC via tonic |
| MLflow |
Sink |
P4 |
0/4 |
23k GitHub stars. 30M+ monthly downloads. Most popular MLOps platform. Stream training metrics. |
reqwest (REST API) |
| Feast |
Source |
P4 |
0/4 |
5k+ GitHub stars. Feature store. Linux Foundation. ML feature serving from Iggy. |
reqwest (REST API) |
15. SaaS & Enterprise
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| Salesforce |
Source+Sink |
P4 |
1/4 |
#1 CRM. $34.9B revenue. 150k+ customers. Stream CRM events. Enterprise-heavy. |
reqwest (REST/Bulk API) |
| ServiceNow |
Source+Sink |
P4 |
1/4 |
Leading ITSM. $10B+ revenue. 85% Fortune 500. Stream incident events. |
reqwest (REST API) |
| Slack |
Sink |
P4 |
0/4 |
65M+ daily active users. Alert notifications from Iggy streams to channels. Simple webhook. |
reqwest (Webhook HTTP POST) |
| Email (SMTP) |
Sink |
P4 |
1/4 |
Universal alert notifications. Stream events matching conditions as email alerts. |
lettre |
| Stripe |
Source |
P4 |
0/4 |
Leading payment processor. $1T+ processed. Ingest payment events for real-time financial analytics. |
reqwest (webhooks) |
| Twilio |
Sink |
P4 |
0/4 |
Leading CPaaS. 300k+ accounts. SMS alerts for IoT scenarios (equipment failure notifications). |
reqwest (REST API) |
16. Stream Processing Integration
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| Apache Flink |
Source+Sink |
P1 |
2/4 |
26k GitHub stars. Leading stream processor. Used by Alibaba, Apple, Netflix, Uber. Needs Flink connector JAR. Enables complex event processing, windowing, joins. Already has #2484 for transport. |
Java connector JAR; Iggy Kafka compat could help |
| Apache Spark Streaming |
Source |
P3 |
2/4 |
41k GitHub stars. Most popular data processing framework. Structured Streaming for batch+stream. |
Java/Scala connector; or Kafka compat layer |
| Apache Beam |
Source+Sink |
P3 |
1/4 |
8k+ GitHub stars. Google-backed. Unified batch+stream. Runs on Flink, Spark, Dataflow. |
Java SDK connector |
| Materialize |
Source |
P4 |
0/4 |
Streaming SQL DB. PostgreSQL-compatible. Incremental materialized views. |
PG wire protocol or Kafka compat |
| RisingWave |
Source |
P4 |
0/4 |
8.6k GitHub stars. Rust-based streaming DB. Natural ecosystem partner. PG-compatible. |
Native Rust integration or PG/Kafka compat |
| Apache Camel |
Framework |
P2 |
n/a |
6.1k GitHub stars. 193+ connectors (Kamelets). An Iggy Camel component would give access to hundreds of systems. Requires JVM. |
Java component (Camel is JVM-based) |
17. Workflow & Orchestration
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| Apache Airflow |
Trigger |
P4 |
1/4 |
44k GitHub stars. Industry standard workflow orchestration. Iggy sensor/trigger for event-driven DAGs. |
reqwest (REST API) |
| Temporal |
Source+Sink |
P4 |
0/4 |
12k+ GitHub stars. Durable execution platform. Bridge between streams and fault-tolerant workflows. |
temporal-sdk-core (official Rust SDK) |
| Prefect |
Trigger |
P4 |
0/4 |
20k GitHub stars. 25k+ community. Modern Airflow alternative. Event-driven workflow triggering. |
reqwest (REST API / webhooks) |
| Dagster |
Source |
P4 |
0/4 |
14k GitHub stars. Asset-based data orchestration. Growing adoption. |
reqwest (REST API) |
18. IoT Protocols
| Target |
Type |
Priority |
Comp |
Why It Matters |
Rust Crate |
| MQTT |
Source+Sink |
P0 |
3/4 |
(See #9.) De facto IoT standard. 50%+ enterprise adoption. $17.6B IoT protocol market. Billions of devices. The single most important missing connector. |
rumqttc |
| CoAP |
Source+Sink |
P3 |
0/4 |
IETF RFC 7252. UDP-based, 4-byte header. For ultra-constrained sensors where MQTT is too heavy. Healthcare, robotics, smart cities. |
coap; coap-lite |
| OPC-UA |
Source |
P2 |
1/4 |
IEC 62541 standard. 40%+ market share in industrial data acquisition. 1,000+ manufacturers. Industry 4.0 standard. Bridges factory floor into streaming. |
opcua (pure Rust client/server) |
| Modbus |
Source |
P3 |
1/4 |
40%+ market share in industrial data. Hundreds of millions of devices. 45+ years old. Legacy factory equipment integration. |
tokio-modbus (async RTU/TCP) |
| LwM2M |
Source |
P4 |
0/4 |
IoT device management protocol on CoAP. 400+ ready-to-use objects. Smart metering, smart cities. Specialized. |
coap crate (LwM2M builds on CoAP) |
Codec / Serialization Format Status
| Format |
Status |
Issue |
Notes |
| JSON |
Done |
-- |
Built-in to connector runtime |
| Protobuf |
Done |
-- |
Built-in to connector runtime |
| FlatBuffers |
Done |
-- |
Unique to Iggy -- zero-copy for IoT |
| Raw / Bytes |
Done |
-- |
Built-in |
| Text / String |
Done |
-- |
Built-in |
| Avro |
In progress |
#1846 |
Critical for Kafka ecosystem interop. apache-avro crate |
| BSON |
In progress |
#1847 |
MongoDB native format. bson crate (14.2M downloads) |
| Parquet |
Proposed |
-- |
Output format for S3/GCS/HDFS sinks. parquet crate |
| CSV |
Proposed |
-- |
Human-readable batch import/export. csv crate |
| MessagePack |
Proposed |
-- |
Compact binary for IoT. rmp-serde crate |
Summary Counts
| Category |
Total |
Done |
In Progress |
Proposed |
| Relational Databases |
7 |
1 (PG) |
1 (JDBC) |
5 |
| NoSQL / Document |
8 |
0 |
1 (MongoDB) |
7 |
| Search & Analytics |
5 |
3 (ES, Quickwit, Pinot) |
0 |
2 |
| Cloud Object Storage |
6 |
0 |
0 |
6 |
| Data Lakehouse |
3 |
1 (Iceberg) |
1 (Delta) |
1 |
| Data Warehouses |
10 |
0 |
2 (CH, Redshift) |
8 |
| Time-Series DBs |
7 |
0 |
1 (InfluxDB) |
6 |
| Graph DBs |
4 |
0 |
0 |
4 |
| Message Queues |
12 |
0 |
0 |
12 |
| Caches & KV |
6 |
0 |
0 |
6 |
| File Systems & Formats |
9 |
0 |
2 (Avro, BSON) |
7 |
| HTTP & API |
4 |
0 |
0 |
4 |
| Observability |
7 |
0 |
0 |
7 |
| AI/ML & Vector DBs |
6 |
0 |
0 |
6 |
| SaaS & Enterprise |
6 |
0 |
0 |
6 |
| Stream Processing |
6 |
0 |
1 (Flink) |
5 |
| Workflow |
4 |
0 |
0 |
4 |
| IoT Protocols |
5 |
0 |
0 |
5 |
| TOTAL |
115 |
5 |
9 |
101 |
How to Contribute
- Check this issue and the linked open issues above to avoid duplicating work
- Comment here to claim a proposed connector
- Reference the existing connectors (PostgreSQL, Elasticsearch) as implementation templates
- See the connector SDK traits (
Sink, Source, Transform) in the SDK crate
- Each connector is a separate Rust cdylib crate under
core/connectors/
- For every P0-P2 target, a mature Rust crate exists on crates.io (linked above)
