Apache Kafka · multi-tenant proxy

Multi-tenant Kafka isolation
on a shared cluster.

Klamm sits between your clients and your Apache Kafka cluster. Each tenant connects with their own credentials and sees their own topics, consumer groups, and transactions. None of them see each other.

Quickstart Source pre-release · 0.x

Clients: unmodified · SASL/PLAIN · SCRAM
Isolation: protocol-layer · per-tenant namespaces
Deployment: single binary · YAML · SIGHUP reload
Status: pre-release · Apache 2.0

01 · why

For platforms that share one Kafka.

Kafka has no native notion of a tenant. Teams that need one usually implement it in one of three places — application code, the broker fleet, or the client SDK. Klamm puts it on the wire.

01

Share topics; enforce tenancy in application code.

Prefix every write, filter every read, and trust that no service ever ships without its isolation logic intact. A single bug leaks one tenant's data into another.
02

Run a broker fleet per tenant.

Operational and financial weight scales with tenant count. Kafka is not cheap at rest, and most tenants do not fill a cluster.
03

Build bespoke isolation into your client SDK.

Every producer and every consumer gets wrapped. Every new Kafka client a team adopts needs the same treatment. The glue outlives the problem.
04

Run Klamm.

Protocol-layer isolation on a single shared Kafka cluster. Clients connect to Klamm as if it were Kafka. Each tenant sees its own virtual cluster. The broker stays one cluster, operated by one team.

02 · capability

What you get.

Four properties that fall out of doing isolation at the protocol layer rather than above it.

Tenant isolation at the protocol layer.

Topic names, consumer group IDs, and transactional IDs are namespaced per tenant on the wire. Two tenants can both publish to orders without ever seeing each other's partitions or offsets.

Unmodified Kafka clients.

Any client that speaks SASL/PLAIN, SCRAM-SHA-256, or SCRAM-SHA-512 connects to Klamm the same way it connects to Kafka. Tenant identity is carried by the SASL credential; no SDK changes are required.

Drop-in deployment.

A single binary and a single YAML file. Point tenants at Klamm's bootstrap address instead of the broker's. Klamm handles upstream broker discovery, broker route generation, and protocol rewriting. SIGHUP reloads tenant changes without dropping connections.

Flexible tenant models.

Dynamic namespaces for quick onboarding, or explicit allow-lists for strict environments. Per-tenant limits on connections and in-flight requests. Topic deletion is a separate permission, disabled by default.

03 · behaviour

What Klamm rewrites on the wire.

A tenant's credentials select a namespace. Klamm translates every identifier that crosses the boundary — in both directions — so each side of the proxy operates in the vocabulary it expects.

tenant view · logical broker view · physical

Topics orders __vt_acme_orders

Consumer groups payments-svc __vt_acme_payments-svc

Transactional IDs orders-producer-1 __vt_acme_orders-producer-1

Broker addresses proxy.example.com:9093 broker-1.kafka.internal:9092

Assignment payloads orders (in payload) __vt_acme_orders (in payload)

04 · surface

One YAML file.
One process.

Klamm reads a single configuration file and negotiates everything else from the cluster. Upstream broker IDs, partition leaders, API versions, and SASL mechanisms are discovered at runtime; route ports are derived deterministically from the broker port range.

Tenant and credential changes reload on SIGHUP. Topology changes require a restart — by design.

Dynamic namespace or explicit allow-list per tenant
Per-tenant limits on connections and in-flight requests
Opt-in broker-delegated ACL admin virtualization
Optional TLS — upstream and SNI-routed downstream

Full config reference

proxy.yaml two tenants · three brokers

proxy:
  listener_sets:
    - name: plaintext
      port_range:
        bind_host: "0.0.0.0"
        advertised_host: "proxy.example.com"
        bootstrap_port: 9092
        broker_port_range: { start: 9093, end: 9095 }
        broker_id_base: 1

upstream:
  bootstrap:
    - "kafka-1.internal:9092"
    - "kafka-2.internal:9092"
    - "kafka-3.internal:9092"

tenants:
  - name: acme
    allowed_topics: [orders, billing, events, audit.log]
    auth:
      credentials:
        - username: acme
          password_file: "secrets/acme.password"

  - name: globex
    allowed_topics: [orders, users]
    auth:
      credentials:
        - username: globex
          password_file: "secrets/globex.password"

05 · coverage

The Kafka wire protocol, virtualized.

Klamm implements the Kafka APIs a tenant uses: produce, fetch, groups, transactions, admin, auth. Versions are intersected with the upstream broker's advertised range at connection time. Production readiness is tracked separately; see status.

Produce / Fetch

Produce v3–12
Fetch v4–18
ListOffsets v1–9
OffsetForLeaderEpoch v3–4

Metadata

Metadata v0–13
DescribeCluster v0–2
DescribeTopicPartitions v0
FindCoordinator v0–6

Groups & Coordination

JoinGroup v0–9
SyncGroup v0–5
Heartbeat v0–4
ConsumerGroupHeartbeat v0–1

Transactions

InitProducerId v0–5
AddPartitionsToTxn v0–5
EndTxn v0–5
TxnOffsetCommit v0–5

Admin

CreateTopics v2–7
DeleteTopics v1–6
DescribeConfigs v1–4
ACL admin (opt-in)

Auth

SaslHandshake v0–1
SaslAuthenticate v0–2
ApiVersions v0–4

Complete list of supported APIs →

06 · compatibility

Compatibility, graded by the clients themselves.

Each of the clients below ships a substantial integration test suite. Klamm runs it with the proxy in the path.

Client Upstream test entrypoint Broker matrix

franz-go go test -timeout 5m ./... upstream root fixture

kafka-go go test ./... · -tags=unsafe 2.7.0 · 2.8.1 · 3.7.0

kafka-python make test 4.0.0 · 3.9.0 · 2.8.2 · PLAIN / SCRAM-256 / SCRAM-512

librdkafka run-test.sh -Q -E -L trivup KafkaCluster fixtures

KafkaJS jest broker group 2.4 (three-broker compose)

07 · status

Scope, openly.

Klamm is pre-release. OAUTHBEARER and GSSAPI are not implemented. Upstream TLS is server-auth only. KIP-848 regex subscriptions are deferred. Connection pooling is deferred. Per-pod limits are not distributed across proxy instances. Each of these gaps is documented on its own page.

Read the limitations

Start reading.

The quickstart brings up Kafka, Klamm, and two tenant consoles with one docker compose command.

Quickstart Configuration All docs