# Self-hosted deployment

In a self-hosted deployment, you host both the **control plane** and the **data plane**. You choose the topology that fits your operational and regulatory requirements — both planes in a single Kubernetes cluster, in separate clusters, in different regions, or any combination. Union.ai gives you complete control over the installation with full data sovereignty.

> [!NOTE]
> Self-hosted deployment is distinct from [self-managed deployment](https://www.union.ai/docs/v2/union/deployment/selfmanaged/_index), where Union.ai hosts the control plane and you manage only the data plane.

## When to use self-hosted deployment

Choose self-hosted deployment when:

- You need full control over both control plane and data plane
- You have strict data locality or sovereignty requirements
- You want to minimize network egress costs
- You are running in an air-gapped or restricted network environment
- You need to colocate the control plane with the data plane for latency, isolation, or regulatory reasons

Choose [self-managed deployment](https://www.union.ai/docs/v2/union/deployment/selfmanaged/_index) when:

- You want Union.ai to manage the control plane
- You need Union.ai's managed services and support

## Topologies

Self-hosted deployments support two topologies, distinguished by how the control plane and data plane are placed relative to each other:

- **Separate-cluster (default and recommended)** — control plane in one Kubernetes cluster, one or more data planes in separate clusters (often per environment, region, or business unit). Communication is via the control plane's external ingress; each data plane authenticates to the CP as a workload identity. Data planes may run on different cloud providers than the CP. This is the topology [Infrastructure requirements](https://www.union.ai/docs/v2/union/deployment/selfhosted/infrastructure-requirements/page.md) is written around.
- **Intra-cluster (special case)** — both planes in the same Kubernetes cluster, communicating over cluster-local Services. The simplest topology; used by the [Getting started](https://www.union.ai/docs/v2/union/deployment/selfhosted/getting-started/page.md) walkthrough and for footprint-constrained or evaluation deployments. Enable by setting `CONTROLPLANE_GRPC_ENDPOINT` + `QUEUE_GRPC_ENDPOINT` to cluster-local svc URLs in your `values.{aws,gcp}.yaml` — see [Getting started → Optional: intracluster topology](https://www.union.ai/docs/v2/union/deployment/selfhosted/getting-started/page.md#optional-intracluster-topology). [Infrastructure requirements → Intra-cluster topology](https://www.union.ai/docs/v2/union/deployment/selfhosted/infrastructure-requirements/page.md#intra-cluster-topology) covers the substrate trade-offs.

The architecture diagram below shows the intra-cluster topology — the simplest layout to visualize. Separate-cluster uses the same chart components in different cluster boundaries.

## Architecture

In an intra-cluster deployment, the control plane and data plane communicate using Kubernetes internal networking rather than external endpoints.

```mermaid
graph TB
    subgraph cluster["Kubernetes Cluster"]
        subgraph cp["Controlplane Namespace"]
            cpingress["NGINX Ingress\n(TLS/HTTP2)\nClusterIP"]
            admin["Admin"]
            identity["Identity"]
            services["Services"]

            cpingress --> admin
            cpingress --> identity
            cpingress --> services
        end

        subgraph dp["Dataplane Namespace"]
            dpingress["NGINX Ingress\nClusterIP"]
            operator["Operator"]
            propeller["Propeller"]
            clusterresource["Cluster Resource\nSync"]

            dpingress --> operator
            dpingress --> propeller
            dpingress --> clusterresource
        end

        subgraph external["External Resources"]
            db["PostgreSQL"]
            storage["Object Storage\n(S3 / GCS)"]
        end

        dpingress -.->|"Internal DNS"| cpingress
        cpingress -.->|"Internal DNS"| dpingress

        admin --> db
        identity --> db
        services --> db
        admin --> storage
        operator --> storage
    end
```

**Key characteristics:**

- **Simplified networking**: All communication stays within the cluster via Kubernetes DNS
- **No external dependencies**: No internet connectivity required for control plane to data plane communication
- **Cost-effective**: No network egress costs between control plane and data plane
- **Self-signed certificates**: Can use self-signed certificates for intra-cluster TLS
- **Single-tenant mode**: Simplified security model with explicit organization configuration

## Deployment guides

Start with [Getting started](https://www.union.ai/docs/v2/union/deployment/selfhosted/getting-started/page.md) for an end-to-end walkthrough. Review [Infrastructure requirements](https://www.union.ai/docs/v2/union/deployment/selfhosted/infrastructure-requirements/page.md) before provisioning your cloud substrate. The remaining pages cover individual deployment components in depth.

### [Getting started](https://www.union.ai/docs/v2/union/deployment/selfhosted/getting-started/page.md)

End-to-end walkthrough: provision, install, configure, smoke test

### [Infrastructure requirements](https://www.union.ai/docs/v2/union/deployment/selfhosted/infrastructure-requirements/page.md)

What to provision — substrate, CP and DP sizing, scaling constraints, topology choice

### [Authentication](https://www.union.ai/docs/v2/union/deployment/selfhosted/authentication/page.md)

Configure OIDC/OAuth2 authentication for your deployment

### [Authorization](https://www.union.ai/docs/v2/union/deployment/selfhosted/authorization/page.md)

Configure authorization mode (Noop, External, or Union built-in RBAC)

### [Identity](https://www.union.ai/docs/v2/union/deployment/selfhosted/identity/page.md)

Configure identity service for User Management

### [Image builder](https://www.union.ai/docs/v2/union/deployment/selfhosted/image-builder/page.md)

Register the image builder for automatic container image builds

### [App serving](https://www.union.ai/docs/v2/union/deployment/selfhosted/app-serving/page.md)

Serve Streamlit, FastAPI, and custom apps from the data plane

### [Operations](https://www.union.ai/docs/v2/union/deployment/selfhosted/operations/page.md)

Operational guides: CI/CD integration, key rotation, and more

## Subpages

- [Getting started](https://www.union.ai/docs/v2/union/deployment/selfhosted/getting-started/page.md)
  - Prerequisites
  - Deployment overview
  - Step 1: Helm repositories and control plane CRDs
  - Step 2: Namespaces and registry pull secret
  - Step 3: TLS certificates
  - Existing certificate
  - cert-manager
  - OpenSSL self-signed
  - Step 4: Database password secret
  - Step 5: Environment overrides
  - AWS
  - GCP
  - Step 6: Install the control plane
  - AWS
  - GCP
  - Step 7: Install data plane CRDs
  - Step 8: Install the data plane
  - AWS
  - GCP
  - Data plane self-registration
  - Step 9: Verify the installation
  - Key configuration
  - Single-tenant mode
  - Service discovery
  - Optional: intracluster topology
  - Key differences from a self-managed deployment
  - Next steps
- [Infrastructure requirements](https://www.union.ai/docs/v2/union/deployment/selfhosted/infrastructure-requirements/page.md)
  - Sizing model
  - Workload signals → size mapping
  - How to read this page
  - Pre-installation checklist
  - Shared substrate (per cluster)
  - Control plane
  - Data plane
  - Shared optional
  - Project APIs (GCP)
  - Kubernetes cluster
  - Substrate
  - AWS
  - GCP
  - Control plane
  - Data plane
  - Networking
  - Allocation strategy
  - Control plane
  - AWS
  - GCP
  - Data plane
  - AWS
  - GCP
  - Scale up when
  - Identity and workload binding
  - Substrate
  - AWS
  - GCP
  - Control plane
  - Data plane
  - AWS
  - GCP
  - AWS
  - GCP
  - Database
  - Control plane
  - Network connectivity
  - AWS
  - GCP
  - Object storage
  - Substrate
  - AWS
  - GCP
  - Control plane
  - Data plane
  - Secrets
  - Workflow secrets (`flyte.Secret`)
  - Container image registry
  - Substrate
  - AWS
  - GCP
  - Data plane
  - Logging
  - Substrate
  - AWS
  - GCP
  - IAM bindings
  - AWS
  - GCP
  - Ingress and DNS
  - Substrate
  - AWS
  - GCP
  - Control plane
  - Data plane
  - Scaling constraints
  - Pod density and IP allocation
  - AWS
  - GCP
  - vCPU quotas
  - Image registry pull rate
  - CoreDNS and conntrack
  - Ephemeral disk
  - etcd ceiling
  - Spot capacity
  - Queue and executor action limits
  - Intra-cluster topology
  - When to use
  - What changes from separate-cluster
  - Scaling considerations specific to intra-cluster
  - Reference
  - Next
- [Authentication](https://www.union.ai/docs/v2/union/deployment/selfhosted/authentication/page.md)
  - Overview
  - Identity provider requirements
  - Authorization server setup
  - Okta
  - Entra ID
  - Generic OIDC
  - Identity type claim requirements
  - Subject claim requirements
  - Step 1: Create OAuth2 applications
  - Application 1: Browser login (Confidential)
  - Application 2: CLI (Public)
  - Application 3: Service-to-service (Confidential)
  - Application 4: Operator (Confidential)
  - Application 5: EAGER (Confidential)
  - Step 2: Configure control plane
  - Okta
  - Entra ID
  - Generic OIDC
  - Step 3: Create Kubernetes secrets (control plane)
  - Step 4: Configure data plane
  - Step 5: Configure EAGER_API_KEY
  - Provision the task-pod key automatically with the operator
  - Seed EAGER credentials without creating an OAuth app (`apiKeyOverrides`)
  - Step 6: Deploy
  - Verification
  - Summary of secrets
  - Self-hosted vs. self-managed authentication
  - Troubleshooting
  - Admin service auth endpoints return 404
  - Token validation fails with "audience mismatch"
  - Data plane cannot authenticate to control plane
  - CLI login fails
  - Entra ID: `AADSTS1002012` invalid_scope for service-to-service
  - Subject not found in token
- [Authorization](https://www.union.ai/docs/v2/union/deployment/selfhosted/authorization/page.md)
  - Prerequisites
  - Architecture
  - Authorization modes
  - Noop (default)
  - Union (built-in RBAC) — recommended
  - External
  - Configuration
  - External authorization server contract
  - gRPC contract
  - Identity resolution
  - Actions
  - Service account permissions
  - Configuring service accounts
  - Reference implementation
  - Observability
  - Key metrics
  - Alerts
  - Verification
  - Troubleshooting
  - All requests denied
  - Dataplane cannot register or heartbeat
  - Workflows fail to launch child tasks
  - "Owned By: Unknown" in the console
  - Authorization component crashlooping
  - High latency on API calls
  - Connection errors to external backend
- [Image builder](https://www.union.ai/docs/v2/union/deployment/selfhosted/image-builder/page.md)
  - How it works
  - Prerequisites
  - Configuration
  - Verify
  - Restricted network environments
  - Container image access
  - Python version constraints
  - AWS VPC endpoints
  - Package index access
  - Manual registration
  - Troubleshooting
  - "remote image builder is not enabled"
  - Build task fails with permission errors
- [App serving](https://www.union.ai/docs/v2/union/deployment/selfhosted/app-serving/page.md)
  - How it works
  - Topologies
  - Separate-cluster data plane
  - Intracluster data plane
  - Prerequisites
  - Configuration
  - Enable the gateway
  - Apps domain
  - Expose a separate-cluster data plane
  - Authentication
  - DNS and TLS
  - Verify
- [Identity](https://www.union.ai/docs/v2/union/deployment/selfhosted/identity/page.md)
  - Prerequisites
  - Overview
  - Identity providers
  - Configuring the Azure provider (Entra ID)
  - Step 1: Create an Entra ID app registration for Graph API
  - Step 2: Configure API permissions
  - Step 3: Create a client secret
  - Step 4: Note the required values
  - Control plane Helm configuration
  - Configuration reference
  - Secret delivery
  - Verifying the configuration
  - Troubleshooting
  - User Management page shows "Unable to load users"
  - Users appear but some are missing
  - Identity pod fails to start
  - "Unauthorized" errors in identity logs
- [Operations](https://www.union.ai/docs/v2/union/deployment/selfhosted/operations/page.md)
  - [CI/CD integration](https://www.union.ai/docs/v2/union/deployment/selfhosted/cicd/page.md)
  - [Task routing](https://www.union.ai/docs/v2/union/deployment/selfhosted/task-routing/page.md)
  - [Troubleshooting](https://www.union.ai/docs/v2/union/deployment/selfhosted/troubleshooting/page.md)
  - [Monitoring](https://www.union.ai/docs/v2/union/deployment/selfhosted/monitoring/_index)

---
**Source**: https://github.com/unionai/unionai-docs/blob/main/content/deployment/selfhosted/_index.md
**HTML**: https://www.union.ai/docs/v2/union/deployment/selfhosted/