Red Hat Connectivity Link

On-premise DNS with CoreDNS

You can self-manage your on-premise DNS by integrating CoreDNS with your DNS infrastructure through access control and zone delegation. Connectivity Link combines the DNS Operator with CoreDNS to simplify your management and security for on-premise DNS servers.

You can use CoreDNS in both single-cluster and multicluster scenarios.

CoreDNS is best used in environments that change often, where using a DNS-as-code approach makes sense. The following situations are example use cases for integrating with CoreDNS:

You need to avoid dependency on external cloud DNS services.
You have regulatory or compliance requirements mandating self-hosted infrastructure.
You need to keep full control over DNS records.
You want to delegate specific DNS zones from existing DNS servers to Kubernetes-managed CoreDNS.
You require consistent DNS management across hybrid or multicloud environments.
You need to reduce DNS operational costs by eliminating per-query charges.
You do not want to directly manage DNS records on the on-premise DNS server.
You need to keep authoritative control on edge DNS servers.

For example:

Configure your authoritative on-premise DNS server to delegate a specific subdomain, such as deployment.example.local, to CoreDNS instances managed by Connectivity Link.
Any pod-level dnsPolicy CR can then interact with the CoreDNS provider within the OpenShift Container Platform cluster. You can specify the DNS provider that handles the records for the targeted gateways in the delegate field of the DNS policy.
The CoreDNS instance becomes authoritative for the delegated subdomain and manages the necessary DNS records for gateways within that subdomain.

CoreDNS integration architecture

CoreDNS is a DNS server that consists of default plugins that do several tasks, for example:

Automatically detects when you add new services to your cluster and adds them to directories.
Caches recent addresses to avoid the latency of repeated lookups.
Runs health checks and skips over services that are down.
Provides dynamic redirects by rewriting queries as they come in.

You can add plugins for observability and other services that you require by updating the CoreDNS with the DNS Operator.

With the DNS Operator, DNS is the first layer of traffic management. You can deploy the DNS Operator to multiple clusters and coordinate them all on a given zone. This means that you can use a shared domain name across clusters to balance traffic based on your requirements.

Technical workflow

To give you integration with CoreDNS, Connectivity Link extends the DNS Operator with the kuadrant CoreDNS plugin that sources records from the kuadrant.io/v1alpha1/DNSRecord custom resource (CR) and applies location-based and weighted response capabilities.

You can create DNS records that point to the CoreDNS secret in one of the three following ways:

Create it manually.
Use a non-delegating DNS policy at a gateway with routes attached. The kuadrant-operator CR creates DNSRecord CRs with the secret.
Use a delegating DNS policy at a gateway. The delegating policy results in the creation of a delegating DNSRecord CR without a secret reference. All delegating DNS Records are combined into a single authoritative DNS Record. The authoritative DNSRecord uses a default provider secret.

The DNS Operator reconciles authoritative records that have the CoreDNS secret referenced and applies labels only to those CRs. CoreDNS watches those records and matches the labels with zones configured in the Corefile. If there is a match, the authoritative DNSRecord CR is used to serve a DNS response.

There are no changes to the dnsPolicy API and no required changes to the policy controllers. This integration is isolated to the DNS Operator and the CoreDNS plugin.

The CoreDNS integration supports both single-cluster and multicluster deployments.

Single cluster

Organizations that want to self-host their DNS infrastructure without the complexity of multicluster coordination can use single-cluster CoreDNS integration. Using delegation is not required.

A single cluster runs both DNS Operator and CoreDNS with the plugin. CoreDNS only serves DNSRecord CRs that point to a CoreDNS provider secret. The CoreDNS plugin watches for DNS records labeled with the appropriate zone name and serves them directly. Any authoritative DNSRecord CR has endpoints from the single cluster.

Multi-cluster delegation

Multiple clusters can participate in serving a single DNS zone through Kubernetes role-based delegation that enables geographic distribution of DNS services and high availability. This implementation enables workloads across multiple clusters to contribute DNS endpoints to a unified zone, with primary clusters maintaining the authoritative view. The role of a cluster is determined by the DNS Operator.

Multi-cluster delegation uses kubeconfig-based interconnection secrets that grant read access to DNSRecord resources across clusters. This approach reuses Kubernetes role-based access (RBAC).

Primary clusters: Run both the DNS Operator and CoreDNS and serve the DNS records that are local. The DNS Operator running on primary clusters that delegate reconciles DNSRecord CRs by reading and merging them. Primary clusters then serve these authoritative DNSRecord CRs. Each CoreDNS instance serves the relevant authoritative DNSRecord for the configured zone. Each primary cluster can independently serve the complete record set.
Secondary clusters: Only run the DNS Operator. These clusters create delegating DNSRecord CRs but do not interact with DNS providers directly. If the secret and subdomain are properly configured, these DNS records are automatically reconciled in the primary cluster.

Zone labeling

CoreDNS integration uses a label-based filtering mechanism. The DNS Operator applies a zone-specific label to DNSRecord CRs when the CRs are reconciled. The CoreDNS plugin only watches for DNSRecord CRs with labels that match configured zones. This method reduces resource use and provides clear zone boundaries.

GEO and weighted routing

GEO and weighted routing use the same algorithmic approach as cloud providers. By using CoreDNS, you can have parity with cloud DNS provider capabilities and maintain full control over your DNS infrastructure.

GEO routing: The CoreDNS geoip plugin uses geographical-database integration to return region-specific endpoints.
Weighted routing: Applies probabilistic selection based on endpoint weights.
Combined routing: First applies GEO filtering, then weighted selection within the matched region.

CoreDNS DNS records security considerations

As an infrastructure engineer or business lead, you can implement several security best practices when using CoreDNS with Connectivity Link.

Zone configuration DNSRecord custom resources (CRs) have full control over a Zone’s name server (NS) records. Anyone who can create or change a DNSRecord that targets the root of the main domain name with NS records can decide where the all Zone traffic goes. Consider this as you plan your access controls.

For example, use the following access-control best practices:

Separate namespaces: Keep zone configuration DNSRecord CRs in a dedicated, restricted namespace
Use least-privilege policies:
- Strict RBAC: Only grant DNSRecord creation permissions to trusted infrastructure engineers and cluster administrators.
- Namespace isolation: Grant application developers DNSRecord permissions only in their own namespaces.
Audit logging: Enable Kubernetes audit logging to track all DNSRecord changes. CoreDNS audit logging is enabled by default for network troubleshooting and traffic pattern observability.
Version control: Use a DNS-as-code approach. Store zone configuration DNSRecord CRs in Git and use standardized review processes.

You can use the following RBAC configuration example to get you started with defining access:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: dns-zone-config-admin
  namespace: kuadrant-coredns
rules:
- apiGroups: ["kuadrant.io"]
  resources: ["dnsrecords"]
  verbs: ["create", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: dns-zone-config-admin-binding
  namespace: kuadrant-coredns
subjects:
- kind: User
  name: dns-admin@example.com  # Only trusted administrators
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: dns-zone-config-admin
  apiGroup: rbac.authorization.k8s.io

Use CoreDNS with a single cluster

You can use CoreDNS as a DNS provider for Connectivity Link in a single-cluster, on-premise environment. This integration allows Connectivity Link to manage DNS entries within your internal network infrastructure.

Important

In a single-cluster setup, ensure that the endpoints IP address value you use is reachable from the kuadrant-system namespace. The default IP address, 10.96.0.10, is the internal cluster-wide DNS address.

Prerequisites

Connectivity Link is installed on the OpenShift Container Platform cluster.
The OpenShift CLI (oc) is installed.
You have administrator privileges on the OpenShift Container Platform cluster.
You are logged in to the cluster you want to configure.
Your OpenShift Container Platform clusters have support for the loadbalanced service type that allows UDP and TCP traffic on port 53, such as MetalLB.
You have access to configure your authoritative on-premise DNS server.
Podman is installed.

Procedure

Set up your cluster. Set the following environment variables for your cluster context:
```
$ export CTX_PRIMARY=$(oc config current-context) \
  export KUBECONFIG=~/.kube/config \
  export PRIMARY_CLUSTER_NAME=local-cluster \
  export ONPREM_DOMAIN=<onprem_domain> \
  export KUADRANT_SUBDOMAIN=""
```
- For the ONPREM_DOMAIN variable value, use your actual root domain.
- For the KUADRANT_SUBDOMAIN variable value, valid values are empty or kuadrant.
Extract the CoreDNS manifests from the dns-operator bundle by running the following commands:
1. Create a writable container layer from your DNS Operator bundle image by running the following command:
  $ podman create --name <bundle> registry.redhat.io/rhcl-1/dns-operator-bundle:rhcl-<1.3.0>
  - Optional. Replace <bundle> with the name you want to use.
  - Replace <1.3.0> with your Connectivity Link version.
2. Copy the CoreDNS manifest file out of the stopped container and onto your local host machine by running the following command:
  $ podman cp <bundle>:/coredns/manifests.yaml ./coredns-manifests.yaml
  Optional. Replace <bundle> with the name you want to use.
3. Delete the stopped container named <bundle> that you created in the first step since you no longer need it by running the following command:
  $ podman rm <bundle>
  Optional. Replace <bundle> with the name you want to use.
Apply the manifests to the cluster by running the following command:
```
$ oc apply -f ./coredns-manifests.yaml
```

Create a ConfigMap object to define the authoritative zone for CoreDNS by using the following example. This minimal configuration enables the kuadrant plugin and GeoIP features.

apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-kuadrant-config
  namespace: kuadrant-coredns
data:
  Corefile: |
    ${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN}:53 {
        debug
        errors
        health {
            lameduck 5s
        }
        ready
        log
        geoip <GeoIP-database-name>.mmdb {
            edns-subnet
        }
        metadata
        kuadrant
    }

Note	For production or exact GeoIP routing, mount your licensed MaxMind GeoIP database into the CoreDNS pod and update the filename in the `data.Corefile.geoip` parameter.

Apply the ConfigMap object by running the following command:

$ oc apply -f --context $CTX_PRIMARY coredns-kuadrant-config.yaml

Update the CoreDNS deployment to use the new configuration by running the following command:

$ oc --context $CTX_PRIMARY -n kuadrant-system patch deployment kuadrant-coredns --patch '{"spec":{"template":{"spec":{"volumes":[{"name":"config-volume","configMap":{"name":"coredns-kuadrant-config","items":[{"key":"Corefile","path":"Corefile"}]}}]}}}}'

Set a watch-and-wait command for the deployment rollout to complete by running the following command:

$ oc --context $CTX_PRIMARY -n kuadrant-system rollout status deployment/kuadrant-coredns

Example output

kuadrant-coredns successfully rolled out

Create the Kubernetes Secret that Connectivity Link uses to interact with CoreDNS. This secret specifies the zones this provider instance is authoritative for. To create the secret, run the following command:
```
$ oc create secret generic coredns-credentials \
  --namespace=<kuadrant_system> \
  --type=kuadrant.io/coredns \
  --from-literal=ZONES="${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN}" \
  --context ${CTX_PRIMARY}
```
Replace <kuadrant_system> with the namespace of your Connectivity Link deployment.

Verification

Check the status of the DNSRecord CR by running the following commands:

$ oc get dnsrecord <name> -n <namespace> -o jsonpath='{.status.conditions[?(@.type=="Ready")]}'

Replace <name> with the name of your DNSRecord CR.

Replace <namespace> with the namespace where you applied your DNSRecord CR.

$ NS1=$(oc get svc kuadrant-coredns -n kuadrant-coredns -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
  ROOT_HOST=$(oc get dnsrecord <name> -n <namespace> -o jsonpath='{.spec.rootHost}')
  dig @${NS1} ${ROOT_HOST}

Expect the Ready condition to be True.

Next steps

Create dnsPolicy custom resources in your OpenShift Container Platform pods, referencing the coredns-credentials secret as the provider. Connectivity Link manages DNS records within the delegated ${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN} zone through CoreDNS.

Troubleshoot CoreDNS with a single cluster

After you set it up, you can troubleshoot CoreDNS as a DNS provider for Connectivity Link in a single-cluster, on-premise environment.

Important

Prerequisites

Connectivity Link is installed on the OpenShift Container Platform cluster.
The OpenShift CLI (oc) is installed.
You have administrator privileges on the OpenShift Container Platform cluster.
You are logged in to the cluster you want to configure.
Your OpenShift Container Platform clusters have support for the loadbalanced service type that allows UDP and TCP traffic on port 53, such as MetalLB.
You have access to configure your authoritative on-premise DNS server.
Podman is installed.

Procedure

If you are having undetermined trouble, view the logs for all CoreDNS pods by running the following command:
```
$ oc logs -n kuadrant-coredns deployment/kuadrant-coredns
```
If the DNSRecord is not appearing in the zone, verify that the record has the zone label by running the following command:
```
$ oc get dnsrecords.kuadrant.io -n dnstest -o jsonpath='{.items[*].metadata.labels}' | grep kuadrant.io/coredns-zone-name
```
The output should include the zone name, for example kuadrant.io/coredns-zone-name: k.example.com.
1. If the output does not show the zone name, check that the DNS Operator is running by using the following command:
  $ oc get pods -n dns-operator-system
2. You can also check the DNS Operator logs by running the following command:
  $ oc logs -n dns-operator-system deployment/dns-operator-controller-manager
A couple of common issues can be missing RBAC and GeoIP database, as shown in the following output examples:
1. RBAC permissions missing.: Check your ClusterRole and ClusterRoleBinding configurations.
2. GeoIP database file not found.: Ensure that your database is accessible.

Use CoreDNS with primary and secondary clusters

You can use CoreDNS as a DNS provider for Connectivity Link in an existing multicluster, on-premise environment. This integration allows Connectivity Link to manage DNS entries within your internal network infrastructure.

Prerequisites

Connectivity Link is installed on two separate OpenShift Container Platform clusters (primary and secondary).
OpenShift CLI (oc) is installed and configured for access to both clusters.
You have administrator privileges on both OpenShift Container Platform clusters.
Your OpenShift Container Platform clusters have support for the loadbalanced service type that allows UDP and TCP traffic on port 53, such as MetalLB.
You have access to configure your authoritative on-premise DNS server to delegate a subdomain.
Podman is installed.
jq is installed.

Procedure

Set up the primary cluster. Set the following environment variables for your primary cluster context:
```
$ export CTX_PRIMARY=<primary_cluster_context_name> \
  export KUBECONFIG=~/.kube/config \
  export PRIMARY_CLUSTER_NAME=<primary_cluster_name> \
  export ONPREM_DOMAIN=<onprem_domain> \
  export KUADRANT_SUBDOMAIN=<kuadrant>
```
- CTX_PRIMARY: Replace <primary_cluster_context_name> with the namespace of the cluster that you are specifying as primary.
- KUBECONFIG: Adjust the path to your config directy as needed.
- PRIMARY_CLUSTER_NAME: Replace <primary_cluster_name> with the name of the cluster that you are specifying as primary.
- ONPREM_DOMAIN: Replace <onprem_domain> with your actual root domain.
- KUADRANT_SUBDOMAIN: List the subdomain to delegate.

Extract the CoreDNS manifests from the dns-operator bundle by running the following commands:

$ podman create --name bundle registry.redhat.io/rhcl-1/dns-operator-bundle:rhcl-<1.3.0>

Replace <1.3.0> with your Connectivity Link version.

$ podman cp bundle:/coredns/manifests.yaml ./coredns-manifests.yaml

$ podman rm bundle

Apply the manifests to the cluster by running the following command:
```
$ oc apply -f ./coredns-manifests.yaml
```
Wait for the CoreDNS service to get an external IP address. You need the IP address to configure delegation on your authoritative on-premise DNS server. Retrieve and store the IP address by running the following command:
```
$ export COREDNS_IP_PRIMARY=$(oc --context $CTX_PRIMARY -n kuadrant-system get service kuadrant-coredns -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
echo "CoreDNS Primary IP: ${COREDNS_IP_PRIMARY}"
```

Create a ConfigMap object to define the authoritative zone for CoreDNS on the primary cluster. The following minimal example configuration enables the kuadrant plugin and GeoIP features:

apiVersion: v1
kind: ConfigMap
metadata:
  name: coredns-kuadrant-config
  namespace: kuadrant-coredns
data:
  Corefile: |
    ${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN}:53 {
        debug
        errors
        health {
            lameduck 5s
        }
        ready
        log
        geoip <GeoIP-database-name>.mmdb {
            edns-subnet
        }
        metadata
        kuadrant
    }

Note	For production or exact GeoIP routing, mount your licensed MaxMind GeoIP database into the CoreDNS pod and update the filename in the `data.Corefile.geoip` parameter.

Apply the ConfigMap object by running the following command:

$ oc apply -f --context $CTX_PRIMARY coredns-kuadrant-config.yaml

Update the CoreDNS deployment to use the new configuration by running the following command:

$ oc --context $CTX_PRIMARY -n kuadrant-system patch deployment kuadrant-coredns --patch '{"spec":{"template":{"spec":{"volumes":[{"name":"config-volume","configMap":{"name":"coredns-kuadrant-config","items":[{"key":"Corefile","path":"Corefile"}]}}]}}}}'

Set a watch-and-wait command for the deployment rollout to complete by running the following command:

$ oc --context $CTX_PRIMARY -n kuadrant-system rollout status deployment/kuadrant-coredns

Example output

kuadrant-coredns successfully rolled out

Create the Kubernetes Secret that Connectivity Link uses to interact with CoreDNS. This secret specifies the zones this provider instance is authoritative for.

$ oc create secret generic coredns-credentials \
  --namespace=kuadrant-system \
  --type=kuadrant.io/coredns \
  --from-literal=ZONES="${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN}" \
  --context ${CTX_PRIMARY}

On your authoritative on-premise DNS server, configure delegation for the ${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN} subdomain to the external IP addresses of the CoreDNS services running on your primary and secondary clusters, $COREDNS_IP_PRIMARY and $COREDNS_IP_SECONDARY. The specific steps depend on your DNS server software, for example, BIND, Windows DNS Server. You typically need to add Name Server (NS) records pointing the subdomain to the CoreDNS IP addresses.

Example delegation

; Delegate kuadrant.example.local to CoreDNS instances
$ORIGIN ${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN}.
@       IN      SOA     ns1.${ONPREM_DOMAIN}. hostmaster.${ONPREM_DOMAIN}. (
                        2023102601 ; serial
                        7200       ; refresh (2 hours)
                        3600       ; retry (1 hour)
                        1209600    ; expire (2 weeks)
                        3600       ; minimum (1 hour)
                        )
        IN      NS      coredns-primary.${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN}.

coredns-primary   IN A ${COREDNS_IP_PRIMARY}

Restart CoreDNS by running the following command:

$ oc -n kuadrant-coredns rollout restart deployment kuadrant-coredns

Note	After configuring delegation, you can test that the DNS resolution for the delegated subdomain works correctly by querying your authoritative DNS server for a record within the `kuadrant` subdomain. One of the CoreDNS instances is expected to refer to and answer the query.

Verification

Launch a temporary pod for testing by running the following command:
```
$ oc debug node/<node_name>
```
Replace <node_name> with the node you are testing on.

Add transfer to your Corefile by running the following command:

$ oc patch cm kuadrant-coredns -n kuadrant-coredns --type merge \
   -p "$(oc get cm kuadrant-coredns -n kuadrant-coredns -o jsonpath='{.data.Corefile}' | \
   sed 's/kuadrant/transfer {\n        to *\n    }\n    kuadrant/' | \
   jq -Rs '{data: {Corefile: .}}')"

Verify zone delegation by running the following command:

$ dig @${EDGE_NS} -k config/bind9/ddns.key -t AXFR example.com

Example output

example.com.            30      IN      SOA     example.com. root.example.com. 17 30 30 30 30
example.com.            30      IN      NS      ns.example.com.
k.example.com.          300     IN      NS      ns1.k.example.com.
ns1.k.example.com.      300     IN      A       172.18.0.16
ns.example.com.         30      IN      A       127.0.0.1
example.com.            30      IN      SOA     example.com. root.example.com. 17 30 30 30 30

In this example, k.example is the delegated zone and ns1.k.example is the primary zone.

Optional. Remove the transfer from your Corefile by running the following command:

$ oc patch cm kuadrant-coredns -n kuadrant-coredns --type merge \
   -p "$(oc get cm kuadrant-coredns -n kuadrant-coredns -o jsonpath='{.data.Corefile}' | \
   sed '/transfer {/,/}/d' | \
   jq -Rs '{data: {Corefile: .}}')"

Verify the start of authority (SOA) record for the delegated zone by running the following command:
```
$ dig @${EDGE_NS} soa k.example.com
```
Example output
```
;; ANSWER SECTION:
k.example.com.          60      IN      SOA     ns1.k.example.com. hostmaster.k.example.com. 12345 7200 1800 86400 60
```
The SOA record is expected to show the primary name server (NS) as confirmation that CoreDNS is responding authoritatively. In this example the primary NS is ns1.k.example.com.

Next steps

Create DNSPolicy resources in your OpenShift Container Platform clusters, referencing the coredns-credentials secret as the provider. Connectivity Link manages DNS records within the delegated ${KUADRANT_SUBDOMAIN}.${ONPREM_DOMAIN} zone through CoreDNS.

Troubleshoot CoreDNS

You can troubleshoot your CoreDNS deployment by restarting the service and by checking the logs. Use the following commands as needed to investigate your specific errors:

Restart CoreDNS by using the following command:

$ oc -n kuadrant-coredns rollout restart deployment kuadrant-coredns

You can view CoreDNS logs by running the following command:

$ oc logs -f deployments/kuadrant-coredns -n kuadrant-coredns

You can get recent logs by running the following command:

$ oc logs --tail=100 deployments/kuadrant-coredns -n kuadrant-coredns

CoreDNS removal or migration

You can remove your CoreDNS integration by deleting the CoreDNS deployment and deleting your DNS policies. To migrate to a different provider, delete existing dnsPolicy CRs and re-create them with the new provider secret reference. No data is permanently locked into CoreDNS.