Skip to content
BifröstDocs

DNS records

Copy-paste examples for the DNS TXT provider. They all publish the same three machines under infra.example.com as one TXT record set at _bifrost.infra.example.com, one value per machine. Such an inline value names its machine with node=. For more than about a dozen machines, see Large fleets.

A complete zone

Section titled “A complete zone”

A standard zone file, as BIND or CoreDNS load it:

db.infra.example.com
$ORIGIN infra.example.com.
$TTL 300
@                   IN SOA ns1 hostmaster 2026092601 3600 600 604800 300
@                   IN NS  ns1
ns1                 IN A   10.0.0.2


; one TXT record set, one value per machine
; agent-01: no host=, so bifrost connects to agent-01.infra.example.com
_bifrost            IN TXT "v=bf1 node=agent-01 tags=dev,agent"
; agent-02: an explicit address
_bifrost            IN TXT "v=bf1 node=agent-02 host=10.0.0.12 tags=dev"
; build-01: every key
_bifrost            IN TXT "v=bf1 node=build-01 host=10.0.0.18 port=2222 user=ci path=/srv/build tags=ci id=vm-4711"


agent-01            IN A   10.0.0.11

How each record becomes a machine

Section titled “How each record becomes a machine”

With this provider in your config:

[[discovery]]
type = "dns"
name = "infra"
domain = "infra.example.com"


[discovery.filter]
include_names = ["agent-*", "build-*"]


[discovery.mount]
user = "ubuntu"
honor_hints = true
Value Machine id Connects to Mounts as Tags
node=agent-01 agent-01 agent-01.infra.example.com, ssh’s port ubuntu, from ~ agent, dev
node=agent-02 agent-02 10.0.0.12, ssh’s port ubuntu, from ~ dev
node=build-01 build-01 10.0.0.18, port 2222 ci, from /srv/build ci
  • The machine id is the node= label, and the mount lands at ~/machines/<id>. id=vm-4711 is only a native id, matched by this provider’s include_ids and exclude_ids.
  • Without host= the node connects to <node>.<domain>, so agent-01 needs an A or AAAA record that your system resolver can answer: ssh resolves it when it connects.
  • host= and port= are always used. user= and path= are hints: with honor_hints = false, build-01 mounts as ubuntu from ~, still on port 2222.

Publish the records

Section titled “Publish the records”

Serve the zone above from CoreDNS with the file plugin:

Corefile
infra.example.com:53 {
    file /etc/coredns/db.infra.example.com
}
.:53 {
    forward . /etc/resolv.conf
}

To try it on your own machine, run it in Docker on port 10053 and set nameservers = ["127.0.0.1:10053"]:

Terminal window
docker run -d --name coredns -p 127.0.0.1:10053:53/udp -p 127.0.0.1:10053:53/tcp \
  -v "$PWD:/etc/coredns:ro" coredns/coredns:1.11.3 -conf /etc/coredns/Corefile

The image runs as a non-root user, so the directory and files must be world-readable. Bump the SOA serial after each edit; the file plugin reloads the zone within a minute.

Check what Bifröst sees

Section titled “Check what Bifröst sees”

  1. Ask DNS directly. Add @10.0.0.2, or @127.0.0.1 -p 10053, to ask one server:

    Terminal window
    $ dig +short TXT _bifrost.infra.example.com
    "v=bf1 node=agent-01 tags=dev,agent"
    "v=bf1 node=agent-02 host=10.0.0.12 tags=dev"
    "v=bf1 node=build-01 host=10.0.0.18 port=2222 user=ci path=/srv/build tags=ci id=vm-4711"

    The values may come back in any order.

  2. Refresh every provider now instead of waiting for the next interval:

    Terminal window
    $ bifrost discover
    PROVIDER  KIND    STATUS  MACHINES  LAST-OK
    static    static  ok      0         -
    infra     dns     ok      3         0s ago

  3. List the machines, then look at one:

    Terminal window
    $ bifrost machines
    NAME      SOURCE  ADDRESS                     STATE    MOUNTED
    agent-01  infra   agent-01.infra.example.com  mounted  yes
    agent-02  infra   10.0.0.12                   mounted  yes
    build-01  infra   10.0.0.18                   mounted  yes
    

    $ bifrost machines show build-01
    id        build-01
    name      build-01
    source    infra
    verdict   allowed (infra.filter.include)
    address   10.0.0.18:2222
    online    unknown
    state     mounted
    tags      ci
    metadata  -
    shadowed  -
    mounts    build-01

  4. Answers are cached for their TTL ($TTL 300 above), so a record you just changed can take that long to show. A machine still missing after that was skipped. The daemon log says why, unless its value lacks node= (see below):

    Terminal window
    journalctl --user -u bifrost | grep bf1

Long records

Section titled “Long records”

One DNS string holds at most 255 bytes. A longer value is several strings in one TXT record, and bf1 joins them with nothing in between, so keep the separating space inside one of the pieces, here at the end of the first. A whole value may be up to 2048 bytes, and each key’s value, the tags= list included, up to 256.

_bifrost            IN TXT ( "v=bf1 node=gpu-01 host=gpu-01.datacenter-eu-west.infra.example.com port=2222 user=ml-runner path=/srv/datasets/imagenet-2012/train/shards "
                             "tags=gpu,cuda,cuda-12,a100,a100-80gb,ml,training,eu-west,rack-12,dev,shared,nightly,long-jobs,team-vision,team-nlp,team-speech,team-robotics id=vm-5821" )

dig shows the pieces:

Terminal window
$ dig +short TXT _bifrost.infra.example.com | grep gpu-01
"v=bf1 node=gpu-01 host=gpu-01.datacenter-eu-west.infra.example.com port=2222 user=ml-runner path=/srv/datasets/imagenet-2012/train/shards " "tags=gpu,cuda,cuda-12,a100,a100-80gb,ml,training,eu-west,rack-12,dev,shared,nightly,long-jobs,team-vision,team-nlp,team-speech,team-robotics id=vm-5821"

A machine is exactly one value. Split a long one into strings, never into two values: two different values for one node are ambiguous, and the node is skipped.

What gets rejected

Section titled “What gets rejected”

Each bad value below is skipped on its own. The rest of the record set still counts, and a skipped node is never partly applied. The nodes= values and _bifrost.db-01 use the index form.

_bifrost            IN TXT "v=bf1 node=dup-01 host=10.0.0.21 host=10.0.0.22"
_bifrost            IN TXT "v=bf1 node=bad-host host=-oProxyCommand=touch${IFS}/tmp/pwned"
_bifrost            IN TXT "v=bf1 node=bad-id host=10.0.0.23 id=../x"
_bifrost            IN TXT "v=bf1 node=twice host=10.0.0.24"
_bifrost            IN TXT "v=bf1 node=twice host=10.0.0.25"
_bifrost            IN TXT "v=bf1 node=both nodes=web-02"
_bifrost            IN TXT "v=bf1 node=web-01 host=10.0.0.26"
_bifrost            IN TXT "v=bf1 nodes=web-01,db-01"
_bifrost            IN TXT "v=bf1 nodes=lab.other"
_bifrost            IN TXT "v=bf1 name=web-03 host=10.0.0.28"


_bifrost.db-01      IN TXT "v=bf1 node=db-01 host=10.0.0.27"
Value Result Logged reason
node=dup-01 node skipped duplicate key "host"
node=bad-host node skipped; ssh never sees the value invalid host "-oProxyCommand=touch${IFS}/tmp/pwned": not an IP literal or hostname
node=bad-id node skipped invalid native id "../x": must match [A-Za-z0-9._:-]{1,128}
two node=twice node skipped as ambiguous ambiguous: 2 distinct bf1 records
node=both nodes=web-02 both skipped; web-02 is never looked up node= and nodes= in one value
node=web-01, and web-01 in nodes= node skipped as ambiguous; never looked up ambiguous: inline and in nodes=
nodes=lab.other that index value is skipped; the other values still count invalid dns label "lab.other": must match [a-z0-9]([a-z0-9-]{0,61}[a-z0-9])?
name=web-03 (no node=) ignored: a root value without node= is an index value, and this one lists no nodes none. no valid bf1 root record appears only when nothing else in the set is valid
_bifrost.db-01 node skipped: node= belongs only in the root record set node= in a node record (root values only)

If nothing valid is left, the provider reports no machines. The same value published twice, even split into strings differently, is not ambiguous.

Large fleets: index + per-node records

Section titled “Large fleets: index + per-node records”

One UDP reply holds 1232 bytes, about 12 to 15 short values (512 bytes, about 5, when Bifröst uses the system resolver on Linux and resolv.conf lacks options edns0). Bifröst retries a truncated answer over TCP, so a bigger record set works as long as your DNS server answers on TCP port 53. If it doesn’t, the lookup fails and the provider keeps its last good view. Use the index form when:

  • a firewall between Bifröst and the server blocks TCP port 53, or
  • each machine publishes its own record, say with a dynamic DNS key scoped to its own name.

An index value lists node labels in nodes=. Each listed node has one record of its own at _bifrost.<node>, with the same keys as an inline value but no node=. Both forms mix in one record set:

_bifrost            IN TXT "v=bf1 node=agent-01 tags=dev,agent"
_bifrost            IN TXT "v=bf1 node=agent-02 host=10.0.0.12 tags=dev"
_bifrost            IN TXT "v=bf1 nodes=build-01,gpu-01"
_bifrost.build-01   IN TXT "v=bf1 host=10.0.0.18 port=2222 user=ci path=/srv/build tags=ci id=vm-4711"
_bifrost.gpu-01     IN TXT "v=bf1 host=10.0.0.40 tags=gpu"
  • A nodes= list holds up to 256 bytes, about 25 short labels, and several index values add up: about 100 indexed machines fit a 1232-byte reply, about 25 a 512-byte one. Each indexed machine costs one more query.
  • 256 machines at most, inline and indexed together.
  • A machine is inline or in nodes=, never both: that is ambiguous, and it is skipped.

Split DNS and custom ports

Section titled “Split DNS and custom ports”

When the zone lives on an internal server that your system resolver doesn’t ask, name the server:

[[discovery]]
type = "dns"
name = "infra"
domain = "infra.example.com"
nameservers = ["10.0.0.2"]            # port 53
# nameservers = ["127.0.0.1:10053"]   # a lab resolver on another port
  • Bifröst then asks only these servers, over UDP and TCP, and caches each answer for its TTL.
  • nameservers is only for the bf1 records. ssh resolves host= names and the <node>.<domain> default with your system resolver, so for a zone only Bifröst asks, publish host=<ip> (or have your system resolver forward that zone).
  • Check against the same server: dig @10.0.0.2 +short TXT _bifrost.infra.example.com.
  • Without nameservers, Bifröst uses the system resolver and re-reads its config on every refresh.

Several environments

Section titled “Several environments”

Two zones, two providers:

[[discovery]]
type = "dns"
name = "staging"
domain = "staging.example.com"


[discovery.filter]
include_tags = ["dev"]


[discovery.mount]
user = "deploy"


[[discovery]]
type = "dns"
name = "lab"
domain = "lab.example.net"
nameservers = ["192.168.1.53"]


[discovery.filter]
include_names = ["lab-*"]


[discovery.mount]
read_only = true

A provider’s name shows in verdicts (allowed (staging.filter.include)), in bifrost discover and in the SOURCE column. A provider without one is named after its type, so two unnamed DNS providers clash:

Terminal window
$ bifrost config check
error: discovery[1].name: duplicate provider name "dns" (a second provider of a type needs a `name`)

Node labels are machine ids across every zone. If both zones publish agent-01:

_bifrost.staging.example.com.   IN TXT "v=bf1 node=agent-01 tags=dev"
_bifrost.lab.example.net.       IN TXT "v=bf1 node=agent-01 host=192.168.1.40"

it is one machine, and as DNS providers are equally trusted, the one whose name sorts first (lab) decides alone. agent-01 is then discover-only, even though staging would allow it, and staging is listed as shadowed. Give each environment its own prefix, such as stg-web-01 and lab-pi-01.