Skip to content

Implement a custom credential backend

credentials ships two backends out of the box: a stub that returns ErrCredentialUnsupported (the regulated default) and a go-keyring wrapper in credentials/keychain (desktop/laptop). Tool authors who need a different secret store — Hashicorp Vault, AWS Secrets Manager / SSM Parameter Store, GCP Secret Manager, 1Password Connect, a bespoke corporate store — implement the Backend interface and register it at process startup.

This guide walks through the full shape of a custom backend using Vault's KV v2 engine as the worked example. The same pattern applies to any store.

This is by design a tool-author concern

The package deliberately ships no bundled vendor adapters — Vault/SSM/etc. pull heavy, deployment-specific SDKs that do not belong in a framework-free core. Backend is the extension point; you own the adapter for your store.

When to build one

Consider a custom backend when:

  • Your organisation mandates a specific secret store (Vault, cloud KMS, internal HSM-backed service) for the credentials your tool handles.
  • You want centralised rotation, audit logs, or policy-based access the OS keychain cannot provide.
  • You run many instances of your tool across a fleet and need a single source of truth.

Stick with the built-ins when you are a single-user desktop tool (OS keychain is fine) or only need env-var references (already the default mode).

The Backend contract

type Backend interface {
    Store(ctx context.Context, service, account, secret string) error
    Retrieve(ctx context.Context, service, account string) (string, error)
    Delete(ctx context.Context, service, account string) error
    Available() bool
}

Required semantics for each method:

Method Must do Must not do
Store Write a secret under service/account. Overwrite any existing entry. Return ctx.Err() on cancellation before commit. Log secret. Embed secret in error messages.
Retrieve Return the stored value when present. Return ErrCredentialNotFound when the backend is reachable but the entry does not exist. Wrap other failures. Signal "missing" by returning an empty string with nil error — resolvers rely on the sentinel to fall through cleanly.
Delete Remove the entry if present. Return nil when it is missing (idempotent). Surface a "not found" error — it confuses callers that re-run setup.
Available Report whether the backend could satisfy calls right now, from a cheap static check (initialised, token present, pool ready). Perform I/O — that is what Probe is for.

Return-value conventions:

  • ErrCredentialNotFound — backend healthy, entry absent. Callers fall through to the next resolution step.
  • ErrCredentialUnsupported — the stub backend. Callers fall through.
  • Any other error — a real failure (auth, network, permission), surfaced by direct Store/Retrieve/Delete callers such as a setup flow.

Worked example: Hashicorp Vault (KV v2)

Uses the official github.com/hashicorp/vault/api client, KV v2 at mount secret/, token auth. The structure adapts to approle, Kubernetes, or cloud IAM auth with only the constructor changing.

1. Skeleton

Create yourtool/credentials/vault/backend.go:

// Package vault provides a credentials.Backend backed by Hashicorp
// Vault's KV v2 secrets engine.
package vault

import (
    "context"
    stderrors "errors"
    "fmt"

    "github.com/cockroachdb/errors"
    vaultapi "github.com/hashicorp/vault/api"

    "gitlab.com/phpboyscout/go/credentials"
)

// Backend writes secrets under <mount>/data/<prefix>/<service>/<account>.
// KV v2 places each secret's payload under a "data" key; we stash the
// single-value secret as {"value": "<secret>"} to match the simple
// Store/Retrieve/Delete contract.
type Backend struct {
    client *vaultapi.Client
    mount  string // e.g. "secret" (no leading/trailing slash)
    prefix string // e.g. "mytool" — namespaces the tool's entries
}

// Config configures the Vault backend.
type Config struct {
    Address string // VAULT_ADDR; e.g. "https://vault.corp:8200"
    Token   string // VAULT_TOKEN; the app's token
    Mount   string // KV v2 mount path; defaults to "secret"
    Prefix  string // namespace inside the mount; defaults to "creds"
}

// New builds a Backend from Config. Vault round-trips happen lazily on
// first call.
func New(cfg Config) (*Backend, error) {
    vc := vaultapi.DefaultConfig()
    if err := vc.Error; err != nil {
        return nil, errors.Wrap(err, "default vault config")
    }

    if cfg.Address != "" {
        vc.Address = cfg.Address
    }

    client, err := vaultapi.NewClient(vc)
    if err != nil {
        return nil, errors.Wrap(err, "vault.NewClient")
    }

    if cfg.Token != "" {
        client.SetToken(cfg.Token)
    }

    mount := cfg.Mount
    if mount == "" {
        mount = "secret"
    }

    prefix := cfg.Prefix
    if prefix == "" {
        prefix = "creds"
    }

    return &Backend{client: client, mount: mount, prefix: prefix}, nil
}

// path returns the KV v2 logical path for a secret.
func (b *Backend) path(service, account string) string {
    return fmt.Sprintf("%s/data/%s/%s/%s", b.mount, b.prefix, service, account)
}

2. Implement the methods

// Store writes a KV v2 entry with payload {"value": secret}.
func (b *Backend) Store(ctx context.Context, service, account, secret string) error {
    _, err := b.client.Logical().WriteWithContext(ctx, b.path(service, account), map[string]any{
        "data": map[string]any{"value": secret},
    })
    if err != nil {
        // Never include `secret` in the wrapped error — Vault sometimes
        // echoes the payload in diagnostics.
        return errors.Wrapf(err, "vault.Write %s/%s", service, account)
    }

    return nil
}

// Retrieve reads a KV v2 entry. Returns ErrCredentialNotFound when Vault
// is reachable but the path has no secret.
func (b *Backend) Retrieve(ctx context.Context, service, account string) (string, error) {
    sec, err := b.client.Logical().ReadWithContext(ctx, b.path(service, account))
    if err != nil {
        return "", errors.Wrapf(err, "vault.Read %s/%s", service, account)
    }

    if sec == nil || sec.Data == nil {
        return "", credentials.ErrCredentialNotFound
    }

    data, ok := sec.Data["data"].(map[string]any)
    if !ok || data == nil {
        return "", credentials.ErrCredentialNotFound
    }

    v, ok := data["value"].(string)
    if !ok {
        return "", credentials.ErrCredentialNotFound
    }

    return v, nil
}

// Delete removes the current version. Idempotent: Vault reports no error
// whether the entry was present or already absent.
func (b *Backend) Delete(ctx context.Context, service, account string) error {
    _, err := b.client.Logical().DeleteWithContext(ctx, b.path(service, account))
    if err != nil {
        if stderrors.Is(err, vaultapi.ErrSecretNotFound) {
            return nil
        }

        return errors.Wrapf(err, "vault.Delete %s/%s", service, account)
    }

    return nil
}

// Available reports whether the backend is ready. Cheap check: a token
// is set. For a true liveness signal use credentials.Probe.
func (b *Backend) Available() bool {
    return b.client != nil && b.client.Token() != ""
}

3. Register at startup

Register the backend before the first credential call. Two patterns:

// yourtool/cmd/yourtool/vault.go — delete this file to fall back to the stub.
package main

import _ "yourtool/internal/vaultinit"

// yourtool/internal/vaultinit/vaultinit.go
package vaultinit

import (
    "log"
    "os"

    "gitlab.com/phpboyscout/go/credentials"
    "yourtool/credentials/vault"
)

//nolint:gochecknoinits // side-effect registration is the whole point
func init() {
    b, err := vault.New(vault.Config{
        Address: os.Getenv("VAULT_ADDR"),
        Token:   os.Getenv("VAULT_TOKEN"),
        Prefix:  "yourtool",
    })
    if err != nil {
        log.Fatalf("vault backend: %v", err)
    }

    credentials.RegisterBackend(b)
}

func main() {
    cfg := loadConfig() // your own plumbing
    b, err := vault.New(vault.Config{
        Address: cfg.VaultAddr,
        Token:   cfg.VaultToken,
        Prefix:  cfg.VaultPrefix,
    })
    if err != nil {
        log.Fatal(err)
    }

    credentials.RegisterBackend(b)
    // ... run your CLI ...
}

The blank-import form composes with the keychain opt-out file — both register their backend via init(), and the later registration wins. Use the explicit form when construction is config-driven (a flag selecting Vault vs OS keychain).

4. Test it

For unit tests of code that depends on a backend (resolvers, setup flows), do not stand up real Vault — use the in-process credtest.MemoryBackend:

import (
    "gitlab.com/phpboyscout/go/credentials"
    "gitlab.com/phpboyscout/go/credentials/credtest"
)

func TestResolverReadsFromBackend(t *testing.T) {
    credtest.Install(t) // registers a fresh MemoryBackend, restores the stub on cleanup

    require.NoError(t, credentials.Store(t.Context(), "mytool", "github.auth", "ghp_test"))
    // ... drive your resolver and assert it picks up the stored value.
}

Round-trip integration tests for the real Vault implementation belong behind an env-var gate, e.g. INT_TEST_VAULT=1 go test ./yourtool/credentials/vault/....

Composing backends

To "try Vault first, fall back to OS keychain", write a wrapping Backend — the registry holds one backend at a time:

type ChainedBackend struct {
    backends []credentials.Backend
}

func (c *ChainedBackend) Retrieve(ctx context.Context, service, account string) (string, error) {
    var lastErr error
    for _, b := range c.backends {
        v, err := b.Retrieve(ctx, service, account)
        if err == nil {
            return v, nil
        }
        lastErr = err // ErrCredentialNotFound or unavailable — try the next
    }
    return "", lastErr
}

// Store typically writes to the first Available() backend only.

credentials.RegisterBackend(&ChainedBackend{
    backends: []credentials.Backend{
        mustBuildVaultBackend(),
        keychain.Backend{}, // from credentials/keychain
    },
})

Current limitations

  • No built-in retry. Transient network errors surface as-is; wrap your client with retry logic before registering.
  • No built-in caching. Each Retrieve hits the backend; caching is your concern — mind token rotation.
  • Single active backend. Composition is via a wrapping Backend, not a registration chain.
  • Thin error taxonomy. ErrCredentialNotFound and ErrCredentialUnsupported are the only sentinels; distinguish auth vs permission failures via errors.As on your own error types.