Backend.AI Confidential AI:
Your Model and User Data,
Hidden from the Host

This document is written for security teams and AI platform teams that serve their own models on GPU servers operated by a cloud provider or another organization and are evaluating how to protect model weights and user data there. It is especially suited to organizations that run model services handling prompts with personally identifiable information (PII) or source code on external GPU infrastructure. Organizations that want to replace contractual assurances from the operator with hardware-based attestation, or that need model keys managed through an existing enterprise key manager or hardware security module (HSM), can also evaluate it.

Backend.AI uses the standard Confidential Containers (CoCo) components, including Kata Containers, the CoCo guest components and Trustee as the key broker. Its own scheduler and node agent place confidential workloads directly on bare-metal servers, so you deploy a confidential model the same way as any other Backend.AI model service. The current scope is a single node with one whole NVIDIA GPU per confidential workload, at sites with internet access. The roadmap includes multi-GPU confidential serving, confidential sessions across several nodes, confidential storage, and AMD SEV-SNP support. See the full document for a comparison of a self-hosted verifier with Intel Trust Authority and an example of launching a confidential workload.