Under Construction: Oryxen
What?
For the better part of 2026, my team and I are building Oryxen. It's an LLM inference system that runs in the "secure enclave" or Trusted Execution Environment (TEE) of AMD EPYC 9000 series CPUs which offer SEV-SNP. It also runs in the Nvidia H100 or newer TEEs of the GPU. We've also started trials with a fully homomorphic encryption (FHE) ASIC/FPGA young company to run "secure RAG" and/or a full large language models (LLM) inside FHE.
If none of that makes sense to you, the following may: I started Oryxen with the idea of private and secure/attestable "AI" because I wanted to create the venn diagram of overlapping privacy enhancing technologies (PETS) with large language model inference and creation; aka, "AI". The hardware guarantees are backed by the Terms of Service and Privacy Policy.
We serve open weight models now. We build custom models for customers. We use Post Quantum Cryptography for all the cryptographic algorithms.
Why?
I want to make data as toxic as nuclear waste, not as it currently is, the new oil. I want to put the user in control of what they disclose to us, not make disclosure mandatory. I want to support private payment methods for AI.
Privacy should be the default, not the exception. The same beautiful Damascus steel carving knife that makes wonderful sushi or filets the mellezane can also be used to do horrible things to people. It's the person who wields that knife that makes the difference. Bad actors will always do bad things. The world needs private, secure AI. We don't want your data, in fact we can't see it.
The needs of the many outweigh the needs of the few..or the one.
Commander Spock, USS Enterprise, NCC-1701, Science Officer. Star Trek II: The Wrath of Khan, directed by Nicholas Meyer (Paramount Pictures, 1982), 1:37:02.
We are bootstrapped. We've taken no outside investment. We control our own destiny.
How?
I architected this solution a few years ago for a client, where they paid for all of the hardware in the prototype. It worked so well, they ordered another one. The original goal was to build a system that could work completely disconnected from any network by a user sitting at a terminal. At that time, my value was to design, test, and implement a series of scripts to control the models, attestation from the TEEs, and get it working end to end. I trained their sysadmins/devops on how to run the system, and update the models.
We've since improved upon those designs with a solution written in safe Rust. It runs on linux for now, but could run on the BSDs if the support for the GPU hardware exists. It runs on ARM CPUs within the TrustZone. The beauty of Rust is the easy cross-compilation for different architectures which scales from low power single board computers to multi-CPU enterprise servers.
The other value added is we went with a simple API key model. No registration beyond creating the API key. This means, no email, no phone number, private payment methods, and it includes tor onion and yggdrasil endpoints. As customers ask for more payment methods or endpoints, we'll seriously consider the pros/cons to adding more complexity to a fairly simple infrastructure. I don't have enough funds to subsidize usage, so it's a simple pay-as-you-go model based on US$10 per one million tokens. The tokens are yours forever. Some people use them in three months, some use them in an hour. You get real human support via Signal and DeltaChat right now. Future platforms could include XMPP, SimpleX, or private webIRC with ephemeral nicknames.
Generating and using an API key doesn't mean solely an OpenAI-compatible API, with optional HPKE support, it also means we built a simple web interface where your API key is the only access. The benefit of API key first (and only) is that you can use our web interface or not. Use your own web interface, agentic platform, etc via the API endpoint. Futures include tying your account to a hardware key, like a Nitrokey or Yubikey. This would support two hardware keys to the same account, so if one key is lost/fails you don't lose your tokens.
All your queries, images, videos, etc are ephemeral. If you want to keep it, use our export function or your own interface to our API. The private keys are on your system, we never see them. Every query is independent and ephemeral.
We started with passkeys, but that quickly became a mess as users/devices would forget or lose their passkey and therefore all their tokens. We tried various wallet solutions, like metamask, but that's more software we have to work with and possibly maintain on multiple operating systems.
We simply want to collect the minimum data points about you as possible. It's up to you if you want to disclose more. We have worked with enterprise clients who wanted to work more closely with us. This is great. It's by design we collect as little as possible, but that doesn't mean it has to stay that way. When someone is doing a billion plus tokens per month, and wants more dedicated support and consulting, you get to know them. It's their choice.
Current
The current prototype system is sort of a mash up of on-device, bare metal models running with an AMD GPU, rented Nvidia B100/B200 GPUs with TEEs, and a proxy set of providers/models with zero data retention policies all proxied through the Oryxen Rust processes. We've worked with some providers to offer dedicated TEEs (CPU and GPU) and HPKE/EBHP between our TEE and theirs. We've also worked with providers offering attestable zero data retention (ZDR) over pinned TLS certs to their API end point.
For transparency, we built the system on a pair of AMD GPUs on the PCIe bus in our AMD EPYC server. The AMD GPUs do not have TEEs, so the user has to trust the CPU TEE, the PCIe bus, and the AMD GPUs aren't recording traffic. They aren't. Our chassis intrusion would detect if someone opend the case (nevermind they'd have to power off the server as the board purposely doesn't support hot-plug PCI devices) to change/insert something on the PCIe bus. We futher built the system using rented AMD EPYC servers with Nvidia H100 and B200 GPUs which do have a TEE. We're confident the system scales up to as many TEEs as possible.
Altogether, we offer around 200 different models for text, image, video, and audio analysis and generation. We offer these through tiered levels of user selectable privacy: our bare metal in our datacenter rack, dedicated rented GPUs with TEEs, trusted TEE providers, and ZDR providers.
Early this year, we bought a nice production server at a discount, which was really a hedge against the expected increases in memory, cpu, and storage. We've been developing on a number of different systems, with and without CPU/GPU TEEs to make the point we can do this at varying levels of attestation and security (and power vs performance tradeoffs).
Here's a table that "scc" produces based on the current codebase:
───────────────────────────────────────────────────────────────────────────────
Language Files Lines Blanks Comments Code Complexity
───────────────────────────────────────────────────────────────────────────────
SVG 56 7,280 0 1,201 6,079 15
DOT 54 2,295 373 237 1,685 0
Rust 51 28,217 2,990 2,809 22,418 1,599
Markdown 32 13,975 3,107 0 10,868 0
Shell 24 3,541 462 519 2,560 520
JavaScript 22 6,234 688 846 4,700 711
TOML 14 451 51 133 267 5
HTML 11 1,865 132 13 1,720 0
Fish 6 827 62 84 681 36
JSON 5 1,752 0 0 1,752 0
Plain Text 4 8,914 791 0 8,123 0
Python 4 396 48 54 294 34
SQL 4 78 11 41 26 0
CSS 3 2,403 344 71 1,988 0
Julia 3 973 25 33 915 28
Systemd 3 156 33 36 87 0
CSV 1 230 0 0 230 0
YAML 1 24 1 1 22 0
───────────────────────────────────────────────────────────────────────────────
Total 298 79,611 9,118 6,078 64,415 2,948
───────────────────────────────────────────────────────────────────────────────
Estimated Cost to Develop (organic) $6,092,376
Estimated Schedule Effort (organic) 18.37 months
Estimated People Required (organic) 10.36
───────────────────────────────────────────────────────────────────────────────
Processed 3736214 bytes, 3.736 megabytes (SI)
───────────────────────────────────────────────────────────────────────────────Future
Short term (1-3 months)
- "Go live" on the aforementioned fancy new servers and move from beta into production.
- Fine-tune the entire stack for performance under production load.
- Publish the source code, possibly with audits (Kerchoff's principle)
- Figure out how to make attestion of the TEE understandable by normal humans.
- Figure out a secure, private, and optimally, self-hosted bitcoin/monero payments system.
- Setup other web interfaces/apps by default for users who want more control/history/power, but still their privacy and security.
Medium Term (3-6 months)
- Find more TEE providers with attestation
- Work with UX/UI people to make sure our docs, website, and web app are usable by non-tech users.
- Buy 2 newer generation GPUs with TEEs that fit in the aforementioned servers (and don't consume all the amps in the datacenter rack)
Longer Term (6 months +)
- Implement and deploy production-ready FHE chips or via trusted FHE providers
- Scale to more servers/GPUs as needed
- Iterate on web app, doc, website usability
- Possibly offer US "Berry Act compliant" models
- Possibly offer EU-sourced models, hosting, hardware via an EU registered company
- Build the system in a box and shrink it down to the minimum necessary to serve offline/DDIL situations as a deployable appliance.
- Move to solely FHE LLMs as hardware becomes available