Why Neural Networks and Cryptographic Ciphers Are So Similar

This connection becomes hard to ignore after reading about some 2024 research where cryptographers were using neural network analysis techniques to attack cipher structures. That shouldn't work. Or at least, it shouldn't work as well as it apparently does. I keep coming back to why these two things — one from machine learning, one from mathematics — keep showing up in each other's papers.

The answer, once you look at it structurally, is that they're solving the same problem from opposite directions. Both take an input, run it through layers of nonlinear transformation, and produce an output that looks nothing like where you started. They differ in intent and control. The architecture is nearly identical.

Rounds and layers: the same structure, different intent

A block cipher like AES takes a 128-bit plaintext block and a key (which may be 128, 192, or 256 bits), runs it through 10 rounds of substitution and permutation for a 128-bit key, and produces a 128-bit ciphertext. Each round applies a substitution layer (the S-box), a permutation layer (ShiftRows, MixColumns), and key material. The output of one round feeds into the next. By the end, every output bit has a complex statistical relationship with every input bit.

A feedforward neural network takes an input vector, multiplies it by a weight matrix, applies a nonlinear activation function, and feeds the result into the next layer. Repeat for however many layers you have. By the end, you have an output that is a deeply nonlinear function of the input.

Write them side by side and the resemblance is hard to ignore. Rounds map to layers. S-boxes and activation functions both introduce nonlinearity. Key addition and weight multiplication are both affine transformations applied at each stage. The diffusion properties cryptographers measure in ciphers — how much a single input bit change propagates through the output — have direct analogues in how gradients flow through neural network layers during backpropagation.

Nonlinearity is doing the heavy lifting in both

In cryptography, a cipher built entirely from linear operations is trivially breakable. Given enough plaintext-ciphertext pairs, an attacker can solve a system of linear equations and recover the key. This is not theoretical — linear cryptanalysis is a real attack class, and the reason ciphers add S-boxes is specifically to introduce algebraic nonlinearity that resists it.

A neural network with no nonlinear activations — just stacked linear transformations — collapses to a single matrix multiplication regardless of how many layers you add. Depth without nonlinearity buys nothing. ReLU, sigmoid, tanh, GELU: all of them exist for the same reason AES has S-boxes. Without them, the model can only learn linear relationships, and almost everything interesting is nonlinear.

The AES S-box is specifically designed to have high nonlinearity: it is derived from the multiplicative inverse over GF(2^8) combined with an invertible affine transformation, a construction known to maximize algebraic nonlinearity and resist both linear and differential cryptanalysis. Neural network activation functions aren't designed with that formalism in mind, but the functional requirement is identical. Break linearity so the composition of layers can represent complex, non-affine mappings.

One-wayness and the problem of inversion

Ciphers: designed to be invertible with the key, hard without it

A block cipher is a keyed pseudorandom permutation. Given the key, decryption is efficient and exact — AES decryption applies the inverse operations (InvSubBytes, InvShiftRows, InvMixColumns) traversing the round keys in reverse order. Without the key, recovering the input from the output should be computationally infeasible. The security proof is essentially: this function behaves like a random permutation to anyone who doesn't know the key.

Neural networks: hard to invert by accident

Neural networks, especially deep ones, are also hard to invert — but not by design in the same sense. Given a trained network and an output, finding an input that produces it is generally a difficult optimization problem. This is why adversarial examples work: you optimize the input rather than the model, finding small perturbations that flip the output. The network's parameters are fixed; you're solving for x in f(x) = y.

This hardness shows up in some security-adjacent contexts. Researchers have explored whether neural networks can approximate hash functions or serve as components in cryptographic constructions. The results are mixed. A neural network trained to approximate SHA-256 outputs will get close on training data and generalize poorly — which tells you something about how the hardness properties differ between the two.

Where the two fields have started borrowing from each other

Neural networks attacking ciphers

The 2019 Gohr paper on applying neural networks to differential cryptanalysis of Speck is the one that made working cryptographers take this connection seriously. Gohr trained a neural network to distinguish Speck ciphertext pairs with a specific input difference from random ciphertext pairs, and it outperformed classical differential distinguishers for reduced-round variants. The network wasn't reasoning symbolically about the cipher's structure — it was learning statistical patterns in the ciphertext that correspond to the cipher's algebraic weaknesses.

Since then there's been a steady stream of follow-on work: deep learning distinguishers applied to Simon, Simeck, PRESENT, and various other lightweight ciphers. The technique doesn't break production-strength ciphers, but it's narrowing the gap between what's theoretically distinguishable and what's practically attackable for reduced-round versions. Cryptographers are paying attention.

Neural cryptography

Going the other direction: can you build a cryptographic primitive out of neural networks? There's been research on this since at least the early 2000s, when Kanter, Kinzel, and colleagues showed that two neural networks can synchronize weights through a public exchange protocol and derive a shared secret — effectively a key agreement protocol built from neural networks trained via mutual Hebbian learning on each other's outputs.

These constructions haven't displaced conventional cryptography, and they're unlikely to in the near term. Formal security proofs for neural cryptographic systems are hard to produce. You can't reduce the security of a neural network construction to a standard hardness assumption the way you can with RSA or elliptic curve systems. That's a real problem for anyone who needs to defend the security of a production system.

Side-channel attacks on both

Both ciphers and neural networks are vulnerable to side-channel attacks, and the attack patterns are analogous. Power analysis on cipher hardware reveals information about key operations through physical measurements. Model extraction attacks on neural networks — querying a black-box model to reconstruct its weights — are structurally similar. In both cases, the algorithm is fine; the implementation leaks information that isn't supposed to be observable.

Differential power analysis against AES implementations and model stealing attacks against commercial ML APIs are both examples of the same principle: the computation itself may be secure, but running it on physical hardware or through an API surface creates measurement channels you didn't account for.

Where the analogy breaks down

The design philosophies diverge sharply, and that's what matters.

Ciphers are designed with formal security goals: IND-CPA security, IND-CCA2 security, resistance to specific attack classes. Every design decision is evaluated against those definitions. AES's S-box wasn't chosen because it performed well on a benchmark — it was chosen because it maximizes algebraic nonlinearity and differential uniformity by formal measures. The security is provable against specific adversary models.

Neural networks are optimized empirically. The activation function is chosen because it trains well and generalizes well. There's no formal theorem that a trained ResNet is secure against any particular adversary. The model that achieves 97% accuracy on ImageNet has no security guarantee. It just works, until it doesn't.

This is why using neural networks as drop-in replacements for cryptographic components is dangerous. Behavioral similarity doesn't imply security equivalence. A network that looks like a cipher from the outside may have exploitable structure that a classical cryptanalyst would find immediately — and that's the constraint most likely to derail neural cryptography as a serious discipline, not the underlying mathematics.

Why this matters now

Large language models are being deployed in contexts where they process sensitive data, and there are active research questions about whether training data can be extracted from model weights — which is essentially asking whether the weights constitute a lossy, non-invertible encoding of the training set. Membership inference attacks and data extraction attacks on LLMs are the current frontier of that question.

On the other side, post-quantum cryptography is pushing cryptographers to borrow intuitions from learning theory. The Learning With Errors problem, which underlies key post-quantum algorithms like CRYSTALS-Kyber, is formally related to a machine learning problem stated in the wrong direction: instead of finding weights that explain observed data, the adversary is trying to find a secret that explains noisy linear observations. The hardness assumption is that this is computationally infeasible. The connection to PAC learning is explicit in the literature.

I genuinely don't know how this plays out over the next decade. The research community treating neural networks as objects of cryptographic study, and cryptographic hardness problems as machine learning problems, is still young. The 2024 work using neural distinguishers against lightweight ciphers suggests the boundary is more porous than most would have expected five years ago.

The short version: both neural networks and ciphers are layered nonlinear transformations with hard-to-invert outputs. The difference is intent and rigor. Ciphers are engineered to specific formal security properties. Neural networks are optimized to generalize. Those are different goals, even when the resulting architectures look remarkably alike.

Understanding the overlap is increasingly useful for anyone working at the intersection of ML systems and security — which, right now, is most of us.

Working at the intersection of this in practice — adversarial ML, post-quantum implementation, side-channel analysis? Tell me what you're actually seeing.