Entropy in Trading
When you think of market pioneers, names like Warren Buffett or George Soros might come to mind.
But the architect of many modern trading strategies can be traced back to someone who wasn’t a financier – Claude Shannon, the mathematician who laid the groundwork for information theory, which is behind every digital device you use today.
Shannon spent decades studying financial markets in his spare time, applying probabilistic models and entropy concepts to uncover patterns in stock prices and investing strategies.
Key Takeaways – Entropy in Trading
- Markets Are Full of Noise – Most trading strategies do poorly because they target high-entropy (random) environments with no discernible edge – or sometimes do but have poor risk management. Understanding entropy helps traders separate signal from noise and avoid unprofitable strategies.
- Probabilities, Not Predictions – Markets can’t be predicted with certainty, but probability-based strategies with measurable edges (in whatever form) improve long-term success.
- Risk Management Over Accuracy – Even a profitable system can fail without proper bet sizing. Use techniques like the Kelly Criterion to manage risk based on entropy levels.
- Backtests Can Be Deceptive – Many backtested strategies are just noise-fitting. Without stress-testing against entropy and Monte Carlo simulations, traders risk overfitting to randomness.
From Information Theory to Market Predictions
Shannon’s 1948 paper, A Mathematical Theory of Communication, didn’t just revolutionize tech – it quietly reshaped finance.
His concept of entropy (a measure of disorder) explains why most trading strategies fail: they target markets drowning in randomness.
Why Traders Overlook Probabilities/Ranges
Most beginners chase “secret indicators,” rather than understanding markets from a probabilistic perspective.
Beginners also often overestimate their ability to predict market movements, without recognizing that markets are complex, changing systems where perfect accuracy is impossible.
Shannon’s MIT lectures on stock pricing (now buried in archives) focused not on patterns, but on uncertainty quantification – which is very important to understand.
Becoming a great trader is analogous to running a casino – you don’t know which individual trades will win on a given day, just like a casino can’t predict which individual hands of blackjack will be profitable.
It’s about consistently applying a structural edge – your strategy and risk management.
Just as casinos rely on the house edge across thousands of bets (different tables, different games, different locations), traders succeed by sticking to a proven strategy over many trades, managing risk, and letting statistical advantages play out.
It’s certainly not like playing in a casino where the edge is against you.
It isn’t predicting individual outcomes but ensuring that, over time, the edge works in your favor.
The Quant’s Trap
Asking the Wrong Questions
A novice quant might ask, “Should I use a neural network or a random forest?”
Shannon might reply: “Does your target variable have an exploitable structure?”
The truth?
No algorithm can extract signal from noise if the system’s entropy is too high.
Shannon’s Three Critical Questions
Shannon’s framework forces traders to confront:
- What exactly are you predicting? (e.g., Bitcoin’s 5-minute volatility, not “the market”)
- How predictable is it? (Calculate its entropy, which is a function of its variance)
- What’s the cost of being wrong? (Risk management > prediction accuracy)
Shannon’s Entropy: The Trader’s Uncertainty Meter
What Is Entropy?
Imagine two slot machines:
- Machine A pays out 50% of the time (high entropy: chaotic).
- Machine B pays out 80% after 3 losses (low entropy: predictable).
Entropy (H) quantifies this difference mathematically:
H = -Σ (p_i * log2 p_i)
For traders, this formula separates gambles (H ≈ 1) from edges (H < 0.7).
Markets as Noisy Communication Channels
Shannon modeled information transfer through noisy channels – like sending a text in a storm.
Applying this to trading:
- Low-entropy signal – Institutional order flow in Apple stock (clear patterns).
- High-entropy noise – Robinhood traders piling into meme stocks (chaotic, more or less unpredictable).
The Illusion of Profit: How Randomness Deceives Traders
Example: The Three Strategy Illusion
Consider three strategies:
These profit/loss graphs show a strategy’s performance over one year (around 252 trading days for most stocks and bonds).
Graph A
This strategy did poorly, down more than 20% after one year, and at one point had a drawdown of nearly 30%.
Graph B
This strategy had mixed results.
It was up almost 20%, before losing the gains (and going below breakeven), before finishing up around 9%.
Graph C
This strategy was best, up over 30% at one point before finishing up around 25% for the year.
After an initial dip at the start of the year, it did well.
Which Strategy Is Best?
The Graph C strategy is, right?
The reality is… these strategies are the exact same.
This is just a very basic strategy where a trader starts with $10,000, bets 1% of his balance on each trade, and has a 50/50 likelihood of being right or wrong with a binary outcome – not far off what most traders deal with.
It’s essentially a coin-flipping strategy.
The Deception of Backtesting
A strategy with, e.g., 55% accuracy might look profitable in backtests.
But without entropy analysis, you’re likely overfitting to noise.
Markets have path dependency – a single Black Swan event can obliterate paper gains.
We covered how long to backtest a strategy here and randomness in markets in more detail here.
The Real Work: Hunting Low-Entropy Variables
Step 1: Uncovering Predictable Patterns
Focus on variables with measurable structure.
For example:
- Order flow imbalance – Large trades leave footprints (e.g., dark pool prints).
- Volatility cycles – Post-earnings calm after IV crush.
- Sentiment extremes – When Reddit’s WallStreetBets hits 1M mentions, mean reversion follows.
Step 2: Stress-Testing Against Entropy
Calculate your target variable’s entropy over 1,000+ samples. Example:
- If SPY has a 53% chance of rising the day after a 2% drop (H = 0.7), proceed.
- If Bitcoin’s 10-minute moves are 50/50 (H = 1), abandon it.
Step 3: Smart Bet Sizing with the Kelly Criterion
Shannon’s colleague John Kelly solved the “how much to bet” problem:
f* = (p*b – q) / b
- f* = % of capital to risk
- p = win probability
- q = loss probability
- b = reward/risk
High entropy? Reduce f* aggressively.
The Kelly Criterion generally bets too much in trading contexts anyway, so concepts like “half-Kelly” and “partial-Kelly” come into play.
Building a Shannon-Inspired Trading Framework
As examples:
1. Map Market Entropy
Use rolling 30-day windows to rank assets by entropy.
Trade EUR/USD at 2 AM EST? Its entropy spikes during low liquidity. Stick to high-volume NYSE open periods instead.
2. Create Predictability Filters
Screen out noise:
- Avoid earnings announcements (high entropy).
- Trade only when VIX > 30 (panic creates order).
- Focus on stocks with >1M daily shares (liquidity = lower entropy).
3. Dynamic Position Sizing
Adjust weekly:
- If entropy drops 20%, increase position size by half-Kelly.
- If entropy spikes, cut exposure by 75%.
4. Validate with Monte Carlo Simulations
Run 10,000 simulations of your strategy.
If the edge is there but any of the paths blow up your account, figure out the risk management.
Conclusion
Claude Shannon’s genius wasn’t in “predicting the future” (as many might think trading is about) – it was in quantifying uncertainty.
Your edge ultimately lies in identifying pockets of predictability others ignore.
Action Plan
- Audit your strategy – Calculate its entropy. If H > 0.8, pivot.
- Trade silence – The biggest edges emerge when others panic (high entropy = opportunity).
- Let math guide greed – Use Kelly to avoid ruin.
Shannon once said, “Information is the resolution of uncertainty.” In trading, that resolution pays the bills. Stop chasing noise. Start measuring it.