The problems where the right answer is a rule

A confession the field does not make often

A large share of shipped machine learning could be replaced by a few hundred lines of ordinary code with no loss of accuracy and a large gain in everything else. Knowing when you are in that situation is a skill, and it is one that makes you more employable rather than less, because the people paying for this work have usually been burned once.

Five situations where a rule wins

1. The rule is already known and exact. Tax brackets. Shipping bands. Eligibility criteria written in a regulation. These are not patterns to be discovered; they are specifications. A model that learns them will get 99.4% of them right, which is strictly worse than the if statement that gets 100%.

2. You have almost no labelled data. Under a few hundred labelled examples with any real complexity, a model’s variance swamps its signal. You will not be able to tell a good model from a lucky one, because your validation set is fifty rows and the confidence interval on any score is enormous. Write the rule, ship it, and let it collect labelled data for the model you build next year.

3. Every decision must be explainable to the person affected, exactly. Some jurisdictions require a specific reason for an adverse decision. A five-condition rule gives you that for free. A gradient boosting model with 400 trees gives you a SHAP plot, which is an approximation of what the model did, and there are contexts where an approximation is not acceptable.

4. The cost of one wrong answer is catastrophic and unbounded. Not “expensive”: catastrophic. A model is a statistical device with a failure rate. If the failure rate must be provably zero on a class of inputs, that class needs a rule, possibly wrapped around a model that handles the rest.

5. The pattern changes faster than you can retrain. If the rules of the game change weekly, a model trained on last month’s data is describing a world that no longer exists, and your monitoring will tell you so a week late.

Five situations where a model wins

The symmetric list, because the point is judgement rather than scepticism.

• The rules exist but nobody can write them down. Handwriting, speech, whether a photo contains a cat. Humans do it effortlessly and cannot articulate how. This is the original case for machine learning and it is still the strongest.
• The rule list has grown past maintenance. When the spam filter has 1,400 conditions and every new one breaks two old ones, you have hand-fitted a model badly. Let the optimiser do it.
• You need a probability, not a decision. Rules give you yes and no. Many businesses need “73% likely”, so they can rank, prioritise or set a threshold that moves with capacity.
• The relationship is multivariate. Twelve weak signals that only mean something in combination. No human writes that rule; a model finds it in an afternoon.
• Personalisation at a scale no human can staff. A different ranking for each of two million users is not a rule problem.

The hybrid is usually the right shape

Most good production systems are not one or the other. They are a rule layer and a model layer with a stated division of labour.

if amount > 500000 and account_age_days < 7:
    route_to_human()        # hard rule, never overridden
elif model.predict_proba(x)[1] > 0.82:
    route_to_review()       # model decides in the grey zone
else:
    approve()

The rules cover the cases you already understand and the ones you cannot afford to get wrong. The model covers the enormous middle where nobody can write the condition. This shape is easier to reason about, easier to explain, and it degrades sensibly: if the model service goes down, the rules still run.

Addaly’s own content moderation is built this way, and the reason is in the failure mode: a rule layer that catches the unambiguous cases means an AI outage does not become a safety outage.

The question to ask first

Before any project, ask: if I gave a careful, experienced person the same information, could they write down how they decide?

If yes, and the list is short, write the list. If yes, but the list would run to hundreds of interacting conditions, build a model. If no (if the person says “I just know”) build a model, and expect to need more data than you think.

The answer is not always flattering to the project. Saying so early is the cheapest thing you will ever do for a team.