The card as an asset, and the sunk-cost trap

Purchase price is not cost

A card you buy for 1,000 and sell for 500 after two years cost you 500 to use for two years, not 1,000. The figure that belongs in a cost-per-token calculation is:

capital cost per month = (purchase price - expected resale price) / months held

This matters because it is usually the largest single line, and because the denominator is a choice you make rather than a fact you look up. The same card, over twelve months, costs more than twice as much per month as it does over thirty-six. Whichever number you pick, write it down and state it, because a cost comparison whose amortisation window is unstated can be made to say anything.

A sensible default for planning is thirty-six months, with an honest acknowledgement that the reason for replacement is rarely failure. It is that a new model you want does not fit.

Why consumer GPUs hold value unusually well

Most computing hardware depreciates steeply because its buyers are narrow. A graphics card has several independent sets of buyers (people who play games, people who render, people who run models) and that breadth supports a resale floor well above what a comparable server component retains.

The risk to that floor is a new generation that changes the VRAM tier. When cards arrive with substantially more memory at a similar price, the previous generation’s value moves down a step, because VRAM is the constraint everybody in this market is buying for. That is a step change rather than a gradual slide, and it is the one worth thinking about if you are buying at the top of the market.

Treat the resale estimate conservatively. Assuming you will recover half is defensible. Assuming you will recover most of it is a forecast, and a cost model built on an optimistic forecast is a decision you have already made and dressed up.

The sunk-cost trap, in both directions

Here is the point most cost comparisons get wrong, and it cuts both ways.

If you already own the card, its purchase price is gone. It does not enter the decision about whether to use it. The marginal cost of running a model on hardware you already own is electricity and your time, and that is genuinely, usually, very cheap. So “should I use the card I have” and “should I buy a card” are different questions with different answers, and people routinely apply the answer from one to the other.

The mirror of that mistake is treating a card bought for other reasons as free. If you bought it to play games and it now runs models overnight, the honest marginal cost is the extra electricity and the wear, which is a strong argument for local inference that no API can match. But if the workload then grows and you buy a second card for it, that second card is a full capital decision and the first card’s history is irrelevant to it.

Sunk costs belong in the accounts and not in the decision.

The accounting treatment differs, and this is not advice

For a business, whether hardware is written off immediately or depreciated over several years, and how that interacts with tax, differs by jurisdiction and changes with policy. Some places allow accelerated write-offs for equipment below a threshold; others require a fixed schedule. The distinction between capital expenditure and operating expenditure also changes how a purchase is approved internally, which is often the real constraint: a monthly API bill passes as an operating cost while a one-off hardware purchase needs a capital approval that takes three months.

This course cannot tell you the treatment in your country and is not giving accounting or tax advice. What it can tell you is to ask the question before the purchase rather than after, because the answer sometimes changes which option is cheaper on paper even when it does not change which is cheaper in cash.

Put it in the model as a range

Because resale value and holding period are both estimates, carry them as ranges rather than as points. Compute the cost per million tokens at a pessimistic pair (low resale, short holding period) and at an optimistic one. If local wins in both cases, the decision is easy. If it wins only in the optimistic case, you have learned that your conclusion rests on a forecast, which is worth knowing before you spend the money.

That pair of numbers is also the most persuasive thing you can hand to whoever approves the purchase. A single figure invites an argument about the assumption; a range with the assumptions named invites a decision.