{"course":{"slug":"fine-tuning","title":"Fine-Tuning, and When Not To","url":"https://addaly.com/learn/fine-tuning","level":"advanced","track":"deep"},"lesson":{"slug":"many-adapters-at-once","title":"Serving many adapters from one base","url":"https://addaly.com/learn/fine-tuning/many-adapters-at-once","position":78,"of":84,"minutes":8,"answer":"One base plus fifty adapters is 24 GB where fifty merged models are 800 GB, at a throughput cost you should measure and a shared rank ceiling you must choose at training time.","keyIdea":"One base plus fifty adapters is 24 GB where fifty merged models are 800 GB, at a throughput cost you should measure and a shared rank ceiling you must choose at training time, but one adapter trained on everyone's data with the customer named in the prompt often matches per-customer adapters, so check whether you are paying for quality or for isolation."},"body":{"format":"markdown","text":"## The arithmetic that makes this interesting\n\nFifty customers, each wanting a model tuned on their data.\n\n- **Fifty merged models**, 8B each at 16 GB: 800 GB of weights and fifty serving processes.\n- **One base plus fifty adapters**: 16 GB plus fifty times 160 MB, so about 24 GB. One process.\n\nThat is the whole argument, and it is decisive. Multi-adapter serving turns per-customer models from an impossible proposition into a routine one.\n\n## How it works\n\nThe base weights stay resident. For each request, the server applies the named adapter's low-rank matrices during the forward pass. Modern implementations batch requests using *different* adapters together, computing the base matrix multiplication once for the whole batch and the small per-adapter corrections separately.\n\n> Figure: Fifty customers, two ways. Bars (GB of weights held): Fifty merged 8B models 800; One base plus fifty adapters 24. One process instead of fifty. Then ask the question nobody asks first: whether one adapter trained on everyone's data, with the customer named in the prompt, would have matched them, because you may be paying for isolation rather than for quality.\n\n```bash\nvllm serve meta-llama/Llama-3.1-8B-Instruct \\\n  --enable-lora --max-loras 8 --max-lora-rank 32 \\\n  --lora-modules acme=/adapters/acme globex=/adapters/globex\n```\n\nThen the request names the adapter as its model:\n\n```json\n{\"model\": \"acme\", \"messages\": [...]}\n```\n\nLoRAX is a dedicated server built around the same idea with dynamic loading from a registry; vLLM's support is built in and sufficient for most cases.\n\n## The costs\n\n**Throughput.** The extra low-rank multiplications cost throughput against a merged model. The cost grows with rank and with the number of distinct adapters in a batch, and published measurements run from small to about half. Measure it on your own traffic before relying on a number. Usually worth paying for the operational simplification.\n\n**Concurrency limits.** `--max-loras` caps how many adapters are active in the GPU at once. Beyond it, adapters are swapped in from host memory or disk, adding latency to the first request that needs one. If you have 500 adapters and 20 are hot, size the cap for the hot set.\n\n**Uniformity requirements.** Every adapter must share the base model, and rank must be at or below `--max-lora-rank`. Plan this at training time: pick one rank for the whole fleet, because a single rank-128 adapter raises the ceiling and the memory cost for all of them.\n\n**Cold start.** Loading an adapter from object storage on first use costs hundreds of milliseconds. Prewarm the ones you know will be needed.\n\n## When to merge instead\n\nMerge when there is one adapter and it is always used. You get the base model's exact serving performance, any inference engine works without adapter support, and you can quantise the result freely.\n\nKeep adapters separate when you have several, when you want to switch without a redeploy, or when you want to A/B two versions against the same base, which is a useful capability that merging removes.\n\n## Composing adapters\n\nThe libraries let you load several adapters at once and combine them, with weights:\n\n```python\nmodel.add_weighted_adapter([\"formal\", \"legal\"], [0.6, 0.4], \"combined\")\n```\n\nIt works in the sense that it runs and produces something. Whether it produces the intended combination is much less certain: adapters trained independently occupy overlapping subspaces, and adding them can interfere in ways that degrade both behaviours rather than combining them. There are methods designed for this (concatenation-based merging, and orthogonality-constrained training) and the results are mixed.\n\nTreat composition as an experiment, not a feature. If you need two behaviours together, train one adapter on both datasets. That reliably works, and it costs one more run.\n\n## Evaluating a fleet\n\nFifty adapters is fifty models, and each needs its own held-out set, its own canary and its own number. That is the cost people discover after building the serving layer.\n\nThe workable approach is a shared harness with per-tenant data: one evaluation script, one command, a results file per adapter, and a single summary that flags any tenant whose metric has fallen since the last build. Without that summary nobody looks at forty-nine of the fifty, and a regression affecting one customer surfaces as a support ticket.\n\nBudget for this before you commit to per-customer adapters. It is usually a larger ongoing cost than the serving.\n\n## The per-customer question nobody asks first\n\nBefore building this, check whether you need it. One adapter trained on all customers' data, with the customer identified in the system prompt, frequently matches per-customer adapters and is enormously simpler: one artefact, one evaluation, one thing to retrain when the base model moves.\n\nPer-customer adapters earn their complexity when the customers differ in ways a prompt cannot express, or when the data may not be mixed for contractual or regulatory reasons. That second reason is often the real one, and it is a good reason. But it is worth knowing that you are paying for isolation rather than for quality."},"check":{"question":"A team serves 50 per-customer adapters from one base with vLLM. One customer's adapter was trained at rank 128; the others at rank 16. What is the consequence?","options":[{"text":"The rank-128 adapter cannot be served alongside the others and must be merged into its own model instance.","correct":false,"tempting":"Concludes incompatibility, when the server accommodates it by raising the shared ceiling rather than rejecting it."},{"text":"The max-lora-rank ceiling must rise to 128 for the whole server, costing memory in every slot.","correct":true},{"text":"Requests to the rank-128 adapter will be 8 times slower, while the other 49 are unaffected.","correct":false,"tempting":"Localises the cost to one adapter; the per-adapter matmuls do scale with rank, but the memory ceiling is shared across all slots."},{"text":"Adapters of different ranks cannot be batched together, so throughput collapses to sequential request handling.","correct":false,"tempting":"Assumes batching requires uniform rank, when implementations pad to the configured ceiling precisely so mixed ranks can batch."}]},"prev":{"slug":"quantising-for-serving","title":"Quantising the model you ship","url":"https://addaly.com/learn/fine-tuning/quantising-for-serving"},"next":{"slug":"cost-at-steady-state","title":"The cost at steady state","url":"https://addaly.com/learn/fine-tuning/cost-at-steady-state"},"authorship":"Founder and series editor: Dr Mustafa Mun. Written by AI models to a written standard. Exam keys checked blind by a second model: 3,665 of 3,666 agreed. Academic review invited.","review":{"status":"Draft, open for academic review"},"cite":"Addaly (2026) 'Serving many adapters from one base', in Fine-Tuning, and When Not To. Edited by M. Mun. Addaly. Available at: https://addaly.com/learn/fine-tuning/many-adapters-at-once (Accessed: 13 October 2026).","licence":{"name":"CC BY-NC-SA 4.0","url":"https://creativecommons.org/licenses/by-nc-sa/4.0/"},"reader":{"schema":1,"title":"Serving many adapters from one base","courseTitle":"Fine-Tuning, and When Not To","module":{"index":9,"slug":"shipping-and-living-with-it","title":"Shipping it, and living with it"},"answer":null,"keyIdea":"One base plus fifty adapters is 24 GB where fifty merged models are 800 GB, at a throughput cost you should measure and a shared rank ceiling you must choose at training time, but one adapter trained on everyone’s data with the customer named in the prompt often matches per-customer adapters, so check whether you are paying for quality or for isolation.","hero":{"kind":"figure","figure":{"t":"figure","label":"Figure","figure":{"type":"bars","unit":"GB of weights held","title":"Fifty customers, two ways","series":[{"label":"Fifty merged 8B models","value":800},{"label":"One base plus fifty adapters","value":24,"highlight":true}],"caption":"One process instead of fifty. Then ask the question nobody asks first: whether one adapter trained on everyone’s data, with the customer named in the prompt, would have matched them, because you may be paying for isolation rather than for quality."},"alt":"Figure: Fifty customers, two ways. Bars (GB of weights held): Fifty merged 8B models 800; One base plus fifty adapters 24. One process instead of fifty. Then ask the question nobody asks first: whether one adapter trained on everyone’s data, with the customer named in the prompt, would have matched them, because you may be paying for isolation rather than for quality."}},"blocks":[{"t":"h2","id":"the-arithmetic-that-makes-this-interesting","runs":[{"text":"The arithmetic that makes this interesting"}]},{"t":"p","runs":[{"text":"Fifty customers, each wanting a model tuned on their data."}]},{"t":"ul","items":[[{"text":"Fifty merged models","b":true},{"text":", 8B each at 16 GB: 800 GB of weights and fifty serving processes."}],[{"text":"One base plus fifty adapters","b":true},{"text":": 16 GB plus fifty times 160 MB, so about 24 GB. One process."}]]},{"t":"p","runs":[{"text":"That is the whole argument, and it is decisive. Multi-adapter serving turns per-customer models from an impossible proposition into a routine one."}]},{"t":"h2","id":"how-it-works","runs":[{"text":"How it works"}]},{"t":"p","runs":[{"text":"The base weights stay resident. For each request, the server applies the named adapter’s low-rank matrices during the forward pass. Modern implementations batch requests using "},{"text":"different","i":true},{"text":" adapters together, computing the base matrix multiplication once for the whole batch and the small per-adapter corrections separately."}]},{"t":"code","lang":"bash","label":"Bash","text":"vllm serve meta-llama/Llama-3.1-8B-Instruct \\\n  --enable-lora --max-loras 8 --max-lora-rank 32 \\\n  --lora-modules acme=/adapters/acme globex=/adapters/globex"},{"t":"p","runs":[{"text":"Then the request names the adapter as its model:"}]},{"t":"code","lang":"json","label":"JSON","text":"{\"model\": \"acme\", \"messages\": [...]}"},{"t":"p","runs":[{"text":"LoRAX is a dedicated server built around the same idea with dynamic loading from a registry; vLLM’s support is built in and sufficient for most cases."}]},{"t":"h2","id":"the-costs","runs":[{"text":"The costs"}]},{"t":"p","runs":[{"text":"Throughput.","b":true},{"text":" The extra low-rank multiplications cost throughput against a merged model. The cost grows with rank and with the number of distinct adapters in a batch, and published measurements run from small to about half. Measure it on your own traffic before relying on a number. Usually worth paying for the operational simplification."}]},{"t":"p","runs":[{"text":"Concurrency limits.","b":true},{"text":" "},{"text":"--max-loras","code":true},{"text":" caps how many adapters are active in the GPU at once. Beyond it, adapters are swapped in from host memory or disk, adding latency to the first request that needs one. If you have 500 adapters and 20 are hot, size the cap for the hot set."}]},{"t":"p","runs":[{"text":"Uniformity requirements.","b":true},{"text":" Every adapter must share the base model, and rank must be at or below "},{"text":"--max-lora-rank","code":true},{"text":". Plan this at training time: pick one rank for the whole fleet, because a single rank-128 adapter raises the ceiling and the memory cost for all of them."}]},{"t":"p","runs":[{"text":"Cold start.","b":true},{"text":" Loading an adapter from object storage on first use costs hundreds of milliseconds. Prewarm the ones you know will be needed."}]},{"t":"h2","id":"when-to-merge-instead","runs":[{"text":"When to merge instead"}]},{"t":"p","runs":[{"text":"Merge when there is one adapter and it is always used. You get the base model’s exact serving performance, any inference engine works without adapter support, and you can quantise the result freely."}]},{"t":"p","runs":[{"text":"Keep adapters separate when you have several, when you want to switch without a redeploy, or when you want to A/B two versions against the same base, which is a useful capability that merging removes."}]},{"t":"h2","id":"composing-adapters","runs":[{"text":"Composing adapters"}]},{"t":"p","runs":[{"text":"The libraries let you load several adapters at once and combine them, with weights:"}]},{"t":"code","lang":"python","label":"Python","text":"model.add_weighted_adapter([\"formal\", \"legal\"], [0.6, 0.4], \"combined\")"},{"t":"p","runs":[{"text":"It works in the sense that it runs and produces something. Whether it produces the intended combination is much less certain: adapters trained independently occupy overlapping subspaces, and adding them can interfere in ways that degrade both behaviours rather than combining them. There are methods designed for this (concatenation-based merging, and orthogonality-constrained training) and the results are mixed."}]},{"t":"p","runs":[{"text":"Treat composition as an experiment, not a feature. If you need two behaviours together, train one adapter on both datasets. That reliably works, and it costs one more run."}]},{"t":"h2","id":"evaluating-a-fleet","runs":[{"text":"Evaluating a fleet"}]},{"t":"p","runs":[{"text":"Fifty adapters is fifty models, and each needs its own held-out set, its own canary and its own number. That is the cost people discover after building the serving layer."}]},{"t":"p","runs":[{"text":"The workable approach is a shared harness with per-tenant data: one evaluation script, one command, a results file per adapter, and a single summary that flags any tenant whose metric has fallen since the last build. Without that summary nobody looks at forty-nine of the fifty, and a regression affecting one customer surfaces as a support ticket."}]},{"t":"p","runs":[{"text":"Budget for this before you commit to per-customer adapters. It is usually a larger ongoing cost than the serving."}]},{"t":"h2","id":"the-per-customer-question-nobody-asks-first","runs":[{"text":"The per-customer question nobody asks first"}]},{"t":"p","runs":[{"text":"Before building this, check whether you need it. One adapter trained on all customers’ data, with the customer identified in the system prompt, frequently matches per-customer adapters and is enormously simpler: one artefact, one evaluation, one thing to retrain when the base model moves."}]},{"t":"p","runs":[{"text":"Per-customer adapters earn their complexity when the customers differ in ways a prompt cannot express, or when the data may not be mixed for contractual or regulatory reasons. That second reason is often the real one, and it is a good reason. But it is worth knowing that you are paying for isolation rather than for quality."}]}],"check":{"question":[{"text":"A team serves 50 per-customer adapters from one base with vLLM. One customer’s adapter was trained at rank 128; the others at rank 16. What is the consequence?"}],"options":[{"text":[{"text":"The rank-128 adapter cannot be served alongside the others and must be merged into its own model instance."}],"correct":false,"tempting":[{"text":"Concludes incompatibility, when the server accommodates it by raising the shared ceiling rather than rejecting it."}]},{"text":[{"text":"The max-lora-rank ceiling must rise to 128 for the whole server, costing memory in every slot."}],"correct":true,"tempting":null},{"text":[{"text":"Requests to the rank-128 adapter will be 8 times slower, while the other 49 are unaffected."}],"correct":false,"tempting":[{"text":"Localises the cost to one adapter; the per-adapter matmuls do scale with rank, but the memory ceiling is shared across all slots."}]},{"text":[{"text":"Adapters of different ranks cannot be batched together, so throughput collapses to sequential request handling."}],"correct":false,"tempting":[{"text":"Assumes batching requires uniform rank, when implementations pad to the configured ceiling precisely so mixed ranks can batch."}]}],"order":[3,1,2,0],"why":null},"prev":{"slug":"quantising-for-serving","title":"Quantising the model you ship","minutes":9},"next":{"slug":"cost-at-steady-state","title":"The cost at steady state","minutes":9}}}