How (and when) to Fine-Tune Google Gemma 4 for Your Business Domain-Specific Use Case

A plain-English guide to the process, the hardware, the real costs, and the savings – written for decision-makers, not just engineers.

Your AI bill went up again this quarter.

And nobody can quite tell you why.

The model you are paying premium rates for spends most of its day doing one boring, repetitive, narrow thing – and doing it no better than a much smaller and much cheaper model would.

Every business leader I talk to has said some version of that to me, usually with a sigh.

So let me tell you straight away: there is a way out, and it is cheaper and far more boring than you think.

Google released Gemma 4 on 2 April 2026 under an Apache 2.0 open-source licence, and that one legal change did more for enterprise AI economics than any performance score in the entire release.

Read this article to the end to find out why.

A promise before we start.

I am going to explain every piece of jargon the first time it appears. If you have ever sat in a vendor meeting nodding along to words like “quantisation” and “p99 latency” while quietly planning to look them up later – this article is for you.

Let us begin with three terms you will need immediately.

A model is the AI itself: a very large file of numbers that turns text you type into text it writes back.

Fine-tuning means taking somebody else’s finished model and training it a little further on your own examples, so it learns your vocabulary, your formats, and your way of doing things.

An open-weight model is one where you can download that file of numbers and run it on your own computers, instead of renting access to it over the internet.

Gemma 4 is an open-weight model.

That is the whole story in one sentence.

Why Gemma 4 Is the Model Worth Fine-Tuning

Let me start with the part everyone skips right past.

The earlier versions – Gemma 1, 2, and 3 – came with a custom Google licence that limited what you could change and what you could pass on to others.

Gemma 4 does not.

Google released it under Apache 2.0, which is one of the most permissive open-source licences there is.

In practical terms it means you can modify the model, deploy it commercially, and even redistribute your modified version, without asking anyone’s permission.

It arrived in four sizes, and a fifth followed shortly after.

Model sizes are measured in parameters – the individual numbers inside the model that get adjusted during training.

More parameters generally means a smarter but slower and more expensive model. “B” means billion.

• E2B and E4B – roughly 2 billion and 4 billion effective parameters, built to run on phones and small devices.
• 12B Unified – a 12-billion-parameter model designed for handling documents, images, and audio together. It arrived later in the release cycle.
• 26B A4B – a 26-billion-parameter Mixture of Experts model. That means the model is internally divided into specialist sections, and only about 4 billion parameters actually fire for any given word it produces. You get the quality of a big model at closer to the speed of a small one.
• 31B Dense – 31 billion parameters, all of them active every time. “Dense” simply means the opposite of Mixture of Experts.

So that is five sizes, and your legal team can finally say yes without a six-week review.

But Thomas, you might say – a licence is a lawyer’s detail, not a business one.

True.

But it is the entire reason your fine-tuned model is yours, permanently, with no clause that can be quietly revised at the vendor’s convenience.

What the Models Can Actually Do

Three capabilities matter for a business project.

A context window of 128K to 256K tokens.

A token is roughly three-quarters of a word, so the smaller models can hold about 100,000 words in mind at once and the larger ones around 200,000.

That is a long contract, or a full quarter of support transcripts, in a single conversation.

Function calling.

This means the model can decide to use a tool – look up a customer record, call your pricing system, query a database – rather than only writing prose.

It is what turns a chatbot into something that can actually do work.

System-role support.

A system role is a standing instruction the model carries into every conversation, separate from what the user types.

Think of it as the job description you hand a new employee, versus the individual requests they get during the day.

Gemma 4 supports this natively, which makes your instructions far more reliable.

There is also Multi-Token Prediction, where a small helper model guesses the next few words ahead so the main model can confirm them in one go.

It makes responses noticeably faster at no cost to quality.

Which size for which problem

One honest warning before you spend anything

If your problem is “the model does not know our latest documents” – fine-tuning is the wrong tool.

What you want there is RAG, which stands for Retrieval-Augmented Generation.

Here is RAG in plain English.

1. Before the model answers a question, your system searches your own documents for the most relevant passages, then pastes those passages into the model’s prompt alongside the question.
2. The model answers using material it has just been handed, rather than material it memorised months ago.

It is the difference between hiring someone who memorised your policy manual and hiring someone who is allowed to look things up.

RAG is cheaper, faster to build, and much easier to keep current – because updating your knowledge means updating a document, not retraining a model.

So build RAG first.

Fine-tune when the problem is behaviour rather than knowledge: the format is wrong, the tone is wrong, the model uses the wrong vocabulary, it makes the wrong judgement calls, or it simply does the task inconsistently no matter how carefully you word the request.

Facts, use RAG.

Behaviour, fine-tune.

Most mature systems end up doing both.

Check Hugging Face First – What You Are Building May Already Exist

Before you spend a single hour collecting data, do this one thing.

Go and look.

Hugging Face is the industry’s public library of AI models – think GitHub, but for trained models instead of source code.

Tens of thousands of them are free to download, and a great many are already fine-tuned for specific industries.

Open Hugging Face, search for your industry plus the word “gemma”, and see what comes back.

Because there is a genuinely embarrassing failure mode in this business, and it goes like this: a team spends four months and a quarter of a million dollars building a model that a stranger already trained, published under a free licence, and gave away eighteen months ago.

It happens far more often than anyone admits at conferences.

Google even maintains the Gemmaverse, a directory of community-built models based on the Gemma family, precisely because so much of this work has already been done by someone else.

Google’s Own Specialized Versions

There are also purpose-built variants straight from Google that might solve your problem outright.

• FunctionGemma – tuned specifically for calling tools and running multi-step tasks.
• EmbeddingGemma – for search and retrieval. An embedding is a numerical fingerprint of a piece of text that lets a computer find similar passages quickly. This is the engine underneath RAG.
• ShieldGemma – a small safety model that reads incoming questions and flags the ones that should be refused.
• PaliGemma – for tasks that combine images and text.

Any one of those could save you a quarter of work.

How to run the check properly

Give this one week. One.

• Search by industry and by task, not just by model name.
  • Try “clinical”, “legal”, “finance”, “support ticket”, and whatever jargon your own industry uses.
  • The person who published the model you need may have described it in language you would not have thought to search for.
• Download the top three candidates and test them against your own examples.
  • You want the same set of test questions you would eventually use to judge your own fine-tuned model, so the comparison is fair.
  • This is an afternoon of work and it can save you a quarter.
• Read the licence and the model card on every single one.
  • A model card is the published documentation that describes how a model was trained and what it should and should not be used for.
  • A model built on a free base can still inherit restrictions from the data it was trained on, and “trained on a mixture of public data” should worry you in any regulated industry.

Now – will an off-the-shelf model beat one properly tuned on your own data?

Usually not.

But here is the thing.

It does not have to win to be worth an afternoon.

If a free public model gets you 85% of the way there, it becomes your new starting point, and now you know precisely how much value your own training project has to add before it justifies its cost.

And sometimes it is simply better than what you would have built, and you get to go and solve a different problem instead.

Which, I think, is pretty awesome.

The Fine-Tuning Process, End to End

Here is the whole sequence, in order.

Follow it in order too, because every step taken out of turn costs money later.

1. Decide what success looks like, before touching any data.

Write it as a number a non-engineer can check – “correctly extracts all five fields from 95 out of 100 invoices”, not “works better”.

2. Measure where you are starting from.

Run the untouched model against your test questions and write the score down. This is your baseline. Without it, you can never prove afterwards that anything improved.

3. Collect and clean your examples.

Strip out personal data, remove near-duplicates, and fix wrong labels. Whatever mistakes are in your data get baked permanently into the model.

4. Put the examples into the format the model expects.

Every model family has its own conventions for marking who said what. Google publishes the official formatting guide for Gemma 4.

5. Split the data three ways.

A training set the model learns from, a validation set you check progress against while training, and a test set that nobody looks at until the very end. That last one is your only honest measure of quality, and looking at it early ruins it.

6. Choose how to train.

This is where a few acronyms show up, so let me deal with them properly.

Full fine-tuning rewrites every number in the model. Maximum control, maximum cost, serious hardware required.

• LoRA – Low-Rank Adaptation. Instead of rewriting the whole model, you freeze it and train a small add-on file, called an adapter, that sits alongside and adjusts the model’s behaviour. The adapter might be a few hundred megabytes against a model of many gigabytes. Cheap, fast, and reversible.

• QLoRA – Quantised LoRA. Same idea, except the frozen base model is first compressed so it takes up less memory. Quantisation means storing each number in the model with less precision – roughly like saving a photo as a smaller JPEG. You lose a little fidelity and save an enormous amount of space.

• DoRA – a refinement of LoRA that separates the size of a change from its direction, giving slightly better results for slightly more cost.

For around 90% of business use cases, QLoRA is simply the right answer.

7. Set your starting dials.

Your engineer will recognise these: rank 16, alpha 32, learning rate 2e-4, 2 to 3 epochs. An epoch is one complete pass through all your training examples. Three passes is usually plenty; ten is usually a mistake.

8. Run the training, and watch the curves.

If the model gets better on the training examples but worse on the validation ones, that is overfitting – it has started memorising your examples rather than learning the pattern behind them. Stop, and reduce the number of epochs.

9. Test it like a sceptic.

Score it on your held-out test set, have a person read a sample blind, and also re-check that it can still do ordinary things.

That last check matters because of catastrophic forgetting, where training a model hard on one narrow speciality causes it to lose general ability elsewhere.

A model that became brilliant at insurance claims and forgot how to count is not a win.

10. Package it, version it, register it.

Treat the model and the data that produced it exactly as you treat source code.

The Tools Your Team Will Use

Google supports Hugging Face Transformers with QLoRA, full fine-tuning through Transformers, Tunix, and Unsloth’s Gemma 4 training path. Axolotl, TRL, PEFT, and Keras all work too.

You do not need to know what any of those are.

Your engineer will.

Here is what a minimal training configuration looks like, purely so you recognise it when it appears in a document.

from peft import LoraConfig
from trl import SFTConfig
peft_config = LoraConfig(
    r=16,
    lora_alpha=32,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
                    "gate_proj", "up_proj", "down_proj"],
)
args = SFTConfig(
    output_dir="gemma4-domain-adapter",
    num_train_epochs=3,
    per_device_train_batch_size=2,
    gradient_accumulation_steps=8,
    learning_rate=2e-4,
    bf16=True,
    logging_steps=10,
    save_strategy="epoch",
)

That is about thirty lines.

The four months of work sit either side of it, not inside it.

Three failure modes account for most disappointing results

Too little data.

Too many epochs.

Test questions that accidentally appeared in the training data – which flatters the score and tells you nothing.

Check all three before anyone blames the model.

Data Requirements

So how much data do you actually need?

It depends on the task, and anybody who gives you one universal number is definitely wrong.

An example here means one input paired with the correct output – one email and the correct category, one invoice and the correctly extracted fields, one customer question and the ideal answer.

Those ranges are wide because the thing that really matters is coverage, not volume.

A thousand examples covering every awkward edge case will beat fifty thousand that repeat the same three ordinary situations.

Good data means correct labels, consistent formatting, real domain vocabulary, difficult cases, and explicit examples of questions the model should decline.

That last category is what stops a model confidently inventing a policy your compliance team never wrote.

About Synthetic Data

Synthetic data means examples generated by another AI rather than collected from real life.

It has a legitimate place – use a larger model to draft candidates, then have a human expert approve or reject each one.

Generate, review, keep the keepers.

Never train on unreviewed AI-generated output, because errors compound quietly and you will not notice until your customers do.

Governance First, Engineering Second

Three questions, all of which come before the first line of code.

Can you legally train on this data?

Do your customer contracts actually permit it?

Does the data have to stay inside a particular country under GDPR, HIPAA, or India’s DPDP Act?

Answer those before you start, not after the audit.

And version your dataset the way you version software – dataset v1.3 produced model v1.3, and a year from now somebody will ask you to prove exactly that.

Hardware Requirements

Two different budgets live in this section, and people constantly confuse them.

Training is the one-off job of teaching the model.

It needs a lot of hardware for a short time.

Inference is the ongoing job of actually answering questions.

It needs less hardware, but permanently.

The measurement that governs both is VRAM – the dedicated memory on a graphics card.

The entire model has to fit in VRAM to run properly, which is why VRAM, not processor speed, is the number that decides what you can and cannot do.

Google publishes the inference figures, so let us start there.

Here is the memory each model needs just to load, before it answers a single question.

Source: Gemma 4 model overview, Google AI for Developers. Figures include roughly 20% overhead and may vary with your serving software.

Precision, in that sentence, means how many digits are kept for each number in the model. Full precision is most accurate and largest; 4-bit is a quarter of the size and slightly less accurate.

Two things in those figures trip almost everyone up.

The “E” means effective, not exact. The small models use a technique called Per-Layer Embeddings, so they take up more memory than “2 billion parameters” would suggest. Do not budget from the name.

The Mixture of Experts model is not a small model. The 26B only uses about 4 billion parameters per word, but all 26 billion must sit in memory ready to be picked from. Budget for the full 26.

And those numbers cover the model alone. On top of them sits the KV cache – working memory that holds the current conversation. The longer the conversation and the more simultaneous users, the more of it you need. Google notes explicitly that memory needed for training is much higher again than memory needed for answering.

What actually fits on what

Small Devices are a Genuine Target Here

Google built the E2B and E4B models in partnership with Qualcomm and MediaTek so they run completely offline, with almost no delay, on phones, on a Raspberry Pi, and on NVIDIA’s Jetson hardware.

Offline is the word to notice. No internet connection, no data leaving the device, no per-request bill. For a factory floor, a field engineer, or a rural clinic, that is not a nice-to-have.

Running on an ordinary processor instead of a graphics card works acceptably for the two small models on overnight batch jobs.

It is a trap for anything a human is waiting on. You will get your answer – just not before they have given up.

And here is what will not work, so nobody wastes a month finding out: a full fine-tune of the 31B on one desktop graphics card, the 26B on a 16 GB card at any compression, or a very-long-conversation service on a single 80 GB card without careful planning.

Cost Budgeting

Seven line items, and the graphics cards are nowhere near the biggest one.

Preparing the data. Training. Testing. Storage. Running the service. Monitoring it. And the salaries of the people doing all of the above.

Here are the market rates you are budgeting against, quoted per GPU-hour – one graphics card rented for one hour.

As of 5 September 2026, the typical on-demand price for an H100 – currently the standard high-end AI chip – sat at $3.32 per GPU-hour across forty providers.

The previous-generation A100 with 80 GB starts around $1.09 per GPU-hour from the cheapest specialist providers, while the big cloud platforms charge $3.43 on AWS and $5.03 on Google Cloud for exactly the same card.

And for the kind of work we are describing, the older A100 usually works out cheaper per completed job than the newer H100.

That is probably the single most useful cost sentence in this article.

You do not need the newest chip to fine-tune a small model.

Three worked examples

These are models with the assumptions stated, not vendor quotes. Treat them as a shape, not a price list.

Assumptions: QLoRA on E4B or 12B, A100 rented at $1.09–$3.43 per GPU-hour, one always-on service at pilot scale growing to a small fleet, and blended engineering cost of $60–$120 per hour depending on region.

Notice the shape of that table.

Compute is a rounding error.

People are the budget.

Rent or Buy

Renting one H100 continuously for a year at typical market rates comes to roughly $29,000.

Buying only wins when your measured usage stays high – not your projected usage, which is almost always optimistic.

The honest rule: rent until you have three consecutive months of real usage data, then decide with evidence.

The costs teams always forget

Training runs that fail. This is completely normal, and you should budget around 30% of your compute for it.

Re-labelling the data, for when your first set of labelling instructions turns out to have been wrong.

Building the test harness, which nobody puts in the plan and everybody needs.

And the second and third fine-tune, because there is always a second and a third.

How Cost Grows as Your Data Grows

Here is the part almost every guide gets wrong.

Compute cost grows in a straight line with the amount of data.

Total project cost does not.

Labelling cost grows in a straight line too, and it never gets meaningfully cheaper per example, because it is human time.

Testing cost grows faster than the data, because more data means more unusual cases, which means more categories to check, which means more human review.

Storage is negligible right up until it suddenly is not.

Assumptions: E4B with QLoRA, three epochs, around 400 tokens per example, A100 rented at $1.09 per GPU-hour, labelling at $1.00 per example dropping to $0.70 at volume.

Now look at the training column.

Across a thousand-fold increase in data, it never once breaks $7,000.

And now look at the labelling column.

That is your project cost, and it is a people problem. Not a tech problem.

Knowing When to Stop

There is a real and measurable point of diminishing returns.

Chart your test-set accuracy against the amount of data you have used, and you will watch the line flatten out – usually somewhere between 5,000 and 30,000 examples for a well-defined task.

Beyond that point, each additional 10,000 examples buys you a fraction of a percentage point for the same labelling cost as the first 10,000.

Stop collecting when the curve flattens.

Not when the budget runs out.

Deployment

Deployment means putting the finished model somewhere your business can actually use it.

Your options, roughly in order of how much operational maturity they demand.

Ollama and LM Studio – free desktop applications that run a model on one computer. Lovely for trying things out. Not built to serve hundreds of people at once.

llama.cpp and MLX – lightweight engines for small devices and Apple hardware.

vLLM and SGLang – proper production serving software, built to handle many simultaneous users efficiently.

Vertex AI and Google Kubernetes Engine – Google runs the hardware, you run the model.

On-premise Kubernetes – your own servers, your own building, when data residency is the entire point of the project.

Google publishes setup guides for LM Studio, Ollama, llama.cpp, MLX, and deployment on Cloud GKE.

Compression, Done Properly

Remember quantisation – shrinking the model by storing its numbers less precisely.

There are two ways to do it, and the difference matters.

Post-training quantisation compresses a finished model. It is easy, and it can measurably damage quality.

Quantisation-Aware Training builds the compression into the training process, so the model learns to work around the loss of precision as it goes. The result is a small model that performs almost identically to the full-size one.

Google publishes official QAT versions of every Gemma 4 model, which means you get that quality improvement simply by downloading the right file rather than the obvious one.

Ask your team whether they are using the QAT checkpoints. It is a free win and it gets missed constantly.

Planning for Load

The arithmetic is simpler than people expect.

Take your number of simultaneous users, multiply by the length of a typical response, divide by how fast one card produces words. That gives you your card count.

Then remember the KV cache. A service holding very long conversations for twenty people at once can need more memory for the conversations than for the model itself.

Budget for it, or your first busy Monday will find it for you.

How to Roll It Out Without a Disaster

Three stages, in order.

Shadow mode – the new model runs alongside the existing system and its answers are recorded, but nobody sees them. You compare quietly, at zero risk.

Canary – 1% of real traffic goes to the new model, with an automatic switch back if anything looks wrong. The name comes from canaries in coal mines, and the logic is the same.

Percentage rollout – 5%, then 25%, then 50%, then everyone, pausing properly at each step.

And a rollback path you have actually tested, not merely documented.

Rollback means reverting to the previous version quickly when something goes wrong, and the first time you try it should never be during an incident.

What to Watch Once It is Live

Log everything: the questions, the answers, the timings, and every refusal.

Then watch three things in particular.

p50 and p99 latency.

Latency is how long a response takes.

p50 is the median – half your users get an answer faster than this.

p99 is the ninety-ninth percentile – only 1 in 100 users waits longer than this.

People obsess over p50 and get burned by p99, because p99 is the experience of your unluckiest customers, and they are the ones who complain.

A system with a p50 of 200 milliseconds and a p99 of twelve seconds feels broken, no matter how good the average looks.

Output length anomalies.

If your model’s answers suddenly get much longer or much shorter, something has changed.

Refusal-rate drift.

If the share of questions the model declines moves noticeably in either direction, quality has shifted.

This one catches problems days before a customer notices.

Guardrails, Security, and Teaching Your Model to Say No

Let us talk about the thing that actually gets projects cancelled.

Not cost.

Not accuracy.

A model that answers a question it should have refused, in front of a customer, on the record.

So how do you teach a model to say no?

The same way you teach it everything else – by showing it examples.

Teaching refusal, concretely

Refusing is a behaviour, and behaviour is precisely what fine-tuning is good at.

Around 5% to 10% of your training examples should be questions the model ought to decline, paired with the exact wording you want it to use when declining.

Four things to include.

1. Genuinely out-of-scope questions, not just harmful ones.

A banking model should politely decline medical questions. It will only learn that if you show it what that looks like.

2. Near-miss pairs.

Two almost identical questions where one is fine and one is not. These are what teach the model where the actual line sits, rather than teaching it to avoid a whole topic.

3. The escalation path inside the refusal.

“I can’t advise on that – please contact the compliance team on extension 4400” is a good answer. A flat “I cannot help with that” is a support ticket waiting to happen.

4. Rephrased versions of the same forbidden request.

Nobody trying to misuse your system will phrase it the way your test set does.

Get the ratio wrong in the other direction and you produce a model that refuses everything, which is its own kind of failure.

So measure your refusal rate on ordinary, legitimate traffic too.

Never Rely on the Model Alone

This is the part to take to your architecture review.

A fine-tuned model is one layer of defence, and one layer is not a defence.

Put a checker in front of the model.

A small classifier reads each incoming question and flags problems before the main model ever sees them.

ShieldGemma is built for exactly this and is cheap to run alongside your main model.

Put a filter behind the model.

Check every outgoing answer for personal data, policy violations, and malformed output before it reaches a human being.

Validate the structure of anything machine-readable.

If your model is supposed to produce a structured record and the record does not parse correctly, reject it.

Never pass a half-formed object downstream and hope.

Control access at the front door. Authentication and rate limits, because none of the model-level safety does anything about somebody hammering your service from outside.

Log every refusal along with what triggered it. That is how you tell a real attack from an over-cautious false alarm.

The security work that is now genuinely yours

When you own the model, you also own the attack surface. Four items belong on somebody’s job description.

Prompt injection testing.

Prompt injection is when instructions hidden inside content the model reads – a document, an email, a web page – hijack its behaviour.

If your model reads anything an outsider can influence, you are exposed to this, and it needs deliberate testing.

Extraction testing.

A model trained on customer data can sometimes be coaxed into repeating that data back. Test for it before somebody else does.

Supply-chain hygiene.

Verify the files you download, pin your versions, and never load a model file from a source you have not checked.

A model file can contain executable code.

Patching.

The software that serves your model ships security fixes, and applying them is now your responsibility rather than your vendor’s.

Google also publishes a Responsible GenAI Toolkit and the Secure AI Framework, and both are worth reading before designing your own.

None of this is glamorous.

All of it is cheaper than the project getting compromised!

Maintenance

A fine-tuned model is a production asset with a named owner.

It is not a project that quietly ends at launch.

The thing you are watching for is drift – the gradual gap that opens between the world the model was trained on and the world it now operates in.

It comes in three flavours.

Data drift – your incoming questions change. New products, new regions, new customer language.

Concept drift – the correct answer changes. A regulation is updated, a policy is revised, and the model is now confidently giving yesterday’s right answer.

Quality regression – nothing obvious changed, but the scores fell anyway. This one is the most unsettling and the most common.

Retrain on Triggers, Not on a Calendar

“We retrain every quarter” is a plan that either wastes money or arrives too late, and often manages both.

Some better triggers:

• The test score drops more than three points from where it launched.
• The refusal rate moves more than five points in either direction.
• More than one in ten incoming questions looks nothing like the training data.
• A new product line, region, or regulation enters scope.

Every retrain runs the full test suite, including the general-ability checks – otherwise you fix one thing and quietly break four others.

Keep the dataset, the settings, the original model version, and the trained adapter together in one record, so any given version can genuinely be reproduced a year later.

When Google Ships the Next Gemma

Please do not migrate on release day!

Run your own test suite against the new base model first, and migrate only if it wins on your numbers, on your task.

A model that scores higher on public benchmarks can easily score lower on your invoices.

The Budget Nobody Plans For

Set aside 20% to 30% of your original build cost, every year, for all of the above.

That number tends to surprise finance teams.

Tell them early, and tell them once.

Where the Savings Come From, Sector by Sector

Three opportunities per sector, each explained properly:

Fintech

• Transaction categorisation. Payment feeds arrive as messy, abbreviated merchant strings that have to be sorted into clean categories, millions of times a month. This is high-volume, narrow, and endlessly repetitive – which is exactly the shape where a small model on your own hardware beats paying per request. The saving comes from converting a variable bill that grows with your business into a fixed infrastructure cost that does not.
• Dispute and chargeback triage. Every disputed transaction needs reading, classifying by reason, and checking for whether the customer supplied enough evidence. A tuned model can do the first pass and route each case to the right queue, so your specialists spend their day deciding rather than sorting. The saving is human hours redirected, not headcount removed.
• Fraud-alert summarisation. Analysts reviewing flagged transactions currently read raw data and reconstruct the story themselves. A model tuned on your own case history can write that story for them in a paragraph, cutting minutes off every review. Across thousands of alerts a week, minutes become full-time equivalents.

Healthcare

• Clinical note structuring. Dictated or typed notes get turned into organised sections – problem, medication, plan – which is tedious, high-volume work that clinicians resent. The decisive advantage here is not cost but location: an open model runs entirely inside the hospital, so patient data never crosses the boundary, and a HIPAA objection becomes an architecture diagram instead of a blocker.
• Coding assistance. Translating a clinical encounter into billing and procedure codes is slow, error-prone, and directly tied to revenue. A model tuned on your own historical coding suggests candidates and a human coder confirms each one, which speeds the work without moving the accountability. Never let this run unsupervised.
• Patient message triage. Inbound patient messages need sorting by urgency before a clinician opens the queue. Getting that ordering right is genuinely valuable clinically, not just operationally, and it can run on-premise at very low cost per message.

Insurance

• First notice of loss extraction. When a claim arrives by email, photo, or phone transcript, somebody has to pull out the policy number, the date, the location, and what happened. This is narrow, repetitive, and enormously high-volume – the classic fine-tuning shape – and automating the extraction shortens the time to a settlement decision, which customers actually feel.
• Policy clause comparison. Comparing versions and endorsements of a policy to find what changed is careful, slow, deeply unglamorous work. A model tuned on your own document family does the first pass and flags differences for a human to confirm, which is faster and considerably more consistent than a tired reader at four in the afternoon.
• Claims file summarisation. When a case moves between adjusters, the incoming person has to read the whole file. A tuned summary written in your house format makes that handover minutes rather than an hour, and handovers happen constantly.

Banking

• Onboarding document extraction. Passports, utility bills, and company registry documents all need reading and turning into structured customer records. High volume, tightly defined, and subject to strict rules about where the data may be processed – which is precisely why an open model you run yourself has an advantage that no API pricing change can match.
• Internal policy question answering. Branch and contact-centre staff constantly need answers from a policy manual nobody has time to read. This is mostly a retrieval problem, but a model tuned on your house tone and your escalation rules makes the answers usable rather than merely correct.
• Credit memo drafting. Turning structured financial data into the standard narrative your credit committee expects is formulaic writing, which is exactly what tuning on house voice is good at. The analyst edits rather than composes, which is a meaningfully faster job.

Manufacturing

• Maintenance log classification. Technicians write free-text notes that contain enormously valuable failure-pattern data nobody can analyse because it is unstructured. A tuned model turns years of notes into categorised failure modes, which feeds directly into reliability engineering.
• Defect report summarisation. Reports pile up across shifts and lines, and the person who needs the overview never has time to read them all. A daily tuned summary in a consistent format makes the pattern visible while it is still actionable.
• Shop-floor assistant. A model running offline on small hardware at the line answers procedure questions without any network connection at all. On a factory floor with poor connectivity and strict security, offline operation is not a cost optimisation – it is the only version that works.

High Technology

• Support ticket routing. Incoming tickets need categorising, deduplicating, and sending to the right team, thousands of times a day. Here latency matters more than brilliance: a fast small model that routes correctly in a fraction of a second beats a slower, cleverer one that routes correctly in four seconds.
• Log and error triage. Turning raw error output into probable-cause categories is the sort of pattern-matching that a model tuned on your own historical incidents does very well, and it puts an engineer closer to the answer before they have finished their coffee.
• Internal documentation. Generating and maintaining documentation from your own code and change history is continuous, low-glamour work that never gets prioritised. A tuned model does the first draft in your house style and a human corrects it, which is the only version of this that has ever actually stuck.

Aerospace and Defence

• Maintenance and airworthiness records. Decades of free-text records need structuring into something searchable, and the environments where this data lives often cannot connect to an external service at all. That makes an open model not merely the cheaper option – it is frequently the only permitted one.
• Requirements traceability. Checking that specification documents remain consistent with each other across thousands of pages is exhausting and unforgiving work. A model tuned on your own specification language flags likely inconsistencies for engineers to adjudicate.
• Air-gapped technical question answering. An air-gapped system has no network connection to the outside world at all. In these facilities, the choice is not between an API and an open model – it is between an open model and nothing.

Crypto and Digital Assets

• On-chain activity summarisation. Blockchain transaction data is structured, enormous, and unreadable to humans. Turning it into plain-language narratives is repetitive at a volume where per-request pricing becomes genuinely painful, which is the textbook case for owning your own model.
• Smart contract documentation. Generating readable documentation from contract source code is formulaic and constant, and a tuned model produces it in your project’s own conventions rather than generic boilerplate.
• Compliance screening narratives. When an address is flagged, an analyst needs a written explanation of why. Producing that first draft consistently and instantly makes the review queue move, and the writing is standardised enough for tuning to handle well.

Cloud Providers and Hyperscalers

• Support ticket deflection. At platform scale, drafting a first response to every inbound ticket is millions of calls a month. A fraction of a cent per call compounds into real money very quickly, which makes the fixed-cost model unusually compelling at this volume.
• Documentation question answering. Product surfaces are far too large for anyone to know completely, including your own staff. A tuned model over your documentation reduces both customer tickets and internal interruptions at once.
• Incident summary drafting. Turning telemetry and status feeds into a customer-facing incident narrative is urgent, repetitive, and follows a strict house format – all three of which favour a tuned model producing the draft while a human approves the wording.

AI Companies and Model Labs

• Data labelling at volume. Pre-labelling examples before human review is one of the largest internal costs at any lab, and paying frontier rates to do it makes very little sense when a tuned small model handles the first pass adequately.
• Evaluation judging. Using an AI to score another AI’s output against a rubric is now standard practice, and it is a narrow, well-defined task where a tuned model is both cheaper and more consistent than a general one.
• Synthetic data generation. Producing training material for downstream models is high-volume generation work where you control the quality bar yourself, so the capability ceiling of a small model matters far less than its cost per million words.

Legal Services

• Clause extraction across portfolios. Finding and tagging obligations, termination rights, and liability caps across thousands of contracts is the classic narrow-task-huge-volume combination, and it is precisely what tuning on your own document set improves dramatically.
• Document review prioritisation. In discovery, ordering documents by likely relevance before human review saves an enormous number of billable hours, and confidentiality argues strongly for running the model inside the firm.
• Redlining against a house playbook. Every firm has its own preferred positions. A model tuned on your own past redlines suggests changes in your house style rather than generic ones, which is the difference between a useful draft and one that gets thrown away.

Retail and E-commerce

• Product attribute extraction. Supplier feeds arrive in every format imaginable and have to become consistent structured records across millions of items. Per-item cost is the whole business case here, and at millions of items the difference between a fraction of a cent and several cents is the entire project.
• Catalogue normalisation. The same product appears under different names from different suppliers, and matching them is endless. A tuned model does this far more consistently than rules, and consistently enough to run unsupervised with spot checks.
• Review summarisation. Condensing hundreds of reviews per product into an honest summary improves conversion and is entirely repetitive – and at catalogue scale, doing it with an expensive model is simply not economic.

Telecommunications

• Network alarm triage. Alarms arrive in floods, and correlating them into probable underlying incidents quickly is what determines whether an outage lasts ten minutes or ninety. Speed matters far more than sophistication, which favours a small fast model close to the network.
• Customer intent classification. Working out what a caller actually wants in the first few seconds shapes the entire interaction, and it happens millions of times. Latency here is felt directly by the customer.
• Field technician notes. Dictated notes from the field become structured records automatically, which improves your asset data without asking technicians to fill in more forms – and models running offline handle poor coverage areas gracefully.

Energy and Utilities

• Sensor anomaly narratives. Control-room operators receive numerical anomalies and have to interpret them under time pressure. A tuned model turning those into a written explanation reduces cognitive load precisely when it is highest.
• Regulatory filing assistance. Filings are long, formulaic, and checked hard for internal consistency. A model tuned on your own past filings catches inconsistencies before a regulator does, which is worth considerably more than the compute it costs.
• Outage communication drafting. Public communications during an outage need to be fast, accurate, and consistent in tone. Drafting them from incident data means the communications team edits under pressure rather than composing under pressure.

Logistics and Supply Chain

• Shipping document extraction. Bills of lading, invoices, and customs forms arrive in dozens of languages and layouts, continuously. Gemma 4’s support for over 140 languages matters here in a way it does not in a single-market business.
• Exception handling. Classifying why a shipment is delayed and routing it to whoever can fix it is high-frequency, low-complexity work where automation directly shortens delay times.
• Multilingual carrier correspondence. Triaging incoming messages from carriers across many countries is exactly the kind of continuous multilingual work where a per-request bill becomes uncomfortable and a fixed-cost model does not.

Pharmaceuticals and Life Sciences

• Literature triage. Screening thousands of papers for relevance is the slowest part of any systematic review. A tuned model does the first pass with a documented, reproducible decision process – which matters here more than raw speed.
• Adverse event coding. Mapping reported events to standard dictionary terms is repetitive, safety-critical, and heavily audited. The decisive advantage of a local versioned model is that you can prove exactly which version produced which output, months later.
• Submission consistency checking. Regulatory submissions must agree with each other across thousands of pages. Finding contradictions before a reviewer does is high-value work that no human reliably completes at that scale.

Government and Public Sector

• Citizen request routing. Requests arrive across many departments in many formats and must reach the right team promptly. Hard data-residency rules often rule out external services entirely, making a self-hosted model the only viable option rather than merely the cheapest.
• Multilingual form processing. Checking submitted forms for completeness across the languages a population actually speaks is repetitive, high-volume, and directly affects how citizens experience the state.
• Redaction assistance. Preparing records for public release requires finding personal information reliably. A model does the first pass and a human signs off – and because the whole thing runs internally, the unredacted material never leaves the building.

Education and EdTech

• Assignment feedback drafting. Producing a first draft of feedback for an instructor to edit turns hours of marking into a review task. Volume here is seasonal and predictable, which is exactly when a fixed-cost service beats variable billing.
• Content levelling. Adapting the same material to different reading levels is formulaic transformation work, and a model tuned on your own curriculum voice does it consistently rather than approximately.
• Multilingual material adaptation. Making course content available in regional languages is a straightforward win for reach, and native support for 140+ languages means the base model already has the foundation.

Media and Publishing

• Archive tagging. Enriching large back-catalogues with topics, entities, and metadata is repetitive work that unlocks search and licensing revenue nobody currently collects, because the archive is not findable.
• Headline variant generation. Producing several headline options in house voice is style-heavy work where tuning genuinely beats prompting – because house voice is a pattern learned from thousands of examples, not a rule you can write down.
• Transcript cleanup. Cleaning transcripts and attributing speakers is continuous, high-volume, and exactly the kind of task where a small model’s cost profile makes previously uneconomic archive work suddenly viable.

Agriculture and Agritech

• Offline advisory generation. Producing crop advice in regional languages on a handset with no signal is the flagship case for the small edge models. There is no API alternative in a field with no coverage.
• Pest and disease report classification. Field submissions arrive as photos and informal text, and sorting them fast gets useful guidance back to farmers within the window where it still helps.
• Market bulletin generation. Turning price data into readable bulletins for cooperatives, in the right language, is formulaic and repetitive, and needs to run cheaply enough that it can be given away.

And this list is not exhaustive – not remotely.

Any organisation with a repetitive, high-volume, domain-specific language task is a candidate, and the twenty sectors above are illustrations, not a boundary.

If your people are doing the same language task a thousand times a week, you are on this list whether or not I happened to name your industry.

10 Pros of Fine-Tuning Gemma 4

1. The licence gives you real commercial freedom. Modify, deploy, and redistribute without a fresh legal review every time your use case shifts.
2. Cost per request collapses at volume. A fixed infrastructure bill beats a variable per-request one the moment you cross your break-even point, and it stops growing with your success.
3. Responses get faster. A small model on your own hardware often beats a remote service on total time, simply because the internet round trip disappears.
4. Data never leaves your building. That turns several long compliance conversations into one architecture diagram.
5. No vendor lock-in. Your model runs on any hardware, in any cloud, at any time you choose.
6. You own it. Nobody can retire it, reprice it, or silently update it out from under you.
7. It works offline. Google built the small models to run completely disconnected, which some environments require absolutely.
8. On your specific task, it can beat a much bigger model. Narrow task plus good data genuinely does win.
9. Costs become predictable. Your finance team gets a number instead of a forecast.
10. The hardware bar is low. A desktop graphics card is enough to get started – piece of cake.

10 Cons of Fine-Tuning Gemma 4

Now; for the other side of the coin!

1. You need real expertise. Somebody on your team has to genuinely understand how training behaves, not merely follow a tutorial.
2. The data is the expensive part, and it is expensive. As that scaling table showed, labelling costs dwarf compute at every single tier.
3. There is a capability ceiling. A small tuned model will not out-think a frontier model on open-ended novel problems, and pretending otherwise ends badly.
4. Training too hard causes forgetting. Push the model too far into your speciality and it loses general competence elsewhere.
5. Maintenance never ends. That 20–30% annual figure is permanent, not a transition cost.
6. New base models keep arriving. Every Gemma release forces an evaluation decision you cannot responsibly skip.
7. Testing is harder than training. Training is a weekend. A trustworthy test harness is a quarter.
8. Hardware is a commitment. Rented or owned, capacity planning is now your problem.
9. Liability moves to you. When the model is wrong there is no vendor to point at. It is your model, your output, your problem.
10. RAG or better prompting might have been enough. This is the most common and most expensive mistake in the entire field, and you should rule it out properly before you start.

I am not softening any of these.

If one of them is a dealbreaker for you – that is genuinely useful information, and it cost you nothing to learn it here.

Frequently Asked Questions

1. Which size should I start with?

Start with E4B, the 4-billion-parameter model.

It fits on hardware you probably already own, and it is capable enough to tell you whether your data is any good – which is the real question at the start.

2. How much data do I really need?

Somewhere between 500 and 20,000 examples, depending on the task.

Covering the awkward cases matters far more than the raw count.

3. Can I do this on a normal desktop computer?

Yes, if it has a good graphics card with 24 GB of memory.

That is enough for the small and mid-sized models. The largest one needs datacentre hardware.

4. What is the difference between RAG and fine-tuning, in one line each?

RAG gives the model documents to read before it answers – use it when the problem is missing facts.

Fine-tuning changes how the model behaves – use it when the problem is format, tone, or judgement.

5. Should I check for an existing free model first?

Absolutely, and it should be the first week of the project.

Search Hugging Face and the Gemmaverse, and test the best candidates against your own examples before committing to anything.

6. How long does the actual training take?

Hours, not days, for a typical job on a few thousand examples.

The data preparation beforehand takes weeks. That is where the project lives.

7. Can I use this commercially?

Yes. It is Apache 2.0 licensed, subject to Google’s prohibited use policy.

8. Does compressing the model hurt quality?

Compressing a finished model can.

Google publishes versions where the compression was built into training itself, and those hold up almost perfectly. Ask your team to use those.

9. What about languages other than English?

Gemma 4 handles over 140 languages natively, which makes it a strong starting point for multilingual work.

10. Can it read images and listen to audio?

All the models handle text and images. Audio works natively on the two small models and the 12B.

11. Can it actually do things, or only write?

It can call tools – look up records, query systems, trigger actions – and Google also publishes FunctionGemma, a version tuned specifically for that.

12. How do I teach it to refuse inappropriate questions?

Include explicit refusal examples in your training data, at roughly 5–10% of the total, and put a separate safety classifier in front of the model.

Never rely on the training alone for safety.

13. On our own servers, or in the cloud?

Your own servers when data residency or disconnected operation is required. Cloud when neither applies and your usage is uneven.

14. How will I know it worked?

Your test-set score beats the baseline you recorded at the start, and a human reviewer agrees when reading a blind sample.

Both, not either.

15. When should we retrain? On triggers – the score dropped, the incoming questions changed, or the business scope changed. Not on a fixed calendar alone.

16. How big a team does this need? One engineer, one subject-matter expert for the data, and part of an infrastructure person for running it.

Three people, not thirty.

17. What goes wrong most often?

Too little data and too many training passes, followed closely by test questions that accidentally appeared in the training data.

18. How do we move off our current AI vendor safely?

Run the new model quietly alongside the old one for a fortnight, compare the answers, then move a small percentage of real traffic before moving more.

19. What is p99 latency and why does everyone mention it?

It is the response time that only 1 in 100 requests exceeds.

Average speed hides your worst experiences, and your worst experiences are the ones customers tell other people about.

20. How do we know when to stop and admit it failed?

If after two full rounds of improving the data you still cannot beat your baseline by a margin that matters commercially, stop.

Killing a project cleanly is a real skill, and it is far cheaper than throwing money away on a failed system.

The Bottom Line

Fine-tune Gemma 4 if you have a narrow, repetitive, high-volume language task, a subject-matter expert who can produce clean examples, and a cost or compliance pressure that a rented service cannot solve.

Do not fine-tune if your problem is missing facts, your volume is low, or you have not yet tried a well-written prompt, a proper retrieval layer, and a serious search of what already exists for free on Hugging Face.

That is the whole decision, really.

The economics genuinely changed this year, and not because of a benchmark score.

They changed because a licence changed.

Own your model.

Own your data.

Own your costs.

Own your response times.

Own your compliance story.

And then measure all five against the baseline you wrote down on day one, because a number you never recorded is a number you cannot defend.

I strongly believe the next decade of enterprise AI belongs to the teams who worked out that the biggest model is rarely the right model.

The quiet little model humming away in your own server room will out-earn the expensive service you are currently too nervous to switch off.

Start small.

Measure everything.

Ship it.

All the very best to you.

And if you are wondering where to spend your engineering effort this year – choose the boring, measurable, domain-specific work.

It compounds.

Cheers!

References

1. Gemma 4 model overview and memory requirements — Google AI for Developers
2. Gemma 4 model card — Google AI for Developers
3. Gemma 4 launch announcement from Google — The Keyword
4. Gemma 4 Apache 2.0 licence terms — Google
5. Gemma prohibited use policy — Google
6. Gemma 4 prompt formatting guide — Google AI for Developers
7. Tuning Gemma with Hugging Face and QLoRA — Google AI for Developers
8. Full model fine-tuning with Hugging Face Transformers — Google AI for Developers
9. Unsloth training guide for Gemma 4 — Unsloth
10. Tunix tuning library documentation — Tunix
11. The Gemmaverse community model directory — Google DeepMind
12. Hugging Face model hub — Hugging Face
13. Gemma 4 model collection on Hugging Face — Hugging Face
14. ShieldGemma safety classifier overview — Google AI for Developers
15. EmbeddingGemma for search and retrieval — Google AI for Developers
16. PaliGemma vision-language models — Google AI for Developers
17. FunctionGemma overview and fine-tuning — Google AI for Developers
18. Responsible Generative AI Toolkit — Google
19. Secure AI Framework — Google
20. Running Gemma with Ollama — Google AI for Developers
21. Running Gemma with LM Studio — Google AI for Developers
22. Running Gemma with llama.cpp — Google AI for Developers
23. Running Gemma with MLX on Apple hardware — Google AI for Developers
24. Deploying Gemma on Cloud GKE — Google AI for Developers
25. Multi-Token Prediction overview — Google AI for Developers
26. Gemma release history and version timeline — Google AI for Developers
27. Gemma 4 quantisation-aware training checkpoints — Hugging Face
28. Gemini Developer API pricing page — Google AI for Developers
29. H100 cloud pricing across 40+ providers — GetDeploying
30. NVIDIA A100 on-demand pricing comparison — Thunder Compute

Appendix A — Installation

This appendix is for whoever on your team will actually do the setup.

If that is not you, hand it over – but the troubleshooting list at the end is worth skimming regardless, because those four failures explain most “it doesn’t work” emails.

What you need first.

Ubuntu 22.04 or later, Windows 11 with WSL2, or macOS 14+ on Apple Silicon.

An NVIDIA driver of version 550 or newer with CUDA 12.4+, or ROCm 6.x on AMD hardware, or Metal on Apple.

CUDA is NVIDIA’s software layer that lets programs use the graphics card for computation – nothing works without a matching version.

Set up an isolated environment. This keeps the project’s software separate from everything else on the machine.

bash

curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv gemma4 --python 3.11
source gemma4/bin/activate

Install the core packages.

bash

uv pip install torch --index-url https://download.pytorch.org/whl/cu124
uv pip install transformers peft trl bitsandbytes accelerate datasets
uv pip install unsloth   # optional, a faster training path

On Apple hardware, install mlx and mlx-lm in place of bitsandbytes.

Sign in to Hugging Face.

bash

uv pip install huggingface_hub
hf auth login

You must accept the Gemma terms on the model’s web page before your first download, or the download will fail.

Verify the setup before training anything. This loads the model and asks it to say two letters. If it prints them, your environment is sound.

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
mid = "google/gemma-4-E4B"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForCausalLM.from_pretrained(
    mid, torch_dtype=torch.bfloat16, device_map="auto"
)
msgs = [{"role": "user", "content": "Reply with OK."}]
ids = tok.apply_chat_template(msgs, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=8)[0]))

The no-code route.

If you just want to try the model out before any of this, install Ollama or LM Studio and download a compressed build.

No setup, working chat in about ten minutes.

The four failures that account for most problems.

1. Version mismatch between the driver and the software. Run nvidia-smi, read the CUDA version it reports, and reinstall to match it exactly.
2. Out of memory when loading. The model is bigger than the card. Use a more compressed version, or step down one model size.
3. Access denied on download. You have not accepted the licence terms on the model page yet. It takes thirty seconds.
4. Compilation errors on bitsandbytes. Install the prebuilt package rather than building from source, and confirm your Python version is supported.

Appendix B — Maintenance Runbook

A runbook is a checklist of routine operational tasks with the frequency attached, so nothing depends on somebody remembering.

Every day.

• Check typical and worst-case response times against normal. If the worst case has crept up, something is queuing.
• Check error and timeout rates.
• Check the refusal rate for any sudden movement in either direction.
• Confirm at least 15% spare memory on the graphics cards.

Every week.

• Read 50 real production answers, chosen at random, without knowing which is which.
• Compare how long the answers are getting against the training examples. Drift here is an early warning.
• Review anything the system flagged as low-confidence.
• Check for security advisories on the serving software.

Every month.

• Run the full test suite and record the score.
• Run the general-ability checks to confirm nothing has been forgotten.
• Run prompt-injection and adversarial tests.
• Compare cost per thousand requests against forecast.

Every quarter.

• Re-benchmark against a commercial service on the same test set, so you know whether owning the model still makes sense.
• Evaluate any newly released Gemma base model, on your data.
• Re-check Hugging Face for newly published models in your domain that might now beat yours.
• Audit data licences and retention compliance.
• Test the rollback procedure end to end. Actually run it.

Triggers that force a retrain.

• Test score falls more than three points from launch.
• Refusal rate moves more than five points either way.
• More than one in ten incoming requests looks unlike anything in the training data.
• A new product, region, or regulation enters scope.

When quality degrades, in order.

1. Freeze the current version so nothing changes underneath you.
2. Roll traffic back to the last version known to be good.
3. Collect the failing inputs into one set.
4. Reproduce the failure offline against the frozen version.
5. Fix the data, retrain, retest, and roll out gradually.
6. Write the post-incident review before closing the ticket.

What the model’s record must contain.

Name and version, which base model it came from, which dataset version trained it, the settings used, the scores at launch, known limitations, what it is for, what it is not for, who owns it, and when it was last reviewed.

What to keep and for how long.

The last three versions and their datasets, indefinitely. All training settings, indefinitely. Raw production logs for 90 days, subject to your privacy policy. Test results, forever.

Who is responsible.

One named model owner. One named data owner. One named infrastructure contact on call.

Escalate to the model owner on any retrain trigger, and to the on-call contact for speed or availability problems.

Never leave a fine-tuned model without a named human attached to it.

That’s all I have for you today, and all the best building your own finetuned SLMs!

All images AI-generated by NightCafe Studio.

The first draft of this article was produced by Claude Opus 5.