NER-aware fine-tuning a Multi-entity Sentiment Model with Gemma 3

July 26, 2026Project

Fine-tuning the Sentiment Analysis revisited

In the previous post, I introduced fine-tuning a local LLM (Llama 3.2) for the multi-entity sentiment analysis task.

The (fine-tuned) model was mainly to be able to extract sentiments of each entity in the text with those entities' relations in mind.

The model was instructed to output the following properties.

• Entity: (Subject and object)
• Polarity: + / - / 0 / ~
• Category: Legal / Business / Performance / Recruitment / NewsRelease / Bankruptcy

The sentiment model had largely been running in isolation for each own purpose, i.e., improved sentiment extractions.

The model's increased roles and the need for a new tuning

NER-aware fine-tuning

As the volume of texts being analyzed grew, this model started to have a larger role in the text analyses pipelines. That is, instead of text analyses being done with base model-only, the task required collaboration of inference with base model along with sentiment model's outputs (i.e., NER value for downstream filtering).

For instance, in the financial literature, it's not uncommon that some person names are being confused by the system as company names (e.g., PulteGroup vs. Bill Pulte).

Initially, a simple prompt modification to the fine-tuned model (i.e., adding NER categorization to the output) worked fine at least to identify PERSON/ORG.

However, as the volume of texts being analyzed grew, more delicate edge cases arose and the model couldn't identify well enough in its own. Specifically, to focus on the financial text only, I needed to filter out geographical entities (e.g., cities, states, countries).

Unfortunately, the model couldn't reliably distinguish the entities as expected (misidentifying them as organizations).

Enriching more various text structures

As text volumes grew and I started to spot-check heavier volumes, I noticed some patterns of the text structures which I expected to be filtered out.

For instance, banks' analysis of economic outlook is a neutral statement, not reflecting the banks' sentiment in itself. The following is one of the training samples to guide the model to better identifying such cases.

{"id": "s768", "text": "Consumer spending is expected to soften in the coming quarter, Citigroup analysts wrote in a note to clients.", "extractions": []}

Model experiment

In previous version, I settled to Llama 3.2 3B model (as in post) previously as it could hit the balance between the model size and the accuracy. However, with the added data samples, I compared different models for a possible transition to more recently released models (i.e., Gemma 3, Qwen 3).

Interestingly, unlike in the previous experiments where there were fewer training samples and one less output property (excluding NER) was produced, recent models seemed to pick up the new task well. To be fair, it was also true that the focus of this tuning step shifted the task to handle more diverse and strict requirements.

For instance, the primary focus of the initial model was to accurately extract the sentiments of each entity with contrasting sentiments as in the following example.

Also dragging on spirits was disappointment over Amazon's earnings released late Thursday. The performance of its AWS cloud unit failed to live up to lofty expectations set by rivals Google and Microsoft, sending its shares down over 8%. But Apple stock rose after its results beat expectations, boosted by surprisingly strong iPhone sales.

The expected outcomes were as follows.

• Amazon: -
• AWS: -
• Google: +
• Microsoft: +
• Apple: +
• iPhone: +

However, now, to better focus on the main parts of the text, the model is expected to output as in the following.

• Amazon: -
• AWS: -
• Apple: +

In short, unlike previously, with inserted clauses (i.e., Google and Microsoft's recent performances), the model is expected to skip those parts and newer models started to prove themselves in handling the revised task.

Revised fine-tuning properties

In addition to the previous three properties (entity, polarity, and category), with the help of Claude, I added a new property entity_type which records NER, making the output properties as below.

• Entity: (Subject and object)
• Polarity: + / - / 0 / ~
• Category: Legal / Business / Performance / Recruitment / NewsRelease / Bankruptcy
• Entity_type: ORG / PERSON / GPE / OTHER

Model Comparisons

While I had been using Llama model primarily, it is known that Meta hasn't released open-source LLMs for a while and the Llama 3.2 is getting older day by day. Therefore, despite its decent performance, I started to reconsider other more recent models including Gemma-family which didn't make good impression in the previous attempts.

Interestingly, with the modified task requirements and added training samples, Gemma started to shine against strengthened evalulation scenarios. With Llama out of the race (admittedly, I put more weights to more recent models as long as they perform comparable to Llama), here is the comparison between Gemma 3 and Qwen 3 (Evaluation sample size: 33).

Comparing these metrics, while Gemma had edges on Recall, Qwen did on Precision. In other words, theoretically Gemma-based model is able to capture more entities while Qwen-based one could do more accurately as long as they were extracted.

That being said, we all know that benchmark doesn't tell the whole story and obviously, the sample size was small. In fact, dry-run on the actual text sample revealed that while neither models were perfect, Gemma extracted the entities better especially with its superior NER extraction ability.

Using the model

From the huggingface repo, the model can be used as follows.

Note: the model hosted on HF is 8bit quantized version, as the same quantization (4bit) is known to underperform of local adapater-based model. I also noted this in previous post.

from mlx_lm import load, generate

model, tokenizer = load("staedi/sentiment-gemma-3")

prompt = (
    "You are a financial analyst specializing in directed sentiment extraction. "
    "Given a financial news text, identify all mentioned entities and determine "
    "the sentiment directed toward each one. Return your answer as a JSON array "
    "where each element has: \"entity\" (name), \"entity_type\" (\"ORG\" for "
    "companies/organizations, \"PERSON\" for individuals, \"GPE\" for countries/"
    "cities/regions, \"OTHER\" for anything else), \"polarity\" (+ positive, "
    "- negative, 0 neutral, ~ context-dependent), and \"category\" (one of: Legal, "
    "Business, Performance, Recruitment, NewsRelease, Bankruptcy)."
)

text = "Apple announced its earnings. The company performed well."
user_content = f"Extract the directed financial sentiment from the following text:\n\n{text}"

if tokenizer.chat_template is not None:
    messages = [{"role": "user", "content": prompt}]
    prompt = tokenizer.apply_chat_template(
        messages, tokenize=Falsse, add_generation_prompt=True, return_dict=False,
    )

response = generate(model, tokenizer, prompt=prompt, verbose=False)