Supervised Fine Tuning (SFT) a Small Language Model (SLM)

Snode recently released our Cyber AI offerings which included (1) HeatSeeker and (2) Firestarter. HeatSeeker is an AI model (in cases - multiple models) for anomaly and threat detection (e.g., not just cyber, can also be used for fraud or collision detection). Firestarter is an AI model specifically for offensive security. Both, include custom SLM technology that can be deployed at the edge (iow, so efficient it runs on an IoT device).

Sections

1. Background for the names (optional)
2. What's a Small Language Model?
3. Why a Small Language Model?
4. How do you train and tune an SLM?
5. My SLM training and tuning methodology.
6. Data collection, synthesis and formatting.
7. Taking your SLM into a live environment.

Background for the names (optional)

This section covers the origin of the model names - feel free to skip to the next section.

You don't know what I'm talking about??? The Prodigy's "Firestarter" is the bizznizz:

HeatSeeker was incorrectly taken from the opening scene of the 1998 movie Blade:

However HeatSeeker used the same sample and he dominated dance floors in 2000:

Nice ne,... sorry,... Heatseeker (Eric from Knysna) track "Xplicit" was #1 in South Africa.

OK, enough instilling culture into the millennials - time to get down - down to business.

What's a Small Language Model?

It's like a LLM (Large Language Model), but,... you guessed it,... smaller! Wait, doesn't the "large" factor make LMMs so great? Yes, but, take a second and think about this:

1. If it's smaller, it's portable (works offline).
2. If it's smaller, it uses less SWaP resource.
3. If it's smaller,... it's probably not as good?

What if I told you: It's not as good,... but, (with some work) it's actually BETTER!!!

There are essentially 2 SLM types with 2 very different implementation use cases:

1. Edge SLM (Heatseeker) - designed for firmware, IoT, OT or mobile applications.
2. GPU SLM (Firestarter) - running on my AMD laptop with a NVIDIA Radeon GPU.

Why a Small Language Model?

So, general LLMs are generally great. They have a wide application scope and make noobs feel powerful - at everything. That's exactly, the problem,... and the opportunity. SLMs, appropriately trained and tuned, can be something specific to someone specific.

1. This makes them better at Agentic Applications (since agents are specific).
2. They can fit onto edge, mobile or IoT devices and operate without latency.
3. If properly trained, tuned and specific to a use case, they outperform LLMs.

How do you train and tune your SLM?

Well, now here is where it gets a bit technical,... there are not only multiple ways to train them. There are multiple types of training - that can happen at specific times - that only work on specific use cases. Here are a few model training & tuning options:

1. Train your own (initial base) SLM;
2. Pre-Training (e.g., CLM/ MLM);
3. Supervised Fine Tuning (SFT);
4. Preference Learning (DPO/KTO);
5. Reinforcement Learning (GRPO);
6. Knowledge Distillation (LLM->SLM);
7. Prefix and Prompt Learning; and
8. Reward Modelling methodologies.

Plus many more,... we keep training & tuning models to improve their performance.

My SLM training and tuning methodology

I have my own method of building, training and tuning models. At a high-level view:

1. Dataset - you need a good dataset in the correct format (I use JSON/JSONL).
2. Device - often overlooked, you need to know OS, RAM, CPU, GPU, SWaP, etc.
3. Model & Method - choose the best model for the use case (e.g., SmolLM+SFT).
4. Adapter (optional) - set the best adapter and parameters for peak performance.
5. Evaluation (mandatory) - you MUST evaluate the model, pivot and re-evaluate.

With experimentation, you will find a feasible setup - specific to your requirement.

Feasible - since this is a cyclic process of constant and never-ending improvement.

Dataset collection, synthesis and formatting

So, depending on the model (& the dataset) - there will be an optimal data format:

1. Often, I use JSON;
2. Or, I'll use JSONL.

I stay away from 1) data prep tools, 2) ML notebooks & 3) their output formats.

This is my personal preference (not advice). I tend to use open, portable formats.

A simple example of a sentiment analysis dataset (for X tweets) for model tuning:

{
  "id": "cb774db0d1",
  "text": " I`d have responded, if I were going",
  "label": 1,
  "label_text": "neutral"
}
{
  "id": "549e992a42",
  "text": " Sooo SAD I will miss you here in San Diego!!!",
  "label": 0,
  "label_text": "negative"
}
{
  "id": "6e0c6d75b1",
  "text": "2am feedings for the baby are fun when he is all smiles and coos",
  "label": 2,
  "label_text": "positive"
}

With regards to the data source - its still garbage-in, garbage-out. So, I will focus on:

1. Data consistency/ normalisation - make sure all the data is the correct format.
2. Data quality/ domain-specific - use domain-specific and accurate data source.
3. Data privacy/ security/ leakage - strip confidential data elements from source.
4. Data distribution/ diversity - balanced training data to ensure optimal results.
5. Data synthesis - augment the training data with high-quality synthesized data.

Taking your new SLM into a live environment

Scaling the model for use in production instances deserves it's own article. However, at a high-level, a deployment process for your SLM should consist of (at a minimum):

1. Design phase - good design & high-quality data is the golden ticket to AI success;
2. Development phase - choosing the right model(s), training and tuning process(es);
3. Deployment phase - choosing the right architecture to scale the SLM solution; &
4. Learning phase - a monitoring and feedback loop for constant SLM improvement.

Conclusion

I'll write an more articles on training edge AI models - subscribe to be notified.

As always, if I got anything wrong,...

References

The following websites serve as appropriate references for additional detail:

Google DeepMind: 01 Build Your Own Small Language Model | Google Skills

<p>In this Google DeepMind course, you will learn the fundamentals of language models and gain a high-level understanding of the machine learning development pipeline. You will consider the strengths and limitations of traditional n-gram models and advanced transformer models. Practical coding labs will enable you to develop insights into how machine learning models work and how they can be used to generate text and identify patterns in language. Through real-world case studies, you will build an understanding around how research engineers operate. Drawing on these insights you will identify problems that you wish to tackle in your own community and consider how to leverage the power of machine learning responsibly to address these problems within a global and local context. </p>

Google SkillsGoogle Skills

Small Language Models (SLM): A Comprehensive Overview

A Blog post by John Johnson on Hugging Face

Hugging FaceJohn Johnson

A Survey of Small Language Models

Small Language Models (SLMs) have become increasingly important due to their efficiency and performance to perform various language tasks with minimal computational resources, making them ideal for various settings including on-device, mobile, edge devices, among many others. In this article, we present a comprehensive survey on SLMs, focusing on their architectures, training techniques, and model compression techniques. We propose a novel taxonomy for categorizing the methods used to optimize SLMs, including model compression, pruning, and quantization techniques. We summarize the benchmark datasets that are useful for benchmarking SLMs along with the evaluation metrics commonly used. Additionally, we highlight key open challenges that remain to be addressed. Our survey aims to serve as a valuable resource for researchers and practitioners interested in developing and deploying small yet efficient language models.

arXiv.orgChien Van Nguyen

Small language model - Wikipedia

Wikimedia Foundation, Inc.Contributors to Wikimedia projects