A news site receives several hundred stories a day and must file each one under the right section. An online store has to sort customer feedback into "delivery", "payment" or "warranty". Both are text classification problems: given a piece of text, predict its label from a fixed set of labels. Before reaching for a large language model, a text classification pipeline built from word segmentation, TF-IDF and a linear model already gives solid results, runs in a few hundredths of a second and is easy to explain. This article builds that text classification pipeline for Vietnamese, measures every decision with real numbers and points out where to doubt the numbers.
This text classification article follows Object detection with YOLO: training on your own data, moving from computer vision to language. It reuses the workflow from Machine learning with scikit-learn: Pipeline, cross-validation and tuning and the metrics from Model evaluation for classifiers: precision, recall, ROC and imbalanced data. Environment: Apple M4, 16 GB RAM, Python 3.12, scikit-learn 1.4, NumPy 1.26, pyvi 0.1.1. All code lives in code/s7-25. The examples are Vietnamese sentences, so a few Vietnamese words stay untranslated in code and output; the prose explains them, and the category names are translated into English.
The text classification task: 10 Vietnamese news categories
The data for this text classification article is VNTC (Vietnamese News Text Classification Corpus), a public corpus by Vu Cong Duy Hoang, released under the MIT license according to the LICENSE file in the repository. Version 1.1 has 33,759 training stories in 10 categories (Politics & Society, Lifestyle, Science, Business, Law, Health, World, Sports, Culture, Computing), taken from four newspapers (VnExpress, Tuoi Tre, Thanh Nien, Nguoi Lao Dong) according to the bundled Stats.txt.
The stories date from the 2000s (one sample mentions the 2004 university admissions season), so the news vocabulary is dated. That does not get in the way of learning the workflow, but do not expect this model to understand today's news. Copyright for each article belongs to the original newspapers, so this text classification article uses only the opening of each story, for research purposes. For internal data such as customer feedback, mask phone numbers and addresses before training.
The Train_Full.rar file is about 49 MB, extracts to about 207 MB of UTF-16 text files and is not shipped in code/s7-25. To reproduce the results, download it from the VNTC repository and run python prepare_data.py Train_Full (the data/ folder keeps only sample.jsonl, about 2 MB). The classes are quite uneven (Sports 5,298, Science 1,820), so prepare_data.py truncates each story to its first 60 syllables (headline plus opening), deduplicates, then draws a stratified sample of 500 stories per category with a fixed seed, 5,000 stories in total. Deduplication runs on the whole set before splitting: the key is the first 25 syllables, lowercased and stripped of punctuation, and every key that appears two or more times (even with different labels) has all its copies removed. Real output of this step:
total files: 33759, remaining after dropping too-short stories: 33759
duplicate keys (first 25 syllables): 62, of which with different labels: 50; all copies of these keys removed
unique stories per label (raw files -> after deduplication):
Politics & Society 5219 -> 5202
Lifestyle 3159 -> 3144
Science 1820 -> 1786
...
saved 5000 articles to data/sample.jsonl
Only 62 duplicate keys, so the effect is small, but 50 of them have different labels: the same story appears in two categories. If such a story leaked into both training and test sets, it would inflate the score.
One limit remains: this deduplication only catches stories that are identical in their first 25 syllables. Near-duplicates (the same event with a few sentences edited) can still land on both sides and make the score slightly higher than reality. After that, an 80/20 stratified split gives 4,000 training stories and 1,000 test stories, exactly 100 per class in the test set. Because the classes are balanced, random guessing gets about 10% and accuracy can be read directly.
Word segmentation for text classification: why whitespace is not enough
Vietnamese is written with spaces between syllables, not between words. "Ngân hàng Nhà nước" (the State Bank) is four syllables but only two words, and "lãi suất" (interest rate) means something quite different from "lãi" (profit) and "suất" (rate, share) taken separately. If you only split() on whitespace, the vocabulary fills up with ambiguous units such as "nhà" (house), "giá" (price) and "hàng" (goods). Word segmentation joins syllables into words and marks them with underscores. Two popular tools are pyvi (pure Python, MIT license, CRF model, installed with pip install pyvi) and RDRSegmenter in VnCoreNLP (needs Java 1.8 or later, NAACL 2018 paper).
For a linear pipeline, pyvi is the compact choice because it adds no Java dependency. A real example:
from pyvi import ViTokenizer
demo = "Ngân hàng Nhà nước giữ nguyên lãi suất điều hành trong quý này"
print(ViTokenizer.tokenize(demo))
# Ngân_hàng Nhà_nước giữ nguyên lãi_suất điều_hành trong quý này
Segmenting 5,000 stories with pyvi takes 1.4 seconds on the M4, so this cost is negligible. Segmentation models also make mistakes, especially on product names and new terms. If you later move to PhoBERT, the README of VinAI PhoBERT requires input text to be pre-segmented with RDRSegmenter, so getting familiar with this step now is not wasted effort.
TF-IDF: turning text into sparse vectors
To do text classification with a linear model, each story must become a numeric vector. The simplest way is a bag of words: each column is a word and each cell counts how often that word occurs. The problem is that common words such as "của" (of), "và" (and) and "trong" (in) appear everywhere and carry little information. TF-IDF multiplies the frequency within a story (tf) by the rarity across the whole set (idf). According to the scikit-learn Feature extraction documentation, with smooth_idf=True (the default), idf(t) = ln((1 + n) / (1 + df(t))) + 1, and each vector is then divided by its L2 norm. This differs slightly from the textbook formula (ln(n / df)), so it is worth verifying by hand on three sentences:
docs = ["giá vàng tăng", "giá xăng giảm", "giá vàng giảm mạnh"]
v = TfidfVectorizer(); X = v.fit_transform(docs)
n = len(docs)
df = np.array([sum(t in d.split() for d in docs) for t in v.get_feature_names_out()])
idf_manual = np.log((1 + n) / (1 + df)) + 1
vocabulary: ['giá', 'giảm', 'mạnh', 'tăng', 'vàng', 'xăng']
idf : [1. 1.2877 1.6931 1.6931 1.2877 1.6931]
idf manual : [1. 1.2877 1.6931 1.6931 1.2877 1.6931] | match: True
sentence 1 vector (manual) : [0.4254 0. 0. 0.7203 0.5478 0. ]
sentence 1 vector (sklearn) : [0.4254 0. 0. 0.7203 0.5478 0. ]
The word "giá" (price) is in all three sentences, so its idf is exactly 1 (the lowest possible), while "tăng" (rise) appears in only one sentence and gets the highest idf. In the vector for sentence 1, "tăng" receives the largest weight (0.7203) even though every word occurs once.
Two other parameters matter. sublinear_tf=True replaces tf with 1 + ln(tf) so a word repeated ten times does not weigh ten times as much. ngram_range=(1, 2) adds adjacent word pairs, which preserves some word order. With min_df=2 we drop pairs that occur in only one story.
Linear models for Vietnamese text classification
Linear models suit text classification because the dimensionality is very high (tens of thousands of words) while there are only a few thousand samples; in that space the classes are often nearly linearly separable. According to the LinearSVC documentation, this class uses liblinear, defaults to loss='squared_hinge', penalty='l2' and C=1.0, and trains one-vs-rest, one classifier per category. In scikit-learn 1.4, leaving dual unset raises a FutureWarning, so the example sets dual="auto". It scales better to many samples than a kernel SVC. We wrap TF-IDF and the classifier in one Pipeline so the transformation learns only from the training set:
pipe = Pipeline([
("tfidf", TfidfVectorizer(ngram_range=(1, 2), min_df=2, sublinear_tf=True)),
("clf", LinearSVC(C=1.0, dual="auto")),
])
pipe.fit(Xtr_seg, ytr) # Xtr_seg: stories already passed through ViTokenizer.tokenize
pred = pipe.predict(Xte_seg)
Because pyvi learns nothing from the labels, segmenting the whole dataset once does not leak information. But TfidfVectorizer must sit inside the pipeline; otherwise idf is computed on the test set too, exactly the mistake described in the article on feature engineering for tabular data.
Text classification results: what segmentation and n-grams add
For this text classification task, the first run fixes the model at LinearSVC(C=1) and varies the representation. The last column is training time, not counting segmentation:
syllables, unigram acc=0.8010 macroF1=0.8005 features= 8624 fit=0.15s
syllables, 1-2 gram acc=0.8230 macroF1=0.8225 features= 31555 fit=0.36s
pyvi words, unigram acc=0.8240 macroF1=0.8235 features= 16679 fit=0.17s
pyvi words, 1-2 gram acc=0.8230 macroF1=0.8222 features= 26641 fit=0.25s
pyvi words, 1-2 gram, no sublinear acc=0.8230 macroF1=0.8228 features= 26641 fit=0.32s
raw-count bag-of-words + LinearSVC acc=0.8050 macroF1=0.8049 features= 26641 fit=0.30s
At a glance, this text classification table shows that segmentation lifts macro-F1 from 0.800 to 0.824 at the unigram level, syllable bigrams do about as well (0.8225), and TF-IDF beats raw counts by about 1.7 points. Dropping sublinear_tf changes almost nothing on this data (0.8228 versus 0.8222). But a single split of 1,000 test stories has a sizeable error (at accuracy 0.82 the theoretical standard deviation of the measurement is about 1.2 points), so I ran a stratified 5-fold cross-validation (cv_check.py, C=0.3) to see which effects hold up:
syllables, unigram macroF1 = 0.8120 ± 0.0061
syllables, 1-2 gram macroF1 = 0.8304 ± 0.0069
words, unigram macroF1 = 0.8285 ± 0.0045
words, 1-2 gram macroF1 = 0.8251 ± 0.0064
The more honest conclusion for text classification: segmentation helps by about 1.7 points when you use only unigrams (0.8285 versus 0.8120, larger than the fold-to-fold standard deviation of 0.005 to 0.007), but once you use syllable bigrams the advantage disappears. Syllable bigrams reach 0.8304, slightly ahead of word bigrams (0.8251), a difference smaller than the standard deviation.
The reason is easy to see: adjacent syllable pairs such as "lãi suất" already capture most compound words. So if your system should avoid an extra dependency, syllable bigrams are a cheap starting point and nearly as good. For Vietnamese text classification, segmentation still has its own value: word unigrams reach almost the same score as syllable bigrams with 16,679 features instead of 31,555 (counted on the training set of the single split), and the top-weighted words are easier to read when you explain the model to users.
Comparing linear models on the same features
LinearSVC C=1 acc=0.8230 macroF1=0.8222 fit=0.34s
LinearSVC C=0.3 acc=0.8270 macroF1=0.8270 fit=0.37s
LogisticRegression C=10 acc=0.8270 macroF1=0.8278 fit=1.70s
ComplementNB acc=0.8250 macroF1=0.8232 fit=0.31s
For text classification, the four models are within 0.6 macro-F1 points of each other (0.8222 to 0.8278) and under half a point of accuracy, much smaller than the error of a single split, so you should not conclude that any one "wins". What matters for text classification is that all four train in under 2 seconds. Logistic regression is about five times slower but gives probabilities that can be calibrated, useful when you need a rejection threshold ("not sure, hand it to a person"). ComplementNB is the cheapest and a good baseline for imbalanced data.
Reading text classification errors: where the model gets confused
The 82% accuracy of this text classification model hides differences between classes. The classification_report for the C=1 model with word 1-2 grams (4 middle classes omitted; all 10 classes are in output.txt):
precision recall f1-score support
Politics & Society 0.753 0.700 0.725 100
Science 0.769 0.700 0.733 100
Law 0.888 0.870 0.879 100
Sports 0.933 0.980 0.956 100
Culture 0.779 0.810 0.794 100
Lifestyle 0.725 0.790 0.756 100
accuracy 0.823 1000
In this text classification run, Sports reaches an F1 of 0.956 because its vocabulary is distinctive (hlv "coach", đội "team", trận "match", cầu_thủ "player"). Politics & Society (0.725), Science (0.733) and Lifestyle (0.756) are lowest. The most frequent confusions in the confusion matrix: Lifestyle predicted as Culture 11 times, Science as Lifestyle 8, Law as Politics & Society 7, Science as Health 7.
These are not entirely model errors, because newsroom category boundaries are blurry: a court case can sit under Law or Politics & Society, and a food story under Lifestyle or Culture. When accuracy stalls at some ceiling, read a few dozen misclassified stories before changing the text classification algorithm, because the labels themselves may be what needs fixing.
What the model learned: top-weighted features and a source-leak warning
A big advantage of linear text classification is that you can read the coefficients. Take the 8 highest-weighted features of each class (coef_ paired with get_feature_names_out()); most look reasonable:
Politics & Society giáo_dục, trường, nlđ, du_lịch, dân, đô_thị, du_học, các trường
Science nghiên_cứu, loài, khoa_học, vũ_trụ, trái_đất, con, các nhà, robot
Law công_an, án, bị_cáo, tand, vụ, nguyên, dũng, tội_phạm
World iraq, tổng_thống, thủ_tướng, hôm_qua, iran, tq, hạt_nhân, palestine
Sports hlv, đội, trận, cầu_thủ, bóng_đá, đội_tuyển, clb, vđv
Computing phần_mềm, internet, máy_tính, website, máy, microsoft, mạng, dữ_liệu
Lifestyle món, chị, món ăn, lòng, anh, ăn, quán, nguyên_liệu
A few signs deserve suspicion. The Politics & Society class contains "nlđ", an abbreviation of a newspaper's name, so the model learned the source name rather than only the topic. The class also contains "giáo_dục" (education), "trường" (school) and "du_học" (study abroad) because VNTC has no separate Education category, so school stories land here (a consequence you will see in the prediction section below). "nguyên" and "dũng" in the Law class are probably personal names in the stories, not topic words. For text classification, I check for source leakage with leak_check.py: count the stories that contain a newspaper name (NLĐ, VnExpress, Tuổi Trẻ online...), delete the names and retrain.
stories containing a newspaper name, by category: {'Politics & Society': 82, 'Law': 71, 'Lifestyle': 5, 'Sports': 25, 'Business': 19, 'Computing': 3, 'Health': 7, 'Culture': 8}
macro-F1 unchanged : 0.8270
macro-F1 after removing names: 0.8270
In this text classification sample, 220 of the 5,000 stories carry a source name (82 in Politics & Society, 71 in Law, mostly "NLĐ"), and removing them does not change macro-F1 at four decimal places. So in this sample the leak exists but is not the source of the 82% figure.
Note that the check only uses a list of newspaper names that I wrote myself. The leak could be in other signals (capitalization style, each newsroom's own punctuation) that this test cannot catch. The lesson for text classification is the habit: always inspect the top features and try removing anything that is not content (newspaper names, email signatures, ticket numbers) before trusting the result.
Saving the text classification model and predicting new stories
The text classification pipeline can be packaged with joblib.dump; the resulting file is about 4.9 MB. At prediction time, remember to apply exactly the same segmentation step used in training, because the model only knows tokens such as lãi_suất. Four new headlines I wrote myself, real output:
Sports <- Đội tuyển Việt Nam thắng đậm trong trận chung kết lượt đi
Business <- Giá vàng SJC tăng mạnh, ngân hàng điều chỉnh lãi suất tiền gửi
Politics & Society <- Bộ Giáo dục công bố phương án thi tốt nghiệp trung học phổ thông
Business <- Căn hộ chung cư tại Hà Nội tăng giá, nhà đầu tư quay lại thị trường
The predictions are worth reading more closely than just marking right or wrong. The headline about the national team winning the first-leg final and the one about gold prices and deposit rates came out as expected. The headline about the high school graduation exam was filed under Politics & Society and the one about apartment prices under Business, not because the model is broken but because the VNTC label set has no Education or Real Estate category: the model can only choose among the 10 labels it learned and picks the nearest one.
This is a lesson about label scope in text classification, and four hand-written headlines do not prove overall quality; the 82% on the test set is the real measure, with the caveat that it was measured on stories from the 2000s. A joblib file is really a pickle, so load only files you created yourself, and pin the scikit-learn version as the article on Python environments for data and AI projects explains.
When not to use this text classification approach
- You need semantics: TF-IDF text classification sees words, not meaning; "không hài lòng" (not satisfied) and "hài lòng" (satisfied) share almost the whole vocabulary, so a TF-IDF model struggles with negation and sarcasm. Sentiment analysis usually needs a contextual model such as PhoBERT.
- Out-of-vocabulary words: new product names, slang, abbreviations and Vietnamese written without diacritics will be missing from the vocabulary; adding character n-grams (
analyzer="char_wb") helps partially. - Blurry labels or too little data: like Politics & Society and Lifestyle above, a stronger architecture cannot rescue contradictory labels.
- Labels that drift over time: topic drift in text classification (epidemics, elections) makes old vocabulary useless; monitor and retrain periodically.
Conversely, for text classification with a few thousand well-labeled samples, millisecond latency on CPU, a need to explain results to operators and a need for a baseline before trying a large model, this approach is the reasonable and cheapest choice.
Common text classification mistakes
- Fitting TF-IDF on both train and test: idf absorbs information from the test set. Always keep the vectorizer inside a
Pipeline. - A stop-word list that does not match the segmentation: the default
token_patternkeepslãi_suấtintact because\wcounts the underscore as a word character, but the scikit-learn documentation warns that stop words must go through the same preprocessing as the vectorizer. - Trusting a single train/test split: a 1-point gap is within noise; use cross-validation and report the standard deviation.
- Not reading the top features: you miss leaks such as newspaper names, ticket IDs and signatures.
- Deduplicating only within each label or after splitting: duplicate stories crossing into both train and test give a falsely high score; deduplicate the whole set before splitting, and remember that near-duplicates still slip past an exact-match key.
- Forgetting to segment at prediction time: the model learned tokens like
lãi_suất, and feeding it the raw string sends every word into the unknown bucket.
Vietnamese text classification checklist
- For every text classification project, balance the classes or at least stratify the split; record the seed.
- Start with TF-IDF on syllable bigrams and compare against segmentation with cross-validation, not with a single split.
- Put the vectorizer and the classifier in the same
Pipeline. - Look at the per-class report, the confusion matrix and the top-weighted features.
- Try removing source names and signatures before trusting a text classification score.
- Pin library versions and load only models you created yourself.
The text classification source code consists of prepare_data.py, classify_demo.py, cv_check.py and leak_check.py, with output.txt and requirements.txt; it ran on an Apple M4, 16 GB, Python 3.12 with a 5,000-story sample. Every number in this article is real output from those runs and will change if you draw a different sample.
Next article
Word embeddings: Word2Vec, FastText and representing word meaning. TF-IDF treats "ô tô" and "xe hơi" (both mean "car") as two unrelated columns; the next article trains embeddings so that words with the same meaning end up close together, and measures that with real numbers.
