Written and maintained byCASRAI Editorial Board
The sklearn f1-score is computed using the f1_score function from Python’s sklearn.metrics module, calculating the harmonic mean of precision and recall to evaluate classification models on imbalanced research datasets.
What it is used for in research
In quantitative empirical research, evaluating supervised machine learning models solely through overall classification accuracy introduces severe methodological bias when target classes are unequally distributed. In domains such as clinical diagnostics, epidemiological disease screening, genomics, social science content analysis, and econometric fraud detection, the condition of interest (such as a rare pathology or event) frequently comprises less than five percent of the overall cohort. Under these conditions of severe class skew, a trivial null classifier that assigns all observations to the negative majority class will exhibit an accuracy exceeding 95 percent while remaining completely incapable of identifying true positive instances.
The sklearn.metrics.f1_score function resolves this deficiency by computing the balanced F-measure, representing the harmonic mean of precision (positive predictive value) and recall (sensitivity). By punishing extreme discrepancies between precision and recall, the F1-score ensures that a model cannot achieve an inflated score if it sacrifices sensitivity for specificity or vice versa. Researchers rely on the scikit-learn implementation to validate classifiers against cross-sectional and longitudinal cohorts, compare algorithmic inductive biases, and report reproducible model performance in peer-reviewed scientific publications.
While the mathematical foundation is covered in our conceptual guide to the F1 score formula, empirical analysis in scientific software requires managing computational workflows, sample weighting, multi-label classifications, and multi-class aggregations. In academic workflows, calculating the F1-score is routinely paired with diagnostic tools such as the sklearn confusion matrix and the sklearn precision-recall curve to understand the full spectrum of true positives, false positives, and false negatives across decision thresholds. A comprehensive understanding of the underlying precision and recall formula allows investigators to choose appropriate threshold cut-offs that align with domain-specific tolerance for Type I versus Type II scientific errors.
Install or access
The f1_score function is distributed as part of the official scikit-learn software library. In compliance with Python packaging standards, the distribution package is named scikit-learn, whereas the internal Python module is imported as sklearn. Researchers should install the package within an isolated virtual environment (such as venv or conda) to ensure computational reproducibility across research group workstations.
To install or update the package from the Python Package Index (PyPI), execute the following command in your terminal:
python -m pip install -U scikit-learnFor scientific environments managed through Conda, install the library from the community-maintained conda-forge channel:
conda install -c conda-forge scikit-learnAccording to the official scikit-learn documentation, the library requires Python (version 3.9 or higher), NumPy (version 1.19.5 or higher), SciPy (version 1.6.0 or higher), joblib (version 1.2.0 or higher), and threadpoolctl (version 3.1.0 or higher). To verify that the software is properly installed and check the active library version in your Python script or Jupyter Notebook, execute:
import sklearn
print("Scikit-learn version:", sklearn.__version__)
from sklearn.metrics import f1_score
print("f1_score import successful.")For researchers configuring complete scientific computing stacks, our broad scikit-learn tutorial for researchers provides detailed guidance on data pipelines, feature preprocessing, cross-validation architectures, and formal citation practices in academic papers.
Step-by-step task
Computing the F1-score in scikit-learn follows an explicit sequential workflow: data preparation, train-test splitting, model training, discrete class prediction, and metric computation. The following numbered steps demonstrate an end-to-end reproducible protocol using an imbalanced synthetic classification benchmark representing a biomedical cohort study.
Step 1: Import required modules and functions
Begin by importing the dataset generation utility, data splitting helper, machine learning classifier, and evaluation metrics from scikit-learn:
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import f1_score, classification_report, confusion_matrixIn scientific studies with imbalanced target classes, preserving class proportions across training and testing partitions is essential. Use stratify=y in train_test_split to prevent test partition distribution shifts:
# Generate an imbalanced synthetic research cohort: 90% negative, 10% positive
X, y = make_classification(
n_samples=1000,
n_features=10,
n_informative=6,
weights=[0.90, 0.10],
random_state=42
)
# Partition data into 70% training and 30% testing subsets
X_train, X_test, y_train, y_test = train_test_split(
X, y,
test_size=0.30,
random_state=42,
stratify=y
)Step 3: Fit the predictive estimator and generate discrete predictions
Train the classifier on the training partition and generate discrete class label predictions on the unseen test set using predict():
# Initialize and fit a random forest estimator
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
# Predict discrete class labels (required by f1_score)
y_pred = model.predict(X_test)Note that f1_score requires discrete binary or multiclass category indicators (such as 0 or 1), not continuous probability estimates. If operating with predicted probabilities from model.predict_proba(X_test)[:, 1], you must apply an explicit decision threshold prior to passing values into f1_score.
Step 4: Compute the binary F1-score
For standard binary classification problems, invoke f1_score with default parameters or explicitly identify the positive target class using pos_label:
# Calculate standard binary F1-score for class 1
binary_f1 = f1_score(y_test, y_pred, pos_label=1)
print(f"Test Partition Binary F1-Score: {binary_f1:.4f}")Step 5: Compute multiclass F1-scores using averaging strategies
When target variables contain three or more diagnostic categories (e.g., disease severity stages), the average parameter dictates how individual class performances are synthesized into a scalar summary:
# Macro-average: unweighted mean of all per-class F1-scores
macro_f1 = f1_score(y_test, y_pred, average='macro')
# Micro-average: global count of TP, FP, and FN across all classes
micro_f1 = f1_score(y_test, y_pred, average='micro')
# Weighted-average: per-class F1 weighted by support (instance frequency)
weighted_f1 = f1_score(y_test, y_pred, average='weighted')
# Per-class scores: returns an array without aggregation
per_class_f1 = f1_score(y_test, y_pred, average=None)
print(f"Macro F1: {macro_f1:.4f}")
print(f"Micro F1: {micro_f1:.4f}")
print(f"Weighted F1: {weighted_f1:.4f}")
print("Per-class F1:", per_class_f1)Step 6: Integrate F1-scoring into k-fold cross-validation
In rigorous academic studies, reporting performance from a single static split is insufficient. Integrate the F1 metric directly into scikit-learn cross-validation pipelines
from sklearn.model_selection import cross_val_score
# Execute 5-fold cross-validation using macro F1 scoring
cv_scores = cross_val_score(model, X, y, cv=5, scoring='f1_macro')
print(f"5-Fold Cross-Validation Macro F1: {cv_scores.mean():.4f} (+/- {cv_scores.std():.4f})")Interpreting the numerical output returned by sklearn.metrics.f1_score requires understanding the mathematical consequences of parameter configurations, particularly the average argument and boundary conditions.
The returned value is either a Python floating-point scalar ranging between 0.0 and 1.0 (when average is set to 'binary', 'macro', 'micro', or 'weighted') or a 1D NumPy array of floats (when average=None). An F1-score of 1.0 indicates perfect classification with zero false positives and zero false negatives, while 0.0 indicates complete failure to correctly identify positive instances.
Averaging options in multi-class classification
When working with multi-class datasets, scikit-learn offers four distinct aggregation options:
- Macro averaging (
average='macro'): Computes the individual F1-score for each category independently and takes their unweighted arithmetic mean. Under macro averaging, every diagnostic class carries equal mathematical weight, regardless of whether a class contains 10 samples or 10,000 samples. In biomedical research, macro averaging is the standard recommendation for imbalanced problems because poor identification of rare disease states immediately penalizes the aggregate score. - Micro averaging (
average='micro'): Aggregates total true positives, false positives, and false negatives globally across all categories before calculating a single combined F1-score. Mathematically, in any single-label multi-class problem where each observation belongs to exactly one category, micro-precision, micro-recall, and micro-F1 are algebraically identical to overall classification accuracy. Researchers should avoid reporting micro-F1 as an alternative to accuracy on imbalanced datasets, as it conceals minority-class failure behind majority-class prevalence. - Weighted averaging (
average='weighted'): Calculates the F1-score for each category independently and calculates the weighted mean, using the true support (number of true instances) of each category as weights. While weighted averaging accounts for unequal group sizes, it can obscure poor performance on clinically critical minority classes if the classifier performs well on abundant classes. - Per-class breakdown (
average=None): Returns an array containing the individual F1-score for each category in numerical or lexicographical label order. This provides transparent visibility into specific class vulnerabilities and should always be examined alongside summary scalars.
Synthesizing F1-score with classification reports
To verify the exact precision, recall, and support values underlying an aggregate F1-score, researchers should combine f1_score with sklearn.metrics.classification_report. The classification report displays a comprehensive tabular summary detailing precision, recall, F1-score, and support counts across all individual classes, along with macro and weighted averages.
Troubleshooting
When applying sklearn.metrics.f1_score to complex research datasets, analysts commonly encounter specific warnings and configuration errors. Below are documented diagnostic protocols for resolving these issues.
1. UndefinedMetricWarning due to zero predicted positive samples
Symptom: The interpreter emits an UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 due to no predicted samples. Use zero_division parameter to control this behavior.
Cause: In binary classification, if the model predicts zero instances for the positive class, both true positives and false positives equal zero (TP + FP = 0). Consequently, precision is mathematically undefined (division by zero), causing the harmonic mean formula to fail.
Resolution: Pass the zero_division argument to define explicit fallback behavior. Setting zero_division=0 silences the warning and returns 0.0, which accurately reflects that no positive cases were recovered. In contrast, setting zero_division=1.0 is appropriate only in specific edge cases where predicting zero positives in a cohort genuinely devoid of positive instances is considered a correct classification.
# Suppress warning and explicitly set undefined scores to 0.0
clean_f1 = f1_score(y_test, y_pred, zero_division=0)2. ValueError: Target is multiclass but average=’binary’
Symptom: Scikit-learn raises ValueError: Target is multiclass but average='binary'. Please choose another average setting, one of [None, 'micro', 'macro', 'weighted'].
Cause: The ground truth vector y_true or prediction vector y_pred contains more than two distinct class values, but the function defaulted to average='binary'.
Resolution: Explicitly designate a multi-class averaging strategy, such as average=’macro’ or average=’weighted’. If you intended to evaluate one specific class in a multi-class problem against all other classes, pass average=’binary’ alongside the designated class label
3. ValueError with non-standard binary labels
Symptom:ValueError: pos_label=1 is not a valid label: array(['control', 'treated']).
Cause: By default, pos_label is initialized to integer 1. When research datasets use string labels, booleans, or alternative integer encodings (such as -1 and 1), the function fails to find label 1 in the label set.
Resolution: Explicitly pass the string or integer corresponding to the target class
# Evaluating binary clinical study with string labels
f1 = f1_score(y_test, y_pred, pos_label='treated')Symptom: High memory consumption, prolonged runtime, or ValueError: continuous is not supported.
Cause: Analysts mistakenly pass continuous predicted probabilities from predict_proba(X_test) or decision_function(X_test) directly into f1_score instead of discrete binary decisions.
Resolution: Convert continuous probabilities to discrete decisions using an explicit decision boundary (e.g., 0.50 or an empirically optimized threshold determined by precision-recall analysis):
# Extract positive class probabilities
y_proba = model.predict_proba(X_test)[:, 1]
# Apply custom decision threshold
threshold = 0.35
y_custom_pred = (y_proba >= threshold).astype(int)
# Calculate F1-score on thresholded discrete predictions
optimized_f1 = f1_score(y_test, y_custom_pred)Free alternatives
While scikit-learn is the standard tool for CPU-based machine learning in Python, alternative scientific computing frameworks implement F1-score calculations for deep learning, distributed environments, and alternative programming languages. The data table below compares scikit-learn with prominent open-pport
Researchers working with deep neural networks implemented in PyTorch typically use Torchmetrics, which computes F1-scores directly on GPU tensors without moving memory back to host RAM. In the R statistical ecosystem, the yardstick package (part of tidymodels) and caret provide native equivalents that integrate with R formula syntax. In lightweight or embedded Python environments where full scientific dependencies cannot be installed, a direct NumPy calculation provides identical numerical results with minimal overhead.
| Software / Library | Primary Language | Hardware Acceleration | Data Structure | Default Averaging | Primary Research Context |
|---|---|---|---|---|---|
| scikit-learn (f1_score) | Python | CPU (NumPy/SciPy) | NumPy ndarray / Pandas | ‘binary’ | Tabular machine learning, empirical benchmarks, cross-validation |
| Torchmetrics (F1Score) | Python / PyTorch | GPU / CUDA / MPS / CPU | PyTorch Tensor | ‘micro’ / ‘macro’ explicit | Deep learning batch evaluation, computer vision, NLP |
| Keras (FBetaScore) | Python / TensorFlow | GPU / TPU / CPU | TensorFlow Tensor | ‘weighted’ / ‘micro’ | TensorFlow neural network training metrics and callbacks |
| yardstick (f_meas) | R | CPU | Tibble / Data Frame | Binary (first level) | Tidymodels statistical modeling, biostatistics, R workflows |
| caret (confusionMatrix) | R | CPU | R Factor / Table | Binary (positive class) | Classical R machine learning pipelines and metric extraction |
| Pure NumPy Implementation | Python | CPU (Vectorized) | NumPy ndarray | Manual vectorization | Embedded scripts, minimal dependency containers, custom formulas |
Frequently asked questions
What is the default averaging method in sklearn f1_score?
The default averaging method in sklearn.metrics.f1_score is average='binary'. Under this default, the function expects binary classification targets and reports the F1-score specifically for the class designated by pos_label (defaulting to 1). If applied to multi-class data without altering this parameter, scikit-learn raises a ValueError.
Why does micro-averaged F1 equal overall accuracy in scikit-learn?
In multi-class single-label classification, every observation is assigned exactly one ground truth label and one predicted label. As a result, every prediction is either a true positive for the correct class or simultaneously a false positive for the predicted class and a false negative for the true class. Globally summing these counts means total false positives mathematically equals total false negatives, causing micro-precision, micro-recall, and micro-F1 to become algebraically identical to overall classification accuracy.
How should researchers choose between macro and weighted F1-score?
Choose average='macro' when all classes are of equal scientific importance or when identifying rare minority classes (such as uncommon adverse events or rare diseases) is critical to the research hypothesis. Choose average='weighted' when class frequencies reflect true population distributions and you wish to evaluate overall system throughput while accounting for sample size differences without letting rare anomalies dominate the metric.
How do I silence the UndefinedMetricWarning in scikit-learn?
To silence the UndefinedMetricWarning when a model predicts zero positive samples, explicitly specify the zero_division parameter. Passing zero_division=0 instructs scikit-learn to set the undefined metric to 0.0 and suppress warning output. This is the recommended practice in automated evaluation scripts and hyperparameter grid searches.
No, sklearn.metrics.f1_score requires discrete binary or multi-class predictions, not continuous probability values or decision scores. If you have predicted probabilities from predict_proba, you must apply a decision threshold (such as 0.50 or a customized threshold) to convert probabilities into binary integers (0 or 1) before calling f1_score.
How do I calculate per-class F1-scores without averaging?
To obtain individual F1-scores for every class in your dataset, pass average=None into sklearn.metrics.f1_score. The function will return a 1D NumPy array containing the exact F1-score for each class in the order specified by the labels argument or in sorted numerical order.
Sources
- Scikit-learn Documentation: sklearn.metrics.f1_score. Available from: https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html (Accessed 2026-11-15).
- Scikit-learn User Guide: Precision, Recall and F-measures (Section 3.3.2.9). Available from: https://scikit-learn.org/stable/modules/model_evaluation.html#precision-recall-f-measure-metrics (Accessed 2026-11-15).
- Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research, 12, 2825-2830. Available from: https://jmlr.csail.mit.edu/papers/v12/pedregosa11a.html (Accessed 2026-11-15).
- Scikit-learn Installation Guide: Installing the Latest Release. Available from: https://scikit-learn.org/stable/install.html (Accessed 2026-11-15).
- Torchmetrics Documentation: Classification F1Score. Available from: https://torchmetrics.readthedocs.io/en/stable/classification/f1_score.html (Accessed 2026-11-15).
Data: Multiclass Averaging Metric Response Across Imbalance Ratios
Empirical comparison of scikit-learn classification evaluation metrics and averaging behaviors across varying levels of cohort class imbalance, compiled from scikit-learn model evaluation benchmarks and scientific literature.
Period covered: 2026. Coverage: Global.
| Imbalance Profile | Class Ratio | Minority Class Prevalence (%) | Accuracy Robustness | Recommended F1 Averaging | Macro-F1 Sensitivity |
|---|---|---|---|---|---|
| Balanced Benchmark | 1:1:1 | 33.3 | High reliability | macro or micro | Moderate |
| Moderate Clinical Imbalance | 5:3:2 | 20 | Moderate distortion | macro | High |
| Severe Epidemiological Skew | 10:1:1 | 8.3 | Severe inflation | macro | Critical |
| Extreme Rare Pathology | 50:5:1 | 1.8 | Extremely misleading | macro or None | Maximum |
- Scikit-learn User Guide: Model Evaluation (accessed 2026-11-15)
- Pedregosa et al. (2011) Scikit-learn: Machine Learning in Python (accessed 2026-11-15)
Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.
Research-administration guidance, standards updates and independent tool reviews.
