Thresholding and image segmentation

Global and Otsu thresholding, adaptive thresholding, watershed for touching objects, GrabCut, and choosing a method from the image rather than the tutorial.

Simple and Otsu thresholding

import cv2
import numpy as np

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)

# a fixed threshold: simple, and fragile to lighting
_, fixed = cv2.threshold(blurred, 127, 255, cv2.THRESH_BINARY)

# Otsu picks the threshold that maximises between-class variance
t, otsu = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
print("otsu threshold:", t)

# Otsu with an inverted result, for dark objects on a light background
_, otsu_inv = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)

# triangle is better than Otsu when one class dominates the histogram
_, triangle = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_TRIANGLE)

# adaptive threshold: a different threshold for every neighbourhood
adaptive_mean = cv2.adaptiveThreshold(blurred, 255, cv2.ADAPTIVE_THRESH_MEAN_C,
                                      cv2.THRESH_BINARY, blockSize=31, C=5)
adaptive_gauss = cv2.adaptiveThreshold(blurred, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
                                       cv2.THRESH_BINARY, blockSize=31, C=5)
print("coverage", [round(float(m.mean()) / 255, 3) for m in
                   (otsu, triangle, adaptive_mean, adaptive_gauss)])

• blockSize must be odd and larger than the features you want to keep. A block smaller than a character produces text that is half background and half foreground.
• C is subtracted from the local mean. Raising it makes the threshold stricter and removes more background; lowering it admits more noise.
• Blur before thresholding. Otsu on an unblurred image recovers every sensor speck as a separate blob.
• Otsu assumes a bimodal histogram. On a histogram with one dominant peak it returns a threshold that splits the peak and produces meaningless output.

Separating touching objects with watershed

# watershed needs a marker image: a distinct seed for each object
binary = cv2.threshold(blurred, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]

# remove noise, then find definite background and definite foreground
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
opened = cv2.morphologyEx(binary, cv2.MORPH_OPEN, kernel, iterations=2)
background = cv2.dilate(opened, kernel, iterations=3)

# the distance transform gives the centre of each blob, which seeds the split
distance = cv2.distanceTransform(opened, cv2.DIST_L2, 5)
_, sure_fg = cv2.threshold(distance, 0.5 * distance.max(), 255, 0)
sure_fg = np.uint8(sure_fg)

unknown = cv2.subtract(background, sure_fg)

# connected components labels each definite foreground blob
_, markers = cv2.connectedComponents(sure_fg)
markers = markers + 1                     # background must not be 0
markers[unknown == 255] = 0               # unknown regions are marked for filling

colour = cv2.cvtColor(binary, cv2.COLOR_GRAY2BGR)
markers = cv2.watershed(colour, markers)
colour[markers == -1] = (0, 0, 255)       # boundaries are drawn as -1

print("objects found:", len(np.unique(markers)) - 2)

# GrabCut: interactive foreground extraction from a rectangle or a mask
rect = (60, 60, colour.shape[1] - 120, colour.shape[0] - 120)
mask = np.zeros(colour.shape[:2], np.uint8)
bgd = np.zeros((1, 65), np.float64)
fgd = np.zeros((1, 65), np.float64)
cv2.grabCut(image, mask, rect, bgd, fgd, 5, cv2.GC_INIT_WITH_RECT)
foreground = np.where((mask == cv2.GC_FGD) | (mask == cv2.GC_PR_FGD), 255, 0).astype("uint8")
print("grabcut coverage", round(float(foreground.mean()) / 255, 3))

• Watershed works from markers, not from intensity. Feed it a bad marker image and it produces a plausible-looking segmentation that is wrong everywhere.
• The distance transform is the practical way to find markers for round, touching objects. The 0.5 threshold is a starting point; raise it to split more aggressively.
• Every object boundary is marked as -1 in the output. If you need clean regions, replace -1 values with a neighbouring label rather than drawing them as red in the final image.
• GrabCut is iterative and slow (seconds per image), and it needs a reasonable initialisation. It is excellent for offline work and unsuitable for a real-time loop.

Choosing and validating a method

def evaluate_mask(predicted, ground_truth):
    """IoU: the standard way to compare two binary masks."""
    predicted = (predicted > 127).astype(bool)
    ground_truth = (ground_truth > 127).astype(bool)
    intersection = np.logical_and(predicted, ground_truth).sum()
    union = np.logical_or(predicted, ground_truth).sum()
    return float(intersection / union) if union else 1.0

def dice(predicted, ground_truth):
    predicted = (predicted > 127).astype(bool)
    ground_truth = (ground_truth > 127).astype(bool)
    return float(2 * np.logical_and(predicted, ground_truth).sum() /
                 (predicted.sum() + ground_truth.sum() + 1e-9))

# threshold stability: a small change in input should not swing the result wildly
def otsu_sensitivity(gray_image, noise_levels=(0, 2, 5, 10)):
    results = []
    for level in noise_levels:
        rng = np.random.default_rng(0)
        noisy = np.clip(gray_image.astype("int16") +
                        rng.normal(0, level, gray_image.shape), 0, 255).astype("uint8")
        value, _ = cv2.threshold(noisy, 0, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)
        results.append((level, round(value, 1)))
    return results

print(otsu_sensitivity(gray))
print("IoU", round(evaluate_mask(otsu, reference_mask), 3))

FAQ

Related

Contours and object detection basics Morphology and noise removal

Last refreshed 2026-09-18.