Transfer learning and fine-tuning

Load a pretrained backbone, freeze and unfreeze it in the right order, match the preprocessing exactly, and know when fine-tuning is worth the cost.

Loading and freezing a backbone

import tensorflow as tf

IMG_SIZE = (224, 224)

# preprocess_input must match the backbone's original training preprocessing
preprocess = tf.keras.applications.mobilenet_v2.preprocess_input

backbone = tf.keras.applications.MobileNetV2(
    input_shape=IMG_SIZE + (3,),
    include_top=False,          # drop the 1000-class head
    weights="imagenet",
)
backbone.trainable = False      # freeze everything before the first fit

inputs = tf.keras.Input(shape=IMG_SIZE + (3,))
x = preprocess(inputs)          # same preprocessing the backbone expects
x = backbone(x, training=False)
x = tf.keras.layers.GlobalAveragePooling2D()(x)
x = tf.keras.layers.Dropout(0.2)(x)
outputs = tf.keras.layers.Dense(1)(x)          # logits for binary classification

model = tf.keras.Model(inputs, outputs)
model.compile(
    optimizer=tf.keras.optimizers.Adam(1e-3),
    loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
    metrics=["accuracy"],
)
model.fit(train_ds, validation_data=val_ds, epochs=5)

• include_top=False removes the classifier so you can attach your own head with the right number of classes.
• Set backbone.trainable = False and pass training=False in the call. Frozen batch-norm layers must stay in inference mode or their running statistics drift and the pretrained features degrade.
• The preprocessing must match how the weights were trained. Using Rescaling(1/255) on a backbone that expects [-1, 1] inputs is a quiet accuracy loss of many points.
• Train the head first with the backbone frozen. Random head weights produce large gradients that would otherwise destroy the pretrained filters.

Unfreezing and fine-tuning

# stage 2: unfreeze the top of the backbone and use a much smaller learning rate
backbone.trainable = True
for layer in backbone.layers[:-30]:        # keep the early, generic layers frozen
    layer.trainable = False

# batch norm layers must remain frozen even inside the unfrozen block
for layer in backbone.layers:
    if isinstance(layer, tf.keras.layers.BatchNormalization):
        layer.trainable = False

model.compile(
    optimizer=tf.keras.optimizers.Adam(1e-5),   # 100x smaller than stage 1
    loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
    metrics=["accuracy"],
)
model.fit(train_ds, validation_data=val_ds, epochs=10,
          callbacks=[tf.keras.callbacks.EarlyStopping(
              monitor="val_loss", patience=3, restore_best_weights=True)])

• You must recompile after changing trainable. Keras builds the optimiser's variable list at compile time, so an unfreeze without a recompile silently keeps training nothing.
• Early layers learn generic edges and textures; later layers learn task-specific structure. Unfreeze from the top down, not all at once.
• Use a much smaller learning rate for fine-tuning than for head training. A large rate on pretrained weights undoes the pretraining within a few hundred steps.
• Fine-tuning helps most when your domain differs from the pretraining data (medical, satellite, industrial imagery) and least when it is close to natural photos.

When it is worth it

# extract features once, then train a small classifier on the frozen vectors
extractor = tf.keras.Model(backbone.input, backbone.output)

def to_features(ds):
    return ds.map(lambda x, y: (extractor(x, training=False), y)).cache().prefetch(
        tf.data.AUTOTUNE)

feat_train = to_features(train_ds)
feat_val = to_features(val_ds)

head = tf.keras.Sequential([
    tf.keras.layers.Input(shape=backbone.output_shape[1:]),
    tf.keras.layers.GlobalAveragePooling2D(),
    tf.keras.layers.Dense(1),
])
head.compile(optimizer="adam", loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
             metrics=["accuracy"])
head.fit(feat_train, validation_data=feat_val, epochs=20)

FAQ

Related

Regularisation and normalisation Text and image preprocessing layers

Last refreshed 2026-09-18.