TensorBoard and experiment tracking

Scalars, graphs, histograms, image summaries and the embedding projector, plus comparing runs so that a change you made is the change that mattered.

Logging during training

import datetime
import tensorflow as tf

log_dir = "logs/fit/" + datetime.datetime.now().strftime("%Y%m%d-%H%M%S")

tensorboard_cb = tf.keras.callbacks.TensorBoard(
    log_dir=log_dir,
    histogram_freq=1,          # weight distributions per epoch
    write_graph=True,
    update_freq="batch",       # per-batch scalars: use a sample rate on big runs
    profile_batch=(5, 10),     # profile steps 5-10 for the profiler view
)

model.fit(train_ds, validation_data=val_ds, epochs=10,
          callbacks=[tensorboard_cb])

# tensorboard --logdir logs/fit     then open the printed localhost URL

tensorboard --logdir logs/fit --port 6006
# compare several runs at once: point logdir at the parent directory
tensorboard --logdir logs/

Each run needs its own timestamped directory; reusing one log dir overwrites history and makes comparison impossible.
update_freq="batch" writes a lot of data. On a long run, set an integer step frequency or a fraction such as 0.01 instead.
Weight histograms add real overhead per epoch. Enable them while diagnosing, then turn them off.
Put the git commit, config and dataset version in the run name or as text summaries. Six runs later, an opaque timestamp is useless.

Custom scalars, images and text

import numpy as np

writer = tf.summary.create_file_writer("logs/custom/run1")

for step in range(100):
    with writer.as_default():
        tf.summary.scalar("loss/train", float(np.exp(-step / 30)), step=step)
        tf.summary.scalar("loss/val", float(np.exp(-step / 40)) + 0.02, step=step)
        tf.summary.histogram("weights/layer1", np.random.default_rng(step).normal(size=500), step=step)

        if step % 20 == 0:
            sample = np.random.default_rng(1).random((1, 28, 28, 1))
            tf.summary.image("input/sample", sample, step=step, max_outputs=4)
            tf.summary.text("config/lr", "0.001", step=step)

writer.flush()

Tab	What you learn	Typical signal
Scalars	How a metric moves over time	Val loss rising while train loss falls: overfitting
Graphs	The actual operations created	An unexpected op or a duplicated subgraph
Histograms	Weight and gradient distributions	Weights collapsing to zero; gradient vanishing
Images	What the model actually sees	Augmentation is destroying the label
Projector	Embedding neighbourhoods	Classes that never separate in representation space
Profiler	Where the time goes	Input-bound pipeline versus kernel-bound model

# a custom callback writing metrics TensorBoard does not track automatically
class LRLogger(tf.keras.callbacks.Callback):
    def __init__(self, log_dir):
        super().__init__()
        self.writer = tf.summary.create_file_writer(log_dir)

    def on_epoch_end(self, epoch, logs=None):
        lr = float(tf.keras.backend.get_value(self.model.optimizer.learning_rate))
        with self.writer.as_default():
            tf.summary.scalar("train/learning_rate", lr, step=epoch)
        self.writer.flush()

💡

TensorBoard shows what you logged, not what actually happened. If the batch-level loss looks smooth and healthy while the epoch-level metric is flat, you are looking at a moving average, and the underlying curve is being hidden by the smoothing slider.

Comparing runs without fooling yourself

Change one thing per run. Two simultaneous changes produce an improvement you cannot attribute, and you will keep both by mistake.
Always plot the baseline alongside the variant on the same axis, with the same step range. Separate charts make small differences look large.
Seeds matter. Run each configuration two or three times with different seeds and plot all of them; a single-seed win is often inside the noise band.
Turn off the smoothing slider before reading a conclusion, then turn it back on for the shape. Smoothed curves can turn a plateau into a rise.
Log the validation metric with the same evaluation frequency across runs, or the comparison is between different amounts of evaluation.

The habit worth building: before any experiment, write the expected outcome in the run name or a text summary. If the result contradicts the expectation, the surprise is the finding, and it is exactly what you would otherwise forget.

FAQ

TensorBoard or Weights and Biases?

TensorBoard is local, free and already integrated with Keras callbacks. Hosted trackers add collaboration, sweep orchestration and artefact storage. Start with TensorBoard and move when a team needs shared history.

Where should the logs go?

A directory outside source control, with one subdirectory per run. Add logs/ to .gitignore, or your repository grows by gigabytes of event files.

Hyperparameter tuning with KerasTuner Debugging, profiling and export formats

Last refreshed 2026-09-18.

TensorBoard and experiment tracking

Logging during training

Custom scalars, images and text

Comparing runs without fooling yourself

FAQ

Related