Choosing SciPy vs specialised libraries
Decide between NumPy, SciPy, scikit-learn, statsmodels and networkx, and understand the real cost of using the wrong one.
Who owns which problem
| Task | Reach for | Why not SciPy |
|---|---|---|
| Array arithmetic, broadcasting | NumPy | SciPy has no general array type of its own |
| Dense and sparse linear algebra | SciPy | This is its core competence |
| Optimisation, ODEs, quadrature | SciPy | optimize and integrate are the reference implementations |
| Supervised learning pipelines | scikit-learn | It wraps estimators, cross-validation and preprocessing coherently |
| Statistical inference and formulas | statsmodels | It reports p-values, confidence intervals and diagnostics |
| Graph algorithms | networkx, python-igraph | SciPy has sparse.csgraph, but a graph library reads better |
| Time series modelling | statsmodels, statsforecast | SciPy has filters, not ARIMA |
| Large-scale numerical work | PyTorch, JAX | SciPy is CPU-first and eager, with no autodiff |
| Table manipulation | pandas, Polars | SciPy has no labelled data structure |
The awkward boundary cases
# least squares with no inference needed: SciPy is enough
from scipy import linalg
import numpy as np
X = np.random.default_rng(0).normal(size=(200, 4))
y = X @ np.array([1.0, -2.0, 0.5, 3.0]) + 0.1
coef, *_ = linalg.lstsq(X, y)
# same data, but you need standard errors and p-values
import statsmodels.api as sm
model = sm.OLS(y, sm.add_constant(X)).fit()
print(model.params.round(3))
print(model.bse.round(3)) # what SciPy would not have given you# a curve fit with bounds and uncertainty: SciPy's natural job
from scipy.optimize import curve_fit
def decay(t, a, tau):
return a * np.exp(-t / tau)
popt, pcov = curve_fit(decay, t, y, p0=[1.0, 1.0], bounds=([0, 0], [np.inf, np.inf]))
perr = np.sqrt(np.diag(pcov))
print(popt, perr)- If you need a
fit,predictandscoreinterface, scikit-learn will save you reimplementing splitting, scaling and metrics. - If you need standard errors, p-values or a formula interface, use statsmodels. Reimplementing inference on top of SciPy is where bugs hide.
- If the computation is a differentiable loss you want to optimise with gradients, use an autodiff framework. Derivative-free SciPy methods do not scale to millions of parameters.
The cost of the wrong tool
Using the wrong library rarely crashes. It costs you in three ways: reimplementing statistics you will get subtly wrong, writing loops that a specialised library would vectorise, and losing the diagnostics that would have told you the model was misspecified.
💡
Prefer the smallest tool that answers the question fully.
scipy.optimize.curve_fit is a better choice than a neural network for fitting four physical parameters, and linalg.lstsq is a better choice than a machine-learning pipeline when there is no inference and no generalisation to worry about.FAQ
When should I move from SciPy to scikit-learn?
When you need to compare models, cross-validate, build pipelines or deploy an estimator. A single fitting routine with fixed parameters is faster to write directly against SciPy and easier to reason about.
Is scipy.sparse.csgraph a replacement for networkx?
For shortest paths and connected components on a numeric adjacency matrix it is fast and sufficient. For graph construction, attributes, traversal algorithms and visualisation, networkx is far more expressive.
Related
Optimisation and curve fitting Statistics and signal processing
Last refreshed 2026-09-18.