Set Up a Python Quant Research Environment

A backtest is a scientific claim: this rule would have earned this Sharpe. Like any scientific claim, it is worthless if nobody, including you, six months later, can reproduce the number. The single most common reason a "great" strategy evaporates is not a subtle statistics error; it's that the environment changed underneath it. A silent pandas upgrade altered how resample labels bins, a numpy release changed a default dtype, and the equity curve you screenshotted no longer exists. This guide sets up the plumbing that makes your results durable before you write a line of strategy code.

Why isolation is non-negotiable

Your operating system ships a Python. Never build research on it. The moment two projects need different pandas versions, and they will, because pandas makes breaking changes, a global install forces one to lose. Worse, upgrading a package for project B silently changes the results of project A. An isolated environment is a private, per-project set of package versions that nothing else can touch.

You have three mainstream choices:

Tool	Best for	Notes
venv	Pure-Python stacks, minimal footprint	Built in (python -m venv); no compiler/BLAS management
conda/mamba	Heavy native deps (MKL, TA-Lib, C++ solvers)	Manages non-Python binaries too; heavier
uv	Modern default, fast, lockfile-first	Rust-based pip/venv replacement; resolves in seconds

For most retail quant work, uv or venv is plenty; reach for conda only when a dependency needs compiled numerical libraries you don't want to build yourself.

# venv (built in)
python -m venv .venv && source .venv/bin/activate
pip install numpy pandas scipy statsmodels scikit-learn matplotlib jupyter

# or uv (faster, lockfile-native)
uv venv && source .venv/bin/activate
uv pip install numpy pandas scipy statsmodels scikit-learn matplotlib jupyter

Pin everything: the lockfile

pip install pandas installs whatever is newest today. Run the same command next month and you may get a different version, and a different backtest. Pinning with == in a requirements.txt fixes your direct dependencies, but not the transitive ones (the packages your packages pull in). The fix is a lockfile that records the exact version of every package in the graph, including hashes:

pip freeze > requirements.lock      # crude but works
uv lock                             # proper resolved lockfile (uv.lock)
conda env export > environment.yml  # conda equivalent

Commit the lockfile. A reviewer (or future you) runs uv sync / pip install -r requirements.lock and gets a bit-for-bit identical package set. This is not bureaucracy, a stray numpy minor bump has changed floating-point reduction order and moved Sharpe ratios in the third decimal. If you can't reproduce your own backtest, you can't debug it.

The core stack, and what each part is for

Package	Role
numpy	Vectorized arrays; the numeric substrate under everything
pandas	Time-indexed tables (Series/DataFrame); your primary data structure
scipy	Stats distributions, optimization, linear algebra
statsmodels	Regression, OLS/GLS, time-series (ADF, ARIMA), proper standard errors
scikit-learn	ML models, pipelines, cross-validation scaffolding
matplotlib	Plots, equity curves, drawdowns, diagnostics
jupyter	Interactive exploration (with caveats, below)

Project structure: separate data, notebooks, and source

Flat folders full of Untitled12.ipynb are where reproducibility goes to die. Impose a boundary:

myquant/
├── data/           # raw + processed data, GITIGNORED, treated as immutable
│   ├── raw/        # exactly as downloaded; never edited in place
│   └── processed/  # cleaned parquet you regenerate from raw
├── notebooks/      # exploration only; import logic, don't define it
├── src/myquant/    # the real, importable, testable code
│   ├── data.py
│   ├── signals.py
│   └── backtest.py
├── tests/
├── pyproject.toml
└── requirements.lock

The rule that pays off: logic lives in src/, notebooks only call it. A function in src/signals.py can be unit-tested, imported from multiple notebooks, and reviewed in a diff. The same code copy-pasted across three notebook cells cannot. Keep data/raw/ immutable and out of git, raw downloads are your ground truth, and you regenerate processed/ from them with a script, never by hand.

Notebooks cause bugs, here's how to mitigate

Notebooks are wonderful for exploration and treacherous for results, because they carry hidden state. A cell's output depends on every cell you have run, in the order you ran them, which may not be top-to-bottom. You can delete the cell that defined a variable and still use it; you can run cell 5, edit cell 2, and never re-run it. The notebook looks consistent but encodes a computation that no longer exists on disk.

Concrete failure: you compute a z-score using a mean variable, then edit an earlier cell to change the lookback but forget to re-run downstream cells. Your printed Sharpe now reflects an inconsistent mix of old and new code, a phantom result. This is a common vector for accidental Look-Ahead Bias: a preprocessing cell run after the split silently contaminates training data.

Mitigations that actually work:

• Restart & Run All before trusting any number. If it doesn't survive a clean top-to-bottom run, it isn't real.
• Push logic into src/ and import it. The notebook becomes a thin orchestration layer.
• Pair with a script via jupytext, so every notebook has a plain-.py twin that diffs cleanly in git.
• Never let a plot be your only output. Persist metrics to disk so they can be regenerated and compared.

Determinism: control your randomness

Any simulation, bootstrap, or train/test shuffle uses a random stream. If you don't fix the seed, you can't reproduce the run. But how you seed matters. The legacy global API (np.random.seed) mutates one shared, process-wide stream, so any library or helper that also draws from it silently shifts your "reproducible" result:

import numpy as np

# GLOBAL state: a shared, mutable stream. An intervening draw silently shifts it.
np.random.seed(0)
sim_A = np.random.normal(size=3)          # your Monte-Carlo run

np.random.seed(0)
_ = np.random.normal(size=1)              # a helper you added later also draws
sim_B = np.random.normal(size=3)          # "same" run, now different!
print("global, drifted:", np.allclose(sim_A, sim_B))   # False

# LOCAL generators: each owns an independent stream. Insertions can't leak in.
def run():
    rng = np.random.default_rng(0)
    return rng.normal(size=3)
print("local, isolated:", np.allclose(run(), run()))    # True

The modern np.random.default_rng(seed) gives each component its own generator. Prefer it everywhere. Even then, be aware that full determinism across machines is not guaranteed: multi-threaded BLAS can reorder floating-point reductions, so a sum can differ in the last bits between a 4-core laptop and a 32-core server. For research this rarely matters; for anything path-dependent (a stop-loss that triggers on a knife-edge), it can.

Performance: vectorize before you optimize

pandas and numpy are fast because operations run in compiled C over whole arrays. A Python for-loop over rows drops back into the interpreter and pays per-element overhead. The gap is not subtle:

import numpy as np, pandas as pd, time
rng = np.random.default_rng(7)
price = pd.Series(100 * np.exp(np.cumsum(rng.normal(0, 0.01, 2_000_000))))

t0 = time.perf_counter()
loop_ret = [np.nan]*len(price); p = price.to_numpy()
for i in range(1, len(p)):
    loop_ret[i] = p[i]/p[i-1] - 1
t_loop = time.perf_counter() - t0

t0 = time.perf_counter()
vec_ret = price.pct_change()
t_vec = time.perf_counter() - t0
print(f"loop {t_loop*1000:.0f} ms | vec {t_vec*1000:.0f} ms | {t_loop/t_vec:.0f}x")

On 2M rows this prints roughly loop 280 ms | vec 9 ms | 30x (exact numbers vary by machine). The vectorized pct_change is ~30× faster and less bug-prone. Reach for a Python loop only when the computation is genuinely path-dependent and cannot be expressed as array operations. For those cases: Numba JIT-compiles a numeric loop to near-C speed with a decorator, and Polars (a Rust-backed DataFrame) beats pandas on large groupby/join workloads and handles out-of-core data. Both are optimizations to reach for after you've profiled, not defaults. Vectorize first; it solves 95% of speed problems and prevents whole classes of off-by-one lookahead bugs that hand-written loops invite.

Where this breaks

• Version drift. No lockfile means "reproducible" is a fiction. Pin, commit, and treat the lockfile as part of your result.
• Notebook state. A number that hasn't survived Restart & Run All is not a finding.
• Cross-machine float noise. Bit-identical results across different CPUs/BLAS are not guaranteed; make your logic robust, not knife-edge dependent.
• Immutable raw data. If you clean data in place, you can never re-derive your dataset. Keep raw/ read-only and regenerate everything downstream.

With the environment nailed down, you're ready to actually pull data, and to meet the traps hiding inside "free" market data in the next guide.