pandas · difficulty ◆◆

pandas mask() — replace values where a condition is true

mask() names the bad cells and replaces them — where() keeps the good ones, and the two are mirror images.

Somewhere in your pipeline a -999 is sitting in a temperature column pretending to be real data. mask() is how you execute it.

fully AI-generated

2026-09-02 · 4 min read

$ df.mask()

0.10
introduced in pandas: 6
parameters (pandas 3.0): 1 line
implementation at birth

What it does

df.mask(cond) returns a frame of the same shape where every cell satisfying cond is replaced with NaN, and everything else keeps its value. Pass other= and the condemned cells get that instead — a scalar, an aligned Series or DataFrame, even a callable. It is exactly where() with the condition inverted: df.mask(c, x) ≡ df.where(~c, x). where() keeps what passes; mask() replaces what fails. Pick the reading that matches how you would describe the bug out loud.

Why it matters

Real feeds lie. Sensors read -1 when the probe disconnects, exports ship with -999 as a missing marker, loyalty points go negative after refunds. Filtering those rows throws away good neighbors; what you want is to keep the row and fix the cell. mask() does that with label alignment: hand it a fallback table keyed by the same columns and it patches the right cells from the right columns in one call — no loops, no column-by-column np.where nesting.

Example

$ df = pd.DataFrame({"city": ["Berlin", "Paris", "Rome", "Oslo"], "pm25": [24, -1, 31, 19]})
df.pm25.mask(df.pm25 < 0)

0    24.0
1     NaN
2    31.0
3    19.0
Name: pm25, dtype: float64

The -1 probe glitch becomes NaN; every row survives for later imputation. The column upcasts to float64 because NaN demands it.

$ points = pd.Series([120, -15, 87, -3], index=["ada", "ben", "cy", "dee"])
points.mask(points < 0, 0)

ada    120
ben      0
cy      87
dee      0
dtype: int64

Refund overdrafts clamp to zero with other= — no NaN, no dtype change, and the ledger stays integer all the way through.

$ reading = pd.DataFrame({"temp_c": [21.3, 55.0], "humidity": [44.0, 99.0]})
fallback = pd.DataFrame({"temp_c": [20.0, 20.0], "humidity": [45.0, 45.0]})
reading.mask(reading > 50, fallback)

   temp_c  humidity
0    21.3      44.0
1    20.0      45.0

Out-of-range readings (55.0, 99.0) get replaced from the fallback table, aligned per column by label. A whole-frame condition, one call.

$ raw = pd.Series([3.2, -999.0, 4.1, -999.0])
raw.mask(raw == -999).fillna(raw[raw > 0].mean()).round(2)

0    3.20
1    3.65
2    4.10
3    3.65
dtype: float64

The classic two-step: mask the sentinel to NaN, then impute from the surviving values. The -999 never contaminates the mean.

Common flags

df.mask(cond): replace cells where cond is True with NaN, keep everything else
df.mask(cond, other): other: scalar, aligned Series/DataFrame, or callable — the replacement values
df.where(~cond): the exact equivalent — mask is where with the condition flipped
cond as callable: df.mask(lambda d: d.temp_c > 50) — condition computed on the whole frame
axis=0|1: alignment axis when other is a Series — axis=0 aligns per column, axis=1 per row
inplace=True: mutate instead of returning a copy; defaults to False and is discouraged

History

Born as a one-liner in pandas 0.10 (December 2012)

DataFrame.where() and mask() both first ship in the v0.10.0 source (v0.7.3 has neither), and mask() was literally one line: return self.where(~cond, NA) — verbatim, docstring and all, at line 5175 of frame.py. There was no other parameter back then; masking to a fallback value came later via the generic NDFrame API once where() grew one.

The naming comes from signal processing

In DSP and NumPy masked-array land, a mask marks the positions to ignore — True means 'this cell is flagged'. pandas kept that convention, which is why the inversion trips people up: NumPy arrays get masked where you hide data, and pandas mask() replaces exactly the True cells. The ~ in the original implementation is the whole story.

Under the hood

Under the hood: the negation, then block dispatch

Calling mask(cond) first computes ~cond — literally the bitwise-not operator on the boolean frame — then hands the whole problem to where(). Downstream, pandas aligns cond and other on index and columns, splits the frame into dtype blocks (float, int, object), and calls the block-level where in pandas/core/internals/blocks.py, which runs NumPy's elementwise where per block. Same numexpr-adjacent plumbing that eval() and query() ride. The practical consequence: masking an int column with NaN upcasts it to float64, one block at a time.

Fun facts

Pros & cons

pros

+ Reads like the bug report: 'mask the readings above 50' maps one-to-one onto the code
+ other= accepts aligned DataFrames, so multi-column fallbacks from a calibration table are one call
+ Same shape and labels out — patch cells without losing rows or breaking joins downstream

cons

− The where()/mask() inversion is a standing mental tax — write the wrong twin and every cell flips
− NaN insertion upcasts int columns to float64, which surprises anyone exporting IDs or counts
− A whole-row condition blanks string columns too — 'Rome' becomes NaN just like the numbers do

Takeaways

1mask(cond) replaces what fails; where(cond) keeps what passes — mask(c) ≡ where(~c)
2Sentinel values (-999, -1, 9999) get masked to NaN first, then fillna() — never averaged raw
3other= takes an aligned DataFrame: multi-column fallbacks in one call, no loops
4mask() never drops rows; if the row must go, that is query()'s job
5Clamping extremes? clip() beats both twins with one argument and zero NaN drama

Related commands

archive entry — written and published by a scheduled job, never editorially reviewed. Kept for reference: out of the site’s search index, and no ads run here. the reviewed library →

← all learning