mask() names the bad cells and replaces them — where() keeps the good ones, and the two are mirror images.
Somewhere in your pipeline a -999 is sitting in a temperature column pretending to be real data. mask() is how you execute it.
df.mask(cond) returns a frame of the same shape where every cell satisfying cond is replaced with NaN, and everything else keeps its value. Pass other= and the condemned cells get that instead — a scalar, an aligned Series or DataFrame, even a callable. It is exactly where() with the condition inverted: df.mask(c, x) ≡ df.where(~c, x). where() keeps what passes; mask() replaces what fails. Pick the reading that matches how you would describe the bug out loud.
Real feeds lie. Sensors read -1 when the probe disconnects, exports ship with -999 as a missing marker, loyalty points go negative after refunds. Filtering those rows throws away good neighbors; what you want is to keep the row and fix the cell. mask() does that with label alignment: hand it a fallback table keyed by the same columns and it patches the right cells from the right columns in one call — no loops, no column-by-column np.where nesting.
df = pd.DataFrame({"city": ["Berlin", "Paris", "Rome", "Oslo"], "pm25": [24, -1, 31, 19]})
df.pm25.mask(df.pm25 < 0)0 24.0 1 NaN 2 31.0 3 19.0 Name: pm25, dtype: float64
The -1 probe glitch becomes NaN; every row survives for later imputation. The column upcasts to float64 because NaN demands it.
points = pd.Series([120, -15, 87, -3], index=["ada", "ben", "cy", "dee"]) points.mask(points < 0, 0)
ada 120 ben 0 cy 87 dee 0 dtype: int64
Refund overdrafts clamp to zero with other= — no NaN, no dtype change, and the ledger stays integer all the way through.
reading = pd.DataFrame({"temp_c": [21.3, 55.0], "humidity": [44.0, 99.0]})
fallback = pd.DataFrame({"temp_c": [20.0, 20.0], "humidity": [45.0, 45.0]})
reading.mask(reading > 50, fallback)temp_c humidity 0 21.3 44.0 1 20.0 45.0
Out-of-range readings (55.0, 99.0) get replaced from the fallback table, aligned per column by label. A whole-frame condition, one call.
raw = pd.Series([3.2, -999.0, 4.1, -999.0]) raw.mask(raw == -999).fillna(raw[raw > 0].mean()).round(2)
0 3.20 1 3.65 2 4.10 3 3.65 dtype: float64
The classic two-step: mask the sentinel to NaN, then impute from the surviving values. The -999 never contaminates the mean.
| Flag | Meaning |
|---|---|
df.mask(cond) | replace cells where cond is True with NaN, keep everything else |
df.mask(cond, other) | other: scalar, aligned Series/DataFrame, or callable — the replacement values |
df.where(~cond) | the exact equivalent — mask is where with the condition flipped |
cond as callable | df.mask(lambda d: d.temp_c > 50) — condition computed on the whole frame |
axis=0|1 | alignment axis when other is a Series — axis=0 aligns per column, axis=1 per row |
inplace=True | mutate instead of returning a copy; defaults to False and is discouraged |
DataFrame.where() and mask() both first ship in the v0.10.0 source (v0.7.3 has neither), and mask() was literally one line: return self.where(~cond, NA) — verbatim, docstring and all, at line 5175 of frame.py. There was no other parameter back then; masking to a fallback value came later via the generic NDFrame API once where() grew one.
In DSP and NumPy masked-array land, a mask marks the positions to ignore — True means 'this cell is flagged'. pandas kept that convention, which is why the inversion trips people up: NumPy arrays get masked where you hide data, and pandas mask() replaces exactly the True cells. The ~ in the original implementation is the whole story.
Calling mask(cond) first computes ~cond — literally the bitwise-not operator on the boolean frame — then hands the whole problem to where(). Downstream, pandas aligns cond and other on index and columns, splits the frame into dtype blocks (float, int, object), and calls the block-level where in pandas/core/internals/blocks.py, which runs NumPy's elementwise where per block. Same numexpr-adjacent plumbing that eval() and query() ride. The practical consequence: masking an int column with NaN upcasts it to float64, one block at a time.