pandas · difficulty ◆
pandas query() — filter DataFrames with expression strings
Write the filter you mean — query() turns selection logic into one readable expression.
There is a line of pandas you have written a thousand times that never needed to be that ugly.
$ df.query()What it does
query() filters the rows of a DataFrame using an expression string: df.query('pm25 > 15') keeps exactly the rows where the column pm25 exceeds 15. Inside the string, column names are plain identifiers — no df. prefix, no repeated bracket indexing. Compound logic is plain English: 'and', 'or', 'not', plus 'in' for membership. Because the expression is parsed once and evaluated by numexpr when installed, it is also typically faster than the equivalent chained boolean masks on large frames.
Why it matters
Filtering is the single most frequent DataFrame operation, and the bracket idiom df[(df.a > 1) & (df.b == 'x')] forces you to repeat the frame name, mind operator precedence (& vs and), and wrap every condition in parentheses. query() removes all of that noise: the expression reads like the question you are actually asking. It chains beautifully too — df.assign(...).query(...) builds transform-then-filter pipelines in one expression chain, and the @name syntax lets SQL-style variables reference Python values without string formatting.
Example
$ df.query('pm25 > 15') city pm25 year
Berlin 24 2024
Paris 19 2024
Rome 31 2025
Berlin 22 2025Every row above the threshold, in one readable line.
$ avg = df.pm25.mean() # 19.5
df.query('pm25 > @avg').city.tolist()['Berlin', 'Rome', 'Berlin']@references a Python variable — the @-prefix is the only escaping rule you need.
$ df.query("city == 'Vienna' and year == 2024")['pm25'].item()12Compound conditions with and / or read like plain English.
$ df.assign(pm25_norm=df.pm25 / df.pm25.max()).query('pm25_norm < 0.5').city.tolist()['Vienna', 'Vienna']Transform-then-filter in one chain: assign a computed column, filter on it, done.
Common flags
- df.query('col > 5')
- basic comparison: > >= < <= == != on any column
- 'a and b', 'a or b', 'not a'
- combine conditions with words instead of & | ~
- 'city in ["Vienna", "Rome"]'
- membership test; also 'not in'
- 'pm25 > @avg'
- @name references a local Python variable inside the expression
- inplace=True
- modify the DataFrame in place instead of returning a copy (defaults to False)
- engine='python'
- force the plain-Python evaluator; default 'numexpr' speeds up large numeric frames
History
Added in pandas 0.13 (January 2014)
query() arrived as a top-level DataFrame method wrapping the numexpr evaluation engine, giving pandas a fast, readable path for medium-to-large boolean selections. It was pitched from the start as the readable alternative to the mask idiom — 'SQL-like' selection for people who think in predicates rather than in array algebra.
The @-syntax (pandas 0.25, 2019)
The LocalVariable reference @name became the official way to inject Python variables into query expressions, replacing the older global-injection tricks. Alongside eval() — its sibling method for column arithmetic — query() is the reason pandas speaks a small expression language at all.
Fun facts
Pros & cons
pros
- + Reads like the question you are asking — filters stay readable at any complexity
- + numexpr acceleration on large numeric frames comes for free
- + Chains cleanly with assign() and eval() into one-line pipelines
cons
- − String expressions get no IDE type-checking or autocomplete
- − Awkward column names (spaces, leading digits) need backtick quoting
- − inplace=True is discouraged by the pandas team — prefer assigning the result
Takeaways
- 1Reach for query() whenever a filter reads like a sentence — it stays readable where brackets collapse into noise
- 2Inject Python values with @var; never f-string numbers into the expression string
- 3'in' / 'not in' work inside expressions — often cleaner than .isin()
- 4Chain assign() → query() for transform-then-filter in a single statement
- 5Skip inplace=True; assign the result instead — that is where pandas is heading