Before you analyze anything, run info() — it tells you what you actually have.
Every mystery DataFrame has a rap sheet — info() prints it in half a second.
df.info() prints a compact report to stdout: the index extent, one line per column with dtype and non-null count, a dtype tally, and total memory usage. It returns None — info() prints, it never hands back a DataFrame, so it can't be chained into a pipeline.
Silent dtype damage is where most pandas pain starts: one stray "N/A" string turns a whole numeric column into object dtype, and everything built on it misbehaves. info() is the cheapest insurance there is — one line flags the damage before you build on it. It also answers the question every new dataset begs: how much cleaning is even needed?
import pandas as pd
df = pd.read_csv('orders.csv') # 12 orders, one blank amount, sparse discount codes
print(df.info())<class 'pandas.DataFrame'> RangeIndex: 12 entries, 0 to 11 Data columns (total 5 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 order_id 12 non-null int64 1 region 12 non-null str 2 amount 11 non-null float64 3 ordered_at 12 non-null str 4 discount_code 7 non-null str dtypes: float64(1), int64(1), str(3) memory usage: 612.0 bytes None
amount is missing one value and discount_code only has 7 of 12 — visible in seconds, no loops, no isna().sum() per column.
import pandas as pd
n = 3_000_000
df = pd.DataFrame({
'customer_id': range(1, n+1),
'segment': ['retail'] * n,
'balance': [100.0] * n,
'opened_at': ['2020-01-01'] * n,
})
df.info(memory_usage='deep')<class 'pandas.DataFrame'> RangeIndex: 3000000 entries, 0 to 2999999 Data columns (total 4 columns): # Column Dtype --- ------ ----- 0 customer_id int64 1 segment str 2 balance float64 3 opened_at str dtypes: float64(1), int64(1), str(2) memory usage: 371.9 MB
memory_usage='deep' walks every object column and asks Python what its strings truly cost — the honest number, at the price of a full pass over the data.
df['segment'] = df['segment'].astype('category')
df['opened_at'] = df['opened_at'].astype('category')
df.info(memory_usage='deep')<class 'pandas.DataFrame'> RangeIndex: 3000000 entries, 0 to 2999999 Data columns (total 4 columns): # Column Dtype --- ------ ----- 0 customer_id int64 1 segment category 2 balance float64 3 opened_at category dtypes: category(2), float64(1), int64(1) memory usage: 51.5 MB
Same 3M rows: 371.9 MB down to 51.5 MB just by categorifying the two string columns. info() before and after is how you prove a downcast worked.
df.info(verbose=False) # skip the per-column table
<class 'pandas.DataFrame'> RangeIndex: 12 entries, 0 to 11 Columns: 5 entries, order_id to discount_code dtypes: float64(1), int64(1), str(3) memory usage: 612.0 bytes
For a 400-column scrape dump, this one-liner is all you want: extent, dtype tally, memory. Drill in per column only where it hurts.
| Flag | Meaning |
|---|---|
(no args) | Full report: per-column non-null counts, dtypes, dtype tally, memory estimate. |
memory_usage=True | False | 'deep' | Default True counts buffers only; 'deep' introspects object strings for the real footprint. |
verbose=True | False | False collapses the per-column table to a one-line summary — ideal for wide frames. |
max_cols=<int> | Columns shown before pandas truncates the table; default 100, falls back to summary style when exceeded. |
show_counts=<bool> | Force non-null counts on or off; on huge frames counting nulls costs a pass, so pandas may omit them above ~1.6M cells. |
buf=<writable> | Redirect the report into a StringIO or log file instead of stdout. |
info() shipped with early pandas — Wes McKinney built it as the R str() analogue for DataFrames. In 0.25.0 (July 2019) it gained show_counts (then null_counts) and the smart skip-counting heuristic for very wide frames.
Pandas 3.0 made str the default dtype for string data, so info() reports str instead of object for text columns — same call, more honest label. Old screenshots showing 'object' are simply pre-3.0.
A DataFrame stores data in blocks of same-dtype arrays, and each block carries an ndarray of validity bits — nulls are known before you ever ask. info() reads those cached counts, so non-null tallies cost nothing per row; default memory_usage just multiplies buffer sizes by itemsize. Only memory_usage='deep' does real work: getsizeof per string plus a pass to hash them for deduplication. That's also why the 'deep' number can shift on interpolated strings — Python interns some, not others.