sysadmin · difficulty ◆◆
lsof — list open files
Everything is a file — lsof tells you who has it open.
You deleted the log, `rm` insists it is gone, and `df` still shows a full disk. Something is holding that file open — lsof names it, by PID.
AI-generated draft · reviewed and editedThis text was generated with AI as a first draft, then reviewed and edited by the author, who holds editorial responsibility for it.$ lsofWhat it does
lsof stands for 'list open files', and in Unix that phrase is much bigger than it sounds: sockets, pipes, device nodes, directories, shared libraries and deleted-but-still-open regular files are all files. On Linux lsof is a /proc archaeologist — it walks every /proc/<pid>/fd, follows each symlink to its target, and prints one row per file descriptor. The columns are the whole story: COMMAND and PID (who), USER (whom it runs as), FD (descriptor number plus the r/w/u access mode), TYPE (REG, DIR, IPv4, unix, FIFO, CHR), and NAME (what the descriptor points at). Run it without root and you see only your own processes; run it with sudo and you see the machine.
Why it matters
lsof answers the questions that page you at 02:00. Which process owns port 8080 — the one systemctl swears is already in use? Which process has config.json open while you try to edit it? Which process still holds the 60 GB log you deleted an hour ago, so the filesystem has not freed a single block? And its quiet superpower: give it a path instead of a process and it reverses the question — `lsof /var/log/nginx/access.log` tells you who is reading your file right now. If a question sounds like 'who has this socket, file or mount open', it is an lsof question, not a ps question.
Reading the output
An empty result is a real answer: lsof exits 1 when nothing matched, which makes it scriptable. In sockets, `(LISTEN)` marks the server side and `->` marks the client side of a connection, so `TCP *:8642 (LISTEN)` is a daemon waiting and `TCP 192.168.1.57:50834->192.168.1.13:8123 (ESTABLISHED)` is a live outbound connection. The NAME column appends `(deleted)` when the inode's link count dropped to zero but a descriptor is still pinned to it. Two habits make the output useful: always add `-P -n` (no port-name or host-name resolution — faster and script-friendly), and remember that several selection options are OR-ed unless you insert `-a` to turn the logic into AND.
Example
$ lsof -i :8191 -P -nCOMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python3 3870937 kmail 3u IPv4 114698613 0t0 TCP 127.0.0.1:8191 (LISTEN)The 'who owns this port' invocation. -i :8191 filters to that port; -P stops lsof translating 8191 into a service name and -n stops it resolving 127.0.0.1 to 'localhost', which also skips the DNS round-trip. 3u = descriptor 3, open for read/write, listening on loopback only. (The example originally used port 8137, which nothing listens on any more — the listener moves, the invocation does not.)
$ lsof -t -i :81913870937-t (terse) prints PIDs and nothing else, which exists precisely so you can write `kill $(lsof -t -i :8137)` — no awk, no process-name guessing. Add -sTCP:LISTEN if you only want the listener and not its clients.
$ lsof +L1COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
python3 1690233 kmail 3w REG 252,0 67108864 0 9710017 /home/kmail/.hermes/cache/scratch/held.tmp (deleted)+L1 lists files with a link count below 1 — deleted files some process still holds open. NLINK 0 plus a 64 MB SIZE/OFF means 64 MB of disk that du cannot see and only a restart (or a kill) will release. This is the single most useful lsof flag on a full disk. Shown is the regular-file row from the run (the full +L1 output on this box also carried two `top` processes' deleted /dev/pts devices, which are the same phenomenon on a character device).
$ lsof -a -u kmail -i -P -n | head -6COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
code-e4c7 1460329 kmail 12u IPv4 25755658 0t0 TCP 127.0.0.1:37609 (LISTEN)
python3 1478166 kmail 8u IPv4 141296009 0t0 TCP 127.0.0.1:56008->127.0.0.1:9119 (ESTABLISHED)
python3 1478166 kmail 15u IPv4 142679063 0t0 TCP 192.168.1.57:39954->104.16.102.112:443 (ESTABLISHED)
hermes 1649116 kmail 42u IPv4 142690664 0t0 TCP 192.168.1.57:44842->104.16.103.112:443 (ESTABLISHED)
python3 1664797 kmail 8u IPv4 142097463 0t0 TCP 127.0.0.1:56236->127.0.0.1:9119 (ESTABLISHED)Without -a this would print every file owned by kmail OR every network file on the box — 3,530 lines at the last check — against 59 with -a, because -a turns the selectors into an AND. That is the difference between a wall of output and the sockets you actually wanted. The rows without -a also drag in other users' processes as `unknown`, which is why the flag is worth remembering before the pipe, not after it.
$ lsof -p 1861744 | head -4COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
node 1861744 kmail cwd DIR 252,0 4096 1336223 /home/kmail/app
node 1861744 kmail rtd DIR 252,0 4096 2 /
node 1861744 kmail txt REG 252,0 124836408 524303 /usr/bin/nodeOne process, every descriptor: cwd and rtd are its working and root directories, txt is the executable image, then come its libraries, logs and sockets. This is how you prove which binary is actually running and from which directory when a service behaves differently than its config suggests.
Common flags
- -i [addr]
- Only network files. `-i` for all of them, `-i :443` for a port, `-i @192.168.1.13:8123` for one remote endpoint, `-i 6` for IPv6.
- -p PID
- Restrict to a process (or a comma list of PIDs). The starting point for 'what has this process opened'.
- -u user
- Filter by user name or UID; prefix with ^ to exclude — `lsof -u ^root` lists everything root does not own.
- -a
- AND the selection options together instead of OR-ing them. Essential the moment you combine -u and -i.
- -t
- Terse mode: print PIDs only. Built for `kill $(lsof -t -i :PORT)`.
- +L1
- Open files whose link count is below 1 — the deleted-but-held files behind a full disk that df and du disagree about.
- +c 0
- Do not truncate command names. The COMMAND column silently stops at 9 characters unless you raise or remove the limit.
History
Written at Purdue, documented forever
Lsof was written by Victor A. Abell at Purdue University; since revision 4.93.0 it is maintained by the lsof-org team on GitHub, the man page on this box is revision 4.95.0. Abell's fingerprints are permanently in the documentation: the examples use his login name `abe`, his home directory `/Homes/abe` and the machine `lsof.itap.purdue.edu`. Very few tools have embedded their author's personal habits in a manual page that ships on millions of servers.
The device cache file Linux never writes
Examining every node under /dev with stat(2) is slow, and the answers rarely change, so lsof caches them: on the dialects that need it, it keeps an ASCII file of device number, inode and path whose name comes from the HASPERSDC format string, '%h/%p.lsof_%L' by default — that is, ~/.lsof_<hostname>. LSOFPERSDCPATH inserts a directory of your choice into that path (the man page's own example turns /Homes/abe into /Homes/abe/LSOF/.lsof_vic), and it is ignored when lsof runs setuid-root or with a real UID of 0. What is worth knowing on Linux is the exception the man page states in brackets: 'lsof normally maintains an ASCII text file of cached /dev (or /devices) information (exception: the /proc-based Linux lsof where it's not needed.)' The Linux build reads /proc/<pid>/fd instead, there is no cache file in your home directory — `ls ~/.lsof*` finds nothing here — and `-D` is not compiled in at all: `lsof -D?` answers 'unsupported option: -D' even though the man page documents the option and its `?` and `b` function letters for the dialects where the cache exists.
Fun facts
Pros & cons
pros
- + Sees every kind of file descriptor — sockets, logs, deleted files, directories, memory-mapped libraries — in one flat table
- + The selectors compose (-i, -p, -u, -c, -a, +D), so almost any 'who has what open' question is a single invocation
- + -t makes it trivially scriptable: PIDs straight into kill or ps with no text surgery
cons
- − Slow on busy machines — it stats every /proc/<pid>/fd, and on a box with tens of thousands of descriptors a bare lsof can take seconds
- − Without root the picture is silently partial, which has sent more than one admin chasing a phantom
- − Old defaults show through: the COMMAND column truncates at nine characters (lsof's documented default, not a terminal width) and Docker hosts get overlay-filesystem warnings on stderr
Takeaways
- 1When a port refuses to bind, run `sudo lsof -i :PORT -P -n` before blaming the application
- 2Close a stray listener in one line: `kill $(lsof -t -i :PORT)`
- 3If `df` says the disk is full but `du` disagrees, `lsof +L1` finds the deleted file a process is still holding
- 4Add `-a` whenever you stack selectors, or you will get an OR-list instead of an AND-answer
- 5`lsof /path/to/file` reverses the question and shows who is reading that exact file right now
