VisiData — a
terminal utility for exploring and arranging tabular data
vd |
[options] --play
cmdlog [-w
waitsecs] [--batch]
[-o output]
[field=value
...] |
VisiData is a
multipurpose tool built on the
vdtui
platform that can be used to explore, clean, edit, and
restructure data. Rows can be selected, filtered, and grouped;
columns can be rearranged, transformed, and derived via regex or Python
expressions; and workflows can be saved, documented, and replayed.
-p,
--play=cmdlog- replay a
saved cmdlog within the
interface
-w,
--replay-wait=seconds- wait
seconds between
commands
-b,
--batch- replay in batch mode (with no interface)
-o,
--output=file- save final visible sheet to
file as .tsv
- --replay-movement
- toggle
--play
to move cursor cell-by-cell
- field
=value
- replace
"{field}" in
cmdlog contents with
value
- ^U
- pause/resume replay
- Tab
- execute next row in replaying sheet
- ^K
- cancel current replay
In most cases, commands that affect selected rows will affect all
rows if no rows are selected.
^H- view this man page
z^H- view sheet of commands and keybindings
^Q- abort program immediately
^C- cancel user input or abort all async threads on current sheet
q- quit current sheet
gq- quit all sheets (clean exit)
Arrow PgUp
Home- move as expected
h j k l- move left/down/up/right
gh gj gk
gl- move all the way to the left/bottom/top/right of sheet
G gg- move all the way to the bottom/top of sheet
^B
^F- scroll one page back/forward
^^
(Ctrl-^)- jump to previous sheet (swaps with current sheet)
/ ? regex- search for
regex forward/backward in current
column
g/ g?
regex- search
for regex
forward/backward over all visible columns
z/ z?
expr- search
by Python expr
forward/backward in current column (with column names as
variables)
n N- move to next/previous match from last search
< >- move up/down to next value in current column
z<
z>- move up/down to next null in current column
{ }- move up/down to next selected row
c regex- move to next column
with name matching regex
r regex- move to next row with
key matching regex
zc zr
number- move to
column/row number
(0-based)
H J K L- slide current row/column left/down/up/right
gH gJ gK
gL- slide current row/column all the way to the left/bottom/top/right of sheet
zh zj zk
zl- scroll one left/down/up/right
[ ]- sort ascending/descending by current column
g[
g]- sort ascending/descending by all key columns
"- open duplicate sheet with only selected rows
g"- open duplicate sheet with all rows
gz"- open duplicate sheet with deepcopy of selected rows
Enter ^C- accept/abort input
^O- open external $EDITOR to edit contents
^R- reload initial value
^A
^E- move to beginning/end of line
^B
^F- move back/forward one character
^H
^D- delete previous/current character
^T- transpose previous and current characters
^U ^K- clear from cursor to beginning/end of line
Backspace
Del- delete previous/current character
Insert- toggle insert mode
Up
Down- set contents to previous/next in history
Tab Shift+Tab- autocomplete input (when available)
o input- open input in
VisiData
^S g^S
filename- save current/all sheet(s) to
filename in format determined by
extension (default .tsv)
- Note: if the format
does not support multisave, or the
filename ends in a
/, a directory will be created.
z^S
filename- save key columns and current column only to
filename in format determined by
extension (default .tsv)
^D
filename.vd- save
CommandLog to
filename.vd file
A
number- open new blank sheet with
number columns
R
number- pushes sheet with random population subset of number
rows
T- open new sheet with rows and columns transposed
+ aggregator- add
aggregator to current column
(see Frequency Table)
z+
aggregator- display
result of aggregator
over values in selected rows for current
column
+ -- increase/decrease zoom level, centered on cursor
_ (underscore)- zoom to fit full extent
s t u- select/toggle/unselect rows on source sheet contained within canvas
cursor
gs gt
gu- select/toggle/unselect rows on source sheet visible on screen
d- delete rows on source sheet contained within canvas cursor
gd- delete rows on source sheet visible on screen
Enter- open sheet of source rows contained within canvas cursor
gEnter- open sheet of source rows visible on screen
1 -
9- toggle display of layers
^L- redraw all pixels on canvas
v- toggle
show_graph_labels
option
mouse
scrollwheel- zoom in/out of canvas
left
click-drag- set canvas cursor
right
click-drag- scroll canvas
Q- quit current
sheet and remove it from the CommandLog
V- view contents of current cell in a new TextSheet
v- toggle sheet-specific visibility (text wrap on TextSheet, legends/axes on
Graph)
Space
longname- execute
command by its longname
^E- view traceback for most recent error
g^E- view traceback for most recent errors
z^E- view traceback for error in current cell
^L- refresh screen
^R- reload current sheet
z^R- clear cache for current column
^Z- suspend VisiData process
^G- show cursor position and bounds of current sheet on status line
^V- show version and copyright information on status line
^P- open
Status History
^Y z^Y g^Y- open current row/cell/sheet as Python object
^X
expr- evaluate
Python expr and opens
result as Python object
z^X
expr- evaluate
Python expr on current
row and shows result on status line
g^X
stmt- execute
Python stmt in the
global scope
- (sheet-specific commands)
- Modifying any cell changes the in-memory value. Changes are only applied
to the filesystem with
^S
Enter gEnter- open current/selected file(s) as new sheet(s)
^O g^O- open current/selected file(s) in external $EDITOR
d gd- schedule current/selected file(s) for deletion
^R z^R gz^R- reload information for all/current/selected file(s), undoing any pending
changes
z^S ^S- apply all deferred changes to current/all file(s)
- Properties of columns on the source sheet can be changed with
standard editing commands (e ge
g= Del) on the Columns Sheet. Multiple aggregators
can be set by listing them (separated by spaces) in the aggregators column.
The 'g' commands affect the selected rows, which are the literal columns on
the source sheet.
- (global commands)
gC- open
Columns Sheet for all visible columns
on all sheets
- (sheet-specific commands)
&- add column from concatenating selected source columns
g!
gz!- toggle/unset selected columns as key columns on source sheet
g+
aggregator- add Ar aggregator No to selected source columns
g-
(hyphen)- hide selected columns on source sheet
g~ g# g% g$
g@ gz#- set type of selected columns on source sheet to
str/int/float/currency/date/len
Enter- open a Frequency Table
sheet grouped by column referenced in current
row
- (global commands)
gS- open
Sheets Graveyard which includes
references to closed sheets
- (sheet-specific commands)
- .
- inner
keep only rows which match keys on all
sheets
- .
- outer
keep all rows from first selected sheet
- .
- full
keep all rows from all sheets (union)
- .
- diff
keep only rows NOT in all sheets
- .
- append
keep all rows from all sheets (concatenation)
- .
- extend
copy first selected sheet, keeping all rows and sheet
type, and extend with columns from other sheets
- (global commands)
- (sheet-specific commands)
Enter e- edit option at current row
- (global commands)
- (sheet-specific commands)
- A Frequency Table groups rows by one or
more columns, and includes summary columns for those with
aggregators.
- (global commands)
gF- open Frequency Table, grouped by all key columns on source sheet
zF- open one-line summary for selected rows
- (sheet-specific commands)
s t u- select/toggle/unselect these entries in source sheet
Enter- open sheet of source rows that are grouped in current cell
- (global commands)
gI- open
Describe Sheet for all visible columns
on all sheets
- (sheet-specific commands)
zs
zu- select/unselect rows on source sheet that are being described in current
cell
!- toggle/unset current column as a key column on source sheet
Enter- open a Frequency Table
sheet grouped on column referenced in current
row
zEnter- open copy of source sheet with rows described in current cell
- Set key column(s) and aggregators on column(s) before
pressing Shift+W
on the column to pivot.
- (sheet-specific commands)
Enter- open sheet of source rows aggregated in current pivot row
zEnter- open sheet of source rows aggregated in current pivot cell
- Open melted sheet (unpivot), with key columns retained and all non-key
columns reduced to Variable-Value rows.
- (global commands)
gM
regex- open melted sheet (unpivot), with key columns retained
and regex capture groups
determining how the non-key columns will be reduced to Variable-Value
rows.
- (sheet-specific commands)
Enter- dive further into Python object
e- edit contents of current cell
v- toggle show/hide for methods and hidden properties
gv zv- show/hide methods and hidden properties
vd
foo.tsv
open the file foo.tsv in the current
directory
vd
-f sqlite bar.db
open the file bar.db as a sqlite database
vd
-b countries.fixed -o countries.tsv
convert countries.fixed (in fixed width format) to
countries.tsv (in tsv format)
vd
postgres://username:password@hostname:port/databasevd
--play tests/pivot.vd --replay-wait 1 --output
tests/pivot.tsv
replay tests/pivot.vd, waiting 1 second between commands, and
output the final sheet to test/pivot.tsv
ls -l |
vd -f fixed --skip 1 --header
0
parse the output of ls -l into usable data
At the start of every session, VisiData
looks for $HOME/.visidatarc,
and calls Python exec() on its contents if it exists. For example:
options.min_memory_mb=100 # stop processing without 100MB free
bindkey('0', 'go-leftmost') # alias '0' to move to first column, like vim
def median(values):
L = sorted(values)
return L[len(L)//2]
aggregator('median', median)
Functions defined in .visidatarc are available in python
expressions (e.g. in derived columns).
These are the supported sources:
- tsv
(tab-separated value)
-
- Plain and simple.
VisiData
writes tsv format by default. See the
--delimiter option.
- csv
(comma-separated value)
-
- .csv files are a scourge upon the earth, and still regrettably
common.
- See the --csv-dialect,
--csv-delimiter, --csv-quotechar,
and --csv-skipinitialspace
options.
- Accepted dialects are
excel-tab,
unix, and excel.
- fixed
(fixed width text)
-
- Columns are autodetected from the first 1000 rows (adjustable with
--fixed-rows).
- json
(single object) and
jsonl
(one object per line).
-
- Cells containing lists (e.g. [3]) or dicts
({3}) can be expanded into new columns with
(
and unexpanded with ).
- yaml/yml
(requires
PyYAML)
- pcap
(requires
xpkt,
dnslib)
-
- View and investigate captured network traffic in a tabular
format.
- png
(requires pypng)
-
- Pixels can be edited and saved in data form. Images can be plotted
with
. (dot).
The following URL schemes are supported:
- http
(requires
requests);
can be used as transport for with another filetype
- postgres
(requires
psycopg2)
- The following sources may include multiple tables. The initial
sheet is the table directory;
- Enter
loads the entire table into memory.
- sqlite
- xlsx
(requires
openpyxl)
- xls
(requires
xlrd)
- hdf5
(requires
h5py)
- ttf/otf
(requires
fonttools)
- mbtiles
(requires
mapbox-vector-tile)
- htm/html
(requires lxml)
- xml
(requires lxml)
-
- v
- show only columns in current row attributes
- za
- add column for xml attributes
- xpt
(SAS; requires
xport)
- sas7bdat
(SAS; requires sas7bdat)
- sav
(SPSS; requires
savReaderWriter)
- dta
(Stata; requires pandas)
- shp
(requires
pyshp)
In addition,
.zip,
.gz,
.bz2, and
.xz
files are decompressed on the fly.
VisiData has an adapter
for pandas. To load a file format which is
supported by pandas, pass -f pandas
data.foo. This will call
pandas.read_foo().
For example,
vd -f pandas
data.parquet loads a parquet file. Note that when
using the pandas loader,
the
.fileformat
file extension is mandatory
These are the supported savers:
- tsv
(tab-separated value)
- csv
(comma-separated value)
- json
(one object with all rows)
-
- All expanded subcolumns must be closed (with )) to
retain the same structure.
- .shp
files can be saved as
geoJSON.
- md
(org-mode compatible markdown table)
- htm/html
(requires lxml)
- png
(requires pypng)
Multisave is supported by
html, md, and
txt;
g^S
will save all sheets into a single output file.