tskymatch2 performs a crossmatch of two tables based on the
proximity of sky positions. You specify the columns or expressions giving
right ascension and declination in degrees for each input table, and a
maximum permissible separation in arcseconds, and the resulting joined table
is output.
If you omit expressions for the RA and Dec, an attempt is made to
identify the columns to use using column Unified Content Descriptors (UCDs)
or names. First columns bearing appropriate UCD1 or UCD1+ values
(POS_EQ_RA, POS_EQ_RA_MAIN, pos.eq.ra or
pos.eq.ra;meta.main and their equivalents for declination) are
sought. If these cannot be found, columns named something like
"RA" or "RA2000" are sought. If either is found, the
column units are consulted and radian->degree conversions are performed
if necessary (degrees are assumed if no unit value is given). If nothing
likely can be found, then the command will fail with an error message. This
search logic is intended as a convenience only; it is somewhat ad hoc and
subject to change. To make sure that the correct angle values are being
used, specify the ra and dec position parameters
explicitly.
tskymatch2 is simply a cut-down version, provided for
convenience, of the more general matching task tmatch2. If you want
more match options or otherwise more configurability, you can probably find
it by using tmatch2.
- in1=<table1>
The location of the first input table. This may take one
of the following forms:
- A filename.
- A URL.
- The special value "-", meaning standard input. In this
case the input format must be given explicitly using the ifmt1
parameter. Note that not all formats can be streamed in this way.
- A scheme specification of the form
:<scheme-name>:<scheme-args>.
- A system command line with either a "<" character at
the start, or a "|" character at the end
("<syscmd" or "syscmd|"). This
executes the given pipeline and reads from its standard output. This will
probably only work on unix-like systems.
In any case, compressed data in one of the supported compression formats (gzip,
Unix compress or bzip2) will be decompressed transparently.
- ifmt1=<in-format>
Specifies the format of the first input table as
specified by parameter
in1. The known formats are listed in SUN/256.
This flag can be used if you know what format your table is in. If it has the
special value
(auto) (the default), then an attempt will be made to
detect the format of the table automatically. This cannot always be done
correctly however, in which case the program will exit with an error
explaining which formats were attempted. This parameter is ignored for
scheme-specified tables.
- in2=<table2>
The location of the second input table. This may take one
of the following forms:
- A filename.
- A URL.
- The special value "-", meaning standard input. In this
case the input format must be given explicitly using the ifmt2
parameter. Note that not all formats can be streamed in this way.
- A scheme specification of the form
:<scheme-name>:<scheme-args>.
- A system command line with either a "<" character at
the start, or a "|" character at the end
("<syscmd" or "syscmd|"). This
executes the given pipeline and reads from its standard output. This will
probably only work on unix-like systems.
In any case, compressed data in one of the supported compression formats (gzip,
Unix compress or bzip2) will be decompressed transparently.
- ifmt2=<in-format>
Specifies the format of the second input table as
specified by parameter
in2. The known formats are listed in SUN/256.
This flag can be used if you know what format your table is in. If it has the
special value
(auto) (the default), then an attempt will be made to
detect the format of the table automatically. This cannot always be done
correctly however, in which case the program will exit with an error
explaining which formats were attempted. This parameter is ignored for
scheme-specified tables.
- omode=out|meta|stats|count|checksum|cgi|discard|topcat|samp|tosql|gui
The mode in which the result table will be output. The
default mode is
out, which means that the result will be written as a
new table to disk or elsewhere, as determined by the
out and
ofmt parameters. However, there are other possibilities, which
correspond to uses to which a table can be put other than outputting it, such
as displaying metadata, calculating statistics, or populating a table in an
SQL database. For some values of this parameter, additional parameters
(
<mode-args>) are required to determine the exact behaviour.
Possible values are
- out
- meta
- stats
- count
- checksum
- cgi
- discard
- topcat
- samp
- tosql
- gui
Use the
help=omode flag or see SUN/256 for more information.
- out=<out-table>
The location of the output table. This is usually a
filename to write to. If it is equal to the special value "-" (the
default) the output table will be written to standard output.
This parameter must only be given if omode has its default
value of "out".
- ofmt=<out-format>
Specifies the format in which the output table will be
written (one of the ones in SUN/256 - matching is case-insensitive and you can
use just the first few letters). If it has the special value
"
(auto)" (the default), then the output filename will be
examined to try to guess what sort of file is required usually by looking at
the extension. If it's not obvious from the filename what output format is
intended, an error will result.
This parameter must only be given if omode has its default
value of "out".
- ra1=<expr>
Right ascension in degrees for the position of each row
of table 1. This may simply be a column name, or it may be an algebraic
expression calculated from columns as explained in SUN/256. If left blank, an
attempt is made to guess from UCDs, column names and unit annotations what
expression to use.
- dec1=<expr>
Declination in degrees for the position of each row of
table 1. This may simply be a column name, or it may be an algebraic
expression calculated from columns as explained in SUN/256. If left blank, an
attempt is made to guess from UCDs, column names and unit annotations what
expression to use.
- ra2=<expr>
Right ascension in degrees for the position of each row
of table 2. This may simply be a column name, or it may be an algebraic
expression calculated from columns as explained in SUN/256. If left blank, an
attempt is made to guess from UCDs, column names and unit annotations what
expression to use.
- dec2=<expr>
Declination in degrees for the position of each row of
table 2. This may simply be a column name, or it may be an algebraic
expression calculated from columns as explained in SUN/256. If left blank, an
attempt is made to guess from UCDs, column names and unit annotations what
expression to use.
- error=<value/arcsec>
The maximum separation permitted between two objects for
them to count as a match. Units are arc seconds.
- tuning=<healpix-k>
Tuning parameter that controls the pixel size used when
binning the rows. The legal range is from 0 (corresponding to pixel size of
about 60 degrees) to 20 (about 0.2 arcsec). The value of this parameter will
not affect the result but may affect the performance in terms of CPU and
memory resources required. A default value will be chosen based on the size of
the
error parameter, but it may be possible to improve performance by
adjusting the default value. The value used can be seen by examining the
progress output. If your match is taking a long time or is failing from lack
of memory it may be worth trying different values for this parameter.
- join=1and2|1or2|all1|all2|1not2|2not1|1xor2
Determines which rows are included in the output table.
The matching algorithm determines which of the rows from the first table
correspond to which rows from the second. This parameter determines what to do
with that information. Perhaps the most obvious thing is to write out a table
containing only rows which correspond to a row in both of the two input
tables. However, you may also want to see the unmatched rows from one or both
input tables, or rows present in one table but unmatched in the other, or
other possibilities. The options are:
- 1and2: An output row for each row represented in both input tables
(INNER JOIN)
- 1or2: An output row for each row represented in either or both of
the input tables (FULL OUTER JOIN)
- all1: An output row for each matched or unmatched row in table 1
(LEFT OUTER JOIN)
- all2: An output row for each matched or unmatched row in table 2
(RIGHT OUTER JOIN)
- 1not2: An output row only for rows which appear in the first table
but are not matched in the second table
- 2not1: An output row only for rows which appear in the second table
but are not matched in the first table
- 1xor2: An output row only for rows represented in one of the input
tables but not the other one
- find=all|best|best1|best2
Determines what happens when a row in one table can be
matched by more than one row in the other table. The options are:
- all: All matches. Every match between the two tables is included in
the result. Rows from both of the input tables may appear multiple times
in the result.
- best: Best match, symmetric. The best pairs are selected in a way
which treats the two tables symmetrically. Any input row which appears in
one result pair is disqualified from appearing in any other result pair,
so each row from both input tables will appear in at most one row in the
result.
- best1: Best match for each Table 1 row. For each row in table 1,
only the best match from table 2 will appear in the result. Each row from
table 1 will appear a maximum of once in the result, but rows from table 2
may appear multiple times.
- best2: Best match for each Table 2 row. For each row in table 2,
only the best match from table 1 will appear in the result. Each row from
table 2 will appear a maximum of once in the result, but rows from table 1
may appear multiple times.
The differences between
best,
best1 and
best2 are a bit
subtle. In cases where it's obvious which object in each table is the best
match for which object in the other, choosing betwen these options will not
affect the result. However, in crowded fields (where the distance between
objects within one or both tables is typically similar to or smaller than the
specified match radius) it will make a difference. In this case one of the
asymmetric options (
best1 or
best2) is usually more appropriate
than
best, but you'll have to think about which of them suits your
requirements. The performance (time and memory usage) of the match may also
differ between these options, especially if one table is much bigger than the
other.
- runner=parallel|parallel<n>|parallel-all|sequential|classic|partest
Selects the threading implementation. The options are
currently:
- parallel: uses multithreaded implementation for large tables, with
default parallelism, which is the smaller of 6 and the number of available
processors
- parallel<n>: uses multithreaded implementation for large
tables, with parallelism given by the supplied value <n>
- parallel-all: uses multithreaded implementation for large tables,
with a parallelism given by the number of available processors
- sequential: uses multithreaded implementation but with only a
single thread
- classic: uses legacy sequential implementation
- partest: uses multithreaded implementation even when tables are
small
The
parallel* options should normally run faster than
sequential
or
classic (which are provided mainly for testing purposes), at least
for large matches and where multiple processing cores are available.
The default value "parallel" is currently limited
to a parallelism of 6 since larger values yield diminishing returns given
that some parts of the matching algorithms run sequentially (Amdahl's Law),
and using too many threads can sometimes end up doing more work or impacting
on other operations on the same machine. But you can experiment with other
concurrencies, e.g. "parallel16" to run on 16 cores (if
available) or "parallel-all" to run on all available
cores.
The value of this parameter should make no difference to the
matching results. If you notice any discrepancies please report
them.