MGD77SNIFFER(1gmt) | GMT | MGD77SNIFFER(1gmt) |
mgd77sniffer - Along-track quality control of MGD77 cruises
mgd77sniffer NGDC-ids [ -Afieldabbrev,scale,offset ] [ -Cmaxspd ] [ -Dd|e|E|f|l|m|s|v[r] ] [ -E ] [ -Gfieldabbrev,imggrid,scale,mode or -Gfieldabbrev,grid ] [ -H ] [ -Ifieldabbrev,rec1,recN ] [ -Lcustom-limits-file ] [ -M ] [ -N ] [ -Rregion ] [ -Sd|s|t ] [ -Tgap ] [ -V[level] ] [ -Wc|g|o|s|t|v|x ] [ -Zlevel ] [ -bobinary ] [ -donodata ] [ -nflags ]
Note: No space is allowed between the option flag and the associated arguments.
mgd77sniffer scans old (pre-Y2K) and new format ASCII MGD77 files for errors using point-by-point sanity checking, along-track detection of excessive slopes, and optional comparison of cruise data with global gravity and predicted bathymetry grids. Detected data problems are output by default as verbose descriptions of each detected error, often resulting in multiple messages per scanned record. Data problems are optionally output (-De option) using a computer-parseable format (see E77 ERROR FORMAT description below). Default error thresholds are derived from histograms of all MGD77 geophysical data collected between 1952 and January, 2006. Thresholds are adjustable with the -L option.
The mgd77sniffer links with Generic Mapping Tools 4.0 or later along with the supplemental GMT packages x2sys and mgd77. See http://gmt.soest.hawaii.edu for GMT details. Grids for comparison with cruise data may be downloaded via the web.
(See mgd77info -L for agency and vessel codes). If no file extension is given then we search for files with one of the four known extensions. The search order (and the extensions) tried is MGD77+ (".nc"), MGD77T (".m77t"), MGD77 (".mgd77" ) and plain text file (".dat"). Use -I to ignore one or more of these file types). Cruise files will be looked for first in the current directory and second in all directories listed in $MGD77_HOME/mgd77_paths.txt [If $MGD77_HOME is not set it will default to $GMT_SHAREDIR/mgd77].
-Dd output differences between cruise and grid data. Requires -G option. Output columns include:
lat lon dist cruiseZ gridZ diff [cruiseZ2 gridZ2 diff2 ...]
Note: grid values are subtracted from cruise data so a positive difference implies cruise > grid. For multiple grid comparison, cruiseZ gridZ diff are repeated for each grid comparison in command line order.
-De output E77 error classification format. Error output is divided into (1) a header containing information globally applicable to the cruise and (2) individual error records summarizing all errors encountered in each cruise record. mgd77sniffer writes E77 directly to <ngdc_id.e77> file handle. See E77 ERROR FORMAT below for additional details.
-DE Same as -De but no regression tests will be carried out.
-Df output delta Z (change in geophysical field) column and delta S (change in distance) for each geophysical field. Distance between observations often differ for different fields depending on instrument sampling rate, so ds is included for each geophysical observation. Output columns include:
d[twt] ds d[depth] ds d[mtf1] ds d[mtf2] ds d[mag] ds d[diur] ds d[msd] ds d[gobs] ds d[eot] ds d[faa] ds
-Dl display mgd77sniffer limits. Customize this output to create a custom limits file for the -L option. No additional arguments are required. Output columns include:
fieldabbrev min max maxSlope maxArea
-Dm output MGD77 format records in Y2K-compliant MGD77 format
-Dn output distance to coast for each record. Requires the -Gnav option. Output columns include:
lat lon dist distToCoast
-Ds output calculated gradients for speed and geophysical fields. Gradients correspond to the gradient type selected in the -S option (spatial derivatives by default). Output columns include:
speed d[twt] d[depth] d[mtf1] d[mtf2] d[mag] d[diur] d[msd] d[gobs] d[eot] d[faa]
See MGD77 FIELD INFO below for field and abbreviations descriptions.
-Dv display values for the twelve position and geophysical fields for each MGD77 data record (in this order):
lat lon twt depth mtf1 mtf2 mag diur msens gobs eot faa
See below for MGD77 FIELD INFO.
Grid comparison activates several additional error checks. (1) Re-weighted Least Squares Regression of ship versus grid data determines slope and DC shift, which when differing from expected 1 and 0, respectively, may indicate incorrectly scaled ship data, including incorrect units or instrument drift as well as erroneous gravity tie-in. (2) Accumulated ship grid offsets are computed along-track and excessive offsets are flagged according to maxArea threshold (use -L option to adjust maxArea). Warning: predicted bathymetry grids are constrained by cruise data so grids and cruise data are not always independent. Comparison of cruise bathymetry with predicted bathymetry grids also activates a "navigation crossing over land" check.
-Hb analyze both (default), report better of two.
-Hd to disable data decimation (equivalent to -H with no argument).
-Hf to force data decimation.
Example custom default file contents (see below for units):
# abbrev | min | max | maxSlope | maxArea |
twt | 0 | 15 | 1 | 0 |
depth | 0 | 11000 | 500 | 5000 |
mag | -800 | 800 | -- | -- |
faa | -300 | 300 | 100 | 2500 |
Use a dash '-' to retain a default limit. Hint: to test your custom limits, try: mgd77sniffer -Dl -L<yourlimitsfile>
Resample or sampling of grids will use various algorithms (see -n) that may lead to possible distortions or unexpected results in the resampled values. One expected effect of resampling with splines is the tendency for the new resampled values to slightly exceed the global min/max limits of the original grid. If this is unacceptable, you can impose clipping of the resampled values values so they do not exceed the input min/max values by adding +c to your -n option.
Field | Abbreviation | Units |
Two-way | Travel | Time |
Corrected | Depth | depth |
Mag | Total | Field1 |
Mag | Total | Field2 |
Residual | Magnetic | mag |
Diurnal | Correction | diur |
Mag | Sensor | Depth/Alt |
Observed | Gravity | gobs |
Eotvos | Correction | eot |
Free | Air | Anomaly |
For -G the grids must eitehr be in the format used by Sandwell & Smith, which is a spherical Mercator 2-byte grid with no header, or any grid type supported by GMT and therefore must contain a GMT header. A correctly formatted *.i2 grid file can be generated using grdraster as shown below.
gmtset GRIDFILE_SHORTHAND TRUE
Create/edit .gmt_io file to include the following rows:
# GMT I/O shorthand file
# suffix format_id scale offset NaN
grd 0 - - - -
i2 2 - - 32767
gmt grdraster 1 -R0/359:55/-90/90 -Getopo5_hdr.i2
The new grid, etopo5_hdr.i2 in this example, contains a GMT header and can be used in the -G option to compare cruise depth with grid values.
Sample
# Cruise 08010039 ID 74010908 MGD77 FILE VERSION: 19801230 N_RECS: 3066
# Examined: Wed Oct 3 16:30:13 2007 by mtchandl
# Arguments: -De -Gdepth,/data/GRIDS/etopo5_hdr.i2
N Errata table verification status
# mgd77manage applies corrections if the errata table is verified (toggle 'N' above to 'Y' after review)
# For instructions on E77 format and usage, see http://gmt.soest.hawaii.edu/mgd77/errata.php
# Verified by:
# Comments:
# Errata: Header
Y-E-08010039-H13-02: Invalid Magnetics Sampling Rate: (99) [ ]
Y-W-08010039-H13-10: Survey year (1975) outside magnetic reference field IGRF 1965 time range (1965-1970)
Y-I-08010039-depth-00: RLS m: 1.00053 b: 0 rms: 127.851 r: 0.973422 significant: 1 decimation: 0
Y-W-08010039-twt-09: More recent bathymetry correction table available
Y-W-08010039-mtf1-10: Integer precision
Y-W-08010039-mag-10: Integer precision
Format <time/distance> <record number> <error code string> <description>
Sample
# Errata: Data
Y 08010039 1975-05-10T22:16:05.88 74 C-0-0 NAV: excessive speed
Error classes: NAV-VAL-GRAD
Error Class Descriptions
NAV (navigation):
0 - fine
A - time out of range
B - time decreasing
C - excessive speed
D - above sea level
E - lat undefined
F - lon undefined
VAL (value):
0 - fine
K - twt invalid
L - depth invalid
O - mtf1 invalid
etc.
GRAD (gradient):
0 - fine
K - d[twt] excessive
L - d[depth] excessive
The NAV error class has unique cases while VAL and GRAD classes are described by alphabetic characters for each of the 24 numeric fields in MGD77 format order.
MGD77 bit-pattern w/ E77 alpha characters
|-------------------------------------------------|----------|
| X W V U T S R Q P O N M L K J I H G F E D C B A | E77 Code |
| - - - - - - - - - - - - - - - - - - - - - - - - | - - - - -|
| n f e g m d m m m m b b d t p l l m h d m y t d | F I |
| q a o o s i s a t t t c e w t o a i o a o e z r | i D |
| c a t b d u e g f f c c p t c n t n u y n a t | e |
| s r n 2 1 t r t r | l |
| s h h | d |
| - - - - - - - - - - - - - - - - - - - - - - - - | - - - - -|
| 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 | Bit place|
| - G C G C C - G G G - - G G - - - T T T T T - - | Bit type |
|-------------------------------------------------|----------|
Bit types: (G)eophysical, (C)orrection, (T)ime
To scan for excessive values or gradients, try
gmt mgd77sniffer 08010001
To dump cruise gradients, try
gmt mgd77sniffer 08010001 -Ds
To compare cruise depth with ETOPO5 bathymetry and gravity with Sandwell/Smith 2 min gravity version 11, try
mgd77sniffer 08010001 -Gdepth,/data/GRIDS/etopo5_hdr.i2 \
-Gfaa,/data/GRIDS/grav.11.2.img,0.1,1
grdraster, mgd77list, mgd77track x2sys_init
The Marine Geophysical Data Exchange Format - MGD77, see http://www.ngdc.noaa.gov/mgg/dat/geodas/docs/mgd77.txt.
2019, P. Wessel, W. H. F. Smith, R. Scharroo, J. Luis, and F. Wobbe
May 21, 2019 | 5.4.5 |