Merge_go provides IDL routines for manipulating and plotting merged SuperDARN HF radar data in a command line environment. The principal program is PLOT_MERGE and it will produce polar or customised plots of all (if requrired) available merged superDARN data in either geographic or geomagnetic coordinates. Below is a discription of the basic routine, how it is constructed, how it is run and how to create the merged data files. You will either need to copy the program from
/people/cutlass/idl/plot_merge/merge_go
or make a link to it. Note that merge_go will also launch go since overplotting line-of-sight data routines uses go programs.
To run the programs you must first set up certain environment variables. These are:
SD_FIT_PATH, SD_FITROPEN_PATH, which you will probably have already set to read "fit" data files
IDL_PLOT_FILE, which specifies the extension on postscript files
and SD_MERGE, which specifies the merged data file directory. This you will not already have. I recomend that you set it to "/cutlass/data/who_you_are/merge"
To run load the IDL plotting routines simply type merge_go in a unix shell. You may need to copy this file from the CUTLASS area.
Please report all spelling mistakes (and any other comments) to the spelling mistake hotline.
Below is how I have set up my (jrt2) paths for CUTLASS plotting routines. You might want to copy something similar at the start of your version of merge_go, or put it into a .cutlass_paths that you may run from your .profile when you login:
unset SD_FITROPEN_PATH unset SD_RAWROPEN_PATH unset SD_FIT_PATH unset SD_LIB_FIT unset SD_MERGE unset IDL_PLOT_FILE unset SD_RAWROPEN_PATH SD_FITROPEN_PATH=/cutlass/data/jrt2/fit/:/cutlass/finland/incoming/:/cutlass/iceland/incoming/:/cutlass/sdmr/fit/:/people/cutlass/Data/:.; export SD_FITROPEN_PATH SD_RAWROPEN_PATH=/cutlass/data/jrt2/dat/:/cutlass/finland/incoming/:/cutlass/iceland/incoming/:/people/cutlass/Data/:.; export SD_RAWROPEN_PATH SD_FIT_PATH=/cutlass/data/jrt2/fit/:/cutlass/finland/incoming/:/cutlass/iceland/incoming/:/people/cutlass/Data/:.; export SD_FIT_PATH SD_LIB_FIT=/people/cutlass/lib/share/libfit.so; export SD_LIB_FIT SD_MERGE=/cutlass/data/jrt2/merge/:/cutlass/data/jrt2/fit/:/cutlass/finland/incoming/:/cutlass/iceland/incoming/:/people/cutlass/Data/:.; export SD_MERGE SD_RAWROPEN_PATH=/people/cutlass/Data/:/cutlass/finland/incoming/:/cutlass/data/jrt2/dat/:/people/jrt2/Cutlass/model/:.; export SD_RAWROPEN_PATH export IDL_PLOT_FILE='ps' IDL_PATH=\+$IDL_DIR/lib export IDL_PATH IDL_PATH=$IDL_PATH:\+/people/nmw/idl export IDL_PATH=$IDL_PATH:/people/jrt2/Cutlass/idl/plot_radar:. export IDL_PATH=$IDL_PATH:/people/cutlass/lib/share:/people/yxo/Idle/lib:/people/cutlass/idl/lib:/people/cutlass/idl/plot_radar:.
Usage
merge_all -f <file_root {yymmddhh}> [ -s <start_time {hhmm}>
-i <integration_time> -d <duration {ddd}> -c -D -S
-p <min power> -v <min velocity> -m <max velocity>
-e <max velocity error> ]
Although superDARN radars have been built in pairs, data from any two radars with
an overlapping field-of-view (f-o-v) can be merged to produce vector components. The
unix shell script "merge_all", via "merge_run", will
automatically run the merge
program
producing files containing standard merged data from all possible (existing) radar
combinations. The only mandatory argument is
-f file_root The filename root is of the form yymmddhh the other arguments are optional
-s start_time (hhmm) Optional start time, unless it is specified, the start time is taken as the start time of the fit file.
-i integration_time (sec) This is the integration time for which data is to be merged and it is in seconds (s). The default is 240 s. -d duration (ddd) the number of hours of data to be merged (if not specified then the default is 2 hours) and an option for concatenated data files <-c>. If no duration is specified then the default time is 2 hours. ddd can be of any length. The output files will be of the form yymmdd??_EXTENSION.mrg where ?? are the letters denoting which pair of radars have been merged (eg ef for Iceland-West and Finland, see radar idenfication ). EXTENSION='stnd' if the -s and -i options are not specified. Otherwise it is set to hhmmddd
For example, typing:
merge_all -f 95081006
will create 2 hour data files:
"950810ab_stnd.mrg", "950810tk_stnd.mrg", "950810gw_stnd.mrg"... and so on
where the letters following the file_root is defined by the radar id. The output merged files will have been generated by the merge standard options.
merge_all -f 95081006 -i 600 -s 0710 -d 1 -c
will create 1 hour data files from concatenated data:
"950810ab_0710600.mrg", "950810tk_0710600.mrg", "950810gw_0710600.mrg"... and so on
The only constraint is that the files must be in an area specified by your SD_FIT_PATH environment variable. The created data files will then found in be the first path specified by your SD_MERGE environment variable
-p <min power> Specifies the minimim power threshold for merging. The default is 3dB
-v <min velocity> Specifies the minimim velocity threshold for merging. The default is 50 m/s to avoid misidentified ground scatter
-m <max velocity> Specifies the maximim velocity threshold for merging.
-e <max velocity error> Specifies the maximum velocity error for merging.
-D Indicates that divergence free analysis is to be used
-S Is single scan merging
Plot_merge is a versatile program which plots snapshots of the convection flow vectors either in the form of polar plots or of a more specified f-o-v. It also alows overlaying of line-of-sight data. The output can either be in the form of polar plots or more localised f-o-vs. Plot_merge is designed to utilise some GO routines. In particular it uses Steve Millan's map_setup routines for non-polar plotting since this gives square plots and can overshoot the pole. If you experience any problems with this routine contact John Taylor.
The aguments specified when you last ran the program are saved as defaults (except for the quality, limits and pole options, the l-o-s overlay options and customising options no_open, no_close, and start_frame, ).The defaults file is $HOME/.merge_preferences/default. If the directory does not already exist then it will be created.
Usage
PLOT_MERGE, 'file_rt' <,hh,mm <,nmn>>, limits='[lat_mid,lat_rad,lon_mid,lon_rad]', frames=frames, pole=pole, lat=lat, start_frame=start_frame, scale=scale, extension='extension', set_prefs='prefs_file', read_prefs='prefs_file', quality=quality, colour='colour', oplot='fit_file_root', ocol=ocol,stations='radars', reverse='radars', start_frame='start_frame', ps_name='ps_name', eiscat='eiscat_tri-vel_file', dmsp=['DMSP_R_files'], iaga=['iaga_station_IDs'], baseline='mag_baseline_file', mag_path='path_to_iaga_files', label='panel_label' /geographic, /magnetic, /tty, /single_frame, /bw, /no_vectors, /no_quality, /all_frames, /post, /no_open, /no_close, /no_erase /tbj, /ls
If no arguments are specified then the above usage is given.
'File_rt': this is either the name of a single file which is to be plotted (e.g. "950810C_stnd.mrg"), or the root name of a collection of files to be plotted (e.g. "950810"). The program automatically knows from the title whether it is plotting data from more than one file. The program then checks its SD_MERGE directories for the file(s). For the multi pair plot files of all possible combinations are sought. Those which are found are then plotted. Note that a merged data file may not contain any data, but will be of a finite size. If the superDARN radars were synchronised this will have no effect on the plotted data. The program will then automatically check if these files are of the same integration time and that the times are synchronised to within the integration time. Non-synchronised radar files but with overlapping times will have an overlap in the frame times.
[,hh,mm [,nmn]]: these are the start hour, and minite and the total number of minutes to be plotted. All paramters are optional. if hh is specified so must mm. In the absence of these parameters the data will be plotted from the start of the data files to the end of the data files.
'Extension': this is the file extension name for multiple pairs. The default is "_stnd.mrg"
Scale: the default scale is 500m/s. This is the size of the scale bar. It is important to remember that the program does not plot velocity vectors, rather (as was agreed by the SuperDARN merging working group) it plots flow directions with the arrow's length determined by the scale length. So, what the program actually does is this. It takes a test particle moving with the bulk flow. It works out where that test particle would be after a time interval based on the scale length (remember that it follows the path of a great circle). It then plots a straight line from the point of origin to this new position. For more information on how the vectors are calcuated click here. Since the map is a 2 dimensional projection of a curved serface, the vector that is plotted will only be an approximation of the true length. The calculations are based on spherical geometry and as long as the latitudinal scale is constant will give a good representation of the vector, so be warned if you want to change the projection type. The scaling is based on the upper and lower latitude difference. For polar plots this is twice the co-latitude.
Pole: The program automatically tests if it has data from both hemispheres. If it does its default is to plot both of them. If POLE is specified then only the requested hemisphere will be plotted.
Limits: This argument is an array of 4 elements which specify [mid lat, lat raduis, mid long , long radius]. If the argument is not specified the default values are used and a multi pair plot will be polar. For a single pair plot of standard merged data, default limits are automatically plotted if the Limits argument is not specified or if the array value is -1.
i.e. PLOT_MERGE, '950810C_stnd.mrg', limits=[75, 10, -1, -1]
will plot a f-o-v based on the default longitude values but a centre lat of 75 deg and a lat raduis of 10 deg
Lat: The lower latitude boundary for polar plots. The default is 65 degrees.
Frames: The default number of frames plotted per page is 8. Other frame numbers can be specified: 1, 2, or 4. plot_merge will then automatically set the page to portrait or landscape. More general page layouts are not currently avilable due to technical complications associated with preserving the correct aspect ratio, but users are invited to modify the program where necessary
Start_frame: Specifies the frame number to start plotting at. The default is zero. If start_frame=0 then a new page is begun unless the keyword no_erase is set. For use with no_open and no_close
set_prefs:This option can be used for saving a set of preferences to a named file 'set_prefs'. This also allows the limits and pole arguements to be saved.The preference file will be located in $HOME/.merge_preferences. If the directory does not already exist then it will be created.
read_prefs:This option can be used for retrieving a set of preferences from a named file 'set_prefs'. This also allows the limits and pole arguments to be retrieved.
Quality: This argument determines which points are to be plotted based upon the map_qlty parameter. The quality index is a four digit number. Each digit is directly comparable to the 4 digits specified in map_qlty. The value of each digit represents the maximum quality value that vectors from each individual frame for each radar will be plotted. It is advised that when producing plots from many stations that the first digit is set to zero. Note also that this value always has a default value of 0000 unless specified when using set_prefs and read_prefs aguments.
Oplot: This is the root name of a fit file if l-o-s data is to be overplotted, eg '95032310' or if the data has been concatenated '95032310_'. The l-o-s data is, by default, averaged over the same interval as the plotted vectors. Ionospheric scatter only is considered. To plot a single scan of data, specify the requried time at the end of the oplot argument. For instance if you want to plot the 1043 UT scan of a concatenated file then the oplot argument is '95032310_1043'.
Stations: This keyword's arguments are the radars for which l-o-s data is to be plotted. Eg for Findland and Goose Bay the argument would be 'fg'. Any number of stations can be plotted.
Reverse: The arguements of this keyword direct the program to reverse the colour bar of radars specified by the stations keyword.
Ocol: Specifies the colour of the overlayed l-o-s data:
ocol=0: two tone grey (or yellow if colour=1)
1: Standard 'go' colour scale
2: red away, blue toward. Good for looking at flow shears.
3: two tone grey (or yellow if colour=1) with l-o-s vectors overlayed
Ps_name: The name of the postscript file. If not set then a default file name is generated
Eiscat: Specifies the name of the EISCAT tristatic velocity file for overlaying EISCAT data
Dmsp: Specifies the names of DMSP R files.
eg: dmsp=['a9508211.f11','a9508211.f12','a9508209.f11','a9508209.f12']
Iaga: Specifies the station IDs of IAGA stations for plotting equivalent convection flow streamlines
eg: ['amk','atu','bjn','con','dmh','dnb']
the root file name is the root file name for the merged file
Baseline: Specifies the name of the file which contains the baseline data for the IAGA mangetometer data files
Mag_path: The path to the IAGA files
label: Labels each page with the apropriate text. ie label='some sort of text'
/Tbj: This keyword over-rides all other l-o-s commands. It produces a single page plot of merged data for a single station pair with the l-o-s data either side of the vector data. Very pretty.
/Geographic: This option plots the data in geographic coordinates complete with coast lines.
/Magnetic: Theis option plots the data in AACGM magnetic coordinates. No coast lines are plotted.
/Post:This option will write the output to a postscript file. For mulit-pair plots the output filename is yymmdd'_polar.'extension where extension is defined by the IDL_PLOT_PATH environment variable. For single pair plots the outfput file is station_file'.'extension.
/Tty: Plots the output to the screen. Note that for technical reasons associated with preserving the correct aspect ratio, plots to the screens will have some anotation cut off.
/Bw: Plots vectors as solid black lines.
/Colour: Plots vectors in colour (red for west, blue for east).
/Vectors: Plots true vectors rather that flow lines. Works only for geographic. This is because the geomagnetic field is kinked and when plotted in magnetic co-ordinates gets distorted into circular. Hence vectors may point in spurious directions. If you really want vectors in magnetic coordinates then there is some commented out code in the program that will do this. Check it out very carefully before using it though.
/No_open: If this keyword is set then no devices will be opened when the program is run. This alows you to add a new figure to an existing plot.
/No_close: If this keyword is set then postscript device will not be closed when the program is run. You may thus add further figures to the page. No open and no close allow you to customise your plots.
/single_fame: If set then only one scan of line-of-sight data is overlayed in each frame
/no_quality: When this flag is set, the quality index is not displayed in the bottom right hand corner
/ls: If this argument is included then the SD_MERGE path is searched and all ".mrg" files are listed. The program then terminates regardless of other options.
Below is a list and a brief discription of the subroutines which are employed by PLOT_MERGE
FIND_FILE ('file_root','file_extension'): This function searches all directories in the SD_MERGE environment variable and returns an array of all files file_root?'.'file_extension complete with the full path name
e.g. files=FIND_FILE ('950810','_stnd.mrg')
where files is an N dimensional array.
FIND_DIRECTORY ('file'): This procedure searches all directories specified by the SD_MERGE environment variable for file and then sets that directory as the present working directory.
GET_COLOURS: This procedure simply sets up the colour table. The table is basically blue and red and is employed to generate red vectors for westward flow and blue vectors for eastward flow.
GET_PAIR_INFO: This routine reads the file getenv('SD_TABLES')+'/pair_plot_lims.dat and fills out the structure 'pair_info' with the following info.
name: a string giving the pair name
Geolim: a 2x2 array giving the latitude limits and longitude limits where (geolim(0:1,0) gives the latitude limits and (geolim(0:1, 1) gives the longitude limits
Maglim: a 2x2 array giving the lat/long limits in magnetic coords.
Misc: an array of dimension 3 giving - the long and lat where to print the L-shell value - the geomagnetic longitude which is in the middle of the common field of view
Common block: the pair information is placed in the common block 'pair_data'
Size :The number of defined pairs is currently 5 (it does not include the southern hemisphere pairs.
MLT, year, t, mlong: This funciton calculates the MLT from the magnetic longitude. The inputs to this function may be
Year - single value of array
t - single value or array (time in seconds since Jan 1st 00:00:00
Mlong, position for calculation single value only
This function returns a -1 if the year and time arrays do not have the same dimensions. Ensure the inputs are of the correct type set mlttime to the same as the input time array.
PLOTSET, 'postscript_file_name',columns,rows: This routine sets up the postscript plotting device to plot the appropriate number of frames in a page without messing up the aspect ratio
READ_MERGE, mrg_unit, nvects,yr,mo,dy,shr,smin,ssec,ehr,emin,esec, niter, map_qlty, height_r, data,ieof, startscan,endscan: This procedure reads the merged data files and returns the data and the start and end time or each record. If the merged files are not quite synchronised then it returns the upper and lower time limits.
GETTIMES, mrg_unit, nvects,yr,mo,dy,shr,smin,ssec,ehr,emin,esec, niter, map_qlty, height_r, data,ieof, startscan,endscan: This routine tests the data files to return the start and end times, integration time (to test that files are synchronised) and sample data (to test which hemisphere)
SET_PREFERENCES, 'file': This procedure saves the current arguments used in PLOT_MERGE to 'file'. The parameters are passed via the common block 'prefs'. 'File' will be located in $HOME/.merge_preferences. If the directory does not already exist then it will be created.
READ_PREFERENCES, 'file': This procedure reads the arguments from 'file' and passes them back to PLOT_MERGE via the common block 'prefs'. 'File' will be located in $HOME/.merge_preferences.
MAGCMP, year, rho, Glat, Glong, Bx, By, Bz, Bt, D, I, H
Calculates the magnetic field for a given geographic location using the IGRF model.
rho is the radial distance from centre of earth.
Glat,Glon Geographic latitudes and longitudes
Bx, By, Bz, Bt, H Geographic field components, the field magnitude and
the horizontal component.
I,D Inclination, declination.
This routine is used to convert geographic vector velocities into magnetic. It is
currently disabled.