Map_potential is a tool developed by JHU/APL to derive large-scale global convection maps from line-of-sight velocities observed by the SuperDARN radars. The model has been explained by Ruohoniemi and Baker (1998). A brief description of the model is presented below.
The SuperDARN radars are arranged such that observations in common-volume area are bidirectional and the 2D ExB velocity can be resolved unambigiously. By merging measurements from all the radars in the Northern or Southern hemisphere, direct mapping of the convection can be extended to almost 12 hours of magnetic local time. However, although the two-dimensional velocity at a point can only be unambigiously resolved if two or more radars make simultaneous measurements, all the available line-of-sight velocity data serves to constrain the possibilities for the large-scale convection velocity. The convection pattern that is most consistent with the measuremets can be determined by mathematical fitting. This is the basis of the map_potential model, described by Ruohoniemi and Baker (1998) as "..a method of filtering, combining and reducing data from the SuperDARN radars which optimizes the mapping of large- scale convection pattern."
The map-potential technique can be used to image flows in the Northern and Southern hemispheres. Identical steps should be followed to image flows in either hemisphere. You do not need to specify which hemisphere you are imaging, but never combing data from Northern and Southern hemipshere radars in the same file.
To run the map_potential model all you need to do is to initialize the .rad_env and .radlogin files.
In the Leicester group this is done by going to the /people/cutlass/new_pot_map directory and simply typing :
. map_setup
The model requires that make_gridmake_grid , combine_grid and trim_grid are run from the directory which contains your SD fit data files. The map_potential model is run from the same directory which contains the radvec file.
N.B. - New Shortcut There is now a unix script called make_radvec whic will grid and combine files for all given radars given the starting time of the files. To use, simply go into area where the fit files are kept and type :
make_radvec -c (if files are concatenated) yyyymmddhh
But be careful, this method makes it more difficult to identify problems with any individual radars. More importantly, if you have data from both Northern and Southern hemisphere radars for the same time interval in the same directory, make_radvec will attempt to combine it all into one file - creating a big mess.
Below is a example of how to use the technique to image flows in the Northern hemisphere on the day 1997/12/01 from 20 to 22 UT.
The first step is to grid the data into a global grid, while spatially and temporally averaging the data. Be warned that the map-potential technique was initially written for normal resolution data. It is possible to grid high-time resolution data (see below), but care should be taken, especially if using concatenated files that might be too large to handle.
Grid files are created using make_grid in the following way :
make_grid filename.fit > filename.grid
e.g. For Finland and Iceland East for 1 Dec 1997, 2000 UT.
make_grid 97120120f.fit > 971201f.grid
make_grid 97120120e.fit > 971201e.grid
The grid file does not need to have this filename.
Normally this would be done for all the available Northern or Southern hemisphere data, remembering that make_grid can be used on concatenated data files.
There are the following options that can be used with make_grid
-st hr:mn set start time
-et hr:mn set end time
-ex hr_mn set extent of time
-i sec set number of seconds (sec) in sampling interval
-tl sec ignoring scan flag (if problem with special modes) and set pseudo scantime (sec)
-vb verbose mode (details data being processed)
-e max maximum vel. error aaccepted (default 200 m/s)
-a abs amsolute minimum magnitude of vel accepted (default 35 m/s)
-r disable temporal averaging
-t disable spatial averaging (presently disabled!)
-rv1 range1 range2 vel1 vel2 - apply range velocity filter (range2 > range1, vel2 > vel1> to remove data
-rv2 apply range/velocity filter (as above)
-rv3 apply range/velocity filter (as above)
Some examples using make_grid
High-time resolution data :
To change the scan period used type make_grid -i (sec), where (sec) is
the number of seconds you want per scan.
Some SuperDARN non-common time modes do not always use the scan flag correctly. If this is a problem, you need to type make_grid -tl (sec) What this does is to ignore the scan flag and instead look at the time stamps for individual records in a fit file to decide which time period they lie in. So to produce a grid file contaning records for each minute of the day use : make_grid -tl 60 -i 60
Beware the make_grid -tl command should not be used on normal resolution data. If this is done the resulting grid file would contain records from alternate parts of the array.
Temporal averaging
When the data is gridded using make_grid, it is both spatially ane temporally
averaged. To switch of the temporal averaging type : make_grid -r
To create 4 min sampling file for a 12 min period on 8th May at Halley, and see the details of the processing
make_grid -st 14:50 -et 15:02 -vb 98050814h.fit > 98050814h.grid
combine_grid filename1.grid filename2.grid filename3.grid .... > radvec.dat
E.g. combine_grid 971201f.grid 971201k.grid ..... > radvec.dat
Any number of grid files can be combine to produce the masterm file. Generally, one grif file for
each radar. Grid files from radars in opposite hemisphere should not be combined. The radvec.dat
filename is flexible.
There are the following options that can be used with combine_grid
-vb verbose mode (details data being processed)
-r replacement mode, overwrites data in master file.
Master files can ne altered using trim_grid in the following way.
trim_grid options radvec.dat > new_radvec.dat
If no options are choosen then the output file is equal to the input file
There are the options that can be used with trim_grid
-st hr:mn set start time
-et hr:mn set end time
-ex hr:mn set extent of time
-vb verbose mode (details data being processed)
-# excludes data from radar with identifier letter #
Example If instead of studying the day 1997/12/01 from 20 to 22 UT, you wanted to study it from 20 to 21 UT and to exclude Goose Bay data.
trim_grid -st 20:00 -et 21:00 -g radvec.dat > radvec.dat
4) Altering master files using trim_grid
5)
When the master file is ready, the potential maps can be created using the map widget. Type map_start and the
map_potential widget should appear.
This is the map_potential widget which is used when deriving
and studying the two-dimensional velocity patterns. The widget reads data from the master file, combines it
with the APL model average convection pattern that mathces the IMF conditions and creates an estiamte of the
potential pattern for the interval concerned. This potential pattern can be plotted and overplotted with
different forms of the velocity vectors in the radar fields-of-view. I have attempted to annotate the widget so that it is fairly
easy to understand, the points are explained in more detail below.
Input file : This is where the name of the master file is entered. After entering the filename remember to
press return, or the software will not register the change (this is the case for all the entry fields on the widget).
Output file: This is the name of the output file. The output file contains all the information in the plot.
It will only be output if return is pressed in the entry field.
Latitude limit: This is the estimated low latitude limit of the convection. If it is constant then the
value can be entered into the fixed entry field. If the value varies across the interval, then the values must be
placed into a file, the filename of which is entered into the filename entry field. Each line of the file contains
a time and a latitude and takes the format hr mn lat, eg 20 00 68 (68 degress at 20:00 UT). A good estimate of the
latitude limit is generally the equatorward limit of he radar backscatter. A wrong positioning of the latitude limit
can lead to very unrealistic potential patterns. Getting it right can sometimes be a matter of trial and error.
Shift pole This is the shift in degrees of the centre of the potential pattern away from the magnetic pole
in the direction of 0 MLT. Values are entered in the same way as for the latitude limit. A nonzero pole shift will affect
the low-latitude convection limit away from magentic local noon. For example, if you have a low-latitude limit
of 65 degrees and a pole shift of 4 degrees, the model will consider scatter down to 65 degrees at noon and 57
(65 -4x2) degrees at midnight.
IMF data Here you can choose whether to use or not to use measured IMF conditions to drive the background model.
If IMF data is selected, there is a choice between the ACE and WIND spacecrafts. The software
will automatically look for the corresponding CDF file containing the IMF data in the data archive. If it does not exist
in this archive the filename can be entered by hand (the CDF file for a particular interval of data can be downloaded
from CDAWEB). The slider sets the time delay (in minutes) between the IMF observations at the satellite and when it
impinges on the subsolar magnetopause. The scale corresponds to the size of the vector in IMF annotation.
Model : There are three options here (Fixed, From IMF, None)
Fixed - The same APL potential model is used for the whole interval. This requires the selection of the model
(IMF clock angle sector & field magnitude) from the pulldown menus.
From IMF - This selects the APL potential model by looking at the (shifted) IMF data for each interval.
None - No APL potential model is used.
Model Weighting : This can either be flat or modified.
Flat - Data points from the statistical model are assigned a fixed weight equal to the average weight of the
data points. At high order of fit, where more data pointsare ised from the statistical mode, this leads to an
increased dependence on the model.
Modified - The statistical model are de-weighted with increase in the fitting order so that the contribution
to the fitting from the model is roughly constant for oders above 4.
Order This represents the potential of the spherical harmonic expansion that will be fitted to the data.
Relatively low order patterns (~4) resolve the global scale of the patterns. Higher orders are often needed to
resolve mesoscale structure in the data (see Ruohoniemi and Baker, 1996)
Output Device There are three choices here, X-windows,Postscript (B&W),Postscript (colour), Output files
are idl.ps
Plot Limits : These represent the maximum and minimum latitudes and longitudes for the plots (can be
absolute values, the master file signifies the hemisphere). Default values are automatically entered. The latitude
limit will automatically go from below the low-latitude convection limit to the pole, whereas the longitude limit
will go from -180 to 180.
Plot Options : This details which data is presented on each plot
Contour - plots contour of constant electric potential.
Fit Vec - plot velocity vectors fitted to the measured and modelled velocity information.
Model vec - plot velocity vectors from the statistical model.
Residual - plot the filtered, gridded and combined line-of-sight vectors.
True Vec - plot the vectors obtained by combining the measured line-of-soght velocity measurement with the
transverse component of the fitted line-of-sight velocity vectors.
Merge Vec - plot merged velocity vectors calculated from overlapping filtered and gridded line-of-sight
measurements from different radars.
Operational buttons : To finally perform any operations
Preview - plots a quick preview of all the line-of-sight data.
Execute - produces the potential maps as requested.
Quit - exits from the map widget.
The velocity analysis begins with a sequence of scan data from the HF radars.
Velocity with error estimates greater than 200 m/s are ignored, as is groundscatter.
Using the make_grid program, data from each radar is processed for a selected
time interval and integration period, The data is first filtered using a boxcar filter involving both spatial and
temporal filtering. The filter is descibed in some detail in Ruohoniemi and Baker (1998). Basically
for a scan collected at time t(k), the filtering samples includes the scans performed at t(k-1) and t(k+1),
where the subscript indicates the scan order. The spatial sampling is performed over a 3x3 beam/gate filter centered on the cell
of interest. After the data has been filtered it is written to a holding file
The next step is to combine the data from the individual radar holding files
into a single file. This is done using the combine_grid program.
A global grid is defined for spatial averaging and a time step for
temporal averaging period. The grid is defined by the spatial scale of 1 degree
of latitude (or abour 111 km projected to the surface of the Earth). For each
integral interval of 1 degree in co-latitude, the number of grid cells distributed
in longitude is set by the requirement that the step in longitude projected onto the
Earth's surface be as close to 111 km as possible. Ruohonimie and Baker (1998) stated
that "it is preferable to average the radar velocity radar than work directly with
the individual velocity values because the latter would lead to oversampling of the
near-radar regions relative to the far-radar regions".
When combining data an average velocity and its uncertainly were obtained for a
given grid cell for a given time if at least 25% of the radar sampling within that
cell returned a velocity value.
Once all the above steps have been done we have produced what are refered to by
Ruohoniemi and Baker as average line-of-sight velocity measurements. These
velocities will be used to derieve two-dimensional convection patterns.
Imaging large-scale convection
Merging common-volume pairs of line-of-sight radar velocity measurements is the
standard method of producing two-dimensional velocity measurements. However, all the
available line-of-sight velocity data serve to constrain the possiblity for the large
scale convection pattern. The pattern that is most consistent with all the average
line-of-sight velocity measurements can be determined by mathematical fitting
procedures. The electrostatoc potential is expanded in terms of spherical harmonic
functions according to e.g. Jackson(1962). The order and degree of the expansion
determine the spatial filtering performed on the velocity data. The global-scale
character of the pattern is resolved by expansion of relatively low order and degree.
Often both the order and degree of the expansion are equal to four, the spatial
scales associated with this fitting are 6 deg. in latitude and 45 deg. in longitude.
The fitting procedure can be applied directly to the preprocessed data points.
In principle it should then be possible to calculate the electrostatic potential
over the entire global grid. However this amounts to extrapolation of the fitting
solution obtained over the limited area covered by the measurements to the uncovered
portions of the high-latitude zone, and these results may be unphysical over the
areas that are distant from the input velocity data.
In order to obtain a global picture of the convection, the map-potential model
fold in velocity information from the statistical model of Ruohoniemi and Baker
(1996). Briefly the statistical model is sampled at the least number of grid
positions that is required to bound each term in the spherical harmonic expansion
of the electrostatic potential. For a given expansion order a set of sampling
points can be determined. If fewer points are taken, there is instability in the
model related to the effect of unconstrained extrapolation. If more points are
taken the global solution owes more of its character to the input from the statistical
model than is necessary to achieve stabilisation.
The map_potential widget
The map_potential model
For more detailed information on the map_potential model, read Ruohoniemi and Baker, 1998.