AS ONE CAN SEE, this document is under construction (pxe 22/01/97 17:15:30 BST last update!)

System specific code alert : the path name of indexscript in rawful.pro. Furthermore, the package was written in an sgi environment for our machines in Leicester. It may be necessary to change the font and add a few spaces here and there for it to look pretty on your machine with all boxes lined up etc.


Rawful : a widget for looking at raw data


  • Realisation
    What the rawful widget is all about
  • Assembly
    How to invoke the application
  • Widget Information
    A glossary of the function of all the buttons and boxes
  • Fish
    How to throw the widget at some data and make it swim
  • Utilisation
    Why look at the raw data anyway when there is all that lovely fit data filling up one's disk.
  • Local Excitement

  • Rawful concept

    Accompanying the wonderful plot_radar software originally crafted by the APL gurus were a number of pieces of idl code for the investigation of raw superDARN radar data. The rawful widget was written as a simple tool which bolts together these various routines to enable a quick look at the raw data to be achieved. The widget does nothing more complicated than existed already in the APL code. The use of repeated calls to routines allows the construction of seeming waterfall plots of HF radar scatter spectra, but such processes are slow since they rely on the exisiting idl interfaces to simple C routines. In future more sophisticated routines may be cojoined with the widget to manufacture such output more rapidly. However, for now, the widgets best application is for quickly finding a time in a file and scanning through the acfs and spectra for the ranges and times in question, producing post script output if requested. Suggestions for improvement and complaints should be addressed to pxe@ion.le.ac.uk. TOP


    Assembly

    Much of the following material contains specific references to the local file store here at Leicester, however the principles in running the widget should be the same if one is elsewhere.

    There has neen no specific startup file constructed for rawful. One needs to have run the standard cutlass setup before entering idl in order that all the low level idl routines are compiled and a number of paths are set. N.B. to run Cutlass setup file: . /people/cutlass/.cutlass_setup

    Many users run this automatically already.

    N.B. For people perhaps running rawful not at Leicester, by the above I refer to the routines in radar.pro rawlib.pro acflib.pro etc which make up the set of routines which comprise plot_radar.

    Raw data files comply with the standard name convention e.g. "95081022f.dat" with the suffix dat. For optimum operation of the rawful widget a raw data index file should also exist, these have names of the form "95081022f.rin". In contrast to fit files where index files (inx) are routinely generated and stored with the data, no raw index files are produced. The raw index file must be produced by the user on an 'as needed' basis. These can be created by the program:

    /people/cutlass/programs/create_raw_index

    A drawback with this system is that the index file will be created where one runs "create_raw_index" and the index file must exist in the same directory where the raw data file resides. It is left to the user to decide how they wish to organise their own file store to cope with this requiremnt. A button on the widget will also create a raw index file once a raw data file has been selected. For this to work, one should run the widget in the same directory as where the raw data file exists, e.g. "/cutlass/data/userid/dat" is a place that everything should work happily. One should also take care that the path SD_RAWROPEN_PATH

    points to where both the "dat" and "rin" file exist. E.g. for user agnes the standard cutlass setup enforces

    SD_RAWROPEN_PATH=/people/cutlass/Data/:/cutlass/finland/incoming/: /cutlass/data/agnes/dat/:.

    A danger with this is that if the raw data that one wishes to examine exists in "/cutlass/finland/incoming/" one would need to make a linked copy to the data in "/cutlass/data/agnes/dat/". i.e. ln -s original.dat linked_copy.dat

    It would then be possible to create a raw index file in "/cutlass/data/agnes/dat/" where agnes has write permission (as opposed to the cutlass directory where the data file exists). However, when the widget is run it will open the file in "/cutlass/finland/incoming" and look for th index file in the same directory and fail to find it. In order to remedy this, one must set the SD_RAWROPEN_PATH to the specific directory where both the index file and the linked copy of the data exist or change the order of the directories in the path so that "/cutlass/data/agnes/dat/" is searched before "/cutlass/finland/incoming". e.g. in the shell where one is to run idl type: SD_RAWROPEN_PATH=.:/cutlass/data/agnes/dat/:/cutlass/finland/incoming/ export SD_RAWROPEN_PATH etcetera

    Once one is happy with all that then enter idl and type the following .run /people/cutlass/idl/rawful/rawful.pro

    This compiles all the routines necessary for the widget. On entry to idl all the cutlass idl code for plot_radar will have been compiled. This includes the file "acflib.pro" which contains the basic routines written by those APL gurus. The colours in this do not appear very well on the Leicester workstations and printers, hence it is also advisable to type

    .run /people/cutlass/idl/rawful/acflib.pro

    which makes everything black and white, which is ideal for the widget.

    Finally type..

    rawful

    and the widget will launch itself. That explains all the nitty gritty and the user can pop all that stuff in a file for speed of use. Some startup code has deliberately not been produced so that the user is forced to implement this software in a manner that best suits their needs, but please ask for help (i.e. pxe@ion.le.ac.uk) if this description has just led to confusion. TOP


    Widget Information

    Here are a couple of pictures one of how the widget looks, too small to see details, and the other a real scale picture of the control panel, main button facet.

    The pictures look pretty awful at the moment, might be improved later, but they give an idication of where everything is to aid this documentation. Looking at the full widget there is the button facet in the top left had corner and underneath there are three text boxes (let them be called top middle and bottom). The (possibly black) area to the right is the draw window where all the acfs and spectra will appear. Attacking the button facet will make things happen in the draw window, the text boxes give warning messages, status information, errors, some extra legends and also some of the raw data parameter block values, e.g transmitter frequency and radar error codes.

    Here is a description of the buttons, jump to the next section if you want to just rummage in some data now. The four horizontally extended boxes in the middle are compound widget including fields into which one can type in values and delete old ones once one has clicked in them. Before these vlues are registered, one must also enter a carriage return. The boxes above and below are either simple buttons or buttons that will pull down menus.

  • RAW DATA On entry to the widget, this field will contain the expression 'please can I have a file'. One approach is to select this expression with the mouse and type in a file name, e.g. 95081022f, and hit return. Files which do not exist in any of the locations specified bt SD_RAWROPEN_PATH will cause a complaint and the user will be able to type in a new file.
  • TIME
    Enter times in the format YYYY MM DD HH MM SS. The widget will update this field when it is given a valid filename in the raw data box. The user can change the HH MM and SS entries for the rapid search to a new time.
  • BEAM
    This field will contain the current beam of the raw data structure loaded into the widget. If one changes the beam, the widget will read through consecutive records until it finds that beam. This is very slow and it is much quicker to navigate to the same beam (say 90s later) by using the time field. This situation may be improved in the future.
  • RANGE
    A range bin (0 to 75ish) of the given beam. This is of relevance if one wishes to look at one ACF and of no consequence if one is plotting the Lag0 powers with range, etc..
  • Make Index
    Please ensure that a file has been loaded before one presses this button. The button will cause a shell script to be run that will create a raw index file (.rin in the current directory). When creating the index file, the widget will close and reopen the raw data file, this is necessary since the low level routines want to open the data file and its index file at the same time. If one watches the window where idl is being run, a Warning message will occur if the Widget fails to find the index file when reopening the raw data file again.
    N.B. If one is installing rawful somewhere else, please make sure that the script "indexscript" has been given execute permission
  • ACF
    Plots the ACF for the current BEAM and RANGE.
  • SPECTRUM
    Plots the SPECTRUM for the current BEAM and RANGE.
  • OTHER
    A menu with the following options
  • ACF power & phase
  • LAG0 powers
  • Four spectra
  • Range waterfall
    One of my favourite buttons.
  • Time waterfall
    Another box will pop up when this happening, trying to dissuade one from this course of action and asking how many time dumps at the given beam. A proper program needs to be written to do this quickly...this is just a quick fix that takes forever. This routine will also save a file to the current directory when run..just in case one wants a plot of the result, this will fail if one has no write permission.
  • Test button (a relic of development...which will go at a future revision)
  • Bad lags
    This plots three panels. The outer panels are plots of range bin against lag quality....that is is it a bad lag or not. Panel 1 just plots bad lags as given by the routine ckrng, which tests for lags due to sampling during pulse transmission and interference from other ranges where there is power. The middle panel is a range waterfall to illustrate where the power is observed. The last panel plots the additional bad lags that arise due to noisy and general wonkiness in the acf. (for more erudition please see the code and/or talk to an expert!)
  • >
    Step one beam forward in time
  • <
    Go backwards in time (actually skips back 2 at the moment...this is a software bug that I am looking at).
  • Run
    Oh my word. This is a menu which allows one to run through whatever type of plot that is selected in either time or range and toggle between the two. The next button (PAUSE) will suspend the operation. Slight delays are inherent in these buttons but they do work....but perhaps not quite how one would expect...try them out and enjoy.
  • Pause
    Pauses and unpauses run operation, this button is a toggle monster.
  • Tools
    A relic from the widget template...cheers all those folk at idl for making this all possible. It is another menu containing Xpalette, XManagerTool and an idl command thing stolen from plot_radar.
  • PS
    Produces a postscript file of whatever is in the draw window, of the name #idl.ps. # can vary from 1 to 999. During each run of the widget a new postscript file is created everytime PS is pressed # will increment in single integer steps.
  • Done
    Closes down the widget
  • TOP

  • Fish

    Here is one example. Agnes wishes to look at the raw data in the file 95101212f, which already exists in /cutlass/finland/incoming, here is what Agnes did, while wearing a delightful sage green hat with an orange bouquet and bow:

    N.B. '$' refers to a shell prompt and 'IDL>' to a prompt in idl. $cd /cutlass/data/agnes/dat $ln -s /cutlass/finland/incoming/95101212f.dat 95101212f.dat $. /people/cutlass/.cutlass_setup $export SD_RAWROPEN_PATH=. <$idl

    IDL>.run /people/cutlass/idl/rawful/rawful IDL>.run /people/cutlass/idl/rawful/acflib IDL>rawful

    And into the widget she was flung. Agnes put the filename '95101212f' into the raw data box and hit return. Next she clicked 'Make Index'; an index file was created and succesfully found by the widget. Agnes adjusted her hat and started clicking at random.......

    If the file is not already on disc, one can just extract it using all the other software as described in local unix commands in the Cutlass documenation.

    TOP


    Utilisation

    To be completed TOP


    Local Excitement

    An experimental version has been installed. This version is called rawful_lemon. It is capable of fitting to the raw data as you browse it with the usual rawful features. Since this required hacking the panoply of fitacf routines, the code is extremely non-standard and is available only to Leicester people. Rawful_lemon has its own special interface to fairly recent fitacf code. For these reasons it remains a local experimental feature. Rawful_lemon can be invoked by typing "rawful_lemon" wherever one would have typed just "rawful" before.


    TOP

    SuperDARN documentation