How to process the IPIX radar files in MATLAB

home | Install MexCDF | Open File | Extract Variables | Load and Pre-process Data

The IPIX radar files are stored in Unidata's NetCDF file format (not to be confused with the NSSDC's CDF file format). This format is ideal for efficient storage of multidimensional array data. The data are easily imported into MATLAB using the freely available MexCDF toolbox. This tutorial describes the four step procedure to start working with the data in MATLAB.

1. MexCDF toolbox

Download the MexCDF toolbox, install it on your system and make sure it is in your MATLAB path. You do not need to understand the structure of netCDF files to work with the IPIX radar data. If you are interested anyway, type help netcdf. It is also very useful to study the netcdf utility ncdump.m. This command prints all the variables, dimensions and global attributes that are in a netCDF file, and also lists the commands used to produce that output.

2. Obtain a pointer to a netCDF data file

To put a pointer to the file '19931118_023604_stareC0000.cdf' into the variable nc, use: 
>> nc=netcdf('19931118_023604_stareC0000.cdf')
If successful, MATLAB will output: 
nc =
          NetCDF_File: '19931118_023604_stareC0000.cdf'
          nDimensions: 4
           nVariables: 22
    nGlobalAttributes: 13
      RecordDimension: ''
             nRecords: 0
           Permission: 'nowrite'
           DefineMode: 'data'
             FillMode: 'fill'
           MaxNameLen: 0
If you want to use a browser to locate a .cdf file for you, use: 
>> nc=netcdf('*.cdf')

3. Extract IPIX variables from a data file

Download the m-files ipixinfo.m and ipixazm.m.

To get an overview of all IPIX variables stored in the cdf-file that nc points to, use: 

>> ipixinfo(nc)
MATLAB will output: 
File: 19931118_023604_stareC0000.cdf

%% Variables:
            RF_frequency =       9.39 GHz
            Pulse_length =        200 nanoseconds
                     PRF =       1000 Hertz
        Unambig_velocity =     7.9872 metres per second
                   range = [2550 2565 ..nrange.. 2745] metres
           azimuth_angle = [170.2606 170.2606 ..nsweep.. 170.2606] degrees
         elevation_angle = [359.5605 359.5605 ..nsweep.. 359.5605] degrees
               radar_lat =      44.62 degrees
               radar_lon =      63.43 degrees
              radar_elev =         30 metres
       antenna_beamwidth =        0.9 degrees
            antenna_gain =       45.7 dB
             like_stc_RF =    0.99854 volts
            cross_stc_RF =    0.99854 volts
             like_stc_IF =    0.84961 volts
            cross_stc_IF =    0.84961 volts
              decimation =          1 none
              adc_like_I =          0 
              adc_like_Q =          1 
             adc_cross_I =          2 
             adc_cross_Q =          3 
                adc_data = [nsweep x ntxpol x nrange x nadc] ADC output

%% Dimensions:
                  nsweep =     131072
                  ntxpol =          2
                  nrange =         14
                    nadc =          4

%% Global attributes:
            Organization = McMaster University
         Instrument_name = IPIX X-band Polarimetric Coherent Radar
                Campaign = OHGR - Dartmouth
                    Site = Osborne Head, NS
    Data_collection_date = 1993/11/18 02:36:04
         TX_polarization =          A
        NetCDF_file_name = 19931118_023604_stareC0000.cdf
 NetCDF_creation_routine = make_Dartmouth_ncdf
    NetCDF_creation_date = Thu May 17 19:40:20 2001
           Field_CD_name =   ohgr0019
          Field_file_dir =   Nov1793a
         Field_file_name = stareC7535901640_0000
      Field_dataset_name = /data/Nov17/stareC2
Note that ipixinfo.m calls ipixazm.m to correct a bug in the azimuth_angle variable. The bug only appears in a few scanning mode data files.

To load the variables listed by ipixinfo.m into the MATLAB workspace, use the following MexCDF syntax: 

>> RF_frequency = nc{'RF_frequency'}(:);
>> Pulse_length = nc{'Pulse_length'}(:);
>> PRF = nc{'PRF'}(:);
etc. etc.

4. Loading the data into the MATLAB workspace

The actual IPIX radar data is stored in the variable adc_data. This array has three or four dimensions, depending on the number of transmit polarizations used, as indicated by the global attribute TX_polarization. This attribute can have the values 'H' (all pulses have horizontal polarization), 'V' (all pulses have vertical polarization) or 'A' (alternating).
The dimensions of the variable adc_data are:
  1. nsweep - the number of sweeps stored in the array. Each sweep contains two pulses if the transmit polarization is alternating, and one otherwise. The number of sweeps measured per second can thus be derived from the variable PRF, the Pulse Repetition Frequency.
  2. ntxpol- the number of different transmit polarizations used, as indicated by the global attribute TX_polarization. This dimension only exists if the transmit polarization alternates (TX_polarization = 'A').
  3. nrange - the number of range bins measured. The variable range contains the starting range of each range bin.
  4. nadc - the number of analog-to-digital-converted numbers stored in the array. This number is always 4, because the radar receiver measures, for each radar return, the horizontally and vertically polarized components. These two components are further divided in their in-phase and quadrature components.
Download the m-file ipixload.m.

The m-file ipixload.m pre-processes the data, and loads all sweeps of a particular polarization and rangebin into MATLAB. The syntax is: 

[I,Q,meanIQ,stdIQ,inbal] = ipixload(nc,pol,rangebin,mode)
The inputs are: The outputs are:

- Automatic Pre-processing -

The automatic preprocessing performed by ipixload.m consists of two steps:
  1. Remove mean and standard deviation from the I and Q channels separately.
  2. Remove the phase inbalance. A phase inbalance occurs when the I and Q channels are not orthogonal to each other, due to hardware imperfections. Let A and φ be the amplitude and phase of the received radar echo, and let β be the phase inbalance in radians, equal to inbal*π/180. Ideally, the I and Q channels measure I=A cos(φ) and Q=A sin(φ). With a phase inbalance of β radians, the I and Q channels are I=A cos(X) and Q=A sin(X+β). Given a long data record and assuming that the phase is uniformly distributed between 0 and 2π, the inbalance β is estimated by sin(β)=2(I,Q)/A2, where (I,Q) is the inner product of I and Q. The phase inbalance is removed by rotating the I component: Irot=[I-Q*sin(β)]/sqrt(1-sin(β)2).
    Derivation: E[ A cos(φ) A sin(φ+β) ] = A2 E[ 1/2 sin(β) + 1/2 sin(2φ+β) ] = A2/2 sin(β), since the expectation of sin(2 φ+β) is zero for uniformly distributed φ.

- Files containing land clutter -

The phase inbalance removal procedure assumes that the phase is uniformly distributed between 0 and 2π. This assumption is true if the distance between the radar antenna and the major scatterers is randomly varying with time, with a variance much larger than the radar wavelength of 3 cm. For ocean wave scatterers this certainly holds, but for land scatterers, the phase may be nearly constant. Therefore, to apply the phase-inbalance removal to scanning radar files which contain land clutter, the phase inbalance is estimated using only that part of the data that is known not to contain land. When calling ipixload.m with the mode switch set to 'dartmouth', azimuth/range combinations known to contain land for the Dartmouth geography are excluded.