suncasa.eovsa.eovsa_pipeline copy
Attributes
Classes
Functions
Module Contents
- tasks
- split
- tclean
- gencal
- clearcal
- applycal
- gaincal
- delmod
- tools
- qatool
- iatool
- mstool
- tbtool
- ms
- tb
- get_tdate_from_basename(vis)
- hostname
- is_on_server
- is_on_inti
- base_dir = '/inti/data/pipeline_mirror'
- class Path_config(base_dir=base_dir)
- paths
- base_dir = '/inti/data/pipeline_mirror'
- udbmsdir
- udbmsscldir
- udbmsslfcaleddir
- udbdir
- caltbdir
- slfcaltbdir
- qlookfitsdir
- qlookfigdir
- synopticfigdir
- workdir_default
- _get_env_var(env_var, default_path)
- _print_summary()
- pathconfig
- udbmsdir
- udbmsscldir
- udbmsslfcaleddir
- udbdir
- caltbdir
- slfcaltbdir
- qlookfitsdir
- qlookfigdir
- synopticfigdir
- workdir_default
- get_local_day_bounds(tim)
Return the local-day bounds used by the daily EOVSA pipeline.
- get_synoptic_output_info(tim)
Return the expected synoptic daily FITS products and status file for one day.
- summarize_synoptic_outputs(tim)
Summarize synoptic daily FITS availability for one pipeline day.
- read_pipeline_status(statusfile)
Read the per-day pipeline status JSON if it exists.
- write_pipeline_status(statusfile, state, **extra)
Persist a compact per-day pipeline status JSON.
- get_calibration_deadline_utc(tim, hour=4, day_offset=2)
Return the UTC hard deadline for one observing day label.
- get_calibration_readiness(tim)
Check whether the observer-written daily calibrations are ready for one day.
- should_enable_smart_cal_check(enable_flag=None)
Resolve the cron-oriented calibration gate behavior.
- getspwfreq(vis)
- Parameters:
vis
- Returns:
mid frequencies in GHz of each spw in the vis
- trange2ms(trange=None, doimport=False, verbose=False, doscaling=False, overwrite=True)
This finds all solar UDBms files within a timerange; If the UDBms file does not exist in EOVSAUDBMSSCL, create one by calling importeovsa Required inputs: trange - can be 1) a single string or Time() object in UTC: use the entire day, e.g., ‘2017-08-01’ or Time(‘2017-08-01’)
if just a date, find all scans withing the same date in local time. if a complete time stamp, find the local date first (which may be different from that provided,
and return all scans within that day
a range of Time(), e.g., Time([‘2017-08-01 00:00’,’2017-08-01 23:00’])
None – use current date Time.now()
- doimport - Boolean. If true, call importeovsa to import UDB files that are missing from
those found in the directory specified in EOVSAUDBMSSCL. Otherwise, return a list of ms files it has found.
doscaling - Boolean. If true, scale cross-correlation amplitudes by using auto-correlations verbose - Boolean. If true, return more information
- calib_pipeline(trange, workdir=None, doimport=False, overwrite=False, clearcache=False, verbose=False, pols='XX', version='v3.0', ncpu='auto', caltype=['refpha', 'phacal'], interp='nearest', force_imaging_rerun=False)
- trange: can be 1) a single Time() object: use the entire day
a range of Time(), e.g., Time([‘2017-08-01 00:00’,’2017-08-01 23:00’])
a single or a list of UDBms file(s)
None – use current date Time.now()
- mk_qlook_image(trange, doimport=False, docalib=False, ncpu=10, twidth=12, stokes=None, antenna='0~12', lowcutoff_freq=3.7, imagedir=None, spws=['1~5', '6~10', '11~15', '16~25'], toTb=True, overwrite=True, doslfcal=False, verbose=False)
- trange: can be 1) a single Time() object: use the entire day
a range of Time(), e.g., Time([‘2017-08-01 00:00’,’2017-08-01 23:00’])
a single or a list of UDBms file(s)
None – use current date Time.now()
- plt_qlook_image(imres, figdir=None, verbose=True, synoptic=False)
- qlook_image_pipeline(date, twidth=10, ncpu=15, doimport=False, docalib=False, synoptic=False, overwrite=True)
date: date string or Time object. e.g., ‘2017-07-15’ or Time(‘2017-07-15’)
- pipeline(year=None, month=None, day=None, ndays=1, clearcache=True, overwrite=False, doimport=True, pols='XX', version='v1.0', ncpu='auto', debugging=False, caltype=['refpha', 'phacal'], interp='nearest', smart_cal_check=None)
Main pipeline for importing and calibrating EOVSA visibility data.
- Name:
eovsa_pipeline — main pipeline for importing and calibrating EOVSA visibility data.
- Synopsis:
eovsa_pipeline.py [options]… [DATE_IN_YY_MM_DD]
- Description:
Import and calibrate EOVSA visibility data of the date specified by DATE_IN_YY_MM_DD (or from ndays before the DATE_IN_YY_MM_DD if option –ndays/-n is provided). If DATE_IN_YY_MM_DD is omitted, it will be set to 2 days before now by default. There are no mandatory arguments in this command.
- Parameters:
year (int, optional) – The year for which data should be processed, defaults to None.
month (int, optional) – The month for which data should be processed, defaults to None.
day (int, optional) – The day for which data should be processed, defaults to None.
ndays (int, optional) – Number of days before the specified date to include in the processing, defaults to 1.
clearcache (bool, optional) – Whether to clear cache after processing, defaults to True.
overwrite (bool, optional) – Whether to overwrite existing files, defaults to True.
doimport (bool, optional) – Whether to perform the import step, defaults to True.
pols (str, optional) – Polarizations to process, can be ‘XX’ or ‘XXYY’, defaults to ‘XX’.
version (str, optional) – Version of the pipeline to use, choices are ‘v1.0’ or ‘v2.0’, defaults to ‘v1.0’.
ncpu (str, optional) – Number of CPUs to use for processing, defaults to ‘auto’.
debugging (bool, optional) – Whether to run the pipeline in debugging mode, defaults to False.
caltype (list, optional) – Calibration types to use, defaults to [‘refpha’,’phacal’].
interp (str, optional) – Interpolation method to use for calibration tables, defaults to ‘nearest’. Options are ‘nearest’, ‘linear’
smart_cal_check (bool, optional) – When True, perform cron-oriented calibration readiness checks and track per-day status before processing, defaults to
None. IfNone, the behavior is enabled automatically whenEOVSA_PIPELINE_CRON=1. The gate waits for same observing-day calibration records until the hard deadline at04:00 UTCtwo days after the observing-day label, then allows the run to proceed with the latest calibration records available in MySQL.
- Raises:
ValueError – Raises an exception if the date parameters are out of the valid Gregorian calendar range.
Example:
To process data for November 24th, 2021 using version 2.0 of the pipeline, with all options enabled:
>>> python eovsa_pipeline.py --date 2021-11-24T20:00 --clearcache --overwrite --doimport --pols XX --version v2.0 --ndays 2
If you want to see the help message, you can run:
>>> python eovsa_pipeline.py -h
- parser