wiki:SSBDEV and SSBX Status

Status of pipeline deployment

The dates listed indicate the date on which a given version of the code became available via the automatic SSB distribution procedures.

SSBDEV

  • 30-May-2015 r3399
    • Added config file for superbias step
  • 29-May-2015 r3396, r3395, r3393
    • Renamed Build 4 version of calwebb_sloper to calwebb_sloper_b4
    • Updated the generic calwebb_sloper to use the new Build 5 order of ramps-to-slopes steps, including the superbias subtraction step
    • Added superbias cal_step indicator
  • 23-May-2015 r3386
    • Fixed an import issue with the referencefile schema in all the reference file data models
  • 16-May-2015 r3372
    • Added data model for pixel area map
  • 15-May-2015 r3370, r3369, r3368, r3367, r3366
    • Added collect_pipeline_cfgs_b4 script to use for retrieving the Build 4 specific cfg files
    • Added pipeline_b4 package to accommodate Build 4 specific pipeline cfg files
    • Photom reference file data model updated to include uncertainty array for Build 5, while preserving the existing version as Build 4
    • Added separate Build 4 specific version of the photom step that uses the old version of the photom data model
    • Updated the default version of the photom step to use the new Build 5 version of the photom data model
  • 14-May-2015 r3364
    • Updated the linearity step to not apply any correction to pixels flagged as NO_LIN_CORR in the linearity reference file
  • 12-May-2015 r3363
    • Updated the saturation step to not apply any correction to pixels flagged as NO_SAT_CHECK in the saturation reference file
  • 07-May-2015 r3362
    • Updated the flat-field step so that no correction is applied to pixels flagged as NO_FLAT_FIELD in the flat-field reference file
  • 06-May-2015 r3360
    • Updated the master DQ flag bit assignment list to match the Build 5 definitions from the PWG
  • 01-May-2015 r3358
    • Added the new superbias subtraction step
  • 25-Apr-2015 r3355, r3354
    • Updated wfs_combine to the use the WCS object values included in the data model and set the S_WFSCOM cal_step indicator upon completion
    • Updated the ramp_fit step to compute slopes for pixels that saturate in group 2, by using the data from group 1 only
  • 14-Apr-2015 r3345
    • Updated the ramp_fit step to compute slopes for integrations with NGROUPS=1, by using the data from group 1 only
  • 11-Apr-2015 r3342
    • Updated the jump step to skip processing of data sets with NGROUPS=1
  • 10-Apr-2015 r3339, r3338, r3336, r3333, r3331
    • Updated the data models to implicitly create zero-filled ERR and DQ arrays (for models containing them) when they're missing from an input model
  • 08-Apr-2015 r3333
    • Updated data models to not create extensions in output files for model arrays that are not initialized
  • 07-Apr-2015 r3329
    • Updated the linearity step to use benign coefficient values in place of NaN's in the reference data
  • 04-Apr-2015 r3325
    • Updated the dark subtraction step to use benign values in place of NaN's in the reference data
  • 02-Apr-2015 r3322
    • Increased the size of the wavelength and response arrays in the NIRISS photom data model
  • 26-Mar-2015 r3317, r3314
    • Added a data model for the GLS ramp_fit optional output product
    • Removed obsolete MIRI MRS grating keywords from the core schema; added new step status keywords to the core schema
  • 25-Mar-2015 r3313, r3312, r3311
    • Increased the size of the wavelength and response arrays in the MIRI imager photom data model
    • Added NRS_MSASPEC as a possible EXP_TYPE value in assign_wcs
    • Updated bias_drift step to use detector name 'NIS' for NIRISS
  • 24-Mar-2015 r3309, r3308
    • Added list of allowed detector names to core schema
    • Updated data models to automatically convert the NIRISS detector value from 'NIRISS' to 'NIS' in input data models
  • 19-Mar-2015 r3302
    • Updated the jump and ramp_fit steps to handle both full-frame and subarray forms of readnoise reference files
  • 13-Mar-2015 r3293, r3292
    • Bug fixes to the IPC step
  • 10-Mar-2015 r3277, r3275
    • Updates to the IPC step to handle both 2-D and 4-D kernels
    • Updated the IPC kernel data model to remove 2-D restriction on array dimensions
  • 28-Feb-2015 r3253
    • Updated the calniriss_ami3 pipeline to allow it to work on a single input image, as well as an association
  • 27-Feb-2015 r3250
    • Fixed definition of reference file types in assign_wcs step so that user-specified overrides will work again
  • 26-Feb-2015 r3247
    • Temporarily removed distortion file from list of NRC_IMAGE reference file types in assign_wcs, until we get some NIRCam distortion files in CRDS
  • 18-Feb-2015 r3229, r3228, r3227
    • Added default configuration (.cfg) files for the coronagraphic klip and hlsp steps
    • Added initial method to coronagraphic hlsp step for computing contrast curve tables
    • Added coronagraphic contrast data model to jwst_lib.models
  • 17-Feb-2015 r3224, r3223
    • Added the calniriss_ami3 pipeline module and all associated cfg files used by the pipeline
    • Added initial versions of the ami_average and ami_normalize steps
  • 14-Feb-2015 r3222, r3221, r3220
    • Fixed a upper/lower case problem in string comparison in the ami_analyze step
    • Added the coron package to the list of installed jwst packages
    • Added initial versions of the stack_refs and hlsp steps in the coron package, as well as updates to the klip step to compute estimated ERR array
  • 13-Feb-2015 r3218, r3217
    • Bug fixes to stpipe/step.py for handling of multiple results from a step
  • 12-Feb-2015 r3216
    • Update the filter data model schema to use a FITS EXTNAME of "THROUGHPUT" instead of "FILTER"
  • 06-Feb-2015 r3201, r3200
    • Updates to log info provided by several steps
    • Changed the default logging level from WARNING to INFO
  • 04-Feb-2015 r3199
    • Fixed an error in bias_drift that was inverting the ramp for the NIRCam B-Long detector
  • 01-Feb-2015 r3197
    • Fixed an error in photom that was preventing a match with reference data for one of the grisms
  • 31-Jan-2015 r3196
    • Added preliminary cube build code
  • 28-Jan-2015 r3194, r3192
    • Removed loading of DATE/TIME in assign_wcs step, at least until it's actually needed
    • Updated models infrastructure to support loading json files
  • 24-Jan-2015 r3188, r3187, r3186
    • Updated bias_drift so that parameter defaults are the same in the code as in the .cfg file
    • Updated core meta data schema to remove time portion from DATE-OBS value and add separate TIME-OBS value
  • 23-Jan-2015 r3185
    • Updated extract_2d to bail out after recognizing an invalid input
  • 17-Jan-2015 r3181
    • Added instrument-specific level-2b imaging pipeline configurations to handle various states of available WCS reference files
  • 15-Jan-2015 r3180
    • Fixed a bug in the MIRI algorithm in the bias_drift step that had the amp-to-column assignments off by 1
  • 10-Jan-2015 r3176
    • Updated jwst_lib.models to save the DATE value in UTC (rather than local clock time)
  • 09-Jan-2015 r3175
    • Updates to jwst_lib.models to be strict about having dimensions of data model arrays match the spec in the schema
  • 06-Jan-2015 r3174
    • Fixed a bug in the bias_drift step that was applying incorrect flips/rotations to NIRISS images
  • 27-Dec-2014 r3173, r3172
    • Initial commits of KLIP routines in the coron package
  • 24-Dec-2014 r3168
    • Fixed bug in the ramp_fit step that was applying masking incorrectly to saturated pixels, causing incorrect slope results
  • 19-Dec-2014 r3165
    • Updates to the MIRI MRS straylight step to handle NaN's that show up in the inter-slice regions
  • 16-Dec-2014 r3163, r3162
    • Updates to photom step to strip whitespace from matching param values
  • 13-Dec-2014 r3161
    • Updates to assign_wcs step to not use direct crds_client calls for retrieving ref files; use normal get_reference_file function
  • 12-Dec-2014 r3157, r3156
    • Updated the flat_field step to replace NaN's in the flat ref file with 1's
    • Updated to the core data model schema to fix a typo in the comment field of the S_REFPIX keyword
  • 09-Dec-2014 r3154
    • Updates to the pipeline user's guide
  • 04-Dec-2014 r3153
    • Updated ramp_fit step to no longer skip any initial or final frames for MIRI exposures
  • 03-Dec-2014 r3152
    • Corrected fits_generator error in MIRI configuration file, which had the 2 IFU detector ID's swapped
  • 26-Nov-2014 r3149, r3148, r3147, r3146
    • Updates to assign_wcs step, configuration files, and stpipe to implement dynamic reference file types that are mode-dependent
  • 25-Nov-2014 r3145, r3144, r3143, r3142, r3141
    • Various updates to assign_wcs tools for building MIRI MRS WCS-related reference files
  • 21-Nov-2014 r3138, r3137, r3136
    • Reinstated previous fix that got lost, which skips CRDS prefetch for steps that are skipped
    • Temporarily backed out r3124 updates to assign_wcs_step, which was checking for 'N/A' returned by reference file fetch
    • Removed unnecessary print statement from data generator
  • 19-Nov-2014 r3135
    • Updated data generator to handle NIRISS spectroscopic modes
  • 15-Nov-2014 r3127
    • Fixed a bug in dynamicdq module so that it returns the input DQ array as-is when the DQ_DEF table is empty
  • 14-Nov-2014 r3124
    • Updated assign_wcs_step to catch an 'N/A' response from CRDS
  • 13-Nov-2014 r3122
    • Updated data generator to correctly populate WCS keywords for MIRI LRS mode exposures
  • 12-Nov-2014 r3120, r3119
    • Updates to data generator to populate additional WCS keywords for both imaging and spectroscopic modes; added MIRI Band and Channel keywords; change FGS instrument name from GUIDER to FGS
    • Updated calwebb_sloper module to always open the input data model before any steps are executed, so that access to meta data is available
  • 11-Nov-2014 r3118, r3117, r3116
    • Updated the straylight step to incorporate review board suggestions
    • Removed debug statement from models/fits.py
    • Fixed the copy method on MultiSlitModel?'s so that the pickled WCS object gets carried along properly
  • 08-Nov-2014 r3112
    • Updates to models and stpipe to eliminate unnecessary arrays in data models and thus reduce memory usage
  • 07-Nov-2014 r3108
    • Removed print statement from bias_drift step MIRI algorithm
  • 06-Nov-2014 r3104
    • Fixed bug in CRDS prefetch when using override="" (null) for a reference file
  • 05-Nov-2014 r3102
    • Refactored CRDS prefetch
  • 04-Nov-2014 r3097
    • Fixed extract_2d so that output multislit data model is still open when attempting to update meta data at end of processing
  • 01-Nov-2014 r3093
    • Upgraded jump step two-point difference method to handle NaN's in input data cube
  • 31-Oct-2014 r3086
    • Upgraded level-2b pipeline modules to no longer keep separate data model results from each step (saves memory)
  • 24-Oct-2014 r3072 r3073 r3074
    • Fixed a typo in the class name in ramp_fit.cfg
    • Upgraded level-2a pipeline modules to no longer keep separate data model results from each step (saves memory) and also updated many step modules to make sure reference file data models get closed at end of step
  • 23-Oct-2014 r3070 r3071
    • Fixed a bug in the FITS data generator so that NIRISS output files have integer values for the subarray start & size keywords
    • Fixed a bug in the extract_1d step that was causing the WCS to not be handled properly when the input is a single 2-D image like MIRI LRS
  • 18-Oct-2014 r3068
    • Fixed passing of pickled WCS object through MultiSlit? data models
  • 17-Oct-2014 r3065
    • Fixed a bug in photom step to allow for matching mode to be found in photom ref data for all NIRCam detectors
  • 19-Apr-2014 r2550
    • Fixed a bug in the flat-field step to handle situations where either the input science data or flat-field data are simple single image models (not multi-slit)
  • 17-Apr-2014 r2547
    • Modified ramp-fit code to use DQ flags to skip first/last frames, instead of explicitly indexing arrays to skip frames in the lower-level fitting routines
  • 12-Apr-2014 r2540
    • Added FRAMES_SKIPPED entry to master list of PIXELDQ flag bit assignments
  • 11-Apr-2014 r2535 r2534
    • Updated DQ array in integration-specific output product from ramp-fit to data type uint16
    • Fixed bug in compression of DQ flags for integration-specific output
    • Updated DQ array in MultiSlitModel to uint16 data type
  • 10-Apr-2014 r2531 r2530
    • Removed extract1d attribute from ImageModel and MultiSlitModel
    • Added first versions of SpecModel and MultiSpecModel
    • Updated extract_1d step to use new Spec data models, instead of extract1d attribute of ImageModel
  • 08-Apr-2014 r2524 r2522
    • Update stpipe so that steps save the jwst svn_revision number in product data models (keyword CAL_VER)
    • Added CAL_VER to core data model schema
  • 05-Apr-2014 r2519 r2518 r2515 r2514 r2513
    • Updated stpipe and crds client to save CRDS software and context map version numbers in data models (CRDS_VER, CRDS_CTX)
    • Updated wcs transforms to accept arrays as region masks
    • Implemented use of mask for MIRI IFU regions definition
    • Updated CubeModel DQ array to data type uint16
  • 02-Apr-2014 r2511 r2510 r2509 r2507
    • Added reset and lastframe data models
    • Added reset and lastframe correction steps (for MIRI)
    • Fixed bug in which setting use_side_ref_pixels=False in bias_drift step threw an exception
  • 01-Apr-2014 r2506
    • Updated jwstlib models to allow for automatic creation of default PIXELDQ array when it doesn't exist in the input
  • 29-Mar-2014 r2502 r2498
    • Updated bias_drift step to correctly implement the side reference pixels correction
    • Enabled IPC correction step in calwebb_sloper pipeline
  • 28-Mar-2014 r2493
    • Fixed bug in jwstlib to correctly load FITS arrays with uint16 data type
  • 27-Mar-2014 r2492
    • Updated ramp_fit step to copy saturation and CR flags from 4-D GROUPDQ array to 2-D output DQ array
  • 25-Mar-2014 r2486
    • Updated ImageModel DQ array to uint16 data type
  • 22-Mar-2014 r2484
    • Added CRDS_VER and CRDS_CTX to core data model schema
  • 20-Mar-2014 r2475
    • Added new IPC correction step (no CRDS files available yet; must use override_ipc)
  • 19-Mar-2014 r2470
    • Added population of stpipe.__svn_revision__ to get jwst repository revision number
  • 15-Mar-2014 r2458 r2457
    • Added saturation data model to jwstlib models and schema
    • Switched saturation step to using dedicated saturation reference files (requires CRDS_CONTEXT jwst_0012.pmap)
    • Removed SAT extension from linearity data model
  • 14-Mar-2014 r2452
    • Added new saturation step to calwebb_sloper pipeline flow
  • 07-Mar-2014 r2439
    • Added new saturation step
    • Removed saturation checking from linearity step
  • 23-Feb-2014 r2419
    • Fixed a bug in copying of HISTORY keywords
  • 22-Feb-2014 r2417
    • Expanded core data model schema to include all defined FITS keywords
    • Changed core data model schema to use '.meta.instrument.name' instead of '.meta.instrument.type'
  • 13-Feb-2014 r2395
    • Implemented support for HISTORY keywords in data models
  • 09-Feb-2014 r2387
    • Ramp_fit updated to collapse 4-D GROUPDQ values into output 2-D DQ array

SSBX

  • 12-Oct-2014 r3042
    • Build 3 release
  • 28-Apr-2014 r2553
    • Flat-field step bug fixed to properly handle non-MultiSlit input data models
    • Upgraded all ramp-fit step output products to use 16-bit DQ arrays
    • Upgraded ramp-fit step to compress 4-D GROUPDQ flags into output 2-D DQ arrays
    • Extract1d extension removed from ImageModel and MultiSlitModel data models
    • Upgraded stpipe to populate CAL_VER, CRDS_VER, and CRDS_CTX keywords in output products, which record the svn repository version numbers of the calibration pipeline, the CRDS software, and the CRDS context map, respectively, that were used in processing
    • Implemented use of mask supplied by IDT to define MIRI IFU slice regions in assign_wcs step
    • Implemented MIRI reset and last-frame correction steps and associated reference file data models
    • Fixed bug in bias_drift step that was causing an error when use_side_ref_pixels=False
    • Upgraded jwstlib data models infrastructure to automatically create a zero-filled PIXELDQ array when it doesn't exist in input RampModel data models
    • Upgraded bias_drift step to correctly implement side reference pixels correction scheme for near-IR detectors
    • Implemented IPC correction step and added it to the calwebb_sloper pipeline
    • Removed SAT extension from linearity reference file data model and implemented independent saturation data model for use in the saturation correction step
  • 17-Mar-2014 r2455
    • Added new saturation step to calwebb_sloper pipeline flow
    • Added new saturation step
    • Removed saturation checking from linearity step
    • Changed core data model schema to use '.meta.instrument.name' instead of '.meta.instrument.type'
    • Expanded core data model schema to include all defined FITS keywords
    • Implemented support for HISTORY keywords in data models
    • Ramp_fit updated to collapse 4-D GROUPDQ values into output 2-D DQ array
Last modified 4 years ago Last modified on 05/29/15 15:49:27