Pipeline

class phrosty.pipeline.Pipeline(diaobj, imgcol, band, science_images=None, template_images=None, science_csv=None, template_csv=None, oid=None, ltcv_prov_tag=None, dbsave=False, dbclient=None, nprocs=1, nwrite=5, verbose=False, memtrace=False, catchfailures=False)

Bases: object

Phrosty’s top-level pipeline

Create the a pipeline object.

Parameters:

• diaobj (DiaObject) – The object we’re building a lightcurve for

• imgcol (ImageCollection) – snappl.imagecollection.ImageCollection

• band (str) – One of R062, Z087, Y106, J129, H158, F184, K213

• science_images (list of snappl.image.Image) – The science images.

• template_images (list of snappl.image.Image) – The template images.

• science_csv (Path or str) –

  CSV file with the science images. The first line must be:

  path observation_id sca mjd band
  

  subsequent lines must have that information for all the science images. path must be relative to ou24.images in config. Pipeline will extract the images from this file whose band matches the band of the pipeline (and ignore the rest)

• template_csv (Path or str) – CSV file with template images. Same format as science_csv.

• oid (str) – Object ID. This is probably a temporary argument. It is only used if diaobj is None, and is really only used to build a filepath without “None” in it when saving the light curve parquet file.

• ltcv_prov_tag (str) – Provenance tag for light curve. Required to use SN PIT database.

• dbsave (bool) – Are we saving to the database? Default False.

• dbclient (snappl.dbclient.SNPITDBClient)

• nprocs (int, default 1) – Number of cpus for the CPU multiprocessing segments of the pipeline. (GPU segments will run a single process.)

• nwrite (int, default 5) – Number of asynchronous FITS writer processes.

• verbose (bool, default True) – Toggle verbose output.

• memtrace (bool, default False) – Toggle memory tracing.

• catchfailures (bool, default False) – Toggle collection of information for images that fail. If true, pipeline will not crash if one image fails. If false, pipeline crashes if one image in a given set fails (useful for debugging and tests).

Methods Summary

__call__([through_step])	Run the pipeline.
add_to_results_dict(one_pair)	Record results from self.make_phot_info_dict() to the aggregate dictionary for the entire light curve.
align_and_pre_convolve(templ_image, sci_image)	Align and pre convolve a single template/science pair.
clear_contents(directory)	Delete contents of a directory.
do_stamps(sci_image, templ_image)	Make stamps from the zero point image, decorrelated difference image, and variance image centered at the location of the supernova.
get_psfs()	Retrieve PSFs for all snappl.image.Image objects in self.science_images and self.template_images.
make_lightcurve()	Collect all results from photometry in one dictionary.
make_phot_info_dict(sci_image, templ_image)
phot_at_coords(img, psf[, pxcoords, ap_r])	Do photometry at forced set of pixel coordinates.
save_stamp_paths(sci_image, templ_image, paths)	Helper function for recording the stamp paths returned in self.do_stamps.
sky_sub_all_images()	Sky subtracts all snappl.image.Image objects in self.science_images and self.template_images using Source Extractor.
write_fits_file(data, header, savepath)	Helper function for writing fits files.

Methods Documentation

__call__(through_step=None)

Run the pipeline.

Parameters:

through_step (str, default None) –

Which step to run thorough? Runs them all if not given.

Steps in order are: * sky_subtract * get_psfs * align_and_preconvolve * subtract * find_decorrelation * apply_decorrelation * make_stamps * make_lightcurve

Returns:

ltcvpath – The path to the output lightcurve file if make_lightcurve was run, otherwise None.

Return type:

pathlib.Path or None

add_to_results_dict(one_pair)

Record results from self.make_phot_info_dict() to the aggregate dictionary for the entire light curve.

Parameters:

one_pair (dict) – Dictionary output from self.make_phot_info_dict().

align_and_pre_convolve(templ_image, sci_image)

Align and pre convolve a single template/science pair.

Parameters:

• sci_image (phrosty.PipelineImage) – The science (new) image.

• templ_image (phrosty.PipelineImage) – The template (ref) image that will be subtracted from sci_image.

Returns:

sfftifier – Use this object for futher SFFT work. Be sure to dereference it to free the prodigious amount of memory it allcoates.

Return type:

SpaceSFFT_Flow

clear_contents(directory)

Delete contents of a directory. Used to clear temporary files.

Parameters:

directory (pathlib.Path) – Path to directory to empty.

do_stamps(sci_image, templ_image)

Make stamps from the zero point image, decorrelated difference image, and variance image centered at the location of the supernova.

Parameters:

• sci_image (snappl.image.Image) – Science image with supernova.

• templ_image (snappl.image.Image) – Template image without supernova.

Returns:

Paths to the stamps corresponding to the zero point image, decorrelated difference image, and variance image centered at the location of the supernova.

Return type:

list of pathlib.Path

get_psfs()

Retrieve PSFs for all snappl.image.Image objects in self.science_images and self.template_images.

Contains its own error logging function, log_error().

make_lightcurve()

Collect all results from photometry in one dictionary. Write the output to a csv as a table.

Contains its own error logging function, log_error().

Returns:

Path to output csv file that contains a light curve.

Return type:

pathlib.Path

make_phot_info_dict(sci_image, templ_image, ap_r=3)

” Do photometry on a difference image generated from sci_image and templ_image. Collect the output in a dictionary.

Parmaeters

sci_image: PipelineImage

science image wrapper

temp_image: PipelineImage

template image wrapper

ap_r: float, default 4

Radius of aperture to use in aperture photometry.

returns:

results_dict – Dictionary with keys sci_name, templ_name, success, ra, dec, mjd, band, observation_id, sca, template_observation_id, template_sca, zpt, aperture_sum, flux_fit, flux_fit_err, mag_fit, and mag_fit_err.

rtype:

dict

phot_at_coords(img, psf, pxcoords=(50, 50), ap_r=3)

Do photometry at forced set of pixel coordinates.

Parameters:

• img (snappl.image.Image) – The image on which to do the photometry

• psf (snappl.psf.PSF) – The PSF.

• pxcoords (tuple of (int, int)) – The position on the image to do the photometry

• ap_r (float) – Radius of aperture.

Returns:

results – Keys and values are: * ‘aperture_sum’: flux in aperture of radius ap_r * ‘flux_fit’: flux from PSF photometry * ‘flux_fit_err’: uncertainty on flux_fit * ‘mag_fit’: instrumental magnitude (i.e. no zeropoint) from flux_fit * ‘mag_fit_err’: uncertainty on mag_fit

All values are floats.

Return type:

dict

save_stamp_paths(sci_image, templ_image, paths)

Helper function for recording the stamp paths returned in self.do_stamps.

Parameters:

• sci_image (snappl.image.Image) – Science image with supernova.

• templ_image (snappl.image.Image) – Template image without supernova.

• paths (list of pathlib.Path) – Output from self.do_stamps().

sky_sub_all_images()

Sky subtracts all snappl.image.Image objects in self.science_images and self.template_images using Source Extractor.

Contains its own error logging function, log_error().

write_fits_file(data, header, savepath)

Helper function for writing fits files.

Parameters:

• data (np.array) – Image array.

• header (_type_) – FITS header.

• savepath (str) – Savepath for FITS file.