ModelTables#

Viterbi-based Tables#

The modeltables module defines FRETBursts tables subclasses for H²MM processing.

The StatePathBase super-set of tables of which the “primary” Table is StatePath. This is a child table, the base table is usually a smfbursts.bursttables.Bursts or smfbursts.burstables.BurstOvlp table, and represents the Viterbi most likely path of H²MM states through each burst. These classes also provide StatePath.optimize() and optimize_models() classmethods which allow for optimization of either a single initial model, or through a range of numbers of states to generate optimal models and statepaths based on a given smfbursts.photondata.PhotonData set of raw data and a given burst search. ntdivStatePath allows incorporation of lifetime data through the divisor aproach while usAlexStatePath implements the shift method for incorporating the acceptor excitation stream of \(\mu s\)ALEX experiments into H²MM analysis.

The Dwells table is a base-table that takes a StatePathBase as it’s only parent, and subdivides each burst into dwells in each state. This makes dwells the rows of the table.

H2MMbursts.modeltables.CCrit#

Options for convergence criterion of StatePath.optimize_models()

alias of Literal[‘BIC’, ‘BICp’, ‘BICph’, ‘ICL’, ‘ICLph’, ‘pathBIC’, ‘pathBICph’] | Callable[[Sequence[h2mm_model], PhotonData | PhotonDataList, Sequence[Param], float | None], int]

H2MMbursts.modeltables.DArray#

Type hint for dets array of H²MM input

alias of ndarray[ndarray[uint8]]

class H2MMbursts.modeltables.Dwells(param: Param, origin: DataSet)#

Defines ranges of time inside of burst that have consecutive photons in the same most-likely state, as determined by the Viterbi algorithm.

Params#

No params exist, Dwells are fully defined by parent statepath

Parents#

statepathStatePathBase
Any Param based on a subclass of StatePathBase (defines bursts and H2MM model) for which to segment state-paths into dwells.

Columns#

Dwells is a BasePhotonTable and thus all columns for BasePhotonTable are also present in Dwells.

The following additional columns are also defined:

rstartint, ()
“Propper” start of a dwell. If dwell is in middle of burst, take midpoint between photons of the transition, if at the beginning of a burst, take the first photon of that burst as the start time.

rstopint, ()
“Propper” stop of a dwell. If dwell is in middle of burst, take midpoint between photons of the transition, if at the end of a burst, take the last photon of that burst as the stop time.

statenp.int8, (i:int, )
State of dwell shifted by i, if that dwell is in another burst, takes value of -1. If i is 0, then state is state of dwell. If i is negative, then the state of the dwell abs(i) dwells before current dwell. If i is possitive, state of i dwells after the current dwell.

iburstint, ()
The index of the burst to which the dwell belongs based on statepath (including gates)

dwell_posnp.int8, ({‘since’, ‘until’, ‘min’, ‘max’, ‘nmin’, ‘pos’}, )
The position of the dwell within a burst. Key indicates direction/function used to evaluate.

‘since’ number of dwells to begginning of burst

‘until’ number of dwells to end of burst

‘min’ min[since, until]

‘max’ max[since, unitl]

‘nmin’ signed min[since, unil] (if maximum is since, value is negative and shifted by -1)

‘pos’ “position” of dwell in burst. Beginning/ending/whole burst dwells have values of -1, -2, and -3 respectively, otherwise give result of min

nph_h2mmint, (phsel:PhSel)
Number of photons based on non-reordered statepath

ratio_h2mmfloat, (phsel_num:Phsel, phsel_dem:PhSel)
Ratio of number of photons in phsel_num to phsel_dem based on non-reordered statepath

anisotropy_h2mmfloat, (phsel_p:Phsel, phsel_s:PhSel)
Anisotropy of number of photons in phsel_p vs phsel_s based on non-reordered statepath

detdef#

class H2MMbursts.modeltables.StatePath(param: Param, origin: DataSet)#

Basic H²MM Viterbi path of each burst based on hm.h2mm_model. the model is defined in model parameter, and the the maping of photon stream to H²MM index is defined by the streams parameter. Photon streams are smf.PhSel objects, the may specify a single detector id or group multiple streams together. If a given detector id is not specified by any photon stream in streams it is ommitted from data given to the viterbi algorithm.

The most common method of creating StatePath objects is StatePath.optimize_models() which

>>> statepaths = bhm.StatePath.optimize_models(data, bursts, max_states=8, to_state=4, conv_crit='pathBIC')

Based on the above call, statepaths will contain at least 4 and up to 8 StatePath Param objects, from 1 state to the point when the convergence criterion was met (pathBIC minimized), maximally 8 states.

Params#

modelhm.h2mm_model
The hm.h2mm_model used in Viterbi processing.

streamstuple[PhSel, …]
tuple of PhSel defining the indexes of photons in H²MM processing.

Parents#

burstsParam[BasePhotonTable]
Usually a :class:smfbursts.datamodel.tables.Bursts` smfbursts.datamodel.tables.Param, defining the time ranges of each “burst” in H²MM processing.

Columns#

indexpathnp.ndarray[np.uint8], ()
Actual indexes used in H²MM processing. Each row is uint8 array.

detpathnp.ndarray[np.uint8], ()
Detector indexes of photons used in H²MM processing, not reassigned by streams. Each row is uint8 array.

timepathnp.ndarray[np.int64], ()
Actual times used in H²MM processing. Each row is int64 array.

statepathnp.ndarray[np.uint8], ()
Most likely states of each photon as processed in Viterbi algorithm. This is the direct output of Viterbi, no sub-selecting photons etc. Each row is 1d uint8 array

scalepathnp.ndarray[np.ndarray[np.float64]], ()
Posterior per photon of Viterbi processing. This is direct output of Viterbi. Each row is 1D float64 array.

pathllpathnp.ndarray[np.float64], ()
Log-likelihood of most likely state of each photon, assuming most likely statepath. This is direct output of Viterbi. Each row is 1D float64 array.

gammapathnp.ndarray[np.float64], ()
Gamma array, giving likelihood per-photon per-state. Each row is 2d float64 array, indexed [photon, state].

ph_indexnp.ndarray[np.int8] (phsel:PhSel, )
Indexes (submitted to H²MM processing) mapped/maskes by phsel. Photons outside of model-stream recieve a value of -1

ph_h2mmtimenp.ndaray[np.int64], (phsel:PhSel, )
Times of photons used in H²MM processing mapped/masked by phsel. Photons outside of model-streams receive value of -1. Rows are 1d int64 arrays.

ph_statenp.ndarray[np.int8], (phsel:PhSel, )
Viterbi state of each photon, mapped/masked by phsel. Photons outside of model-streams receive value of -1. Rows are 1d int8 arrays.

ph_scalenp.ndarray[np.float64], (phsel:PhSel, )
Posterior likelihood of each phton, mapped/masked by phsel. Photons outside of model-streams receive value of nan. Rows are 1d float64 arrays.

ph_pathllnp.ndarray[np.float64], (phsel:PhSel, )
log-likihood of state-assignment of photon for Viterbi path, mapped/masked by phsel. Photons outside of model-streams receive value of nan. Rows are 1d float64 arrays

ph_gammanp.ndarray[np.float64], (phsel:PhSel, )
Gamma array, giving likelihood per-photon per-state mapped/masked by phsel. Each row is 2d float64 array, indexed [photon, state].

llfloat, ()
loglikelihood of each burst, over all statepaths.

llphfloat ()
The loglikelihood per photon over all statepaths of each burst.

pathllfloat, ()
loglikelihood of each burst, of most likely statpath.

pathllphfloat ()
The loglikelihood per photon of most likely statpath of each burst.

eff_statenp.ndarray[np.int8], (phsel:PhSel, )
“Effective” state of each photon in burst maped/masked by phsel. If a photon is in a stream not present in model streams, then infer state by nearest photon that is in streams. This is essentially ph_state with -1s replaced with an infered state. Rows are 1d int8 arrays.

bstatesint, ()
Bitcode indicating which states present in burst. If given bit position is present, then that state is present in burst, ie if states 0 and 2 are in a given burst, the the value is 0b00000101 = 5. Rows are int.

transcountnp.ndarray[np.int64], ()
Number of transisions from state to state in a given burst. 2D square rows, size of nstate.

ntransint, ()
Number of transitions in burst, sum of off-diagonal of transcount.

nanohist_statenp.ndarray[np.int64], (phsel:PhSel, thresh:float, discr:Literal[‘ph_gamma’, ‘ph_state’, ‘ph_pathll’], full:bool)
Histogram of state x nanotime of photons in a burst, with photons filtered by thresh and discr, only photons with a likelihood of state assignment greater than or equal to thresh according to the dicr (either ph_gamma, ph_scale or ph_pathll) are included in the histogram. If full, the return histogram using TCSPC raw bins, if full=False, then trim to excitation range.

classmethod get_ndet(param: Param) → int#

From Param based on StatePath, get the number of indexes assigned to photon streams.

Parameters:: param (Param) – Param: based on Statepath defining number of indexes in H2MM processing.
Returns:: Number of indexes used in H2MM processing defined by param.
Return type:: int

model_streams#

Determine index(es) of phsel in the hm.h2mm_model based on a given param definition.

Parameters:

statepath (Param) – smfbursts.datamodel.tables.Param of type smfburts.datamodel.tables.StatePath with model.
phsel (PhSel) – Stream selection to map to model.
origin (PhotonData, optional) – If specified, the smfbursts.photondata.PhotonData object from which statepath is assumed to have been optimized. The default is None.
strict (bool, optional) – Whether to check if the phsel contains only streams specified in the streams param of statepath. The default is True.

Raises:

ValueError – phsel contains streams not used in statepath.

Returns:

used_streams – Array of indexes which contribute to phsel in statepath.model.obs. statepath.model.obs[:,used_strams].sum(axis=1) will return the probability that a photon arises from phsel per state in the statepath.model.

Return type:

np.ndarray[np.int64]

classmethod optimize(origin: PhotonData | PhotonDataList, bursts: Param, model: h2mm_model, streams: Sequence[PhSel] = None, gate: GateGroup = None, **kwargs) → Param#

Optimize a model against data defined by bursts in origin, using indexes defined by streams

Parameters:

origin (PhotonDataS) – Source of burst data.
bursts (Param) – Time (and thus photon arrays) defining data to optimize.
model (hm.h2mm_model) – Initial model to start optimization.
streams (Sequence[PhSel], optional) – Sequence of PhSel, defines indexes given to H²MM. The default is None.
gate (GateGroup, optional) – Gate to apply to bursts. The default is None
**kwargs (Any) – Additional kwargs handed to model.optimize.

Returns:

StatePath based Param of optimized model.

Return type:

Param

classmethod optimize_models(origin: PhotonData | PhotonDataList, bursts: Param, streams: Sequence[PhSel] = None, min_states: int = 1, max_states: int = 8, to_state: int = 4, conv_crit: Literal['BIC', 'BICp', 'BICph', 'ICL', 'ICLph', 'pathBIC', 'pathBICph'] | Callable[[Sequence[h2mm_model], PhotonData | PhotonDataList, Sequence[Param], float | None], int] = 'pathBIC', thresh: None | float = None, gate: GateGroup = None, **kwargs) → list[Param]#

Given data and burst definition, optimize successive state-models and return the cooresponding state-path Param s in a list.

Parameters:

origin (PhotonDataS) – Source of photon data to optimize.
bursts (Param) – Time ranges to optimize.
streams (Sequence[PhSel], optional) – Sequence of PhSel, defines indexes given to H²MM. If None, use each detector ID as separate stream. The default is None.
min_states (int, optional) – Number of states in first model optimized. The default is 1.
max_states (int, optional) – Maximum number of states, regardless of conv_crit in models to optimize, sets an upper limit of the number of states to optimize. The default is 8.
to_state (int, optional) – Minimum number of states in models guaranteed to optimize, regardless of conv_crit, it is guaranteed that the returned sequence will contain StatePaths with models with from min_states to to_state (inclusize) numbers of states. The default is 4.
conv_crit (Literal['BIC','BICp','BICph','ICL','ICLph','pathBIC','pathBICph'] | Callable, optional) –
Criterion to use to determine of best model has been found, the convergence criterion. This can either be a string defining the function or a callable. Options are:
- ’BIC’ Choose based on BIC, uses conv_BIC()
- ’BICp’ Choose based on modified BIC, uses conv_BICp()
- ’BICph’ Choose based on BIC per photon, uses conv_BICph()
- ’ICL’ Choose based on ICL, uses conv_ICL()
- ’ICLph’ Choose based on ICL per photon, uses conv_ICLph()
- ’pathBIC’ Choose based on the BIC of most likely state path,
  uses conv_pathBIC()
- ’pathBICph’ Choose based on BIC of most likely state path, uses conv_pathBICph()
If specified as a callable, should have a signature like that of conv_BIC() ie conv_crit(models:Sequence[hm.h2mm_model], origin:PhotonDataS, params:Sequence[Param], thresh:None|float)->int The default is ‘pathBIC’.
thresh (float, optional) – If specified, the threshold to consider a model converged, ie if the difference in the statistical disciminator between a model and the minimum is less than thresh, then that model is considered ideal. The default is None.
gate (GateGroup, optional) – Gate to apply to bursts. The default is None.
**kwargs (Any) – Additional kwargs handed to model.optimize() .

Returns:

List of StatePath based Params of optimized models.

Return type:

list[Param]

classmethod param_idx_to_det_map(params: dict[slice(<class 'str'>, typing.Any, None)], detdef: ~smfbursts.ph_sel.DetDef) → ndarray[uint8]#

Returns 1D numpy array that should map h2mm index to detector index- ie idxmap[index] = detector where detector should match the ph_dets array

Parameters:

params (dict[str:Any]) – param dictionary definition for class, may omit keys not requied to compute the idx_to_det_map.
detdef (DetDef) – DetDef of data for which param is expected to be based. Needed to determing dets

Returns:

mapping of H2MM idx to det based on detdef.

Return type:

np.ndaray[np.uint8]

classmethod param_model(statepath: Param, origin: PhotonData | PhotonDataList = None) → h2mm_model#

Return the H²MM model used by statpath

Parameters:

statepath (Param) – StatePath Param from which to retrieve H²MM model.
origin (PhotonDataS, optional) – Data on which model will be used/is based. The default is None.

Returns:

Raw H²MM model.

Return type:

H2MM_C.h2mm_model

classmethod param_ndet(param: dict) → int#

Get number of indexes for H2MM processing defined by param dictionary. This dictionary need only define the parts of the param in a StatePath Param needed to determine the number of indexes. Other keys may be omitted.

Parameters:: param (dict) – params dict to be used to create StatePath Param, unnecessary keys may be ommited.
Returns:: Number of indexes in H2MM processing as defined by param.
Return type:: int

classmethod param_preprocess(params: Sequence[tuple[str, Any]] | tupledict, parents: dict[str, Param]) → tuple[dict, dict]#: Preprocess auto-filling detdef and sorting model :meta private:

classmethod param_sort_process(params: dict[slice(<class 'str'>, typing.Any, None)], detdef: ~smfbursts.ph_sel.DetDef) → tuple[dict[slice(<class 'str'>, typing.Any, None)], ~numpy.ndarray[~numpy.int64]]#

In a params dictonary, sort the streams so that streams are in ascending order based on DetDef

Parameters:

params (dict[str:Any]) – params dictionary to be used in StatePath based Param.
detdef (DetDef) – DetDef of expected Param.

Returns:

params (dict[str,Any]) – Sorted params dictoinary.
sort (np.ndarray[np.int64]) – Re-sorting array, value is original index, position is index for destination. Therefore new = old[sort]

param_streams#

Return the streams used by statepath

Parameters:

statepath (Param) – StatePath Param from which to retrieve photon streams.
origin (PhotonDataS, optional) – Data on which model will be used/is based. The default is None.

Returns:

Sequence of PhSel defining streams of each H²MM index.

Return type:

Sequence[PhSel]

phsel_select(phsel: PhSel, col: str, fill: Any, dtype: dtype) → ndarray[object_]#

Maps the of the inputs to H²MM evaluation to the “unprocessed” output shape.

Parameters:

phsel (PhSel) – A phsel object defining the output streams to return.
col (str) – Name of column being returned.
fill (Any) – Value to fill any photons that are in phsel but outside of phsel_span.
dtype (np.dtype) – Data-type of output array.

Returns:

If implemented should return object array of column maped to phsel

Return type:

np.ndarray[np.object_]

classmethod sort_photons(origin: PhotonData | PhotonDataList, statepath: Param = None, bursts: Param = None, streams: Sequence[PhSel] = None) → dict[slice(<class 'str'>, numpy.ndarray[numpy.ndarray], None)]#

Sort photons into indexes/times arrays for processing with H²MM.

Parameters:

origin (PhotonDataS) – Data from which to sort photons.
statepath (Param, optional) – Definition of streams, if used cannot use bursts or streams kwargs. The default is None.
bursts (Param, optional) – bursts definition, must be used with streams, and . The default is None.
streams (Sequence[PhSel], optional) – Streams to inlcude in H²MM processing. The default is None.

Returns:

dict[str – Dictionary of sorted photons, each key contains a particular sort type. has the following keys:

indexes : np.ndarray[np.ndarray[np.uint8]] Photon indexes for H²MM processing.
times : np.ndarray[np.ndarray[np.int64]] Photon arrival times for H²MM processing

Return type:

np.ndarray[np.ndarray]]

classmethod validate_param(param: Param) → None#: Validate a StatePath Param :meta private:

class H2MMbursts.modeltables.StatePathBase(param: Param, origin: DataSet)#

Base class for tables storing Viterbi path of H²MM model.

Required param_defs:

ParamDef(‘model’, TV_H2MMModel),
ParamDef(‘streams’, TV_tuple(typedefs=TV_PhSel))

Required parent_defs (automatically included):

ColumnDef(‘bursts’, BasePhotonColumn, is_base=True)

Required columns

Non-reordered array columns (all have rows of arrays of size of burst):

indexpath indexes submitted to h2mm
timepath times submitted to h2mm
statepath simple output of most-likely states set in init columns, store all
scalepath verterbi posterior probability set in init columns, store all
pathllpath path photon likelihood, based on Viterbi path, should be store all
gammapath h2mm state likelihood should be store all

Reordered array columns (include ph_sel argument) size matches ph_times of base:

ph_index (phsel:PhSel): index mapped to phsel
ph_h2mmtime (phsel:PhSel): times mapped to phsel (for alex, arrays may not be monotonic)
ph_state (phsel:PhSel): state of each photon mapped to phsel
ph_scale (phsel:PhSel): viterbi posterior probabilit mapped to phsel
ph_pathll (phsel:PhSel): Viterbi path loglikilihood
ph_gamma (phsel:PhSel): h2mm state likelihood

Processed columns:

ll (): loglikelihood of burst, given model and all possible state paths.
llph (): loglikelihood per photon included in H²MM of burst, given model and all possible state paths.
eff_state (): effetive state (if state not know, infer from dwell time) computed columns
bstates (): bitwise truthtable of states present
transcount (): number of transitions between states in burst (2D array)
ntrans (): number of transitions in burst, single integer
pathll (): Viterbi path loglikelihood of the burst
pathllph (): Viterbi path loglikelihood of the burst per photon

BIC#

This is a tableproperty.

Bayes Information Criterion of given StatePath (model + burst selection from origin)

Parameters:

statepath (Param) – StatePath which defines burst selection, H²MM model and photon selection params.
origin (PhotonDataS) – Raw Data.
as_array (bool, optional) – (Only for when origin is PhotonDataList). If True return BIC of each element in The default is False.
from_flag (bool, optional) – If True, check flags of statepath, if dataID of origin is present, and BIC present, return already computed, if False, force computation of BIC new. The default is False

Returns:

BIC.

Return type:

float | np.ndarray[np.float64]

static BICp(statepaths: Sequence[Param], origin: PhotonData | PhotonDataList, from_flag: bool = True) → ndarray[float64]#

Compute a the Bayes Information Criterion per photon of the models defining the input sequence of StatePath based Params .

Parameters:

statepaths (Sequence[Param]) – Sequence of StatePath based Params, used to specify H²MM models..
origin (PhotonDataS) – Data against which to compute the BIC per photon.

Returns:

1D array of BIC per photon for each model.

Return type:

np.ndarray[np.float]

BICph#

Bayes Information Criterion per photon of given StatePath (model + burst selection from origin). That is the \(BIC / no.photons\)

Parameters:

origin (PhotonDataS) – Raw Data.
statepath (Param) – StatePath which defines burst selection, H²MM model and photon selection params.

Returns:

BIC per photon.

Return type:

float

ICL#

Get the Integrated Complete Likelihood (ICL) of the data in origin specified by the model in statepath. Note that this is a classmethod.

Parameters:

origin (PhotonDataS) – Data for which to compute the ICL.
statepath (Param) – Param defining the burst selection and model.

Returns:

ICL of data/model combination.

Return type:

float

ICLph#

Get the Integrated Complete Likelihood (ICL) per photons of the data in origin specified by the model in statepath. Note that this is a classmethod.

Parameters:

origin (PhotonDataS) – Data for which to compute the ICL.
statepath (Param) – Param defining the burst selection and model.

Returns:

ICL/photon of data/model combination.

Return type:

float

classmethod find_saved(data: PhotonData | PhotonDataList, states: int | Sequence[int] = None) → tuple[Param, ...]#

Retrieve all saved tables matching class and return as tuple of params

Parameters:

data (PhotonDataS) – DESCRIPTION.
states (int | Sequence[int], optional) – If specified, restrict parmas returned to those with models matching states. The default is None.

Returns:

Tuple of all matching Params.

Return type:

tuple[Param,…]

classmethod get_ndet(param: Param) → int#

The number detector indexes for a smfbursts.datamodel.tables.Param of the given subclass. Implemented per sublcass.

Note

Should be based on parameters in param other than “model” so that it can be used to validate the model in validate_param

Parameters:: param (Param) – smfbursts.datamodel.tables.Param of type matching class for which to determine the number of detector indices.
Returns:: Number of indeces in H2MM_C.h2mm_model of the input H2MM_C.h2mm_model.
Return type:: int

classmethod get_phsel_span(param: Param) → PhSel#

Get the smfbursts.ph_sel.PhSel object that spans all streams used to compute the phsel.

Parameters:: param (Param) – StatePath for which to derive the “span” of phsels.
Returns:: PhSel which includes all streams used in H2MM processing.
Return type:: PhSel

classmethod load_saved_models(data: PhotonData | PhotonDataList, states: int | Sequence[int] = None) → tuple[Param, ...]#

Load saved models into memory, and return tuple of their Params.

Parameters:

data (PhotonDataS) – Data with linked HDF5 file from which to load models (models saved in HDF5 file).
states (int | Sequence[int], optional) – If specified, restrict parmas returned to those with models matching states. The default is None.

Returns:

Tuple of all matching Params.

Return type:

tuple[Param,…]

model_streams#

Determine index(es) of phsel in the hm.h2mm_model based on a given param definition.

Parameters:

statepath (Param) – StatePath Param to derive model from.
phsel (PhSel) – phsel to extract model detector indexes.
origin (PhotonDataS, optional) – Origin Data. The default is None.
strict (bool, optional) – Whether to require phsel in phsel_span of statepath. The default is True.

Returns:

array of indexes- statepath.model.obs[:,out] will return array of only obs columns in phsel.

Return type:

np.ndarray[np.int64]

model_value#

A parammethod that retrives the expected value of column per state based on the model of statepath.

Parameters:

statepath (Param) – StatePath based param defining H2MM model.
col (Column) – Column to compute expected values based on H2MM model.
origin (PhotonDataS, optional) – Origin data. The default is None.
strict (bool, optional) – Ensure value can be fully calculated from model (ie phsel does not contain streams not used in model-streams). The default is True.

Raises:

ValueError – One or more phsel of col outside of model-strams.

Returns:

Per-state expected value of column.

Return type:

np.ndarray[np.float64]

model_values#

Get expected values of multiple columns of Table based on model.

Parameters:

*args (Column) – Columns to get expected values based on model.
strict (bool, optional) – Whether or not to raise an error if any col uses streams outside of model-streams. The default is True.

Returns:

Arrays of per-state expected values of each column based on model.

Return type:

np.ndarray[np.float] …

ndet#: Number of states in model

nstate#: Number of states in model

classmethod param_idx_to_det_map(params: dict[slice(<class 'str'>, typing.Any, None)], detdef: ~smfbursts.ph_sel.DetDef) → ndarray[uint8]#

Subclasses must implement this method

Returns 1D numpy array that should map h2mm index to detector index- ie idxmap[index] = detector where detector should match the ph_dets array

Parameters:

params (dict[str:Any]) – param dictionary definition for class, may omit keys not requied to compute the idx_to_det_map.
detdef (DetDef) – DetDef of data for which param is expected to be based. Needed to determing dets

Returns:

mapping of H2MM idx to det based on detdef.

Return type:

np.ndaray[np.uint8]

classmethod param_model(statepath: Param, origin: PhotonData | PhotonDataList = None) → h2mm_model#

Get the model from a given statepath of matching type. Implemented per subclass Implemented so that statpaths can define the model either in params or parents.

Parameters:

statepath (Param) – Param from which to extract model.
origin (PhotonDataS, optional) – Origin data. The default is None.

Returns:

Model of statepath

Return type:

hm.h2mm_model

param_streams#

Get the sequence of phsels specifying each stream in obs. Implemented per subclass

Parameters:

statepath (Param) – StatePath base Param from which to extract streams.
origin (PhotonDataS, optional) – Origin Data. The default is None.

Returns:

Sequence of phsel, specifying map of stream -> H²MM index.

Return type:

Sequence[PhSel]

pathBIC#

Get the BIC of likelihood of most-likely path of the data in origin specified by the model in statepath. Note that this is a classmethod.

Parameters:

origin (PhotonDataS) – Data for which to compute the ICL.
statepath (Param) – Param defining the burst selection and model.

Returns:

path BIC of data/model combination.

Return type:

float

pathBICph#

Get the BIC of likelihood of most-likely state-path per photon of the data in origin specified by the model in statepath. Note that this is a classmethod.

Parameters:

origin (PhotonDataS) – Data for which to compute the ICL.
statepath (Param) – Param defining the burst selection and model.

Returns:

path BIC of data/model combination.

Return type:

float

phsel_select(phsel: PhSel, col: str, fill: Any, dtype: dtype) → ndarray[object_]#

Subclasses must implement this method. This method should implement the mapping of the inputs to H2MM evaluation to the “unprocessed” versions.

Parameters:

phsel (PhSel) – A phsel object defining the output streams to return.
col (str) – Name of column being returned.
fill (Any) – Value to fill any photons that are in phsel but outside of phsel_span.
dtype (np.dtype) – Data-type of output array.

Raises:

NotImplementedError – Subclass does not allow use of this function, usually indicates abstract class.

Returns:

If implemented should return object array of column maped to phsel

Return type:

np.ndarray[np.object_]

property phsel_span: PhSel#: The smfbursts.ph_sel.PhSel object that includes all streams used in computing the H2MM_C.h2mm_model used to determine if a given stream is included or excluded from H²MM processing

transrate#

Table property.

Compute the transition rates of the model in units of s.

Parameters:

statepath (Param) – Param specifying H²MM model
origin (PhotonDataS) – Data used to optimize model, used to define clock rate.

Returns:

Transition rates in seconds.

Return type:

np.ndarray[np.float64]

class H2MMbursts.modeltables.StatePathFilter(param: Param, origin: DataSet)#

Bursts with state(s) removed. Motivated by Kache & Hendrix, 2025. This BasePhotonTable defines the contiguous time periods where:

The photons are defined to be “in burst” as defined by the burst defintion of the parent StatePath

The system is not in the excluded state(s).

The result of this is that bursts starting or ending with the excluded state(s) are truncated, and bursts with a transition into and out of the excldued state(s) are split into two or more bursts with the transitions into the excludes state(s) removed.

Parents#

statepathStatePath: Defines the model and bursts by which photon states are defined.

Params#

exclude: frozenset[int]: All states to be ecluded

detdef#: paramproperty that gives the DetDef of the current Param

H2MMbursts.modeltables.TArray#

Type hint for times array of H²MM input

alias of ndarray[ndarray[int64]]

H2MMbursts.modeltables.TV_H2MMModel = <class 'smfbursts.datamodel.immutabledata.TypeValidator'> type_ = <class 'H2MM_C.h2mm_model'> check = <function _check_h2mm_model> encoder = <function encode_h2mmmodel> decoder = <function decode_h2mmmodel> write = functools.partial(<function write_byteslike at 0x7882b13271a0>, convert=<function dwrite_h2mm_model at 0x7882a0863e20>, title_func=<function _val_type_name at 0x7882b1327060>) node_prefix = None node_repr = None node_read = None node_check = None data_proc = <function _emptydict> ckwargs = {} validator = <function _return_first> validator_dict = {}#

TypeValidator for H²MM models. Uses check_h2mm_model() as check func. The options are:

nstate (int) Number of states that must be in the model (the nstate attribute)
ndet (int) Number of detector indexes that must be in the model (the ndet attribute)

H2MMbursts.modeltables.TV_str_dwelldist = <class 'smfbursts.datamodel.immutabledata.TypeValidator'> type_ = <class 'str'> check = <function check_str> encoder = None decoder = None write = functools.partial(<function write_byteslike at 0x7882b13271a0>, convert=<function dwrite_str>, title_func=<function _val_type_name>) node_prefix = str node_repr = <function _echo> node_read = <function _echo> node_check = <function node_check_str> data_proc = <function _emptydict> ckwargs = {'isin': ('since', 'until', 'min', 'max', 'nmin', 'pos')} validator = <function _return_first> validator_dict = {}#: TypeValidator for string of dwell_pos column first key

H2MMbursts.modeltables.TV_str_dwellstart = <class 'smfbursts.datamodel.immutabledata.TypeValidator'> type_ = <class 'str'> check = <function check_str> encoder = None decoder = None write = functools.partial(<function write_byteslike at 0x7882b13271a0>, convert=<function dwrite_str>, title_func=<function _val_type_name>) node_prefix = str node_repr = <function _echo> node_read = <function _echo> node_check = <function node_check_str> data_proc = <function _emptydict> ckwargs = {'isin': ('istart', 'istarttime', 'rstart')} validator = <function _return_first> validator_dict = {}#: TypeValidator for start time type in Dwells

H2MMbursts.modeltables.TV_str_dwellstop = <class 'smfbursts.datamodel.immutabledata.TypeValidator'> type_ = <class 'str'> check = <function check_str> encoder = None decoder = None write = functools.partial(<function write_byteslike at 0x7882b13271a0>, convert=<function dwrite_str>, title_func=<function _val_type_name>) node_prefix = str node_repr = <function _echo> node_read = <function _echo> node_check = <function node_check_str> data_proc = <function _emptydict> ckwargs = {'isin': ('istop', 'istoptime', 'rstop')} validator = <function _return_first> validator_dict = {}#: TypeValidator for stop time type in Dwells

H2MMbursts.modeltables.calc_BIC(origin: PhotonData | PhotonDataList, statepaths: Sequence[Param], **kwargs) → ndarray[float64]#

Compute Bayes Information Criterion for sequence (presumably from same optimization) of statepaths

Parameters:

origin (PhotonDataS) – Source of data (photons) for computation.
statepaths (Sequence[Param]) – Sequence of StatePathBase based Param defining h2mm models.

Returns:

BICs of each model in statpaths.

Return type:

np.ndarray[np.float64]

H2MMbursts.modeltables.calc_BICp(origin: PhotonData | PhotonDataList, statepaths: Sequence[Param]) → ndarray[float64]#

Compute modified Bayes Information Criterion per photon for sequence (presumably from same optimization) of statepaths

Parameters:

origin (PhotonDataS) – Source of data (photons) for computation.
statepaths (Sequence[Param]) – Sequence of StatePathBase based Param defining h2mm models.

Returns:

modified BIC per photon of each model in statpaths.

Return type:

np.ndarray[np.float64]

H2MMbursts.modeltables.calc_BICph(origin: PhotonData | PhotonDataList, statepaths: Sequence[Param], **kwargs) → ndarray[float64]#

Compute Bayes Information Criterion per photon for sequence (presumably from same optimization) of statepaths

Parameters:

origin (PhotonDataS) – Source of data (photons) for computation.
statepaths (Sequence[Param]) – Sequence of StatePathBase based Param defining h2mm models.

Returns:

BIC per photon of each model in statpaths.

Return type:

np.ndarray[np.float64]

H2MMbursts.modeltables.calc_ICL(origin: PhotonData | PhotonDataList, statepaths: Sequence[Param], **kwargs) → ndarray[float64]#

Compute Integrated Complete Likelihood for sequence (presumably from same optimization) of statepaths

Parameters:

origin (PhotonDataS) – Source of data (photons) for computation.
statepaths (Sequence[Param]) – Sequence of StatePathBase based Param defining h2mm models.

Returns:

ICLs of each model in statpaths.

Return type:

np.ndarray[np.float64]

H2MMbursts.modeltables.calc_ICLph(origin: PhotonData | PhotonDataList, statepaths: Sequence[Param], **kwargs) → ndarray[float64]#

Compute Integrated Complete Likelihood per photon for sequence (presumably from same optimization) of statepaths

Parameters:

origin (PhotonDataS) – Source of data (photons) for computation.
statepaths (Sequence[Param]) – Sequence of StatePathBase based Param defining h2mm models.

Returns:

ICL per photon of each model in statpaths.

Return type:

np.ndarray[np.float64]

H2MMbursts.modeltables.calc_pathBIC(origin: PhotonData | PhotonDataList, statepaths: Sequence[Param], **kwargs) → ndarray[float64]#

Compute path Bayes Information Criterion for sequence (presumably from same optimization) of statepaths

Parameters:

origin (PhotonDataS) – Source of data (photons) for computation.
statepaths (Sequence[Param]) – Sequence of StatePathBase based Param defining h2mm models.

Returns:

path BICs of each model in statpaths.

Return type:

np.ndarray[np.float64]

H2MMbursts.modeltables.calc_pathBICph(origin: PhotonData | PhotonDataList, statepaths: Sequence[Param], **kwargs) → ndarray[float64]#

Compute path Bayes Information Criterion per photon for sequence (presumably from same optimization) of statepaths

Parameters:

origin (PhotonDataS) – Source of data (photons) for computation.
statepaths (Sequence[Param]) – Sequence of StatePathBase based Param defining h2mm models.

Returns:

path BIC per photon of each model in statpaths.

Return type:

np.ndarray[np.float64]

H2MMbursts.modeltables.conv_BIC(models: Sequence[h2mm_model], origin: PhotonData | PhotonDataList, params: Sequence[Param], thresh: float = None) → int#

Determine ideal model in sequence of optimizations based on Bayes Information Criterion.

Parameters:

models (Sequence[hm.h2mm_model]) – Optimized models, in ascending number of states.
origin (PhotonDataS) – Data on which optimizations were performed.
params (Sequence[Param]) – StatePath Parameters of each optimized model.
thresh (float, optional) – Minimum difference to accept model as ideal. If None, last model must have larger BIC than ideal (equivalent to thresh = 0.0) The default is None.

Returns:

Index of ideal model, if no ideal model found return -1.

Return type:

int

H2MMbursts.modeltables.conv_BICp(models: Sequence[h2mm_model], origin: PhotonData | PhotonDataList, params: Sequence[Param], thresh: float = None) → int#

Determine ideal model in sequence of optimizations based on modified Bayes Information Criterion.

Parameters:

models (Sequence[hm.h2mm_model]) – Optimized models, in ascending number of states.
origin (PhotonDataS) – Data on which optimizations were performed.
params (Sequence[Param]) – StatePath Parameters of each optimized model.
thresh (float, optional) – Minimum difference to accept model as ideal. If None, last model must have larger BIC than ideal (equivalent to thresh = 0.0) The default is None.

Returns:

Index of ideal model, if no ideal model found return -1.

Return type:

int

H2MMbursts.modeltables.conv_BICph(models: Sequence[h2mm_model], origin: PhotonData | PhotonDataList, params: Sequence[Param], thresh: float = None) → int#

Determine ideal model in sequence of optimizations based on Bayes Information Criterion per photon.

Parameters:

models (Sequence[hm.h2mm_model]) – Optimized models, in ascending number of states.
origin (PhotonDataS) – Data on which optimizations were performed.
params (Sequence[Param]) – StatePath Parameters of each optimized model.
thresh (float, optional) – Minimum difference to accept model as ideal. If None, last model must have larger BIC than ideal (equivalent to thresh = 0.0) The default is None.

Returns:

Index of ideal model, if no ideal model found return -1.

Return type:

int

H2MMbursts.modeltables.conv_ICL(models: Sequence[h2mm_model], origin: PhotonData | PhotonDataList, params: Sequence[Param], thresh: float = None, **kwargs) → int#

Determine ideal model in sequence of optimizations based on Integrated Complete Likelihood.

Parameters:

models (Sequence[hm.h2mm_model]) – Optimized models, in ascending number of states.
origin (PhotonDataS) – Data on which optimizations were performed.
params (Sequence[Param]) – StatePath Parameters of each optimized model.
thresh (float, optional) – Minimum difference to accept model as ideal. If None, last model must have larger BIC than ideal (equivalent to thresh = 0.0) The default is None.

Returns:

Index of ideal model, if no ideal model found return -1.

Return type:

int

H2MMbursts.modeltables.conv_ICLph(models: Sequence[h2mm_model], origin: PhotonData | PhotonDataList, params: Sequence[Param], thresh: float = None, **kwargs) → int#

Determine ideal model in sequence of optimizations based on Integrated Complete Likelihood per photon.

Parameters:

models (Sequence[hm.h2mm_model]) – Optimized models, in ascending number of states.
origin (PhotonDataS) – Data on which optimizations were performed.
params (Sequence[Param]) – StatePath Parameters of each optimized model.
thresh (float, optional) – Minimum difference to accept model as ideal. If None, last model must have larger BIC than ideal (equivalent to thresh = 0.0) The default is None.

Returns:

Index of ideal model, if no ideal model found return -1.

Return type:

int

H2MMbursts.modeltables.conv_pathBIC(models: Sequence[h2mm_model], origin: PhotonData | PhotonDataList, params: Sequence[Param], thresh: float = None, **kwargs) → int#

Determine ideal model in sequence of optimizations based on path Bayes Information Criterion.

Parameters:

models (Sequence[hm.h2mm_model]) – Optimized models, in ascending number of states.
origin (PhotonDataS) – Data on which optimizations were performed.
params (Sequence[Param]) – StatePath Parameters of each optimized model.
thresh (float, optional) – Minimum difference to accept model as ideal. If None, last model must have larger BIC than ideal (equivalent to thresh = 0.0) The default is None.

Returns:

Index of ideal model, if no ideal model found return -1.

Return type:

int

H2MMbursts.modeltables.conv_pathBICph(models: Sequence[h2mm_model], origin: PhotonData | PhotonDataList, params: Sequence[Param], thresh: float = None, **kwargs) → int#

Determine ideal model in sequence of optimizations based on path Bayes Information Criterion per photon.

Parameters:

models (Sequence[hm.h2mm_model]) – Optimized models, in ascending number of states.
origin (PhotonDataS) – Data on which optimizations were performed.
params (Sequence[Param]) – StatePath Parameters of each optimized model.
thresh (float, optional) – Minimum difference to accept model as ideal. If None, last model must have larger BIC than ideal (equivalent to thresh = 0.0) The default is None.

Returns:

Index of ideal model, if no ideal model found return -1.

Return type:

int

H2MMbursts.modeltables.decode_h2mmmodel(val: tuple[str, bytes]) → h2mm_model#

Decode a tuple from msgpack bytes as an H²MM model.

Parameters:: val (tuple[str,bytes]) – msgpack tuple representing H²MM model.
Returns:: H²MM model of val.
Return type:: hm.h2mm_model

H2MMbursts.modeltables.dread_h2mm_model(arr: bytes, dct: dict) → h2mm_model#: Read H2MM model from bytes, for TypeValidator

H2MMbursts.modeltables.dwell_distmax(burst_num: ndarray[int8]) → ndarray[int8]#

Maximum number of dwells to reach transition (from or until)

Parameters:: burst_num (np.ndarray[np.int64]) – Burst number of each dwell.
Returns:: dwell_max – Maximum number of dwells (from or until) to reach a new burst.
Return type:: np.ndarray[np.int8]

H2MMbursts.modeltables.dwell_distmin(burst_num: ndarray[int64]) → ndarray[int8]#

Minimum number of dwells to reach new burst (from or until)

Parameters:: burst_num (np.ndarray[np.int64]) – Burst number of each dwell.
Returns:: dwell_min – Minimum number of dwells (from or until) to reach new burst.
Return type:: np.ndarray[np.int8]

H2MMbursts.modeltables.dwell_distminneg(burst_num: ndarray[int64]) → ndarray[int8]#

Minimum number of dwells to a different burst, with sign indicating direction. If nearest different burst is before dwell, value is positive, if nearest transition is after dwell, then value is negative, and shifted by -1 ie if a dwell is 1 transition away from the next burst, it will have a value of \(-2 = -1 + -1\), This means that the ending dwell of a burst has a value of -1.

Parameters:: burst_num (np.ndarray[np.int64]) – Burst number of each dwell.
Returns:: dwell_pos – Signed minimum number of dwells to new burst.
Return type:: np.ndarray[np.int8]

H2MMbursts.modeltables.dwell_distminpos(burst_num: ndarray[int64]) → ndarray[int8]#

Determine dwell “position” in burst, negative values indicate beginning or ending dwells, while positive values are minimiumn number of dwells to reach new burst (ie same values as would be returned by dwell_distfrmin()). -1 indicates a beginning dwell, -2 an ending dwell, and -3 a dwell that is both beginning and ending (whole burst dwell).

Parameters:: burst_num (np.ndarray[np.int64]) – Burst number of each dwell.
Returns:: burst_min – Burst position of each .
Return type:: np.ndarray[np.int8]

H2MMbursts.modeltables.dwell_distsince(burst_num: ndarray[int64]) → ndarray[int8]#

Find number of dwells since the last burst based on burst number of dwells.

Parameters:: burst_num (np.ndarray[np.int64]) – Burst number of each dwell.
Returns:: dwell_f – Number of dwells since last new burst.
Return type:: np.ndarray[np.int8]

H2MMbursts.modeltables.dwell_distuntil(burst_num)#

Find number of dwells until the next burst based on burst number of dwells.

Parameters:: burst_num (np.ndarray[np.int64]) – Burst number of each dwell.
Returns:: dwell_r – Number of dwells until next new burst.
Return type:: np.ndarray[np.int8]

H2MMbursts.modeltables.dwrite_h2mm_model(val: h2mm_model) → bytes#: Get bytes representation of hm.h2mm_model for writing to HDF5 file

H2MMbursts.modeltables.encode_h2mmmodel(val: h2mm_model) → tuple[bytes]#

Encode a H²MM model object as a tuple for packing into msgpack bytes object.

Parameters:: val (hm.h2mm_model) – H²MM model to be stored as tuple in msgpack bytes.
Returns:: single tuple containing the bytes representation of the H²MM model.
Return type:: tuple[bytes,]

H2MMbursts.modeltables.get_nanohist_thresh(nanos: ndarray[int64], states: ndarray[int8], discr: ndarray[float64], thresh: float, nstate: int, ln: int, nanosub: int) → ndarray[int64]#

Generate the 2D histogram of states x tcspc bin, given the photon nanotimes in nanos and the Viterbi state assignment in states, while filtering out photons whose likelihood (gamma) for the Viterbi state is less than thresh (if certainy is greater than or equal to the thresh, photon is included) Arguments ln and nstate are the expected number of tcspc bins and states respetively, used as not all bursts have nanotimes covering all tcspc bins or include all states.

Parameters:

nanos (np.ndarray[np.uint16]) – Nanotimes of burst.
states (np.ndarray[np.uint8]) – Viterbi assigned state of burst.
discr (np.ndarray[np.float64]) – Photons x states 2D array of likelihood for each state and each photon.
thresh (float) – Threshold of discr to include photon in histogram.
nstate (int) – Number of states.
ln (int) – Number of TCSPC bins.
nanosub (int) – Amount to subtract from nanos to set threshold

Returns:

2D histogram, organized (state, tcspc bin).

Return type:

np.ndarray[np.int64]

H2MMbursts.modeltables.get_nanohist_thresh_gamma(nanos: ndarray[int64], states: ndarray[int8], discr: ndarray[float64], thresh: float, nstate: int, ln: int, nanosub: int) → ndarray[int64]#

Generate the 2D histogram of states x tcspc bin, given the photon nanotimes in nanos and the Viterbi state assignment in states, while filtering out photons whose certainty of verterbi assigments (should either be posterior probability or path loglikelihood) is less than thresh (if certainy is greater than or equal to the thresh, photon is included) Arguments ln and nstate are the expected number of tcspc bins and states respetively, used as not all bursts have nanotimes covering all tcspc bins or include all states.

Parameters:

nanos (np.ndarray[np.uint16]) – Nanotimes of burst.
states (np.ndarray[np.uint8]) – Viterbi assigned state of burst.
discr (np.ndarray[np.float64]) – Statistical likelihood of each state assignement of Viterbi.
thresh (float) – Threshold of discr to include photon in histogram.
nstate (int) – Number of states.
ln (int) – Number of TCSPC bins.
nanosub (int) – Amount to subtract from nanos to set threshold

Returns:

2D histogram, organized (state, tcspc bin).

Return type:

np.ndarray[np.int64]

H2MMbursts.modeltables.infer_state(states: ndarray[int8], times: ndarray[int64]) → ndarray[int8]#

For a given state path where some photons are misssing, infer state of missing photons

Parameters:

states (np.ndarray[np.int8]) – State path with -1 values for all photons whose state is to be infered.
times (np.ndarray[np.int64]) – Arrival times of photons.

Returns:

out – Infered state path.

Return type:

np.ndarray[np.int8]

H2MMbursts.modeltables.make_h2mm_columndefs(skip: str | Sequence[str] = None) → tuple[ColumnDef, ...]#

Make tuple of columndefs for StatePathBase subclass. Defines the folloing columns:

indexpath

detpath, subclass should implement _get_detpath getter function

timepath

statepath

scalepath

pathllpath

pathll

pathllph

gamapath

ph_index

ph_h2mmtime

ph_state

ph_scale

ph_pathll

ph_gamma

bstates

eff_states

Parameters:: skip (str | Sequence[str], optional) – name(s) of ColumnDef s to skip. The default is None.
Returns:: tuple of ColumnDef for use in StatePathBase.column_defs.
Return type:: tuple[ColumnDef,…]

class H2MMbursts.modeltables.ntdivStatePath(param: Param, origin: DataSet)#

Table of statepath where photons are processed using the divisor. In addition to the standard StatePath parameters, it also takes the divs parameter, which defines the position of divisors in each stream.

With the inclusion of divisors, it is possible with ntdivStatePath to compute model nanomean values with StatePath.model_value(). StatePath.model_value() and StatePath.model_values() methods.

Params#

modelhm.h2mm_model
The hm.h2mm_model used in Viterbi processing.

streamstuple[PhSel, …]
tuple of PhSel defining the indexes of photons in H²MM processing.

divstuple[np.ndarray[np.int16],…]
tuple of arrays of positions of divisors per photon stream. Divisors set by raw nanotime. For each stream there will be 1 more indexes compared to the size of the cooresponding divs array

Parents#

burstsParam[BasePhotonTable]
Usually a :class:smfbursts.datamodel.tables.Bursts` smfbursts.datamodel.tables.Param, defining the time ranges of each “burst” in H²MM processing.

Columns#

Note that these are identical to StatePath

indexpathnp.ndarray[np.uint8] ()
Actual indexes used in H²MM processing. Each row is uint8 array.

detpathnp.ndarray[np.uint8] ()
Detector indexes of photons used in H²MM processing, not reassigned by streams. Each row is uint8 array.

timepathnp.ndarray[np.int64] ()
Actual times used in H²MM processing. Each row is int64 array.

statepathnp.ndarray[np.uint8] ()
Most likely states of each photon as processed in Viterbi algorithm. This is the direct output of Viterbi, no sub-selecting photons etc. Each row is 1d uint8 array

scalepathnp.ndarray[np.float64], ()
Posterior per photon of Viterbi processing. This is direct output of Viterbi. Each row is 1D float64 array.

pathllpathnp.ndarray[np.float64], ()
Log-likelihood of most likely state of each photon. This is direct output of Viterbi. Each row is 1D float64 array.

gammapathnp.ndarray[np.float64], ()
Gamma array, giving likelihood per-photon per-state. Each row is 2d float64 array, indexed [photon, state].

ph_indexnp.ndarray[np.int8], (phsel:PhSel, )
Indexes submitted to H²MM processing, mapped/masked by phsel.

ph_h2mmtimenp.ndarray[np.int64], (phsel:PhSel, )
Times of photons used in H²MM processing mapped/masked by phsel. Photons outside of model-streams receive value of -1. Rows are 1d int64 arrays.

ph_statenp.ndarray[np.int8], (phsel:PhSel, )
Viterbi state of each photon, mapped/masked by phsel. Photons outside of model-streams receive value of -1. Rows are 1d int8 arrays.

ph_scalenp.ndarray[np.float64], (phsel:PhSel, )
Posterior likelihood of each phton, mapped/masked by phsel. Photons outside of model-streams receive value of nan. Rows are 1d float64 arrays.

ph_llnp.ndarray[np.float64], (phsel:PhSel, )
log-likihood of state-assignment of photon for Viterbi path, mapped/masked by phsel. Photons outside of model-streams receive value of nan. Rows are 1d float64 arrays

ph_gammanp.ndarray[np.float64], (phsel:PhSel, )
Gamma array, giving likelihood per-photon per-state mapped/masked by phsel. Each row is 2d float64 array, indexed [photon, state].

bstates: int, ()
Bitcode indicating which states present in burst. If given bit position is present, then that state is present in burst, ie if states 0 and 2 are in a given burst, the the value is 0b00000101 = 5. Rows are int.

eff_stateint, (phsel:PhSel, )
“Effective” state of each photon in burst maped/masked by phsel. If a photon is in a stream not present in model streams, then infer state by nearest photon that is in streams. This is essentially ph_state with -1s replaced with an infered state. Rows are 1d int8 arrays.

classmethod get_ndet(param: Param) → int#: Get the number of detector indexes of a ntdivStatePath Param

model_streams#

Determine index(es) of phsel in the hm.h2mm_model based on a given param definition.

Parameters:

statepath (Param) – smfbursts.datamodel.tables.Param of type smfburts.datamodel.tables.StatePath with model.
phsel (PhSel) – Stream selection to map to model.
origin (PhotonData, optional) – If specified, the smfbursts.photondata.PhotonData object from which statepath is assumed to have been optimized. The default is None.
strict (bool, optional) – Whether to check if the phsel contains only streams specified in the streams param of statepath. The default is True.

Raises:

ValueError – phsel contains streams not used in statepath.

Returns:

Return type:

np.ndarray[np.int64]

param_defs: ClassVar[tuple[..., ParamDef]] = (<class 'smfbursts.datamodel.tables.ParamDef'> name = model type_validator = <class 'smfbursts.datamodel.immutabledata.TypeValidator'> type_ = <class 'H2MM_C.h2mm_model'> check = <function _check_h2mm_model at 0x7882a0863ba0> encoder = <function encode_h2mmmodel at 0x7882a0863ec0> decoder = <function decode_h2mmmodel at 0x7882a0863f60> write = functools.partial(<function write_byteslike at 0x7882b13271a0>, convert=<function dwrite_h2mm_model at 0x7882a0863e20>, title_func=<function _val_type_name at 0x7882b1327060>) node_prefix = None node_repr = None node_read = None node_check = None data_proc = <function _emptydict> ckwargs = {} validator = <function _return_first> validator_dict = {} required = True unit =, <class 'smfbursts.datamodel.tables.ParamDef'> name = streams type_validator = <class 'smfbursts.datamodel.immutabledata.TypeValidator'> type_ = <class 'tuple'> check = <function make_arbsequence.<locals>._check_arbsequence at 0x7882b1360d60> encoder = None decoder = None write = <function write_arbsequence> node_prefix = tuple node_repr = <function node_repr_arbsequence> node_read = functools.partial(<function node_read_arbsequence at 0x7882b1360c20>, <class 'tuple'>) node_check = <function node_check_arbsequence> data_proc = <function _emptydict> ckwargs = {'typedefs': <class 'smfbursts.datamodel.immutabledata.TypeValidator'> type_ = <class 'smfbursts.ph_sel.PhSel'> check = <function check_PhSel at 0x7882a07c7b00> encoder = <function encode_phsel at 0x7882a07d0900> decoder = <function decode_phsel at 0x7882a07d09a0> write = functools.partial(<function write_byteslike at 0x7882b13271a0>, convert=<function dwrite_PhSel at 0x7882a07d0720>, title_func=<function _val_type_name at 0x7882b1327060>) node_prefix = phsel node_repr = <function node_repr_PhSel> node_read = <function node_read_PhSel> node_check = None data_proc = <function _emptydict> ckwargs = {} validator = <function _return_first> validator_dict = {}} validator = <function _return_first at 0x7882b1325c60> validator_dict = {} required = True unit =, <class 'smfbursts.datamodel.tables.ParamDef'> name = divs type_validator = <class 'smfbursts.datamodel.immutabledata.TypeValidator'> type_ = <class 'tuple'> check = <function make_arbsequence.<locals>._check_arbsequence at 0x7882b1360d60> encoder = None decoder = None write = <function write_arbsequence> node_prefix = tuple node_repr = <function node_repr_arbsequence> node_read = functools.partial(<function node_read_arbsequence at 0x7882b1360c20>, <class 'tuple'>) node_check = <function node_check_arbsequence> data_proc = <function _emptydict> ckwargs = {'typedefs': <class 'smfbursts.datamodel.immutabledata.TypeValidator'> type_ = <class 'numpy.ndarray'> check = <function check_array> encoder = <function encode_ndarray> decoder = <function decode_ndarray> write = <function write_array> node_prefix = None node_repr = None node_read = None node_check = None data_proc = <function _emptydict> ckwargs = {'dtype': dtype('uint16'), 'superdtype': <class 'numpy.integer'>, 'mn': 0} validator = <function _return_first> validator_dict = {}} validator = <function _return_first> validator_dict = {} required = True unit =)#: meta private:

classmethod param_idx_to_det_map(params: dict[str, Any], detdef: DetDef) → ndarray[uint8]#

Returns 1D numpy array that should map h2mm index to detector index- ie idxmap[index] = detector where detector should match the ph_dets array

Parameters:

params (dict[str:Any]) – param dictionary definition for class, may omit keys not requied to compute the idx_to_det_map.
detdef (DetDef) – DetDef of data for which param is expected to be based. Needed to determing dets

Returns:

mapping of H2MM idx to det based on detdef.

Return type:

np.ndaray[np.uint8]

classmethod param_ndet(param: dict) → int#: Get number of parameters from Param of type ntdivStatePath

classmethod param_sort_process(params: dict[str, Any], detdef: DetDef)#

In a params dictonary, sort the streams so that streams are in ascending order based on DetDef

Parameters:

params (dict[str:Any]) – params dictionary to be used in ntdivStatePath based Param.
detdef (DetDef) – DetDef of expected Param.

Returns:

params (dict[str,Any]) – Sorted params dictoinary.
sort (np.ndarray[np.int64]) – Re-sorting array, value is original index, position is index for destination. Therefore new = old[sort]

classmethod sort_photons(origin: PhotonData | PhotonDataList, statepath: Param = None, bursts: Param = None, streams: Sequence[PhSel] = None, divs: Sequence[ndarray[uint16]] = None) → dict[slice(<class 'str'>, numpy.ndarray[numpy.ndarray], None)]#

Get divisor based inputs for H²MM optimization.

Parameters:

origin (PhotonDataS) – Data from which to sort photons.
statepath (Param, optional) – Definition of streams, if used cannot use bursts or streams kwargs. The default is None.
bursts (Param, optional) – bursts definition, must be used with streams, and . The default is None.
streams (Sequence[PhSel], optional) – Streams to inlcude in H²MM processing. The default is None.
divs (Sequence[np.ndarray[np.uint16]], optional) – Per stream divisor, specified in order of streams, 1 array per stream. Each element indicates a divisor threshold. The default is None.

Returns:

dict[str –

Contains at least the folling keys:

indexesnp.ndarray[np.ndarray[np.uint8]]
Photon indexes for H²MM processing.

timesnp.ndarray[np.ndarray[np.int64]]: Photon arrival times for H²MM processing

Return type:

np.ndarray[np.ndarray]

classmethod validate_param(param: Param)#: Validate valid parameter :meta private:

H2MMbursts.modeltables.sort_index_time_div(id_map: ndarray[int8], dets: ndarray[uint8], times: ndarray[int64], nanos: ndarray[uint16], divs: ndarray[uint16]) → tuple[ndarray[uint8], ndarray[int64], ndarray[bool]]#

Single burst sort function. Sort photons into function returns input for hm.h2mm_model.optimize which splits detectors by divisions from nanotime. Usually called internally by sort_indexes_times_divs()

Parameters:

id_map (np.ndarray[np.int8]) – Detector index map, maps dets to group.
dets (np.ndarray[np.uint8]) – Original detector indexes.
times (np.ndarray[np.int64]) – Times of photons.
nanos (np.ndarray[np.uint16]) – Nanotimes.
divs (np.ndarray[np.uint16]) – Per stream in id_map, divisions within nanotimes.

Returns:

ndets (np.ndarray[np.uint8]) – Indexes to submit to hm.h2mm_model.optimize().
times (np.ndarray[np.int64]) – Times to submit to hm.h2mm_model.optimize().
mask (np.ndarray[np.bool_])

H2MMbursts.modeltables.sort_indexes_times(table: BasePhotonTable, streams: Sequence[PhSel]) → tuple[ndarray[uint8], ndarray[int64]]#

Get numpy arrays of indices and times for H²MM analysis from a base table, and stream definition of streams (tuple of smfbursts.ph_sel.PhSel)

Parameters:

table (BasePhotonTable) – Base table defining the bursts over which to create index/time arrays.
streams (Sequence[PhSel]) – Streams in order of output index over which to produce index/time arrays.

Returns:

out_dets (np.ndarray[np.ndarray[np.uint8]]) – Array of array of index for each photon in each burst.
out_times (np.ndarray[np.ndarray[np.int64]]) – Array of arrays of times for each photon in each burst.

H2MMbursts.modeltables.sort_indexes_times_divs(table: BasePhotonTable, streams: Sequence[PhSel], divs: ndarray[object_]) → tuple[ndarray[ndarray[uint8]], ndarray[ndarray[int64]]]#

Retreive arrays of bursts information for optimization in H²MM from a BasePhotonTable using the divisor approach (Harris 2022 )

Parameters:

table (BasePhotonTable) – Table to retrieve H²MM inputs from.
streams (Sequence[PhSel]) – Streams to include as distinct indexes (further subdivided by divs) in output.
divs (np.ndarray[np.object_]) – Per photon stream, nanotime thresholds to use as divisions creating new index.

Returns:

out_dets (np.ndarray[np.ndarray[np.uint8]]) – Indexes to submit to hm.h2mm_model.optimize().
out_times (np.ndarray[np.ndarray[np.int64]]) – Times to submit to hm.h2mm_model.optimize().

H2MMbursts.modeltables.sort_usALEX_times_indexes(table: BasePhotonTable, phsels: Sequence[PhSel], shifts: Sequence[str]) → tuple[ndarray[ndarray[int8]], ndarray[ndarray[uint64]], ndarray[ndarray[int64]]]#

Generate time-shifted times, indexes, and sort order arrays for H²MM analysis of usALEX data. This function is used by usAlexStatePath to generate the underlying mapping of raw data to H²MM

Parameters:

table (BasePhotonTable) – BasePhotonTable that is the source of the burst data (defines time ranges to analyze).
phsels (Sequence[PhSel]) – Defines the phsel->|H2MM| index map, ie [phsel0, phsel1] map to [0, 1] in H²MM index
shifts (Sequence[str]) – Definition for how to shift each index in phsels, this is the .

Returns:

out_idx (np.ndarray[np.ndarray[np.int8]]) – Time shifted indexes for use in H²MM analysis.
out_times (np.ndarray[np.ndarray[np.int64]]) – Time shifted times for use in H²MM analysis.
out_sort (np.ndarray[np.ndarray[np.int64]]) – Map from H²MM shifted order to original order, ie out_idx[out_sort] = original_order_idx.

H2MMbursts.modeltables.time_even_c(times: ndarray[int64], timesper: ndarray[int64], timesmod: ndarray[int64], mask: ndarray[bool], ex_ref: int, ex_start: int, ex_end: int) → None#

Apply the contiguous even- style shift to photons in times Note that shift is applied inplace in times array.

Parameters:

times (np.ndarray[np.int64]) – Unshifted photon times.
timesper (np.ndarray[np.int64]) – Time of beginning of alex period of each photon.
timesmod (np.ndarray[np.int64]) – Time since beginning of alex period of each photon.
mask (np.ndarray[np.bool_]) – Photons to shift (matching det).
ex_ref (int) – Start of excitation period of source.
ex_start (int) – Start of destination excitation period.
ex_end (int) – Ratio of duration of destination excitation period to source excitation period.

H2MMbursts.modeltables.time_even_n(times: ndarray[int64], timesper: ndarray[int64], timesmod: ndarray[int64], mask: ndarray[bool], thresh: int, ex_startr: int, ex_endr: int, ex_startf: int, ex_endf: int) → None#

Apply the nearest even- style shift to photons in times Note that shift is applied inplace in times array.

Parameters:

times (np.ndarray[np.int64]) – Unshifted photon times.
timesper (np.ndarray[np.int64]) – Time of beginning of alex period of each photon.
timesmod (np.ndarray[np.int64]) – Time since beginning of alex period of each photon.
mask (np.ndarray[np.bool_]) – Photons to shift (matching det).
thresh (int) – Time of division between photons to shift backwards vs forwards.
ex_startr (int) – Start time of destination period for shifting backwards.
ex_endr (int) – End time of destination period for shifting backwards.
ex_startf (int) – Start time of destination period for shifting forwards.
ex_endf (int) – End time of desintation period for shifting forwards.

H2MMbursts.modeltables.time_rand_c(times: ndarray[int64], timesper: ndarray[int64], timesmod: ndarray[int64], mask: ndarray[bool], ex_start: int, ex_end: int, gen: Generator) → None#

Apply continguous random style shift to photons in times. Note that the shift is applied inplace in the times array.

Parameters:

times (np.ndarray[np.int64]) – Unshifted photon times.
timesper (np.ndarray[np.int64]) – Time of beginning of alex period of each photon.
timesmod (np.ndarray[np.int64]) – Time since beginning of alex period of each photon.
mask (np.ndarray[np.bool_]) – Photons to shift (matching det).
ex_start (int) – Start of destination excitation period.
ex_end (int) – Stop of destination excitation period.
gen (np.random.Generator) – random number generator.

H2MMbursts.modeltables.time_rand_n(times: ndarray[int64], timesper: ndarray[int64], timesmod: ndarray[int64], mask: ndarray[bool], thresh: int, ex_startr: int, ex_endr: int, ex_startf: int, ex_endf: int, gen: Generator) → None#

Apply nearest random style shift to photons in times. Note that the shift if applied inplace in the times array.

Parameters:

times (np.ndarray[np.int64]) – Unshifted photon times.
timesper (np.ndarray[np.int64]) – Time of beginning of alex period of each photon.
timesmod (np.ndarray[np.int64]) – Time since beginning of alex period of each photon.
mask (np.ndarray[np.bool_]) – Photons to shift (matching det).
thresh (int) – Time of division between photons to shift backwards vs forwards.
ex_startr (int) – Start time of destination period for shifting backwards.
ex_endr (int) – End time of destination period for shifting backwards.
ex_startf (int) – Start time of destination period for shifting forwards.
ex_endf (int) – End time of desintation period for shifting forwards.
gen (np.random.Generator) – random number generator.

H2MMbursts.modeltables.time_shift_c(times: ndarray[int64], timesper: ndarray[int64], timesmod: ndarray[int64], mask: ndarray[bool], ex_ref: int, ex_start: int, scale: float) → None#

Apply the contiguous shift- style shift to photons in times. Note that shift is applied inplace in times array.

Parameters:

times (np.ndarray[np.int64]) – Unshifted photon times.
timesper (np.ndarray[np.int64]) – Time of beginning of alex period of each photon.
timesmod (np.ndarray[np.int64]) – Time since beginning of alex period of each photon.
mask (np.ndarray[np.bool_]) – Photons to shift (matching det).
ex_ref (int) – Start of excitation period of source.
ex_start (int) – Start of destination excitation period.
scale (float) – Ratio of duration of destination excitation period to source excitation period.

H2MMbursts.modeltables.time_shift_n(times: ndarray[int64], timesper: ndarray[int64], timesmod: ndarray[int64], mask: ndarray[bool], ex_refb: int, ex_startb: int, scaleb: float, ex_reff: int, ex_startf: int, scalef: float) → None#

Apply the nearest shift- style shift to photons in times. Note that shift is applied inplace in times array.

Parameters:

times (np.ndarray[np.int64]) – Unshifted photon times.
timesper (np.ndarray[np.int64]) – Time of beginning of alex period of each photon.
timesmod (np.ndarray[np.int64]) – Time since beginning of alex period of each photon.
mask (np.ndarray[np.bool_]) – Time since beginning of alex period of each photon.
ex_refb (int) – Time of beginning of back shift source excitation period.
ex_startb (int) – Time of beginning of previous destination excitation period subsection.
scaleb (float) – Ratio of size of previous destination excitation period subsection to source excitation period subsection before division.
ex_reff (int) – Time in source excitation period to split between backward and forward shift.
ex_startf (int) – Time of beginnig of next destination excitation period subsection.
scalef (float) – Ratio of size of next destination excitation period subsection to source excitation periods subsection after division.

class H2MMbursts.modeltables.usAlexStatePath(param: Param, origin: DataSet)#

Viterbi StatePath for H²MM analysis where certain streams are shifted according to several available schemes so that excitiation periods overlap.

This is used primarily to implement the method in Harris et. al. 2022 for dealing with \(\mu s\) ALEX measurements.

The parameters are the same as StatePath with the addition of the shifts parameter. This is a tuple of the same length as streams which defines the function

For non-shifted streams, the code should be “base” For shifted photon streams, the string specifies 3 pieces of information

Direction of shift options are “n” and “c” for nearest, and contiguous, respectively.
Style of shift, may be “shift”, “even” or “rand0x<hex>”
Destination excitation period (an integer).

These are assembled as follows <direction><style>:<destination> So for a contiguous, even shift into excitation period 0, the string would be "ceven:0".

Regarding the direction of the shift

“c” (continguous) shift results in all photons from a given excitation period will be shifted into the same destination excitation period
“n” (nearest) shift results in photons being shifted to the temporally nearest destination excitation period.

Regarding the styles of shifts

“shift” is the simplest shifting technique, where an apporpirate linear shift is applied to each photon, so that spacing of shifted photons is proportionally not changed (a scaling factor is necessary to account for differences in the size of excitation periods)
“even” results in the even distribution of shifted photons into their destination period.
“rand0x<hex>” note that “<hex>” shoudl be replaced by a hexidecimal specification. Photon time are randomly distributed in the destination period. “<hex>” value is used as the seed for the random number generator generating the times. So long as numpy version is the same, this ensures reproducability of results. A random shift into excitation period 0 and using the nearest excitation period and a seed of 0x2f4c the string would be "nrand0x2f4c:0".

Params#

modelhm.h2mm_model
The hm.h2mm_model used in Viterbi processing.

streamstuple[PhSel, …]
tuple of PhSel defining the indexes of photons in H²MM processing.

shiftstuple[str, …]
String describing how to shift (which shift function to use) the given stream, tuple must be same length as streams.

Parents#

Note that these are the same as StatePath

burstsParam[BasePhotonTable]
Usually a :class:smfbursts.datamodel.tables.Bursts` smfbursts.datamodel.tables.Param, defining the time ranges of each “burst” in H²MM processing.

Columns#

Note that these are the same as StatePath

indexpathnp.ndarray[np.uint8], ()
Actual indexes used in H²MM processing. Each row is uint8 array.

detpathnp.ndarray[np.uint8], ()
Detector indexes of photons used in H²MM processing, not reassigned by streams. Each row is uint8 array.

timepathnp.ndarray[np.int64], ()
Actual times used in H²MM processing. Each row is int64 array.

statepathnp.ndarray[np.uint8], ()
Most likely states of each photon as processed in Viterbi algorithm. This is the direct output of Viterbi, no sub-selecting photons etc. Each row is 1d uint8 array

scalepathnp.ndarray[np.float64], ()
Posterior per photon of Viterbi processing. This is direct output of Viterbi. Each row is 1D float64 array.

pathllpathnp.ndarray[np.float64], ()
Log-likelihood of most likely state of each photon. This is direct output of Viterbi. Each row is 1D float64 array.

gammapathnp.ndarray[np.float64], ()
Gamma array, giving likelihood per-photon per-state. Each row is 2d float64 array, indexed [photon, state].

ph_indexnp.ndarray[np.ndarray[np.int8]], (phsel:PhSel, )
Indexes of photons used in H²MM processing, mappped/masked by phsel. Photons ouside of model-streams receive a value of -1. Each row is 1D int8.

ph_h2mmtimenp.ndarray[np.ndarray[np.int64]], (phsel:PhSel, )
Times of photons used in H²MM processing mapped/masked by phsel. Photons outside of model-streams receive value of -1. Rows are 1d int64 arrays.

ph_statenp.ndarray[np.ndarray[np.int8]], (phsel:PhSel, )
Viterbi state of each photon, mapped/masked by phsel. Photons outside of model-streams receive value of -1. Rows are 1d int8 arrays.

ph_scalenp.ndarray[np.ndarray[np.float64]], (phsel:PhSel, )
Posterior likelihood of each phton, mapped/masked by phsel. Photons outside of model-streams receive value of nan. Rows are 1d float64 arrays.

ph_llnp.ndarray[np.ndarray[np.float64]], (phsel:PhSel, )
log-likihood of state-assignment of photon for Viterbi path, mapped/masked by phsel. Photons outside of model-streams receive value of nan. Rows are 1d float64 arrays

ph_gammanp.ndarray[np.ndarray[np.float64]], (phsel:PhSel, )
Gamma array, giving likelihood per-photon per-state mapped/masked by phsel. Each row is 2d float64 array, indexed [photon, state].

bstates: ()
Bitcode indicating which states present in burst. If given bit position is present, then that state is present in burst, ie if states 0 and 2 are in a given burst, the the value is 0b00000101 = 5. Rows are int.

eff_state(phsel:PhSel, )
“Effective” state of each photon in burst maped/masked by phsel. If a photon is in a stream not present in model streams, then infer state by nearest photon that is in streams. This is essentially ph_state with -1s replaced with an infered state. Rows are 1d int8 arrays.

In a params dictonary, sort the streams so that streams are in ascending order based on DetDef

Parameters:

params (dict[str:Any]) – params dictionary to be used in StatePath based Param.
detdef (DetDef) – DetDef of expected Param.

Returns:

params (dict[str,Any]) – Sorted params dictoinary.
sort (np.ndarray[np.int64]) – Re-sorting array, value is original index, position is index for destination. Therefore new = old[sort]

phsel_select(phsel: PhSel, col: str, fill: Any, dtype: dtype) → ndarray[object_]#

Maps the of the inputs to H²MM evaluation to the “unprocessed” output shape.

Parameters:

phsel (PhSel) – A phsel object defining the output streams to return.
col (str) – Name of column being returned.
fill (Any) – Value to fill any photons that are in phsel but outside of phsel_span.
dtype (np.dtype) – Data-type of output array.

Returns:

If implemented should return object array of column maped to phsel

Return type:

np.ndarray[np.object_]

classmethod sort_photons(origin: PhotonData | PhotonDataList, statepath: Param = None, bursts: Param = None, streams: Sequence[PhSel] = None, shifts: Sequence[str] = None) → dict[slice(<class 'str'>, numpy.ndarray[numpy.ndarray], None)]#

Sort photons into indexes/times arrays for processing with H²MM.

Parameters:

origin (PhotonDataS) – Data from which to sort photons.
statepath (Param, optional) – Definition of streams, if used cannot use bursts or streams kwargs. The default is None.
bursts (Param, optional) – bursts definition, must be used with streams, and . The default is None.
streams (Sequence[PhSel], optional) – Streams to inlcude in H²MM processing. The default is None.
shifts (Sequence[str], optional) – How to shift each selected stream. The default is None.

Returns:

dict[str – Dictionary of sorted photons, each key contains a particular sort type. has the following keys:

indexes : np.ndarray[np.ndarray[np.uint8]] Photon indexes for H²MM processing.
times : np.ndarray[np.ndarray[np.int64]] Photon arrival times for H²MM processing

Return type:

np.ndarray[np.ndarray]]