OpenADMET Models documentation

openadmet-models

Binary Evaluation Metrics

Posthoc binary metrics evaluation.

class openadmet.models.eval.binary.PosthocBinaryMetrics(*, n_resamples: int = 9999, **extra_data: Any)

Bases: EvalBase

Posthoc binary metrics.

Intended to be used for regression-based models to calculate precision and recall metrics for user-input cutoffs

Not intended for binary models

evaluate(y_true: list = None, y_pred: list = None, cutoff: float = None, report: bool = False, output_dir: str = None)

Evaluate the precision and recall metrics for the model with user-input cutoffs.

Parameters:

• y_true (array-like) – True values or labels.

• y_pred (array-like) – Predicted values or labels.

• cutoff (float, optional) – Cutoff value to calculate precision and recall.

• report (bool, optional) – Whether to save JSON files of the resulting precision/recall metrics. Default is False.

• output_dir (str, optional) – Directory to save the output plots and report. Default is None.

Raises:

ValueError – If y_true or y_pred is not provided.

get_precision_recall(y_pred: list, y_true: list, cutoff: float)

Calculate precision and recall metrics for a given cutoff.

Parameters:

• y_pred (array-like) – Predicted values.

• y_true (array-like) – True values.

• cutoff (float) – Cutoff value to calculate precision and recall.

Returns:

A tuple containing: - precision : float

Precision value.

• recallfloat

  Recall value.

Return type:

tuple

model_config: ClassVar[ConfigDict] = {'extra': 'allow'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

plot_confusion_matrix(y_true: list, y_pred: list, cutoff: float, output_dir: str = None)

Plot the confusion matrix for a given cutoff.

Parameters:

• y_true (list or array-like) – True values or labels.

• y_pred (list or array-like) – Predicted values or labels.

• cutoff (float) – Cutoff value to binarize predictions and true values.

• output_dir (str, optional) – Directory to save the confusion matrix plot. If None, the plot is not saved.

plot_posthoc_classification(y_true: list, y_pred: list, cutoff: float, output_dir: str = None)

Plot the post-hoc classification scatter plot with cutoff lines.

Parameters:

• y_true (list or array-like) – True values or labels.

• y_pred (list or array-like) – Predicted values or labels.

• cutoff (float) – Cutoff value to draw threshold lines.

• output_dir (str, optional) – Directory to save the classification plot. If None, the plot is not saved.

report(write=False, output_dir=None, precision=None, recall=None)

Report the evaluation results, optionally saving them to JSON.

Parameters:

• write (bool, optional) – Whether to write the results to a JSON file. Default is False.

• output_dir (str, optional) – Directory to save the JSON file if write is True.

• precision (float or array-like, optional) – Precision value(s) to report.

• recall (float or array-like, optional) – Recall value(s) to report.

stats_to_json(data_df, output_dir)

Save the precision-recall DataFrame to a JSON file.

Parameters:

• data_df (pandas.DataFrame) – DataFrame containing precision and recall metrics.

• output_dir (str) – Directory to save the JSON file.