Regression Metrics

Regression metrics and plots for model evaluation.

class openadmet.models.eval.regression.RegressionMetrics(*, n_resamples: int = 9999, bootstrap_confidence_level: float = 0.95, use_wandb: bool = False, pXC50: bool = False, **extra_data: Any)[source]

Bases: EvalBase

Compute and report regression metrics such as MSE, MAE, R2, Kendall’s tau, and Spearman’s rho.

Variables:

  • bootstrap_confidence_level (float) – Confidence level for the bootstrap.

  • use_wandb (bool) – Whether to use wandb for logging.

  • _evaluated (bool) – Whether the model has been evaluated.

  • _metrics (dict) – Dictionary of metrics to compute.

property active_metrics

Return metrics applicable to the current target scale.

bootstrap_confidence_level: float
evaluate(y_true=None, y_pred=None, use_wandb=False, tag=None, target_labels=None, **kwargs)[source]

Evaluate the regression model and compute metrics.

Parameters:

  • y_true (array-like) – True values.

  • y_pred (array-like) – Predicted values.

  • use_wandb (bool, optional) – Whether to log metrics to Weights & Biases.

  • tag (str, optional) – Tag for the evaluation run.

  • target_labels (list of str, optional) – List of target names.

  • kwargs (Dict) – Additional keyword arguments.

Returns:

Dictionary of computed metrics and confidence intervals.

Return type:

dict

get_stat_caption(t_label)[source]

Get a formatted statistics caption for a given task.

Parameters:

t_label (str) – Task label.

Returns:

Caption string with statistics.

Return type:

str

get_stat_dict(t_label)[source]

Get a statistics dictionary for a given task.

Parameters:

t_label (str) – Task label.

Returns:

Dictionary of statistics for the task.

Return type:

dict

property metric_names

Return the metric names.

Returns:

List of metric names.

Return type:

list of str

model_config: ClassVar[ConfigDict] = {'extra': 'allow'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Args:

self: The BaseModel instance. context: The context.

pXC50: bool
report(write=False, output_dir=None)[source]

Report the evaluation results, optionally writing to disk.

Parameters:

  • write (bool, optional) – Whether to write the report to disk.

  • output_dir (str, optional) – Output directory for the report.

Returns:

Dictionary of computed metrics.

Return type:

dict

property task_names

Return the task names.

Returns:

List of task names.

Return type:

list of str

use_wandb: bool
write_report(output_dir)[source]

Write the evaluation report to a JSON file and optionally log to wandb.

Parameters:

output_dir (str) – Output directory for the report.

class openadmet.models.eval.regression.RegressionPlots(*, n_resamples: int = 9999, axes_labels: list[str] = ['Measured', 'Predicted'], title: str = 'Pred vs ', do_stats: bool = True, pXC50: bool = False, plot_errbars: bool = False, plots: dict = {}, min_val: float = None, max_val: float = None, fit_reg: bool = False, use_wandb: bool = False, dpi: int = 300, **extra_data: Any)[source]

Bases: EvalBase

Generate and save regression plots such as regression scatter plots and confidence interval plots.

Variables:

  • axes_labels (list of str) – Labels for the axes.

  • title (str) – Title for the plot.

  • do_stats (bool) – Whether to compute and display statistics on the plots.

  • pXC50 (bool) – Whether to highlight pXC50 log unit ranges.

  • plot_errbars (bool) – Whether to plot error bars for ensemble predictions.

  • plots (dict) – Dictionary of plot functions.

  • min_val (float) – Minimum value for the axes.

  • max_val (float) – Maximum value for the axes.

  • use_wandb (bool) – Whether to use wandb for logging.

  • dpi (int) – DPI for the plot.

axes_labels: list[str]
static ciplot(stat_dict={})[source]

Create a confidence interval plot for regression metrics.

Parameters:

stat_dict (dict) – Dictionary containing metrics, means, confidence intervals, and task name.

Returns:

The confidence interval plot figure.

Return type:

matplotlib.figure.Figure

do_stats: bool
dpi: int
evaluate(y_true=None, y_pred=None, y_std=None, use_wandb=False, target_labels=None, **kwargs)[source]

Generate regression plots and optionally compute statistics.

Parameters:

  • y_true (array-like) – True values.

  • y_pred (array-like) – Predicted values.

  • y_std (array-like) – Standard deviation of predictions if ensemble is specified.

  • use_wandb (bool, optional) – Whether to log plots to Weights & Biases.

  • target_labels (list of str, optional) – List of target names.

  • kwargs (Dict) – Additional keyword arguments.

Returns:

Dictionary of plot figures.

Return type:

dict

fit_reg: bool
max_val: float
min_val: float
model_config: ClassVar[ConfigDict] = {'extra': 'allow'}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

pXC50: bool
plot_errbars: bool
plots: dict
static regplot(y_true, y_pred, y_pred_err=None, y_true_err=None, data_labels=None, xlabel='Measured', ylabel='Predicted', title='', stat_dict={}, confidence_level=0.95, pXC50=False, min_val=None, max_val=None, fit_reg=False, plot_errbars=False)[source]

Create a regression scatter plot with optional confidence intervals and statistics table.

Parameters:

  • y_true (array-like) – True values.

  • y_pred (array-like) – Predicted values.

  • y_pred_err (array-like, optional) – Prediction error bars.

  • y_true_err (array-like, optional) – Experimental error bars.

  • data_labels (list, optional) – Labels for each data point.

  • xlabel (str, optional) – Label for the x-axis.

  • ylabel (str, optional) – Label for the y-axis.

  • title (str, optional) – Title for the plot.

  • stat_dict (dict, optional) – Dictionary of statistics to display on the plot.

  • confidence_level (float, optional) – Confidence level for the regression line.

  • pXC50 (bool, optional) – Whether to highlight pXC50 log unit ranges.

  • min_val (float, optional) – Minimum axis value.

  • max_val (float, optional) – Maximum axis value.

  • fit_reg (bool, optional) – Whether to fit and plot a regression line.

  • plot_errbars (bool, optional) – Whether to plot model error bars from ensemble predictions.

Returns:

The regression plot object.

Return type:

seaborn.axisgrid.JointGrid

report(write=False, output_dir=None)[source]

Report the generated plots, optionally writing to disk.

Parameters:

  • write (bool, optional) – Whether to write the plots to disk.

  • output_dir (str, optional) – Output directory for the plots.

Returns:

Dictionary of plot figures.

Return type:

dict

title: str
use_wandb: bool
write_report(output_dir)[source]

Write the generated plots to PNG files and optionally log to wandb.

Parameters:

output_dir (str) – Output directory for the plots.

openadmet.models.eval.regression.pct_within_1_log_unit(y_true, y_pred)[source]

Compute the fraction of predictions within +/-1 log unit of the true value.

Parameters:

  • y_true (array-like) – True values (assumed to be on a log scale, e.g. pXC50).

  • y_pred (array-like) – Predicted values.

Returns:

Fraction (0-1) of predictions within 1 log unit.

Return type:

float

openadmet.models.eval.regression.relative_absolute_error(y_true, y_pred)[source]

Compute Relative Absolute Error (RAE).

RAE = sum(|y_true - y_pred|) / sum(|y_true - mean(y_true)|). Lower is better; RAE < 1.0 means the model outperforms a naive mean predictor.

Parameters:

  • y_true (array-like) – True values.

  • y_pred (array-like) – Predicted values.

Returns:

Relative absolute error.

Return type:

float