Skip to content

metricviscomponent

metricviscomponent

Module for MetricVisComponent for the dashboard

Classes

MetricVisComponent 

MetricVisComponent(
    component_name=None,
    meta_function=None,
    target_value_list=None,
    assert_on_error=False,
)

Bases: ExpVisComponent

Dashboard ExpVisComponent to visualize all metrics in two columns (the right side starting with val_)

Source code in niceml/dashboard/components/expviscomponent.py 
21
22
23
24
25
26
27
28
29
30
31
32
33
def __init__(
    self,
    component_name: Optional[str] = None,
    meta_function: Optional[MetaFunction] = None,
    target_value_list: Optional[List[Any]] = None,
    assert_on_error: bool = False,
):
    # Create empty list for chart images
    self.component_name: Optional[str] = component_name
    self.chart_images_list: List[Image.Image] = []
    self.meta_function = meta_function
    self.target_value_list = [] if target_value_list is None else target_value_list
    self.assert_on_error = assert_on_error

Functions

generate_charts_for_metrics 

generate_charts_for_metrics(
    exp_ids, exp_manager, metric_list
)

Generates all charts for selected experiments and metrics and caches them

Parameters:

  • exp_ids (List[str]) –

    Experiment Ids to get the metrics charts for

  • exp_manager (ExperimentManager) –

    Experiment Manager to get the data from

  • metric_list (List[str]) –

    Metrics to get the charts of

Returns:

  • list –

    List of charts for each metric

Source code in niceml/dashboard/metricviscomponent.py 
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
def generate_charts_for_metrics(
    exp_ids: List[str], exp_manager: ExperimentManager, metric_list: List[str]
) -> list:
    """
    Generates all charts for selected experiments and metrics and caches them

    Args:
        exp_ids: Experiment Ids to get the metrics charts for
        exp_manager: Experiment Manager to get the data from
        metric_list: Metrics to get the charts of

    Returns:
        List of charts for each metric
    """

    @st.cache_data()
    def _get_metrics_charts(*args) -> list:  # pylint: disable = unused-argument
        """Generates all charts for selected experiments and metrics"""
        sorted_metrics = sort_metric_list(metric_list)
        chart_data_list = []
        for metric in sorted_metrics:
            cur_df = exp_manager.get_visu_df(metric, list(exp_ids))
            if cur_df is None:
                continue
            chart_data = generate_chart(cur_df, metric)
            chart_data_list.append(chart_data)
        return chart_data_list

    return _get_metrics_charts(exp_ids, metric_list)

sort_metric_list 

sort_metric_list(metric_list)

Sorts the metric list so that validation metrics are together with train metrics.

Parameters:

  • metric_list (List[str]) –

    List[str]: list of metrics

Returns:

  • List[str] –

    A list of strings where the validation metrics are together with the train metrics

Source code in niceml/dashboard/metricviscomponent.py 
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
def sort_metric_list(metric_list: List[str]) -> List[str]:
    """
    Sorts the metric list so that validation metrics are together with train metrics.

    Args:
        metric_list: List[str]: list of metrics

    Returns:
        A list of strings where the validation metrics are together with the train metrics

    """

    train_metrics = sorted([x for x in metric_list if not x.startswith("val_")])
    sorted_metrics = []
    for met in train_metrics:
        sorted_metrics.append(met)
        sorted_metrics.append(f"val_{met}")
    return sorted_metrics