Skip to content

uniformdistributionshuffler

uniformdistributionshuffler

Module for the UniformDistributionShuffler and helper methods

Classes

UniformDistributionShuffler 

UniformDistributionShuffler(class_attr, mode='max')

Bases: DataShuffler

A shuffler which generates uniform distributed indexes

Parameters

class_attr: str Classttribute name of the datainfo mode: str How the target amount of each class should be calculated such that they are evenly distributed (max, min, avg)

Source code in niceml/data/datashuffler/uniformdistributionshuffler.py 
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
def __init__(self, class_attr: str, mode: str = "max"):
    """
    A shuffler which generates uniform distributed indexes

    Parameters
    ----------
    class_attr: str
        Classttribute name of the datainfo
    mode: str
        How the target amount of each class should be calculated
        such that they are evenly distributed (max, min, avg)
    """
    check_mode(mode)
    self.class_attr = class_attr
    self.mode = mode
Functions

Functions

check_mode 

check_mode(mode)

Checks if mode is available otherwise raises ModeNotImplementedError

Source code in niceml/data/datashuffler/uniformdistributionshuffler.py 
18
19
20
21
22
23
def check_mode(mode: str):
    """Checks if mode is available otherwise raises ModeNotImplementedError"""
    if mode not in MODE_DICT:
        mode_list: str = ",".join(MODE_DICT.keys())
        message: str = f"{mode} not available, please use: {mode_list}"
        raise ModeNotImplementedError(message)

classdict_to_indexes 

classdict_to_indexes(class_dict, mode)

Uses the class dict to return a list of indexes

Parameters

class_dict: Dict[Any, List[int]] Contains for each class the list of indexes referring to it mode: str How the target amount of each class should be calculated such that they are evenly distributed (max, min, avg)

Returns
A shuffled list of indexes (each index can occur multiple times)
Source code in niceml/data/datashuffler/uniformdistributionshuffler.py 
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
def classdict_to_indexes(class_dict: Dict[Any, List[int]], mode: str) -> List[int]:
    """
    Uses the class dict to return a list of indexes

    Parameters
    ----------
    class_dict: Dict[Any, List[int]]
        Contains for each class the list of indexes referring to it
    mode: str
        How the target amount of each class should be calculated
        such that they are evenly distributed (max, min, avg)

    Returns
    -------
        A shuffled list of indexes (each index can occur multiple times)
    """
    check_mode(mode)
    class_count_dict = {x: len(y) for x, y in class_dict.items()}
    cur_mode = MODE_DICT[mode]
    max_count = int(cur_mode(list(class_count_dict.values())))
    out_list: List[int] = []
    for class_list in class_dict.values():
        count, parts = divmod(max_count, len(class_list))
        out_list += class_list * count
        out_list += sample(class_list, parts)
    shuffle(out_list)
    return out_list