Learning strategy submodule#

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Strategies for the learning rate evolution which can be used in the SOM.

How to include a custom learning strategy ?#

To implement a custom learning strategy, please inherit from LearningStrategy and implement your own LearningStrategy.__init__ and LearningStrategy.__call__ methods as follows

class NewLearningStrategy(LearningStrategy):

   def __init__(self, lr: Union[int, float]=1, **kwargs) -> None:

      super().__init__(lr)

      ...

   def __call__(step, *args, **kwargs) -> float:

      ...

      return learning_rate

The LearningStrategy.__call__ method must always have step as its first argument.

Note

Additional arguments with *args and **kwargs can be present in LearningStrategy.__call__ method but should not be used.

By default, the super().__init__(lr) line will initialise the initial learning rate (self.initial_lr) and the total number of iterations. (self.ntot). Note that the SOM will automatically set self.ntot when starting the fitting procedure. So if your strategy requires to know the total number of iterations, you can directly use self.ntot without having to set its value by hand as the SOM will do it for you.

API#

class SOMptimised.learning_rate.ExponentialLearningStrategy(lr: Union[int, float] = 1, tau: Union[int, float] = 1, **kwargs)[source]#

Bases: SOMptimised.learning_rate.LearningStrategy

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Class implementing an exponential learning rate strategy given by

\[\eta \times \exp \left \lbrace - t / \tau \right \rbrace,\]

where \(\eta\) is the initial learning rate, \(t\) is the time step and \(\tau\) is the decay time scale.

Parameters

  • lr (float) – (Optional) intial learning rate

  • tau (float) – (Optional) decay time scale

__call__(step: int, *args, **kwargs) → float[source]#

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Provide the learning rate at the given step.

Parameters

step (int) – iteration step

property tau: Union[int, float]#

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Provide the value of the decay time scale.

Returns

Decay time scale

Return type

int or float

class SOMptimised.learning_rate.LearningStrategy(lr: Union[int, float] = 1)[source]#

Bases: abc.ABC

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Abstract class for learning rate strategies.

Parameters

lr (int or float) – (Optional) intial learning rate

abstract __call__(step: int, *args, **kwargs) → float[source]#

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Provide the learning rate at the given step.

Parameters

step (int) – time step during the learning process

Returns

Learning rate at the given time step

Return type

float

static _check_int_float(param: Any, name: str, *args, **kwargs) → None[source]#

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Check if a parameter is int or float.

Parameters

  • param – value to check

  • name – name of the parameter

Type

str

Raises

TypeError – if not isinstance(param, (int, float))

_check_lr(value, name='lr', **kwargs) → None[source]#

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Check if a value is acceptable for the learning rate, that is, either a positive int or a positive float.

Parameters

  • value – value to check

  • name – (Optional) name of the parameter

Type

str

Raises

  • TypeError – if not isinstance(value, (int, float))

  • ValueError – if lr < 0

static _check_negative(param: Any, name: str, *args, **kwargs) → None[source]#

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Check if a parameter is negative.

Parameters

  • param – value to check

  • name – name of the parameter

Type

str

Raises

ValueError – if lr < 0

property initial_lr: Union[int, float]#

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Provide the value of the initial learning rate.

Returns

Initial learning rate

Return type

int or float

property ntot: int#

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Provide the value of the total number of iterations.

Returns

Total number of iterations

Return type

int

class SOMptimised.learning_rate.LinearLearningStrategy(lr: Union[int, float] = 1, **kwargs)[source]#

Bases: SOMptimised.learning_rate.LearningStrategy

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Class implementing a linear learning rate strategy given by

\[\eta \times \left ( 1 - t / t_\rm{tot} \right ),\]

where \(\eta\) is the initial learning rate, \(t\) is the time step and \(t_{\rm{tot}}\) is the total number of iterations during the learning process.

Parameters

  • lr (int or float) – (Optional) intial learning rate

  • ntot (int) – (Optional) maximum number of iterations

__call__(step: int) → float[source]#

Code author: Wilfried Mercier - IRAP <wilfried.mercier@irap.omp.eu>

Provide the learning rate at the given step.

Parameters

step (int) – iteration step

Returns

Learning rate at the given step

Return type

float