pyswarms.utils.search package¶
The pyswarms.utils.search
module implements various techniques in
hyperparameter value optimization.
pyswarms.utils.search.base_search module¶
Base class for hyperparameter optimization search functions
-
class
pyswarms.utils.search.base_search.
SearchBase
(optimizer, n_particles, dimensions, options, objective_func, iters, bounds=None, velocity_clamp=(0, 1))[source]¶ Bases:
object
-
__init__
(optimizer, n_particles, dimensions, options, objective_func, iters, bounds=None, velocity_clamp=(0, 1))[source]¶ Initialize the Search
-
optimizer
¶ pyswarms.single – either LocalBestPSO or GlobalBestPSO
-
n_particles
¶ int – number of particles in the swarm.
-
dimensions
¶ int – number of dimensions in the space.
-
options
¶ dict with keys
{'c1', 'c2', 'w', 'k', 'p'}
– a dictionary containing the parameters for the specific optimization technique- c1 : float
- cognitive parameter
- c2 : float
- social parameter
- w : float
- inertia parameter
- k : int
- number of neighbors to be considered. Must be a
positive integer less than
n_particles
- p: int {1,2}
- the Minkowski p-norm to use. 1 is the sum-of-absolute values (or L1 distance) while 2 is the Euclidean (or L2) distance.
-
objective_func
¶ function – objective function to be evaluated
-
iters
¶ int – number of iterations
-
bounds
¶ tuple of np.ndarray, optional (default is None) – a tuple of size 2 where the first entry is the minimum bound while the second entry is the maximum bound. Each array must be of shape
(dimensions,)
.
-
velocity_clamp
¶ tuple (default is
None
) – a tuple of size 2 where the first entry is the minimum velocity and the second entry is the maximum velocity. It sets the limits for velocity clamping.
-
-
assertions
()[source]¶ Assertion method to check
optimizer
inputRaises: TypeError
– Whenoptimizer
does not have an ‘optimize’ attribute.
-
generate_score
(options)[source]¶ Generate score for optimizer’s performance on objective function
Parameters: options (dict) – a dict with the following keys: {‘c1’, ‘c2’, ‘w’, ‘k’, ‘p’}
-
search
(maximum=False)[source]¶ Compare optimizer’s objective function performance scores for all combinations of provided parameters
Parameters: maximum (bool) – a bool defaulting to False, returning the minimum value for the objective function. If set to True, will return the maximum value for the objective function.
-
pyswarms.utils.search.grid_search module¶
Hyperparameter grid search.
Compares the relative performance of hyperparameter value combinations in optimizing a specified objective function.
For each hyperparameter, user can provide either a single value or a list of possible values. The cartesian products of these hyperparameters are taken to produce a grid of all possible combinations. These combinations are then tested to produce a list of objective function scores. The search method default returns the minimum objective function score and hyperparameters that yield the minimum score, yet maximum score can also be evaluated.
>>> options = {'c1': [1, 2, 3],
'c2': [1, 2, 3],
'w' : [2, 3, 5],
'k' : [5, 10, 15],
'p' : 1}
>>> g = GridSearch(LocalBestPSO, n_particles=40, dimensions=20,
options=options, objective_func=sphere, iters=10)
>>> best_score, best_options = g.search()
>>> best_score
0.498641604188
>>> best_options['c1']
1
>>> best_options['c2']
1
-
class
pyswarms.utils.search.grid_search.
GridSearch
(optimizer, n_particles, dimensions, options, objective_func, iters, bounds=None, velocity_clamp=(0, 1))[source]¶ Bases:
pyswarms.utils.search.base_search.SearchBase
Exhaustive search of optimal performance on selected objective function over all combinations of specified hyperparameter values.
pyswarms.utils.search.random_search module¶
Hyperparameter random search.
Compares the relative performance of combinations of randomly generated hyperparameter values in optimizing a specified objective function.
User provides lists of bounds for the uniform random value generation of ‘c1’, ‘c2’, and ‘w’, and the random integer value generation of ‘k’. Combinations of values are generated for the number of iterations specified, and the generated grid of combinations is used in the search method to find the optimal parameters for the objective function. The search method default returns the minimum objective function score and hyperparameters that yield the minimum score, yet maximum score can also be evaluated.
>>> options = {'c1': [1, 5],
'c2': [6, 10],
'w' : [2, 5],
'k' : [11, 15],
'p' : 1}
>>> g = RandomSearch(LocalBestPSO, n_particles=40, dimensions=20,
options=options, objective_func=sphere, iters=10)
>>> best_score, best_options = g.search()
>>> best_score
1.41978545901
>>> best_options['c1']
1.543556887693
>>> best_options['c2']
9.504769054771
-
class
pyswarms.utils.search.random_search.
RandomSearch
(optimizer, n_particles, dimensions, options, objective_func, iters, n_selection_iters, bounds=None, velocity_clamp=(0, 1))[source]¶ Bases:
pyswarms.utils.search.base_search.SearchBase
Search of optimal performance on selected objective function over combinations of randomly selected hyperparameter values within specified bounds for specified number of selection iterations.
-
__init__
(optimizer, n_particles, dimensions, options, objective_func, iters, n_selection_iters, bounds=None, velocity_clamp=(0, 1))[source]¶ Initialize the Search
-
n_selection_iters
¶ int – number of iterations of random parameter selection
-
-