Package 'EMNA'

Estimation of Multivariate Normal Algorithm

Description

An Estimation of Distribution Algorithm (EDA) that models a population using a multivariate normal distribution.

Usage

EMNA(fn, lower, upper, n, d, iter=100, eeea = TRUE,alpha=NULL,epsilon=NULL)

Arguments

fn	A function representing the objective function to be minimized. The function must receive a numeric vector which represents an individual and return a numeric fitness value.
lower	Lower bounds for the variables. A single numeric value can be provided and will be replicated across all d dimensions. Alternatively, a numeric vector of length d can be used.
upper	Upper bounds for the variables. A single numeric value can be provided and will be replicated across all d dimensions. Alternatively, a numeric vector of length d can be used.
n	The population size. Must be a positive integer indicating the number of individuals generated in each iteration.
d	The dimensionality of the optimization problem, corresponding to the number of variables in each individual.
iter	The number of generations or iterations of the algorithm, it must be provided, default value is based at 100.
eeea	Selecte whether to use EEEA (Explicit Exploration Strategy for Evolutionary Algorithms) to generate the initial population (TRUE or FALSE) or not, if EEEA is not used, then the population is generate by using an uniform distribution.
alpha	A convergence speed control parameter used a lineal combination of the new and old poblation whith alpha as parameter of porpotion. Higher values correspond to faster convergence. If this value is NULL, the algorithm proceeds using the standard EMNA method. alpha must be a value between (0,1). The default value is set at $\alpha = \code{NULL}$.
epsilon	A tolerance threshold parameter for distribution changes. It represents the maximum allowable limit for the Kullback-Leibler divergence $D_{KL}$ to determine if the distribution at the current step $i$ has not changed significantly compared to the previous step $i-1$. If this value is NULL, the algorithm proceeds using the standard EMNA method. The default value is set at $\epsilon = \code{NULL}$.

Value

par	Best population obtained by the estimation at the end.
value	Values of the best population obtained by the estimation at the end.

Note

If both alpha and epsilon are assigned a value, the function employs the slow-convergence method that utilizes alpha.

Author(s)

Rogelio Salinas Gutiérrez, Manuel Alonso Segoviano Baltazar, Jhon Daniel Aguilar Payares, Juan Alberto Dávila del Alto, Jonathan Michell Jauregui CarranzA, Hiram Efraim Macias Ruelas, Byron Axel Morales Gutiérrez, María Fernanda Nieto Guerrero, Pedro Abraham Montoya Calzada.

References

Larranaga, P. and Lozano, J. A. (2002). Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Kluwer Academic Publishers.

See Also

https://CRAN.R-project.org/package=EEEA

Examples

# Sphere objective function

sphere <- function(x) {
return(sum(x^2))
}

# Run EMNA using

set.seed(123)
result <- EMNA(
fn = sphere,
lower = -50,
upper = 50,
n = 20,
d = 2,
iter = 100,
eeea = FALSE,
alpha=NULL,
epsilon=2.5)

# Best individual
result$par
# Best objective function value
result$value

---

Kullback-Leibler Divergence Between Two Multivariate Normal Distributions

Description

Computes the Kullback-Leibler divergence between two multivariate normal distributions with given mean vectors and covariance matrices.

Usage

kldivergence(mu1, sigma1, mu2, sigma2)

Arguments

mu1	A numeric vector representing the mean of the first (reference) multivariate normal distribution.
sigma1	A symmetric positive-definite numeric matrix representing the covariance matrix of the first distribution. Its dimension must match the length of mu1.
mu2	A numeric vector representing the mean of the second (approximating) multivariate normal distribution. Its length must match the length of mu1.
sigma2	A symmetric positive-definite numeric matrix representing the covariance matrix of the second distribution. Its dimension must match the length of mu1.

Details

Given two multivariate normal distributions

$Q \sim N(\mu_1, \Sigma_1), \qquad P \sim N(\mu_2, \Sigma_2)$

of dimension $d$, the Kullback-Leibler divergence of $Q$ from $P$ is given by

$D_{KL}(Q \| P) = \frac{1}{2}\left[\mathrm{tr}(\Sigma_2^{-1}\Sigma_1) + (\mu_2 - \mu_1)^\top \Sigma_2^{-1}(\mu_2 - \mu_1) - d + \ln\frac{\det \Sigma_2}{\det \Sigma_1}\right].$

The function evaluates this expression directly from the provided means and covariances. The parameter $d$ is taken as the length of mu1.

The KL divergence is non-negative and equals zero if and only if $\mu_1 = \mu_2$ and $\Sigma_1 = \Sigma_2$. It is not symmetric, i.e., $D_{KL}(Q \| P) \neq D_{KL}(P \| Q)$ in general.

For the formula to be valid, both sigma1 and sigma2 must be symmetric and positive-definite, so that their inverses and determinants are well-defined.

Value

A numeric scalar giving the Kullback-Leibler divergence $D_{KL}(Q \| P)$.

Author(s)

Rogelio Salinas Gutiérrez, Manuel Alonso Segoviano Baltazar, Jhon Daniel Aguilar Payares, Juan Alberto Dávila del Alto, Jonathan Michell Jauregui CarranzA, Hiram Efraim Macias Ruelas, Byron Axel Morales Gutiérrez, María Fernanda Nieto Guerrero, Pedro Abraham Montoya Calzada.

References

Cover, T. M., & Thomas, J. A. (1991). Elements of Information Theory. John Wiley & Sons.

Examples

# Bivariate case

mu1 <- c(0, 0)
mu2 <- c(1, -1)
sigma1 <- matrix(c(1, 0.2, 0.2, 1), nrow = 2)
sigma2 <- matrix(c(2, 0.1, 0.1, 1.5), nrow = 2)

kldivergence(mu1, sigma1, mu2, sigma2)

# Multivariate case

mu1<-c(1,-1,2,0)
mu2<-c(0,0,1,2)
sigma1 <- matrix(c(3,0.5,0.2,0.1,0.5,2,0.3,0.2,0.2,0.3,4,0.5,0.1,0.2,0.5,1.5),4,4,byrow = TRUE)
sigma2 <- matrix(c(2,0.1,0.1,0,0.1,1.5,0.2,0.1,0.1,0.2,2.5,0.3,0,0.1,0.3,2),4,4,byrow = TRUE)

# kldivergence = 0

kldivergence(mu1,sigma1,mu1,sigma1)

# kldivergence is not symetric

kldivergence(mu1,sigma1,mu2,sigma2)
kldivergence(mu2,sigma2,mu1,sigma1)

---