Hybrid Genetic Algorithm and Simulated Annealing for Function Optimization

. The optimization problems on real-world usually have non-linear characteristics. Solving non-linear problems is time-consuming. Thus heuristic approaches usually are being used to speed up the solution’s searching. Among of the heuristic-based algorithms, Genetic Algorithm (GA) and Simulated Annealing (SA) are two among most popular. The GA is powerful to get a nearly optimal solution on the broad searching area while SA is useful to looking for a solution in the narrow searching area. This study is comparing performance between GA, SA, and three types of Hybrid GA-SA to solve some non-linear optimization cases. The study shows that Hybrid GA-SA can enhance GA and SA to provide a better result.


Introduction
In recent years, Evolutionary Algorithm (EA) has already become an important tool to solve optimization problem such as multi-objective problems.This kind of problem sought to look for an optimal solution or nearest optimal solution to be achieved.However, sometimes that solution is hard to obtain if we use a common mathematical approach which may lead into local optimum only.To solve this kind of problems, we can use a heuristic algorithm such as Genetic Algorithm (GA) [1].
The genetic algorithm (GA) is a search algorithm that emulates the heredity and advancement of biology in a common habitat with the attributes of solid parallels, randomness, and self-adaption likelihood.The hypothetical establishments of the GA are the Darwinian biological evolution standard and Mendel's heredity hypothesis.The GA controls the search procedure utilizing existing data, merging to ideal or acceptable solutions by evaluating the fitness of chromosomes and also performing the selection, crossover, mutation and other genetic operations [2,3].
Simulated annealing (SA) is a single objective combinatorial improvement algorithm utilizing an analogy to the measurable mechanics of tempering in solids.It was presented in 1982 by Kirkpatrick et al. [4].Annealing defines a procedure in material science where a solid is warmed to a temperature that grants various atomic adjustments.The solid is then gradually cooled in a controlled procedure until the solid crystallizes [5].It is easy for SA to be modified, one of them is using functions or operators that are produced particularly according to the problem which used to generate the neighbor solution [6].
The SA can adequately escape local optima and meet to the global optima by joining a probabilistic hopping property with changing time qualities and an inclination to converge to zero in the inquiry procedure.The benefit of the GA and SA is that they are not restricted by the coherence or differentiability of the optimization issues.Thus, they are reasonable for complicated and nonlinear issues that conventional search strategies cannot solve.Nevertheless, the GA has poor nearby pursuit capacity, and the SA has low proficiency [7].
To overcome the both shortcomings of GA and SA, there are several studies proposed the hybrid between GA and SA.Junghans & Darde [5] compare between GA and hybrid GA with modified SA (MSA).The SA used in their experiment has been modified which control the reduction of the temperature.They found that hybrid GA-MSA provides higher reliability than GA.
Another research conducted by Chen & Shahandashti [8] which also compare the GA, SA, a hybrid of GA-SA, and MSA.They found that hybrid GA-SA is outperformed the GA, SA, and MSA.However, they ensure the fairness through the same total number of iterations, which is set to 10,000.This setting will lead to the unfairness as the GA is a population-based meta-heuristics algorithm and SA is a non-population-based metaheuristics algorithm.
In this paper, we hybrid the GA-SA with three scenarios, namely hybrid GA-SA, hybrid SA-GA, and cyclic GA-SA.The hybrid GA-SA process is to get the best solution in GA and use it as the initial population in SA.Otherwise, the hybrid SA-GA process is using the result from SA as the initial population for GA.While the cyclic GA-SA process is to randomly select the individual for interval 1000 iteration and improved by the SA, then processed again using GA.

Genetic Algorithm (GA)
Genetic Algorithms (GA) depend on the mechanics of natural selection and natural hereditary qualities [3].The GA works with an initial population of a series of variables known as chromosomes which hold the parameters or qualities and the measure of population.There are three operators specifically, selection, crossover, and mutation to create new population from the old populace.In the selection, an arrangement of chromosomes is chosen as initial parents at the generation stage based on their fitness.The fittest are given a more prominent chance of survival and also greater probability of reproducing more off-springs.The way toward mating is executed through the crossover.Mutation, a discretionary change of the genes, is actualized to safeguard the differing genetic qualities in the population.Mutation occurrence probability can be kept low as it can disturb the best solution [9,10].The process of GA can be seen in Figure 1.

Fig. 1. The GA process
The GA will be processed some individuals, and they do crossover among individuals then mutate every child, result of the previous crossover.By calculating fitness among the children and the parents, the most 'elite' individual will be selected as new population, replace the old.This new population will be through crossover, mutation, and selection again until the stopping criteria met.

Simulated Annealing
Simulated Annealing is a local search algorithm that simulates the melting and cooling in metal processing.It has a variable initial temperature that set very high and gradually cooling by time to time [11].Simulated Annealing usually implemented to search optimal solution on short range area even it sometimes also performs equally or even better than GA in some cases like research conducted by [12] to solve 8-Queens Problem or research by [13] that is looking for optimal reservoir operation by using GA and SA.This crystallization frames an assortment of rigid, and, subsequently, mechanically enhanced structures.The SA optimization algorithm utilizes a closely resembling "controlled cooling" handle for nonphysical optimization issues.In the improvement issue, the adjustment of the low vitality bonds at the warming stages is the fitness function of the optimization issue [5].
The controlled cooling procedure is the improvement procedure itself.In every emphasis step, the temperature is expected to diminish so that a low vitality obligation of the molecules is accepted.Inside the emphasis step, the algorithm seeks the neighbors of the present solution for a preferred arrangement.The iterative process is rehashed until a more proper arrangement is found [5].The process of SA can be seen in Figure 2.

Fig. 2. The SA process
The SA is different from GA. SA have only one solution's candidate and will be looking for its neighbors to achieve an optimum solution.How long SA will be searching the optimum solution depends on the temperature.This initial temperature will be gradually down over time and after the final temperature bound is being reached, the process will be stopped.

Hybrid GA-SA
A hybrid between GA and SA may reach balanced the power of algorithms to let GA explore huge search of space and let SA exploit local search areas.While the standard SA typically begins with random initial parameter configuration of parameter settings, in the proposed solution, the best solution of the GA is utilized as the initial configuration in SA.This combination already studied by [14] which proved that Hybrid GA-SA outperforms common GA and SA.Another study by [15] shows that GA-SA hybrid can solve Permutation Flowshop Scheduling Problem.
Another research which utilizes local search-based algorithm conducted by [16].The researchers implements hybridization between GA to explore huge search space and Variable Neighborhood Search (VNS) to exploit local search areas.They found that the hybridization delivers better outcomes comparable with the sequential approach.
For the sake of simplicity, we called this kind of hybridization as GASA1.The process of this hybrid method is starting by processing population on GA then processing the output of GA in SA to check whether the most optimal solution can be found.The process is being shown in Figure 3.After processed under GA for some time, 40% best among the last population selected to be treated by SA, and after that, we may get the optimum solution.

Hybrid SA-GA
Almost same with hybrid GA-SA, hybrid SA-GA processing the input on SA first before continuing to GA.For the sake of simplicity, we called this kind of hybridization as GASA2.The process can be shown in Figure 4. To implement this hybrid, firstly we generate some initial population for SA.Then each of them will be handled by SA, and when they have been done, they will be GA's initial population and addressed through GA until time allocation expiry.

Cyclic GA-SA
Another type of hybrid GA and SA is Cyclic GA-SA.For the sake of simplicity, we called this kind of hybridization as GASA3.This model will consist of some GA-SA iteration.On the first iteration, the input will be processed through GA first then some of best GA's output -10% to be exactwill be processed in SA process.After that about 10% of best solution from SA's first iteration's will be addressed through GA again then the new best solution will be addressed through SA.This process will be repeated until stopping criteria is being met.Time allocation is the most important on this model.So we are going to test some scenarios based on time allocation later.The detailed process of this model can be seen in Figure 5.

Experimental Result and Discussion
Besides using real-world problems, some papers also use test function.Test functions are artificial issues.It can be utilized to assess the conduct of an algorithm in some of the time various and troublesome circumstances [17].Numerical experiments carried out by [18] shows that the evolved EA has better performance than the GA.They use unimodal and highly multimodal test function.
In this study, the comparison between GA, SA, hybrid GA-SA, hybrid SA-GA and cyclic GA-SA, to test its robustness and validate its performance is conducted using five test functions.There are Sphere, Rosenbrock, Rastrigin, Cosine Mixture, and Schwefel function.

Experimental Scenario
We experiment with GA, SA, and hybridization of both algorithms which are GASA1, GASA2, and GASA3.All experimental algorithms were tested using five different test function which is described in Table 1.In each testing scenario, we tune the different number of the random rate which is ranged between 0.1 and 1 to obtain the minimum value.We use random rate parameter to select the number of individual randomly that will be processed in GASA1, GASA2, and GASA3.The random rate of 0,1 in GASA1 means that the 10% of the population in GA will be addressed through SA while the random rate of 0,2 on GASA2 means that 20% of the number of individual SA will be dealt with through GA.While on the GASA3 random rate of 0,1 means only 10 % of the population that selected randomly will be put on GA then processed through SA and returned to GA.
For a fair comparison, we test all algorithm in the same environment.GA, SA, GASA1, GASA2, and GASA3 were coded in Scala programming language which provides both functional and object-oriented paradigm.This experiment uses multithreading concept.The first thread is working as a timer to stop the second thread that working as optimization process of each algorithm.
The timer is set to 60 seconds which cause the algorithm process will be terminated exactly after 60 seconds.After terminating the process, a necessary procedure in GA like selection may not be executed.Thus, we select the lowest value as the best individual to be compared with other algorithms. JITeCS

Result and Discussion
We conducted a test for 5 seconds and 60 seconds for all of the algorithms in ten attempts.The 5 seconds test is to analyze the best random rate for each algorithm in each function which will be used to analyze the 60 seconds computation time.The best random rate parameter for all hybrid algorithms in 5 seconds is described in detail in Table 2 -4.It shows that on each test function, all hybrid algorithms have a different best parameter which is shown in bold type value.
In GASA1, the best random rate for F2 and F5 is found on 0.4.While for F1 and F3, the best random rate is found on 0.5 and 0.6 respectively.For F4, the best random rates are 0.4 and 0.9.In GASA2, 0.8 is considered as the best random rate for F1 and F2.The best random rate for F4 is 0.1.The best random rates for F3 are 0.1 and 1.0.While for F5, the best random rates are 0.2, 0.8, and 0.9.In GASA3, test function of F1, F2, and F3 use 0.1 as the best random rate.For F4, 0.6 is found as the best random rate.For F5, the best random rates are 0.1 and 0.4.Furthermore, the summary of all best random rate is described in Table 5.
Even though each hybridization algorithm needs different random rate, they share the same behavior.In GASA1, about a half random rate is necessary to provide a better result.While in GASA2, the high number of random rate produce a better result.In GASA3, almost all of test function only need a low number of random rate.From all tables, it can be concluded that generally, GA, GASA1 with a random rate of 0.5 and GASA2 with a random rate of 0.8 returned the best random rate for those functions.While SA produces the worst result for all function.The standard deviation for SA for each function consists of higher value than another algorithm.It shows that the result is widely spread which caused the obtained result is bad.The standard deviation for the best algorithm for each function consists of lower value which caused the obtained result is good.

Conclusion
This paper presents the three kinds of hybridization of GA and SA.The obtained result from 5 benchmark functions shows that hybridizing GA with SA can enhance both performances to solve a nonlinear problem.Using cyclic GA-SA process show the minimum average value in two benchmark function.We also define the random rate which is sensitive with provided result.In the different case, random rate parameter needs to be tuned to obtain the best result.

Table 2 .
The GASA1's random rate result

Table 3 .
The GASA2's random rate result

Table 4 .
The GASA3's random rate result

Table 5 .
Best random rate for GASA1, GASA2, and GASA3The best random rate for each algorithm in 5 seconds is used to test the best algorithm for each function in 60 seconds.The best algorithms for each benchmark functions in 60 seconds computation time described in detail in Table6-11.It shows that on each test function has a different best algorithm with the different best random rate.The bold type value denotes the best value and the suffix "_0.1R" denotes random rate.In Sphere function, GASA2 with a random rate of 0.8 provide the best result.In Rastrigin and Schwefel function, GASA1 with a random rate of 0.5 provide the best result.However, in Rosenbrock and Cosine Mixture function, all hybrid algorithms fail to provide better result comparing to GA.In this case, GA provides the best result.

Table 6 .
The Comparison of Algorithms on the Sphere Function

Table 7 .
The Comparison of Algorithms on the Rosenbrock Function

Table 9 .
The Comparison of Algorithms on the Cosine Mixture Function

Table 10 .
The Comparison of Algorithms on the Schwefel Function Alfarisy et al., Hybrid Genetic Algorithm and Simulated... 97