CURVE FITTING THERMAL DATA USING SIMULATED ANNEALING ALGORITHM FOR UROP 0 Author: Lock Hun Yi / Assistant Professor George Chung Kit Chen School of Electrical & Electronic Engineering Nayang Technological University Nanyang Avenue, Singapore 639798 Keywords: Simulated Annealing, Thermal conductivity and diffusivity of Tetrahedral Amorphous Carbon (ta-c) film Abstract Simulated Annealing optimisation technique is applied to curve fit the surface thermal profile of Gold/ta-C/Silicon sample for extracting the thermal properties of ta-c film. The fitted thermal conductivity of gold and ta-c are determined to be 75 W/m K and.35 W/m K respectively. Thermal diffusivity of ta-c is found to have very little effect on the optimization result. Three approaches of Simulated Annealing are investigated, with the first two approaches using random variables in different sequences and the third approach combining the Nelder-Mead Simplex direct search with the Simulated Annealing. Our results of ten trails show that the mean and standard of the thermal conductivity of gold and ta-c films are about the same for the first two approaches, while significant difference between these two approaches is obtained for the thermal diffusivity of ta-c film. 1 Introduction Tetrahedral amorphous carbon (ta-c) film has been an active research topic in recent years [5]. This is mainly due to the high sp 3 content in the ta-c film, which results in unique properties that range from extreme hardness, chemical inertness, high electrical resistivity and breakdown strength. Despite many research efforts have been done on investigating various ta-c film s properties, studies on the film s thermal properties are scarce. Using the pulsed photothermal reflectance technique, the surface thermal profiles of various thickness of ta-c film, ranging from 0nm to 100nm, on Gold/ta-C/Si samples were obtained [5]. By applying a -layer heat conduction model (treating ta-c film as a thermal resistor. This is justified by the fact that the ta-c film s thickness is much smaller than the Gold film and Si substrate) and curve fit the surface thermal profile using the Nelder-Mead Direct Simplex optimisation technique, thermal conductivity of ta- ta-c) was then ta-c is not thickness dependent and hence one can use any thickness of ta-c film to determine its thermal characteristics. For curve fitting the experimental thermal profile, instead of using a -layer model one could replace it with a 3-layer heat conduction model. In the 3-layer model, the ta-c film is treated as a layer along with the overlay Gold layer and the Si substrate layer to form the 3 layers, whereas in the -layer model the ta-c film is treated as a thermal resistor between the Gold and Si layers. The advantage of using a 3-layer model is that it allows the flexibility of film thickness in the samples, ie. the Au film does not necessary have to be much thicker as compare to the ta-c film. As for the optimisation technique, depending on the initial setting for the variables, the Direct Simplex method would yield a diverse result for a 3-layer model. This is because the Direct Simplex method is basically a local descent optimisation technique and when being applied to situation where there are multiple variables, as in the case of 3-layer model, a variety of local minima could be obtained and sometimes it is difficult to decide which is the true solution. This problem can be resolved by applying global optimisation technique for curve fitting. In this study, Simulated Annealing global optimisation technique together with the 3-layer model is applied to curve fit the experimental thermal profile of Gold/ta-C/Si sample. Three approaches of varying the variables along with some of the key parameters of Simulated Annealing are investigated. Simulated Annealing.1 Theory The concept of growth of a crystal from a liquid is used solves a difficult optimization problem. If the liquid was instantaneously frozen, the atoms would be trapped in the configuration they had in liquid state. This arrangement has a higher energy than the crystalline ordering, but there is an energy barrier to be surmounted to reach the crystalline state from a glassy one. If instead the liquid was slowly cooled, the atoms would start at a high temperature exploring many local rearrangements, then at lower temperature the atoms become trapped in the lowest energy configuration that was reached. The slower the cooling rate, the more likely it is that they will find the ordering that has the lowest global energy. This process of slowly cooling a system to eliminate defects is called annealing. This concept of annealing is actually Simulated Annealing, which is introduced by Metropolis et al.[1], and is used to approximate the solution of very large combinatorial optimization problems.
To further elaborate this concept in detail, let us assume we are looking for the configuration that minimizes a certain cost function E. The algorithm can then be formulated as follows. Starting off at an initial configuration, a sequence of iterations is generated. Each iterations consists of the random selection of a configuration from the neighbourhood of the current configuration and the calculation of the corresponding change in the cost functio E. The neighbourhood is defined by the choice of a generation mechanism, i.e. a prescription to generate a transition from one configuration into another by a smaller perturbation. If the change in cost function is smaller than a reference value, the transition is unconditionally accepted; if the cost function, increases the transition is accepted with a probability P acc E) based upon the Boltzmann distribution, P acc E exp (- E/ T), where is a constant and the annealing temperature T is a control parameter. This temperature is gradually lowered throughout the algorithm from a sufficiently high starting value (i.e. a temperature where almost every proposed transition, both positive and negative, is accepted) to a freezing temperature, where no further changes occur. In practise, the temperature decreased in stages, and at each stage the temperature is kept constant until thermal quasi-equilibrium is reached. All of the parameters determining temperature decrement (initial temperature, stop criterion, temperature decrement between successive stages, number of transitions for each temperature value) is called the cooling schedule.. Approach The experiment surface thermal profile which was obtained earlier by using the pulsed reflectance techniques on the 80nm thickness of the ta-c film is shown in Figure 1. This film is sandwiched between a 1.77 film on top and a thick silicon substrate the bottom. This profile is made up of 500 samples time interval points. Using Eq. (1) and () to plot The surface thermal profile of the experiment Figure 1 Example of curve fitted equation with experiment results The temperature excursion profile of 80nm ta-c film is fitted with 3-Layer Transmission-Line Theory of Heat Conduction Model, with the silicon substrate model as an infinite medium. The Laplace transformed surface temperature excursion, T(s), can be described by [6] Q( s) (cosh η s + e3 sinh η s ) cosh η1 s + ( e31 cosh η T ( s) = e1 s ( e31 cosh η s + e1 sinh η s ) cosh η1 s + (cosh η s + e s + e 1 3 sinh η sinh η s )sinh η 1 s )sinh η 1 s (1) s where e 1 = ρ1c1 κ, ei di 1 eij =, η i =, i, j = 1,, 3 e α j d 1 and d is the thickness of the gold layer and ta-c layer respectively. Q(s) is the Laplace transformed Gaussian pulse (Nd:YAG) which can be described by 1 b cs 1 Q ( s) = 1 + Erf exp bs + c s () c 4 where b=.3 10-8 s, and c=5.86 10-9 s are the fitted values to our Nd:YAG pulse. i For fitting to the temperature excursion profile, Eq. (1) is inversely Laplace transformed into time domain using Stehfest Numerical method.[5] The fit utilizes gold bulk density value (1.93 10 4 kg/m 3 ) [5] and specific heat (19 J/kg K).[5] since these parameters do not vary much with thickness.[7] For silicon substrate, its effusivity (e ) is taken as 1.57 10 4 Ws 1/ m - K -1 [5]. This fitting temperature excursion profile can be plotted out as shown in Figure 1(continuous line). The errors between the experimental and fitting profiles are calculated at the 500 sampled time interval. Each of this errors
at each time interval is being squared first and then sum of all the squared errors. This calculated result is the cost, Eq. (3), which is used in the Simulated Annealing to find the global minima solutions. t = end error _ between _ both _ the _ profiles cos t = (3) t= start at _ each _ time _ int erval There are three approaches were formed using the Simulated Annealing and the cost...1 Using all random variables The first approach starts off by substituting a set of initial variables, K gold, K ta-c ta-c, into Eq. (1) and () for computing the cost. This cost is termed as first cost. Then, three random numbers for K gold, K ta-c ta-c, limited by their individual boundaries, are generated and are substituted into Eq. (1) and () for computing another cost, which is termed as second cost. If the second cost is smaller than the first cost, the randomly generated values for K gold, K ta-c ta-c will be saved. In addition, the second cost will replace the first cost. But if the second cost is larger than the first cost, the second cost will go through the Simulated Annealing acceptance process, ie. P acc E), to determine whether to save the randomly generated K gold, K ta-c ta-c and replace the first cost or not. This process is considered as a single iteration. After that, another set of K gold, K ta-c ta-c will be randomly generated and a second cost will be computed, comparison with the first cost will be made as before. This process will repeat for a certain number of iteration before the annealing temperature is lowered and then the process will continue again. The entire process will stop once 100 times of iteration of reducing the annealing temperature is completed. Each reduction of the annealing temperature is 10%.... Using Single variable for each set of iteration The next approach is very similar to the first except that only one variable will be randomly generated in one set of iteration and the other variables remain the same. After the iteration, it will follow by the next variable until all the variable has gone through the same set of iteration, before reducing the annealing temperature. Example, first the value of K gold is randomly generated and all else remain the same. After certain number of iteration and comparing, the value of K ta-c is randomly generated and the rest remain fixed for a same fixed number of iteration. Next, the ta-c will be randomly generated and the rest of variable is fixed going through the same amount of iteration. Once all the variables has randomly generated, the annealing temperature is reduced and the rest the same as using all random variables in one iteration...3 Simplex and Simulated Annealing Together The last approach is to combine Nelder-Mead simplex direct search with simulated annealing. It also has similar steps as the first approach The first step is to use Nelder-Mead simplex direct search with initial variables to find the local minima. Next, randomly generated values of variables (K gold, K ta-c ta-c ) and use the simplex direct to find the next local minima. Calculate the difference of both the costs. If the cost which uses the randomly generated variables is smaller, then s the local minima variables is been saved. If not, the error will go through Simulated Annealing process to determine whether to accept these local minima variables. The process of next iteration as well as the reducing the annealing temperature is the same as the first approach. 3 Results Each of the approaches is further being divided into different program to analyse in greater detail. There are basically two types of parameters; one of them is number of iteration, the other is initial annealing temperature. The different number of iterations is to analyse the effect on the variation of each variables and the cost when having different number of iteration in simulation annealing. This like the atoms, if there is not enough time for it to explore the local rearrangement and therefore it will freeze close to its liquid arrangement. For the setting of different initial annealing temperature, this affects the probability of saving results in simulated annealing that has a higher cost. If the initial annealing temperature is set too low, the results will end up at local minima instead of a global minimum. This related to the atoms that moves more freely at higher temperature, but when the initial temperature is very low, the atoms were not able to move much. Therefore, it could not able to explored significant local rearrangement to find the global energy as mention in the introduction. Mathlab programs were written and each program was ran for 10 trails. The results were consolidated and from this results the mean and standard for each of variables (K gold, K ta-c ta-c ) were calculated and presented in the tables below. All the initial values are set as K gold =300, K ta-c- =5 and ta-c =1 10-6. The program was ran on a AMD 1.6GHz PC. 3.1 First Approach The first two sets of results are obtained by programming the first approach mention earlier (..1) except that the second sets have twice the number of iteration as the first set. This is to examine the effect of larger number of iteration.(table 1 and ) The next two set are the sets are the same as the first two set except that the initial temperature is lower. (Table 3 and 4)This is to examine the effect of lowering initial annealing temperature with the same number of iteration. The final two set of results(table 5 and 6) has the number of iteration
reduce such that the final annealing temperature will be the same as the final annealing temperature of the first two set.(table 1 and ) This program is similar to the second set of program but the cooling cycle for the annealing temperature is shorted such that the final annealing temperature is the same as the first set of programming. Initial Annealing Temperate=500 Iteration=10000 Running time for each program=11 minutes Mean 76.168.368468 9.9E-04.6E-01 4.61394859 0.44938858.50E-03 1.75E-0 Table 1 Initial Annealing Temperate=500 Iteration=0000 Running time for each program=3 minutes Mean 76.6363.34895 1.68E-04.54E-01 34.49178908 0.40431737.84E-04 1.53E-0 Table Initial Annealing Temperate=300 Iteration=10000 Running time for each program=11 minutes Mean 8.8893.5193 1.46E-03.67E-01.7099965 0.459190139.81E-03 3.8E-0 Table 3 Initial Annealing Temperate=300 Iteration=0000 Running time for each program= minutes Mean 66.354.18594 1.18E-03.49E-01 3.1979488 0.35764767.31E-03 9.45E-03 Table 4 Initial Annealing Temperate=300 Iteration=9500 Running time for each program=10 minutes Mean 59.1578.309987 5.49E-05.75E-01 36.0179078 0.491788853 1.11E-04 4.04E-0 Table 5 Initial Annealing Temperate=300 Iteration=19000 Running time for each program=1 minutes Mean 80.9407.376594 7.85E-04.53E-01.67531351 0.418591178.E-03 1.57E-0 3. Second Approach Table 6 The four set of results below have been run the similar way as the first four set of the first approach. (the first two set have only the difference in number of iteration and the last two set is having the initial temperature difference) Initial Annealing Temperate=500 Iteration=30000 Running time for each program=34 minutes Mean 79.9405.41839 9.15E-04.60E-01 6.7915367 0.353175688 1.66E-03.18E-0 Table 7 Initial Annealing Temperate=500 Iteration=60000 Running time for each program=70 minutes Mean 75.604.386857 1.8E-03.47E-01 19.66974 0.16885993 3.7E-03 7.66E-03 Table 8 Initial Temperate=300 Iteration=30000 Running time for each program=34 minutes Mean 70.96.313687 1.55E-03.45E-01 17.8551844 0.766841.58E-03 5.85E-03 Table 9 Initial Annealing Temperate=300 Iteration=60000 Running time for each program=70 minutes Mean 76.01.6313.0E-03.45E-01 1.0791759 0.3053784 3.69E-03 5.78E-03 Table 10 3.3 Third Approach This program has only run 3 times due to time constraint, but all the set of results are the same. Therefore, the results were
not compiled to its mean and standard and shown below directly. Initial Annealing Temperate=500 Total iteration=10000 Time K gold K ta-c > ta-c Cost 19 hours Table 11 3.4 Using the Nelder-Mead simplex direct search only A set of random values of the variables (K gold, K ta-c ta-c ) is being substituted into Eq. (1) and () and using Nelder- Mead Simplex Direct Search to find the optimal local minima. This is to analyse the difference and the effect in using the Simplex method. This program was run for ten times. The results have been consolidated and calculated into its mean and standard for each variables(k gold, K ta-c ta-c ) as shown in Table 1. Running time for each program = 1 seconds K gold K ta-c ta-c Cost Mean 85.8958 693.33 1.98E-04 3.8 78.06.369 7.904e-5 0.3661 13.355 1000.8 3.11E-04.59 Table 1 However, for different initial annealing temperature (first and second approach) and having the same final annealing temperature (first approach last set) for different initial annealing temperature setting, there is no significant change in the mean and standard in the cost, K gold and K ta-c. From this result, it can be sure that with a lower annealing temperature (300) and less iteration (to have the same final annealing temperature as the first two set of the first approach) the results will remain about the same. It possible to use even lower initial annealing temperature and iteration to save even more computing time, but due time constraint the finding of lowest number of iteration and initial annealing temperature was not considered. The diffusivity of the ta-c has significant difference for all the approaches. For the first approach, it has lower mean and also standard for higher number iteration, especially for the initial annealing temperature of 500 degrees. But it can be seen that the results for the second approach is opposite of the first approach. It has a lower mean and standard for lower number of iteration. To further analyse this, the cost of the experiment and the calculated surface thermal profile ta-c varying from 1 10-9 to 1 10-3 m /sec, the K ta-c fixed at.35w/m K and K Gold fixed at 76.79W/m K) is being plotted out in figure. 4 Discussions The results of the Simplex search have a very large standard for all the variables (K gold, K ta-c ta-c ), which can be seen from Table 1, this is probability due to the fact that there are many local minima in the solution. Another point to take note of is that both the standard and the mean of the cost is also large. This reason is that there is a very large difference in the experiment surface thermal profile and the calculated surface thermal profile in general.(there is a mismatch between the experiment and the fitting surface thermal profiles) Therefore, many of the results cannot be used to determine the thermal conductivity of gold and ta-c film and also the thermal diffusivity of ta-c film. Compare the above with the 3 approaches of simulated annealing, it can be seen that all the values of the variables and cost except for the ta-c, have all quite small standard (less than 10%). This proves that simulated annealing is a better way of finding the solution with many local minima. The higher the iteration, in general, the mean and standard of the cost is lower for all three approaches. This shows that both the experiment and calculated surface thermal profile are having less difference if there were more iteration. It is like the atoms were able to have more time are more chances to find a better local rearrangement in the same temperature. Figure!" # $ %&'#(! & )* +,-*! &.* / ta-c Figure shows that the thermal diffusivity of ta-0 13 45 6 ta-c, has very little effect on the cost over a wide range of values. This means that for a larger 7 89 :;< : 6 ta-c there will be very little change in the value of cost. This is possible to see why there are large standard for the values =1 6 ta-c in the first two approaches. The first approach is using all random variables to find the minimum cost, which has a larger number of possible solutions in a single iteration as compared to the second approach. This is because the total number of solutions has to be considered is actually the number of each possible variable multiply by the other. However, the second approach has only
considered only the number of each variable individually for the total solutions in a single iteration. It is possible that for the second approach to reach a point where the global minimum for K gold and K ta-c is found after certain number of ta-c will end up greater mean and standard as it has negligible cost over large values.(figure ) This is possible why for higher iteration the mean and standard is greater at higher iteration for the second approach. 5 Conclusion The results obtained by using Simulated Annealing Optimization Technique have shows clearly that the values of K Gold is about 75 W/m K and K ta-c is about.35 W/m K. For ta-c varies from 1 10-3 to 9 10-7 m /sec. But using the combination of both Simulated Annealing and Nelder-Mead simplex direct search, the result ta-c is likely to be 7.9 10-5 m /sec. This result can only be confirmed if there is more time to run more trials. ta-c variation does not affect much on the cost (Figure ), it seems to have a very small cost with a large ta-c. It is possible to due to the thickness of ta-c film is much smaller as compare to the gold film and silicon substrate. The ta-c film did not hold much heat and allow most of it to pass through to silicon from gold.!" ta-c is not possible as it is too small or negligible. Time to compute the result greatly depends on the amount of iterations, the more the iteration the greater amount of time required to complete the program. It is possible to reduce this computation time by further reducing the initial temperature and number of iteration. Lastly, comparing all the approaches, the best approach that uses the shortest computation time and produces results that is satisfying would be the first approach with initial temperature of 300 degrees and number of iteration is 9500. This is because all the approaches have almost the same results except for the value of ta-c, which can be neglected; however the first approach having the shortest time due to smallest number of iteration. Simulated Annealing Parallelization Techniques, pp. 1-10, (199). [4] William H. Press, Brian P. Flannery, Saul A. Teukolsky, William T. Vetterling. Simulated Annealing Methods, Numerical Recipes in C: The Art of Scientific Computing, pp. 444-455, (1993). [5] George Chen, Ping Hui, Shi Xu. Thermal conduction in metalized tetrahedral amorphous carbon(ta-c) films on silicon, www.elsevier.com/locate/tsf, (1999) [6] P. Hui and H.S. Tan, "A transmission-line theory for heat conduction in multi-layer thin films", IEEE Trans. on Components, Hybrids, and Manufacturing Technology, Part B, vol.17(3), pp.46-434, (1994) [7] D.L. Decker, L.G. Koshigoe, E.J. Ashley, NBS Special Publication 77, "Laser Induced Damage in Optical Material", Government Printing Office, pp 91-97, (1994) References [1] Metroplis N., Rosenbluth A.W., Rosenbluth M.N., Teller A.N., & Teller E. Equation of State Calculations by Fast Computing Machines, Journal of Chemical Physics, volume 1, pp. 1087-109, (1993). [] S. Kirkpatrick, C. D. Gelatt, Jr., M.P. Vecchi. Optimization by Simulated Annealing, Science, volume 0, pp. 671-678, (1983). [3] Robert Azencott. Sequential Simulated Annealing: Speed of convergence and acceleration Techniques,