A REINFORCEMENT LEARNING APPROACH TO SCHEDULING DUAL-ARMED CLUSTER TOOLS WITH TIME VARIATIONS
|
|
- Morgan Cole
- 5 years ago
- Views:
Transcription
1 A REINFORCEMENT LEARNING APPROACH TO SCHEDULING DUAL-ARMED CLUSTER TOOLS WITH TIME VARIATIONS Ji-Eun Roh (), Te-Eog Lee (b) (),(b) Deprtment of Industril nd Systems Engineering, Kore Advnced Institute of Science nd Technology (KAIST), Dejeon, Republic of Kore () (b) ABSTRACT The cluster tool, which consists of multiple process chmbers re widely used in the semiconductor industry. As the process of wfers becomes more sophisticted, the opertion of cluster tools is lso being improved. To effectively operte cluster tools, severl rule-bsed schedules, such s the swp sequence hve been developed. However, scheduling in time vrince environment is not fully considered. In this pper, we propose cluster tool modeling method, which cn hndle time vrince in dul-rmed cluster tool. Then, we present reinforcement lerning process bsed on the proposed cluster tool model to find new opertionl schedules in specific configurtions. To mesure the performnce of the newly obtined schedule, mkespn is compred under the new policy nd the swp policy. The mkespn reduced compred to the conventionl swp policy, which implies tht the reinforcement lerning well lerned the opertion schedule in the time vrince environment. Keywords: cluster tool, Mrkov decision process, scheduling, reinforcement lerning 1. INTRODUCTION Along with the innovtive development in the semiconductor mnufcturing industry, qulity issues in the wfer mnufcturing process hve been discussed. The technologies of ech process hve rpidly developed nd improved the production qulity of the wfers. To void qulity issues due to btch production, the cluster tool tht process one wfer t time re now widely used in the semiconductor industry (Lee 2008). Chmbers in cluster tools do not process wfers in units of btches but process them individully, so they cn meet the qulity stndrd. The cluster tool consists of usully four to six processing chmbers nd one trnsport robot. In ech chmber, only one wfer is processed t time, nd one of the process steps specified before the strt of the process is processed. In ddition, the trnsport robot moves the wfers inside the cluster tool. To strt the processes, wfer enters the chmber tht is responsible for the first process step. After the process is completed in the chmber, the wfer is trnsported to the next chmber by trnsport robot. The trnsport robot repets the process of unloding the processed wfer from the chmber nd loding the wfer in the next pproprite chmber which is in chrge of the next process step. Once ll the required processes re completed with the proper chmber sequence, the wfer process in the cluster tool finlly is completed. Such configurtion leds cluster tools to hve severl issues in opertion. Since cluster tools consist of only processing chmbers nd single trnsport robot, only one robot opertion is possible t time. Hence, the wfers cnnot be moved t ny time, nd cn only be moved when the trnsport robot moves them. Even if the multiple chmbers re redy for the process, multiple wfers cnnot be delivered t the sme time. In ddition, the cluster tools do not contin ny buffer spce in their interior spce, so the only wy to store the wfers outside the chmber within the cluster tool is to hold it with the trnsport robot. When the robot lods wfer into the chmber, the chmber strts the process, nd the process ends nturlly fter the process time. The chmber processes for period of time without ny decision. Therefore, we cn sy tht the overll cluster tool schedule depends on the order decisions of the trnsport robot. As the trnsport robot unlods or lods the chmbers, the sttus of the cluster tool system hs chnged. This cn be seen s discrete event system becuse the stte of the system chnges s the robot tkes ction. Petri nets, finite stte mchines (FSMs), Mrkov decision processes (MDPs), etc. re used to model discrete event systems (Murt 1989; Choi nd Kng 2013; Putermn 2014). Severl studies bout cluster tools scheduling hve used timed Petri net (TPN) model, in which the plces or trnsitions hve time dely. A TPN is clssified into deterministic nd stochstic TPN whether the time dely is deterministic or stochstic (Murt 1989). Ech of them is modeling technique tht dels with different kinds of time delys. Mny studies hve been conducted on finding policies for the deterministic environment. For exmple, (Lee, Lee nd Shin 2004, Zuberek 2004) hve modeled nlyzed cluster tools by the deterministic TPNs, nd Jung nd Lee (2012) 153
2 proposed mixed integer progrmming (MIP) model to find the optiml policies in deterministic TPNs. In stochstic environment, there lso hs been severl studies. To del with time vrition in cluster tool, (Kim nd Lee 2008) suggested n extended Petri net nd developed grph-bsed procedure to verify the schedulbility condition. Qio, Wu, nd Zhou (2012) hve introduced two-level rchitecture to del with time vrition, nd proposed some heuristic lgorithms to find schedule for one of the rchitectures. Molly (1982) suggested tht the stochstic TPNs re isomorphic to finite Mrkov processes under the certin conditions. However, new methods for finding good policies in specific systems re studying nowdys. Reinforcement lerning (RL) is widely used in finding policies in mny fields such s mnufcturing systems, utonomous vehicle control, finnce, nd gmes. This reinforcement lerning is one of the solutions to find optiml policies in MDP models (Sutton nd Brto 1998). Hence, modeling the system behvior by MDPs, then find the policies for the system by RL is widely used (Moody et l. 1998; Mnih et l. 2015). In this pper, we propose model using MDPs, which cn hndle time vritions in cluster tools. Then, we report how we lern the new robot sequence using the MDP model to minimize the verge mkespn in time vrition environment, nd mesure the performnce of the obtined sequence. This study is n ttempt to schedule the cluster tool using reinforcement lerning. Since the model is designed to represent the stochstic process time of the cluster tool, the schedules obtined from reinforcement lerning cn be expected to be well pplied in stochstic environment. 2. MODELING DUAL-ARMED CLUSTER TOOLS WITH TIME VARIATIONS To model dul-rmed cluster tools with time vritions, we use MDP model. After introducing the problem of configuring the MDP model, the model we proposed is reported Mrkov Decision Process A Mrkov decision process (MDP) is tuple < S, A, P, r >, where S denotes set of sttes, A set of ctions, P stte trnsition probbility distribution, nd r rewrd function, respectively. The process follows the Mrkov property, which mens the trnsition probbility nd rewrd functions depend only on the current stte nd the ction, not the pst history. In this pper, we ddress only the sttionry environments, which mens the system properties does not chnge s time goes by. According to Putermn (2014), t every decision epoch, stte s S, which is representtion of system, is observed nd n ction A s hs to be chosen by decision mker from the set of llowble ctions in the stte s. As result of the ction, the system stte t the next decision epoch chnges by some trnsition probbility P S A S [0, 1]. P s,s is determined by the current stte s nd the chosen ction. The decision mker gets rewrd r S A S R, nd r s,s is determined by the current stte s, ction, nd the next stte s. Here, we hve concept of decision rule, which describes procedure for ction selection in ech stte. This rule is clled sttionry policy, d S A s. The decision mker psses through sequence of sttes s t, which re determined by trnsition probbilities P s,s nd the ctions t = d(s t ) the decision mker chooses, then the sequence of rewrd r s,s is obtined. The populr performnce metric is discounted rewrd, which is the sum of the discounted rewrd gined over the entire time horizon when we llow the specific policy. The discounted rewrd of policy d strting t stte i is defined s k d(s j ) J d (i) lim Ε [ j=1 γ (j 1) r k sj,s j+1 s 1 = i], where γ [0,1] is discount fctor. By Bertseks (1995), it is d(s) d(s proved tht J d (i) j ) = [ s S P s,s ( r sj,s j+1 + γj d (s ))]. The function J d is clled the vlue function for policy d. There re mny vrints of MDP model, nd one of the them is semi-mrkov decision process (semi-mdp). In semi-mdp, temporl fctors re included in the modeling. The originl MDP ssumes tht ech trnsition tkes the sme time through ll the sttes; however, semi-mdp considers the trnsition time to follow rbitrry probbility distributions Cluster tool modeling with MDP If we simply insert the stte of the chmbers nd ctions the robot tkes into MDPs s previous studies hve done with deterministic TPN models, the MDPs does not properly represent the behvior of the cluster tools. This is due to chrcteristics of cluster tools. The cluster tools re not simply chnging the systems by discrete event; cluster tools re chnging the systems by the time element. The generl MDP ssumes tht every trnsition tkes the sme time, however, the trnsitions in cluster tools cnnot be ssumed to be the sme. Considering the different trnsition times between sttes, we my think of simple wys to model the cluster tool system. 1. Model with MDPs which hve constnt trnsition time, one second. Adjust trnsition probbilities to represent the time element. 2. Model with semi-mdps so we cn insert time informtion to the trnsition distribution. To verify bove two methods, we first consider the exmple cse s shown in Figure 1. The exmple represents sttus chnge in single chmber, which hs two sttes A nd B, nd two ctions 1 nd 2. Consider the cse where the environment stte chnges from A to B fter 50 seconds. By using the first modeling method listed bove, we cn set trnsition probbility to 1/50; thus, the stte chnges from A to B occurs fter 50 seconds in verge. However, the verge cnnot express the ctul individul trnsition time explicitly. 154
3 Figure 1: Simple Exmple for Cluster Tool System By using the second modeling method, we cn set trnsition time distribution s 50 seconds. When using the second modeling method, we cn express tht the trnsition occurs fter 50 seconds in Figure 1 if we set the trnsition time distribution to 50 seconds (s constnt). However, the cluster tool is tool with multiple chmbers, so the model should reflect chnges in stte in other chmbers. If 20 seconds hve elpsed while the sttus chnges in nother chmber, nd so the chmber of the exmple now needs to chnge its stte fter 30 seconds, the trnsition time distribution should be chnged to 30 seconds. Every time the trnsition occurs in the other chmber, we hve to clculte nd chnge the distribution; the environment of the system chnges, which does not stisfy the mrkov property. The two MDP modeling methods bove re not pproprite for modeling the cluster tool opertions in time vritions, since they do not represent the trnsition time explicitly or do not stisfy the Mrkov property. To stisfy the Mrkov property, stte should contin sufficient informtion, so tht the stte does not require pst history to obtin the next stte or rewrd. Therefore, stte is designed to contin informtion bout chmber nd robot usge, nd the remining process time. In this study, We propose to represent the stte s (W, C 1, C 2,, C n, R, S 1, S 2, Z 1, Z 2,, Z n ), where n is the number of processing chmbers in cluster tool. W is the number of wfers remining in the lodlock. C i is the stte of the chmber, which represents whether the i th chmber is empty ( C i = 0), full ( C i = 1), or completed processing (C i = 2) for i {1, 2,, n}. R is the number of wfers held by the trnsport robot. Since we re deling with dul-rmed cluster tool, R cn hve vlue of 0, 1, or 2. S 1 nd S 2 re the next process steps of the wfers held by the robot. These represent the process steps tht ech wfer should visit in the next step. Z i is the expected remining process time of the ith chmber for i {1, 2,, n}. The vlue is clculted by subtrcting the elpsed time from the initilly set process time, becuse it is not known wht the ctul process time will be in time vrition environment. In ddition, define S to be the stte spce of the proposed stte. Hence, the proposed stte structure contins the informtion bout the chmber nd the wfer inside it, nd includes informtion on which chmbers the wfers on the robot should be sent. Finlly, it roughly contins informtion when the chmbers re going to be finished. Then, the model stisfies the Mrkov property. To better understnd the structure of the sttes, we show stte representtion of Figure 2 by using the proposed sttes structure. Suppose tht the blck nd white wfers indicte tht they re in the first nd second process steps, respectively, nd the htched wfer indictes tht the process is in progress. According to the proposed structure, the stte is (W = 3, C 1 = 2, C 2 = 1, C 3 = 1, C 4 = 2, R = 1, S 1 = 2, S 2 = 0, Z 1 = 0, Z 2 = 4, Z 3 = 4, Z 4 = 0). This mens tht the three wfers re remined in the lodlock, the process is going on in the second nd the third chmbers, wheres the first nd the forth chmber hve completed its process. The robot holds single wfer witing to enter the second process step, nd the remining process times of the chmbers re four for the second nd the third chmbers. Figure 2. A Dul-rmed Cluster Tool with Four Chmbers A is set of ctions tht the trnsport robot cn perform, which is {Wit, U j, L j, SW i }, where j {0,1, 2,, n} nd i {1, 2,, n}. They ll represent the robot tsks: U j, L j, nd SW i indicte unlod, lod, nd swp opertions on the jth or i th chmber. The unlod, lod, nd swp opertions re the common robot tsks in other modeling methods. However, in this study, Wit ction is dded. Wit represents the robot witing until the stte chnges, such s finishing the process in chmber. To mke the model trnsition independent on remining process time, we suggest Wit to be n ction of the robot. If without Wit ction, the stte of the cluster tool chnges when the process completes, even if no ction is selected. Furthermore, for ll sttes {Wit, U j, L j, SW i } cn lwys be selected s n ction; therefore, 3n + 3 ctions re vilble to be chosen in ll sttes. Trnsition probbility is denoted by P s,s, where s is the current stte, is the current ction nd s is the next stte. It stnds for probbility for trnsition to stte s when we choose ction in stte s. In this study, we do not consider unexpected event occurrence such s robot filure, so the system lwys trnsposes exctly to the specified stte. Hence, when the ction is one of U i, L i, nd SW i, P s,s hs vlue of 1 for the pproprite s. Here, the pproprite stte s is when s 155
4 is likely to be the next stte in stte s. Furthermore, when the s is not likely to be the next stte in s, P s,s = 0. However, when the ction is Wit, trnsition probbility P s,s does not only hve vlue of 0 or 1. Assume tht the ction is Wit. If t lest one chmber is full, which stnds for C i = 1, the chmbers should be selected which chnge their stte to completed processing. In other words, it chooses which chmber the process ends with. Choosing the chmber with the shortest remining process time is the simplest nd most intuitive. However, since we re deling with time vrition environment, we cnnot sy tht the process is finished fter Z i, exctly the expected remining process time. Therefore, the chmber selection should reflect the fct tht unexpected chmber processes my end sooner. In ddition to selecting the chmber, the trnsition time needed to be stochsticlly generted to reflect the time vrition environment. In generl, the process of the chmber is completed within rnge of times, including the verge time. Therefore, we choose to generte the trnsition time using bet distribution tht cn generte the verge vlue within the bounded rnge. Rewrd function is set to get 1 if wfer completes ll processes, otherwise, the function is set to get 0.1 in ll trnsitions except for the Wit ctions. When the Wit ction is selected, the rewrd is set to be 0.1 (trnsition time). Detiled trnsition rules nd rewrd definitions re introduced in Sections 3.2 nd LEARNING POLICIES FOR ROBOT MOVES IN CLUSTER TOOLS BY REINFORCEMENT LEARNING There re vrious wys to solve MDP. First, when we know the model, we cn define totl return. Then by dynmic progrmming, we cn find the optiml vlue function of Bellmn optimlity eqution (Bellmn 1954). However, the proposed model hs time element in the stte, it is not possible to solve the MDP through dynmic progrmming due to the curse of dimensionlity. Therefore, we pplied reinforcement lerning (RL) to find the policy for the proposed MDP model. RL requires n environment to interct with. So we built n interctive environment bsed on the proposed MDP model. We then set up tsks to perform the lerning, nd lerned the robot policy using Q- lerning. After lerning, we performed nother experiment to mesure the performnce of the policy Reinforcement lerning Reinforcement lerning (RL) is useful when the clssicl dynmic progrmming is not enough to solve the MDP problems. Dynmic progrmming is not suitble for the problem, when the environment is difficult to build perfect model due to its unknown trnsition probbility (curse of modeling), or when the environment hs too mny sttes to solve (curse of dimensionlity). RL dds the concepts of stochstic pproximtion, temporl differences, nd function pproximtion to clssicl dynmic progrmming, so tht it cn solve the MDP problem even if the trnsition probbility is not explicitly represented, nd the stte dimension is too lrge (Gosvi 2014) The environment Since we find the robot policy by RL, we need n environment tht returns the next stte nd rewrd for ction ccording to stte for lerning progress. The environment required for RL is either n environment consisting of rel equipment or n environment built by computer simultion. The best wy to interct with the environment is to build the rel world environment, then conduct the experiments with it. However, using rel world cluster tool is nerly impossible due to its vlue. A single tool is so expensive nd needs lrge re to instll; therefore, even the compny is not ble to rrnge enough cluster tools for their mnufcturing fcilities. To replce the rel world environment, we built virtul environment, which responds to some stimultion. The simultion environment mimics the rel world; however, not fully, only prtilly. To mke the prtilly reflected simultion environment to involve the core elements we wnt, we needed to be creful mking the simultion environment. We mde some ssumptions in building n environment. First, we do not consider mchine filure. Process time cn vry depending on mchine nd time; however, we do not hndle mchine filure in this study. Second, the environment is sttionry, which mens system dynmics does not chnge s time goes by. No mtter how much time hs pssed, they follow the sme distribution. Third, the time dynmics of the systems re governed by the rules we set; hence, it my not fully reflect the rel world cluster tool behviors. The simultion environment shows the chnge of the environment depending on the gent s ction. It shows the pproprite rewrd nd the next stte ccording to the selected ction. Some ctions re vlid only in prticulr sttes. For exmple, robots cn tke Wit ction only when more thn one chmber re filled with the wfer, nd re not completed processing, which mens tht chmber i with C i vlue of 1 exists. Detiled rules for robot ctions re specified in Tble 1. Tble 1: Possible Situtions for the Robot Actions Robot Possible Situtions Actions Wit When t lest one chmber is filled with wfer hs not been completed the process Unlod When the chmber hs processed wfer, but still hve the wfer inside in it Lod When the robot holds some wfers, nd one of the chmbers corresponding to the steps tht the wfers re going to enter is empty Swp When the chmber hs processed wfer, but still hs the wfer inside in it. At the sme time, the robot holds only one wfer to enter the chmber 156
5 Tble 1 shows the possible situtions for the ction. The ction is vlid only when the right box condition is stisfied. If we select n ction tht is not in those possible situtions, the stte will not chnge. We set the environment to prevent stte trnsitions when inpproprite ctions re tken. We cll n ction tken in n impossible sitution n invlid ction, nd n ction tken in possible sitution vlid ction. For exmple, if chmber hs processed wfers but the wfers re still in it nd t the sme time the robot hs only one wfer in its rm, swp ction on the chmber is vlid ction, while the rest ctions re not. All trnsitions re treted to be deterministic move except for Wit ction, s mentioned in Section 2.2. Other ctions cuse the environment to hve the sme response ech time stte is entered. However, when Wit ction is chosen, the stte trnsition is performed stochsticlly nd the trnsition probbility is defined differently depending on the stte. When only one chmber is in the full stte, the chmber is selected to complete the process. However, when more thn one chmbers re in full stte, chnges in environment is determined by the lgorithm below. With probbility 0.9, the chmber with the minimum remining process time completes the process With probbility of 0.1, the selected chmber completes the process with probbility of inversely proportionl to the remining process time Every time the Wit ction is chosen, the trnsition time is set to be stochsticlly selected long the bet(50,50) distribution Tsk description In generl, set of 25 wfers becomes 'wfer cssette', which is the unit of production for cluster tools. Therefore, the gol is to quickly finish single cssette process rther thn single wfer process. In fct, in rel compnies usully produces 100 to 200 wfers once, which mens produces 4~8 wfer cssette without stopping the opertion of the cluster tool. The problem of minimizing the totl mkespn cn be thought of s reduction in the totl mkespn itself nd reduction in the mkespn of ech subtsk. When the whole mkespn is continuously mesured nd returned t the end, the rewrd structure is hevily sprse to lern the policy. Therefore, in this study, we follow the method of minimizing the mkespn of ech subtsk, which is completing single wfer. In ddition, we follow the structure of receiving rewrd t the end of ech wfer process. In order for ll subtsks to ultimtely reduce the totl mkespn, the totl return is defined s the discounted rewrd. So lerning is done in the direction of receiving the rewrd s soon s possible. Furthermore, fter 100 ~ 200 wfers re produced in rel world, the equipment configurtion my be chnged. In the cse of n episodic tsk tht ends fter the production of certin number of wfers, the whole process cn be divided into strt-up, stedy, nd closedown cycles (Kim, Lee nd Kim 2016). However, in this study, we exmine whether there is policy chnge when the environment chnged from deterministic to stochstic in the stedy cycle. Therefore, we consider the lerning tsk s continuous tsk to lern stedy cycle. The most importnt gol of this problem is to lern to choose n ction tht mximizes the totl return. But we should lern to prevent the system from selecting behvior tht mkes it dedlock nd lern to prevent invlid ctions Lerning procedure The cluster tool behvior ws lerned by computer simultion-bsed reinforcement lerning. Lerning is chieved through the interction of gents nd environments. The gent repets the process of selecting the ction, receiving the corresponding response, nd updting the Q vlue for the stte nd the ction. The ction selection nd Q vlue updte follow the trditionl Q-lerning method (Sutton nd Brto 1998). The lgorithm for Q-lerning is illustrted in Figure 3. We used n ε-greedy policy to select n ction from the current stte, where the ε decresed over the time steps. Even if using the sme lerning method, depending on how the environment is set, the selectble ction set chnges nd the response to the ction chnges. In this study, we set the environment to hve two chmbers, nd wfer flow pttern (1,1) with ll process times to be 4 seconds. Here, the wfer flow pttern refers to the number of prllel chmbers in ech process. The initil stte is set to be full loded, so ll the chmbers re filled with wfers tht hve not yet begun to process. Since we only del with the stedy stte, we only used (C 1, C 2, R, S 1, S 2, Z 1, Z 2 ) s the stte. With this environment settings, the gent continued lerning using Q-lerning in the direction of mximizing the totl return. Figure 3: Q-Lerning Algorithm However, there re other issues tht need to be tken cre of by gents: dedlock ctions nd invlid ctions. 157
6 We wnt to ensure tht when n gent chooses n ction in the current stte, it does not choose n invlid ction nd dedlock ction tht cuses the next stte to become dedlock. The gent goes through the process of defining possible ction sets for the current stte by two-step look hed method. To remove the invlid ctions for the current stte, the gent requests the environment for the results for ll the ctions, then sves only which ctions re vlid (one-step look hed). For n ction tht is vlid for current stte, second look hed is needed to check if the dedlock occurs. It is time-consuming to lwys hve two-step look hed in every lerning step. To further speed up the procedures, we followed the dedlock prevention rules to the second look hed procedure. As we pply the rules, dedlock ction checking is needed only when ll of the loding ctions for the current stte re invlid nd the number of the holding wfer ( R ) is 1. In ddition, only unloding ctions mong vlid ctions need to be checked. If ny of the loding ctions re vlid, the gent does not need to do dedlock check. A wy to check if given unloding ction is dedlock is to request response for n pproprite loding ction. If the unloded wfer hs process step k to be visited, check tht it is possible to lod to the chmbers tht re in chrge of the kth process step. If ny of the loding ctions re vlid, the given unloding ction is not dedlock ction, otherwise, the given unloding ction is dedlock ction. For exmple, consider the cse illustrted in Figure 2, where chmber size (n) is 4, the current stte is (3, 2, 1, 1, 2, 1, 2, 0, 0, 4, 4, 0), nd the wfer flow pttern is (2,2). Since the vlue of n is 4, the crdinlity of the vilble ction sets for ll sttes is 15, which is the vlue of 3n + 3. Among these 15 ctions, the gent hs to void invlid ctions nd dedlock ctions. The gent first removes the invlid ctions from the ction set, nd then removes the dedlock ctions from the remining ction set. Through interction with the environment, the gent gets the invlid ctions nd stores only the vlid ctions, which is {Wit, U 0, U 1, U 4, SW 4 }. Since there is no loding ction mong the vlid ctions, ctions {U 0, U 1, U 4 } become the cndidtes for the dedlock ctions, by the dedlock checking rule tht we mentioned bove. Therefore, the gent needs to check the vlidity of the pproprite loding ctions for sttes (b), (c), nd (d) in Figure 4. In stte (b), unloded wfer hs to visit the 1 st process step; hence, request responses for ctions {L 0, L 1 }, which re in chrge of the 1 st process step. In this wy, responses for ctions {L 2, L 3 } nd {L 4 } re requested to check the dedness of the stte (c) nd (d), respectively. Then, the gent gets tht ny loding ctions re not vlid for stte (b) nd (c), nd {L 4 } is vlid for stte (d). Therefore, the unloding ctions corresponding to the stte (b) nd (c) re the dedlock ctions, so the ctions {U 0, U 1 } should be removed from the remining vlid ction set, {Wit, U 0, U 1, U 4, SW 4 }. Finlly, ctions {Wit, U 4, SW 4 } cn be chosen from the current stte S 0. Figure 4: Checking Dedlock Action Cndidtes We conducted Q-lerning bsed on the bove ction selection rule. The gent lwys chooses non-invlid nd non-dedlock ction, regrdless of the explortion rte ε. The reson for constructing this dditionl ction selection prt before the ction selection prt in Q- lerning is the lerning time. In the cluster tool, there re too mny invlid or dedlock ctions in the whole ction. If we do not forbid those inpproprite ctions, computtion expense to explore for unnecessry ction occurs nd unnecessry lerning time occurs. Therefore, by dding hnd-crfted djustments specific to this system, lerning time hs been shortened. 158
7 Tble 2: Q-vlues for Some Sttes 3.5. Performnce mesurement After the lerning is performed bsed on the proposed MDP model, Q(s, ) vlues for every stte s S nd ction A, re obtined. The policy we obtined is greedy policy tht selects the ction with the highest Q- vlue in ech stte, i.e., mx Q(s, ). To report the policy, we hve to list the selected ctions for sttes s shown in Tble 2. However, there re too mny sttes to list ll of them s tble, nd it is difficult to intuitively understnd wht ctions to tke in some cses. Therefore, fter we obtined the policy, we used the mesures tht represent the performnce of the policies, then compred the performnce of the cquired policy ginst the existing swp sequence. Through lerning, Q(s, ) for s S, A is continuously updted; define Q(s, ) for ech thousnd wfers produced s q 0, q 1,..., q t R S A. However, we do not use ll q 0, q 1,..., q t to get the policy; we only use the Q(s, ) vlue fter the convergence. To verify the convergence of sequence q 0, q 1,..., q t, q k q k 1 for k {1,, t} ws checked. If ll elements in the mtrix q k q k 1 is reltively smll, the q k cn be considered to be converged. Figure 5 shows the mximum element in mtrix q k q k 1 for k {1,,543}. The vlue converges to 0, which mens the sequence q 0, q 1,..., q t converged. We used the converged Q(s, ) vlue fter the 543 th itertion, i.e., used q 543 to obtin the policy. To indicte the performnce of the obtined policy, the time it tkes to process 50 wfers, mkespn, is used. Mkespn is mesured in the sme environment s the lerning period. We used two processing chmbers, wfer flow pttern (1,1), verge process time of four, nd the chmber process time is set to follow the bet(50,50) distribution. Becuse the process time is rndomly generted, mkespn cn be mesured differently, even if the mesurement is implemented under the sme policy. Therefore, we mesured mkespn thousnd times under the designted policies nd compred the verges. Figure 5: Q-vlue difference mximum vlue 4. ANALYSIS OF ROBOT POLICY OBTAINED BY LEARNING Figure 6 represents the verge q-vlues of the obtined policy nd the swp policy. Figure 7 nd Tble 3 represent the mkespn differences of the two policies s histogrm nd tble, respectively. M p nd M s represent mkespn under the obtined policy nd the swp policy. We confirmed tht the mkespn of the policy obtined through Q-lerning bsed on the proposed MDP model 159
8 is on verge shorter thn the swp policy s shown in Figure 6. Tble 3: Frequency Distribution of the Q-vlue Difference Figure 6: Averge Q-vlues of Two Policies Except for 2.3% cses, the others cquired the sme or shorter mkespn with the obtined policy. This mens tht the bove reinforcement lerning hs found good robot sequence nd obtined slightly better policy thn the widely used swp policy. The verge of mkespn is not significntly different s illustrted in Figure 6. It seems tht the smll difference is due to the settings, the verge process time nd the number of the chmbers in our lerning environment. Both of the settings my hve reduced the time vrition effect to the environment. If we were to increse the verge process time nd number of chmbers s much s the ctul cluster tool, then the time vrince would hve incresed, nd the difference in mkespn my hve incresed. However, the fct tht mkespn is reduced in most cses mens tht s policy run in time vrince environment, the policy obtined through lerning in time vrince environment is better thn the policy obtined in deterministic environment. 5. CONCLUSIONS Cluster tool scheduling hs been usully studied under the deterministic environment. In this pper, we modeled dul-rmed cluster tool behviors s Mrkov decision process in the time vrince environment, then pplied reinforcement lerning to obtin robot policies. The newly obtined robot policies reduces the mkespn of 50 wfer processes compre to the conventionl swp sequence in most cses. From this study, it seems tht the proposed MDP models cn dequtely express the cluster tool behviors in time vrince environment. This kind of view mkes it possible to look t the cluster tool from slightly different perspective; hence we cn continue to try scheduling cluster tools through reinforcement lerning. However, since the bove model includes the remining time in the stte in order to consider the time vrince, the stte spce cn be very lrge. Furthermore, the environment settings we used in lerning couldn t fully express the rel world cluster tool behviors. Therefore, the lerning with more generl environment settings is needed. To conduct such lerning in further reserch, functionl pproximtion seems to be pplied due to the time elements in the sttes. Figure 7: Histogrm Grph Showing the Q-vlue Differences ACKNOWLEDGMENTS This reserch ws supported by the Bsic Science Reserch Progrm through the Ntionl Reserch Foundtion of Kore (NRF) funded by the Ministry of Eduction, Science nd Technology (2015R1D1A1A ). REFERENCES Bellmn, R., The theory of dynmic progrmming (No. RAND-P-550). RAND CORP SANTA MONICA CA. 160
9 Bertseks, D. P., Dynmic progrmming nd optiml control (Vol. 1, No. 2). Belmont, MA: Athen scientific. Choi, B. K., & Kng, D., Modeling nd simultion of discrete event systems. John Wiley & Sons. Gosvi, A., Simultion-bsed optimiztion. prmetric optimiztion techniques nd reinforcement lerning. Jung, C., & Lee, T. E., An efficient mixed integer progrmming model bsed on timed Petri nets for diverse complex cluster tool scheduling problems. IEEE Trnsctions on Semiconductor Mnufcturing, 25(2), Kim, D. K., Lee, T. E., & Kim, H. J., Optiml scheduling of trnsient cycles for single-rmed cluster tools with prllel chmbers. IEEE Trnsctions on Automtion Science nd Engineering, 13(2), Kim, J. H., & Lee, T. E., Schedulbility nlysis of time-constrined cluster tools with bounded time vrition by n extended Petri net. IEEE Trnsctions on Automtion Science nd Engineering, 5(3), Lee, T. E., Lee, H. Y., & Shin, Y. H., 2004, December. Worklod blncing nd scheduling of singlermed cluster tool. In Proceedings of the 5th APIEMS Conference (pp. 1-15). Gold Cost, Austrli. Lee, T. E., 2008, December. A review of scheduling theory nd methods for semiconductor mnufcturing cluster tools. In Proceedings of the 40th conference on winter simultion (pp ). Winter Simultion Conference. Mnih, V., Kvukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemre, M. G.,... & Petersen, S., Humn-level control through deep reinforcement lerning. Nture, 518(7540), Molloy, M. K., Performnce nlysis using stochstic Petri nets. IEEE Trnsctions on computers, 31(9), Moody, J., Wu, L., Lio, Y., & Sffell, M., Performnce functions nd reinforcement lerning for trding systems nd portfolios. Journl of Forecsting, 17(56), Murt, T.,1989. Petri nets: Properties, nlysis nd pplictions. Proceedings of the IEEE, 77(4), Putermn, M. L., Mrkov decision processes: discrete stochstic dynmic progrmming. John Wiley & Sons. Qio, Y., Wu, N., & Zhou, M., Rel-time scheduling of single-rm cluster tools subject to residency time constrints nd bounded ctivity time vrition. IEEE Trnsctions on Automtion Science nd Engineering, 9(3), Sutton, R. S., & Brto, A. G., Reinforcement lerning: An introduction (Vol. 1, No. 1). Cmbridge: MIT press. Zuberek, W. M., Cluster tools with chmber revisiting-modeling nd nlysis using timed Petri nets. IEEE Trnsctions on semiconductor mnufcturing, 17(3),
A New Learning Algorithm for the MAXQ Hierarchical Reinforcement Learning Method
A New Lerning Algorithm for the MAXQ Hierrchicl Reinforcement Lerning Method Frzneh Mirzzdeh 1, Bbk Behsz 2, nd Hmid Beigy 1 1 Deprtment of Computer Engineering, Shrif University of Technology, Tehrn,
More informationComplete Coverage Path Planning of Mobile Robot Based on Dynamic Programming Algorithm Peng Zhou, Zhong-min Wang, Zhen-nan Li, Yang Li
2nd Interntionl Conference on Electronic & Mechnicl Engineering nd Informtion Technology (EMEIT-212) Complete Coverge Pth Plnning of Mobile Robot Bsed on Dynmic Progrmming Algorithm Peng Zhou, Zhong-min
More informationLecture 10 Evolutionary Computation: Evolution strategies and genetic programming
Lecture 10 Evolutionry Computtion: Evolution strtegies nd genetic progrmming Evolution strtegies Genetic progrmming Summry Negnevitsky, Person Eduction, 2011 1 Evolution Strtegies Another pproch to simulting
More informationMidterm 2 Sample solution
Nme: Instructions Midterm 2 Smple solution CMSC 430 Introduction to Compilers Fll 2012 November 28, 2012 This exm contins 9 pges, including this one. Mke sure you hve ll the pges. Write your nme on the
More informationINTRODUCTION TO SIMPLICIAL COMPLEXES
INTRODUCTION TO SIMPLICIAL COMPLEXES CASEY KELLEHER AND ALESSANDRA PANTANO 0.1. Introduction. In this ctivity set we re going to introduce notion from Algebric Topology clled simplicil homology. The min
More informationA Heuristic Approach for Discovering Reference Models by Mining Process Model Variants
A Heuristic Approch for Discovering Reference Models by Mining Process Model Vrints Chen Li 1, Mnfred Reichert 2, nd Andres Wombcher 3 1 Informtion System Group, University of Twente, The Netherlnds lic@cs.utwente.nl
More information1. SEQUENCES INVOLVING EXPONENTIAL GROWTH (GEOMETRIC SEQUENCES)
Numbers nd Opertions, Algebr, nd Functions 45. SEQUENCES INVOLVING EXPONENTIAL GROWTH (GEOMETRIC SEQUENCES) In sequence of terms involving eponentil growth, which the testing service lso clls geometric
More informationDynamic Programming. Andreas Klappenecker. [partially based on slides by Prof. Welch] Monday, September 24, 2012
Dynmic Progrmming Andres Klppenecker [prtilly bsed on slides by Prof. Welch] 1 Dynmic Progrmming Optiml substructure An optiml solution to the problem contins within it optiml solutions to subproblems.
More informationECE 468/573 Midterm 1 September 28, 2012
ECE 468/573 Midterm 1 September 28, 2012 Nme:! Purdue emil:! Plese sign the following: I ffirm tht the nswers given on this test re mine nd mine lone. I did not receive help from ny person or mteril (other
More informationAn Efficient Divide and Conquer Algorithm for Exact Hazard Free Logic Minimization
An Efficient Divide nd Conquer Algorithm for Exct Hzrd Free Logic Minimiztion J.W.J.M. Rutten, M.R.C.M. Berkelr, C.A.J. vn Eijk, M.A.J. Kolsteren Eindhoven University of Technology Informtion nd Communiction
More informationUnit #9 : Definite Integral Properties, Fundamental Theorem of Calculus
Unit #9 : Definite Integrl Properties, Fundmentl Theorem of Clculus Gols: Identify properties of definite integrls Define odd nd even functions, nd reltionship to integrl vlues Introduce the Fundmentl
More information12-B FRACTIONS AND DECIMALS
-B Frctions nd Decimls. () If ll four integers were negtive, their product would be positive, nd so could not equl one of them. If ll four integers were positive, their product would be much greter thn
More information2 Computing all Intersections of a Set of Segments Line Segment Intersection
15-451/651: Design & Anlysis of Algorithms Novemer 14, 2016 Lecture #21 Sweep-Line nd Segment Intersection lst chnged: Novemer 8, 2017 1 Preliminries The sweep-line prdigm is very powerful lgorithmic design
More informationDQL: A New Updating Strategy for Reinforcement Learning Based on Q-Learning
DQL: A New Updting Strtegy for Reinforcement Lerning Bsed on Q-Lerning Crlos E. Mrino 1 nd Edurdo F. Morles 2 1 Instituto Mexicno de Tecnologí del Agu, Pseo Cuhunáhuc 8532, Jiutepec, Morelos, 6255, MEXICO
More information1 Quad-Edge Construction Operators
CS48: Computer Grphics Hndout # Geometric Modeling Originl Hndout #5 Stnford University Tuesdy, 8 December 99 Originl Lecture #5: 9 November 99 Topics: Mnipultions with Qud-Edge Dt Structures Scribe: Mike
More informationMATH 25 CLASS 5 NOTES, SEP
MATH 25 CLASS 5 NOTES, SEP 30 2011 Contents 1. A brief diversion: reltively prime numbers 1 2. Lest common multiples 3 3. Finding ll solutions to x + by = c 4 Quick links to definitions/theorems Euclid
More informationMA1008. Calculus and Linear Algebra for Engineers. Course Notes for Section B. Stephen Wills. Department of Mathematics. University College Cork
MA1008 Clculus nd Liner Algebr for Engineers Course Notes for Section B Stephen Wills Deprtment of Mthemtics University College Cork s.wills@ucc.ie http://euclid.ucc.ie/pges/stff/wills/teching/m1008/ma1008.html
More informationIn the last lecture, we discussed how valid tokens may be specified by regular expressions.
LECTURE 5 Scnning SYNTAX ANALYSIS We know from our previous lectures tht the process of verifying the syntx of the progrm is performed in two stges: Scnning: Identifying nd verifying tokens in progrm.
More informationEngineer To Engineer Note
Engineer To Engineer Note EE-169 Technicl Notes on using Anlog Devices' DSP components nd development tools Contct our technicl support by phone: (800) ANALOG-D or e-mil: dsp.support@nlog.com Or visit
More information9 Graph Cutting Procedures
9 Grph Cutting Procedures Lst clss we begn looking t how to embed rbitrry metrics into distributions of trees, nd proved the following theorem due to Brtl (1996): Theorem 9.1 (Brtl (1996)) Given metric
More informationMisrepresentation of Preferences
Misrepresenttion of Preferences Gicomo Bonnno Deprtment of Economics, University of Cliforni, Dvis, USA gfbonnno@ucdvis.edu Socil choice functions Arrow s theorem sys tht it is not possible to extrct from
More informationOn the Detection of Step Edges in Algorithms Based on Gradient Vector Analysis
On the Detection of Step Edges in Algorithms Bsed on Grdient Vector Anlysis A. Lrr6, E. Montseny Computer Engineering Dept. Universitt Rovir i Virgili Crreter de Slou sin 43006 Trrgon, Spin Emil: lrre@etse.urv.es
More informationCSEP 573 Artificial Intelligence Winter 2016
CSEP 573 Artificil Intelligence Winter 2016 Luke Zettlemoyer Problem Spces nd Serch slides from Dn Klein, Sturt Russell, Andrew Moore, Dn Weld, Pieter Abbeel, Ali Frhdi Outline Agents tht Pln Ahed Serch
More informationTool Vendor Perspectives SysML Thus Far
Frontiers 2008 Pnel Georgi Tec, 05-13-08 Tool Vendor Perspectives SysML Thus Fr Hns-Peter Hoffmnn, Ph.D Chief Systems Methodologist Telelogic, Systems & Softwre Modeling Business Unit Peter.Hoffmnn@telelogic.com
More informationFig.25: the Role of LEX
The Lnguge for Specifying Lexicl Anlyzer We shll now study how to uild lexicl nlyzer from specifiction of tokens in the form of list of regulr expressions The discussion centers round the design of n existing
More informationP(r)dr = probability of generating a random number in the interval dr near r. For this probability idea to make sense we must have
Rndom Numers nd Monte Crlo Methods Rndom Numer Methods The integrtion methods discussed so fr ll re sed upon mking polynomil pproximtions to the integrnd. Another clss of numericl methods relies upon using
More informationA Transportation Problem Analysed by a New Ranking Method
(IJIRSE) Interntionl Journl of Innovtive Reserch in Science & Engineering ISSN (Online) 7-07 A Trnsporttion Problem Anlysed by New Rnking Method Dr. A. Shy Sudh P. Chinthiy Associte Professor PG Scholr
More informationCOMP 423 lecture 11 Jan. 28, 2008
COMP 423 lecture 11 Jn. 28, 2008 Up to now, we hve looked t how some symols in n lphet occur more frequently thn others nd how we cn sve its y using code such tht the codewords for more frequently occuring
More informationPointwise convergence need not behave well with respect to standard properties such as continuity.
Chpter 3 Uniform Convergence Lecture 9 Sequences of functions re of gret importnce in mny res of pure nd pplied mthemtics, nd their properties cn often be studied in the context of metric spces, s in Exmples
More informationsuch that the S i cover S, or equivalently S
MATH 55 Triple Integrls Fll 16 1. Definition Given solid in spce, prtition of consists of finite set of solis = { 1,, n } such tht the i cover, or equivlently n i. Furthermore, for ech i, intersects i
More informationSolving Problems by Searching. CS 486/686: Introduction to Artificial Intelligence Winter 2016
Solving Prolems y Serching CS 486/686: Introduction to Artificil Intelligence Winter 2016 1 Introduction Serch ws one of the first topics studied in AI - Newell nd Simon (1961) Generl Prolem Solver Centrl
More informationCS321 Languages and Compiler Design I. Winter 2012 Lecture 5
CS321 Lnguges nd Compiler Design I Winter 2012 Lecture 5 1 FINITE AUTOMATA A non-deterministic finite utomton (NFA) consists of: An input lphet Σ, e.g. Σ =,. A set of sttes S, e.g. S = {1, 3, 5, 7, 11,
More informationRevisiting the notion of Origin-Destination Traffic Matrix of the Hosts that are attached to a Switched Local Area Network
Interntionl Journl of Distributed nd Prllel Systems (IJDPS) Vol., No.6, November 0 Revisiting the notion of Origin-Destintion Trffic Mtrix of the Hosts tht re ttched to Switched Locl Are Network Mondy
More informationExam #1 for Computer Simulation Spring 2005
Exm # for Computer Simultion Spring 005 >>> SOLUTION
More informationRepresentation of Numbers. Number Representation. Representation of Numbers. 32-bit Unsigned Integers 3/24/2014. Fixed point Integer Representation
Representtion of Numbers Number Representtion Computer represent ll numbers, other thn integers nd some frctions with imprecision. Numbers re stored in some pproximtion which cn be represented by fixed
More informationSection 10.4 Hyperbolas
66 Section 10.4 Hyperbols Objective : Definition of hyperbol & hyperbols centered t (0, 0). The third type of conic we will study is the hyperbol. It is defined in the sme mnner tht we defined the prbol
More informationThe Distributed Data Access Schemes in Lambda Grid Networks
The Distributed Dt Access Schemes in Lmbd Grid Networks Ryot Usui, Hiroyuki Miygi, Yutk Arkw, Storu Okmoto, nd Noki Ymnk Grdute School of Science for Open nd Environmentl Systems, Keio University, Jpn
More informationBall. Player X. Player O. X Goal. O Goal
Generlizing Adversril Reinforcement Lerning Willim T. B. Uther nd Mnuel M. Veloso Computer Science Deprtment Crnegie Mellon University Pittsburgh, PA 15213 futher,velosog@cs.cmu.edu Abstrct Reinforcement
More informationFile Manager Quick Reference Guide. June Prepared for the Mayo Clinic Enterprise Kahua Deployment
File Mnger Quick Reference Guide June 2018 Prepred for the Myo Clinic Enterprise Khu Deployment NVIGTION IN FILE MNGER To nvigte in File Mnger, users will mke use of the left pne to nvigte nd further pnes
More informationDigital Design. Chapter 6: Optimizations and Tradeoffs
Digitl Design Chpter 6: Optimiztions nd Trdeoffs Slides to ccompny the tetbook Digitl Design, with RTL Design, VHDL, nd Verilog, 2nd Edition, by Frnk Vhid, John Wiley nd Sons Publishers, 2. http://www.ddvhid.com
More informationSection 3.1: Sequences and Series
Section.: Sequences d Series Sequences Let s strt out with the definition of sequence: sequence: ordered list of numbers, often with definite pttern Recll tht in set, order doesn t mtter so this is one
More informationSmall Business Networking
Why network is n essentil productivity tool for ny smll business Effective technology is essentil for smll businesses looking to increse the productivity of their people nd business. Introducing technology
More informationFig.1. Let a source of monochromatic light be incident on a slit of finite width a, as shown in Fig. 1.
Answer on Question #5692, Physics, Optics Stte slient fetures of single slit Frunhofer diffrction pttern. The slit is verticl nd illuminted by point source. Also, obtin n expression for intensity distribution
More informationthis grammar generates the following language: Because this symbol will also be used in a later step, it receives the
LR() nlysis Drwcks of LR(). Look-hed symols s eplined efore, concerning LR(), it is possile to consult the net set to determine, in the reduction sttes, for which symols it would e possile to perform reductions.
More informationPreserving Constraints for Aggregation Relationship Type Update in XML Document
Preserving Constrints for Aggregtion Reltionship Type Updte in XML Document Eric Prdede 1, J. Wenny Rhyu 1, nd Dvid Tnir 2 1 Deprtment of Computer Science nd Computer Engineering, L Trobe University, Bundoor
More informationPolicy-contingent state abstraction for
Policy-contingent stte bstrction for hierrchicl MDPs oelle Pineu nd Geoffrey Gordon School of Computer Science Crnegie Mellon University Pittsburgh PA 15213 jpineuggordon @cs.cmu.edu Abstrct Hierrchiclly
More informationEpson Projector Content Manager Operation Guide
Epson Projector Content Mnger Opertion Guide Contents 2 Introduction to the Epson Projector Content Mnger Softwre 3 Epson Projector Content Mnger Fetures... 4 Setting Up the Softwre for the First Time
More informationA HYDRAULIC SIMULATOR FOR AN EXCAVATOR
P-06 Proceedings of the 7th JFPS Interntionl Symposium on Fluid Power TOYAMA 008 September 5-8 008 A HYDRAULIC SIMULATOR FOR AN EXCAVATOR Soon-Kwng Kwon* Je-Jun Kim* Young-Mn Jung* Chn-Se Jung* Chng-Don
More informationPolicy-contingent state abstraction for hierarchical MDPs
Policy-contingent stte bstrction for hierrchicl MDPs Joelle Pineu nd Geoffrey Gordon School of Computer Science Crnegie Mellon University Pittsburgh, PA 15213 jpineu,ggordon@cs.cmu.edu Abstrct Hierrchiclly
More informationSlides for Data Mining by I. H. Witten and E. Frank
Slides for Dt Mining y I. H. Witten nd E. Frnk Simplicity first Simple lgorithms often work very well! There re mny kinds of simple structure, eg: One ttriute does ll the work All ttriutes contriute eqully
More informationSmall Business Networking
Why network is n essentil productivity tool for ny smll business Effective technology is essentil for smll businesses looking to increse the productivity of their people nd processes. Introducing technology
More informationSmall Business Networking
Why network is n essentil productivity tool for ny smll business Effective technology is essentil for smll businesses looking to increse the productivity of their people nd business. Introducing technology
More informationMATH 2530: WORKSHEET 7. x 2 y dz dy dx =
MATH 253: WORKSHT 7 () Wrm-up: () Review: polr coordintes, integrls involving polr coordintes, triple Riemnn sums, triple integrls, the pplictions of triple integrls (especilly to volume), nd cylindricl
More informationVoIP for the Small Business
Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become vible solution for even the
More informationL. Yaroslavsky. Fundamentals of Digital Image Processing. Course
L. Yroslvsky. Fundmentls of Digitl Imge Processing. Course 0555.330 Lecture. Imge enhncement.. Imge enhncement s n imge processing tsk. Clssifiction of imge enhncement methods Imge enhncement is processing
More informationTree Structured Symmetrical Systems of Linear Equations and their Graphical Solution
Proceedings of the World Congress on Engineering nd Computer Science 4 Vol I WCECS 4, -4 October, 4, Sn Frncisco, USA Tree Structured Symmetricl Systems of Liner Equtions nd their Grphicl Solution Jime
More informationComputing offsets of freeform curves using quadratic trigonometric splines
Computing offsets of freeform curves using qudrtic trigonometric splines JIULONG GU, JAE-DEUK YUN, YOONG-HO JUNG*, TAE-GYEONG KIM,JEONG-WOON LEE, BONG-JUN KIM School of Mechnicl Engineering Pusn Ntionl
More informationEngineer To Engineer Note
Engineer To Engineer Note EE-186 Technicl Notes on using Anlog Devices' DSP components nd development tools Contct our technicl support by phone: (800) ANALOG-D or e-mil: dsp.support@nlog.com Or visit
More informationpdfapilot Server 2 Manual
pdfpilot Server 2 Mnul 2011 by clls softwre gmbh Schönhuser Allee 6/7 D 10119 Berlin Germny info@cllssoftwre.com www.cllssoftwre.com Mnul clls pdfpilot Server 2 Pge 2 clls pdfpilot Server 2 Mnul Lst modified:
More informationTransparent neutral-element elimination in MPI reduction operations
Trnsprent neutrl-element elimintion in MPI reduction opertions Jesper Lrsson Träff Deprtment of Scientific Computing University of Vienn Disclimer Exploiting repetition nd sprsity in input for reducing
More informationVoIP for the Small Business
VoIP for the Smll Business Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become
More informationVoIP for the Small Business
VoIP for the Smll Business Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become
More informationII. THE ALGORITHM. A. Depth Map Processing
Lerning Plnr Geometric Scene Context Using Stereo Vision Pul G. Bumstrck, Bryn D. Brudevold, nd Pul D. Reynolds {pbumstrck,brynb,pulr2}@stnford.edu CS229 Finl Project Report December 15, 2006 Abstrct A
More informationSmall Business Networking
Why network is n essentil productivity tool for ny smll business Effective technology is essentil for smll businesses looking to increse the productivity of their people nd business. Introducing technology
More information2014 Haskell January Test Regular Expressions and Finite Automata
0 Hskell Jnury Test Regulr Expressions nd Finite Automt This test comprises four prts nd the mximum mrk is 5. Prts I, II nd III re worth 3 of the 5 mrks vilble. The 0 Hskell Progrmming Prize will be wrded
More informationVoIP for the Small Business
Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become vible solution for even the
More informationSmall Business Networking
Why network is n essentil productivity tool for ny smll business Effective technology is essentil for smll businesses looking to increse the productivity of their people nd business. Introducing technology
More informationSmall Business Networking
Why network is n essentil productivity tool for ny smll business Effective technology is essentil for smll businesses looking to increse the productivity of their people nd processes. Introducing technology
More informationVoIP for the Small Business
Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become vible solution for even the
More informationVoIP for the Small Business
Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become vible solution for even the
More informationVoIP for the Small Business
Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become vible solution for even the
More informationVoIP for the Small Business
Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. TechAdvisory.org SME Reports sponsored by Voice over Internet Protocol (VoIP)
More informationPerformance Evaluation of Dynamic Reconfiguration in High-Speed Local Area Networks
Performnce Evlution of Dynmic Reconfigurtion in High-Speed Locl Are Networks Rfel Csdo, Aurelio Bermúdez, Frncisco J. Quiles, JoséL.Sánchez Depto. de Informátic Universidd de Cstill-L Mnch 271- Albcete,
More informationSubtracting Fractions
Lerning Enhncement Tem Model Answers: Adding nd Subtrcting Frctions Adding nd Subtrcting Frctions study guide. When the frctions both hve the sme denomintor (bottom) you cn do them using just simple dding
More informationAI Adjacent Fields. This slide deck courtesy of Dan Klein at UC Berkeley
AI Adjcent Fields Philosophy: Logic, methods of resoning Mind s physicl system Foundtions of lerning, lnguge, rtionlity Mthemtics Forml representtion nd proof Algorithms, computtion, (un)decidility, (in)trctility
More informationOptimization of Air Bearing Slider Design
Proceedings of TC2005 orld Tribology Congress III Proceedings of TC2005 September 2-6, orld 2005, Tribology shington, Congress D.C., III SA September 2-6, 2005, shington, D.C., SA Optimiztion of Air Bering
More informationVoIP for the Small Business
Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become vible solution for even the
More informationDefinition of Regular Expression
Definition of Regulr Expression After the definition of the string nd lnguges, we re redy to descrie regulr expressions, the nottion we shll use to define the clss of lnguges known s regulr sets. Recll
More informationVoIP for the Small Business
VoIP for the Smll Business Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become
More informationDr. D.M. Akbar Hussain
Dr. D.M. Akr Hussin Lexicl Anlysis. Bsic Ide: Red the source code nd generte tokens, it is similr wht humns will do to red in; just tking on the input nd reking it down in pieces. Ech token is sequence
More informationUnit 5 Vocabulary. A function is a special relationship where each input has a single output.
MODULE 3 Terms Definition Picture/Exmple/Nottion 1 Function Nottion Function nottion is n efficient nd effective wy to write functions of ll types. This nottion llows you to identify the input vlue with
More informationVoIP for the Small Business
Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become vible solution for even the
More informationSmall Business Networking
Why network is n essentil productivity tool for ny smll business Effective technology is essentil for smll businesses looking to increse the productivity of their people nd processes. Introducing technology
More informationComputer-Aided Multiscale Modelling for Chemical Process Engineering
17 th Europen Symposium on Computer Aided Process Engineesing ESCAPE17 V. Plesu nd P.S. Agchi (Editors) 2007 Elsevier B.V. All rights reserved. 1 Computer-Aided Multiscle Modelling for Chemicl Process
More informationTO REGULAR EXPRESSIONS
Suject :- Computer Science Course Nme :- Theory Of Computtion DA TO REGULAR EXPRESSIONS Report Sumitted y:- Ajy Singh Meen 07000505 jysmeen@cse.iit.c.in BASIC DEINITIONS DA:- A finite stte mchine where
More informationVoIP for the Small Business
Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become vible solution for even the
More informationVoIP for the Small Business
VoIP for the Smll Business Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become
More information10.5 Graphing Quadratic Functions
0.5 Grphing Qudrtic Functions Now tht we cn solve qudrtic equtions, we wnt to lern how to grph the function ssocited with the qudrtic eqution. We cll this the qudrtic function. Grphs of Qudrtic Functions
More informationImproper Integrals. October 4, 2017
Improper Integrls October 4, 7 Introduction We hve seen how to clculte definite integrl when the it is rel number. However, there re times when we re interested to compute the integrl sy for emple 3. Here
More informationSmall Business Networking
Why network is n essentil productivity tool for ny smll business Effective technology is essentil for smll businesses looking to increse the productivity of their people nd processes. Introducing technology
More information4452 Mathematical Modeling Lecture 4: Lagrange Multipliers
Mth Modeling Lecture 4: Lgrnge Multipliers Pge 4452 Mthemticl Modeling Lecture 4: Lgrnge Multipliers Lgrnge multipliers re high powered mthemticl technique to find the mximum nd minimum of multidimensionl
More informationPNC NC code PROGRAMMER'S MANUAL
PNC-3200 NC code PROGRAMMER'S MANUAL Thnk you very much for purchsing the PNC-3200. To ensure correct nd sfe usge with full understnding of this product's performnce, plese be sure to red through this
More informationAddress Register Assignment for Reducing Code Size
Address Register Assignment for Reducing Code Size M. Kndemir 1, M.J. Irwin 1, G. Chen 1, nd J. Rmnujm 2 1 CSE Deprtment Pennsylvni Stte University University Prk, PA 16802 {kndemir,mji,guilchen}@cse.psu.edu
More informationCS 221: Artificial Intelligence Fall 2011
CS 221: Artificil Intelligence Fll 2011 Lecture 2: Serch (Slides from Dn Klein, with help from Sturt Russell, Andrew Moore, Teg Grenger, Peter Norvig) Problem types! Fully observble, deterministic! single-belief-stte
More informationVoIP for the Small Business
Reducing your telecommunictions costs Reserch firm IDC 1 hs estimted tht VoIP system cn reduce telephony-relted expenses by 30%. Voice over Internet Protocol (VoIP) hs become vible solution for even the
More informationApproximation by NURBS with free knots
pproximtion by NURBS with free knots M Rndrinrivony G Brunnett echnicl University of Chemnitz Fculty of Computer Science Computer Grphics nd Visuliztion Strße der Ntionen 6 97 Chemnitz Germny Emil: mhrvo@informtiktu-chemnitzde
More informationAn introduction to model checking
An introduction to model checking Slide 1 University of Albert Edmonton July 3rd, 2002 Guy Trembly Dépt d informtique UQAM Outline Wht re forml specifiction nd verifiction methods? Slide 2 Wht is model
More informationCHANGING STRATA AND SELECTION PROBABILITIES* Leslie Kish, The University of Michigan. Summary
124 CHANGING STRATA AND SELECTION PROBABILITIES* Leslie Kish, The University of Michign Summry Survey smples re often bsed on primry units selected from initil strt with probbilities proportionl to initil
More informationParallel Square and Cube Computations
Prllel Squre nd Cube Computtions Albert A. Liddicot nd Michel J. Flynn Computer Systems Lbortory, Deprtment of Electricl Engineering Stnford University Gtes Building 5 Serr Mll, Stnford, CA 945, USA liddicot@stnford.edu
More informationCSCI 446: Artificial Intelligence
CSCI 446: Artificil Intelligence Serch Instructor: Michele Vn Dyne [These slides were creted by Dn Klein nd Pieter Abbeel for CS188 Intro to AI t UC Berkeley. All CS188 mterils re vilble t http://i.berkeley.edu.]
More information