Patent Application Publication. Apr. 24, 2014 Sheet 1 0f 6 US 2014/ A1. mm msm E996. Em:

Transcription

1 Patent Appication Pubication Apr. 24, 2014 Sheet 1 0f 6 US 2014/ A1 N@ msm mm msm E996 mm r 4 _1 _Q:mEoO om Em: É

2 Patent Appication Pubication Apr. 24, 2014 Sheet 2 of 6 US 2014/ A1 90 Software Appication 100 t Guest (Virtua Hardware Operating \ Machine) Architecture System n 94 Emuation Program (e.g., Hypervisor of Host Operating System) J / /'/ Physica Hardware Architecture 92 /// FIG. 2

3 Patent Appication Pubication Apr. 24, 2014 Sheet 3 0f 6 US 2014/ A1

4 Patent Appication Pubication Apr. 24, 2014 Sheet 4 0f 6 US 2014/ A1...w-"_ mm wor Emwîm ÉS _œës.42 NS

5 Patent Appication Pubication Apr. 24, 2014 Sheet 5 of 6 US 2014/ A1 LBQiëâi ***** L ` g Core *Processor M Core *P_ffîâsâêsß M 4 ìphysioa Processor L@ " ' rios 'y Physica L3 / Physica L3 /V/ Processor I Processor,A/ /? g Physica L3 Physica L3 Processor Processor Physica Physica L. - 3 Processor Processor Physica L3 Physica L3 Processor Processor Node Mem Node Mem m Two Node System 1_ 412 ' Node 1_ 412

6 ê Patent Appication Pubication Apr. 24, 2014 Sheet 6 of 6 US 2014/ A1 FIG. 5 _ Virtuaizer (VM) ÉVirtuaizer changes Virtuaizer directy becomes aware of virtuaìzed updates guest GS resource change processor topoogy interna tabes FIG. 6 suegros 63? 6.02 OS \ Scheduer 1 M ß 635 ' È\/ 3\(3him3 virtua " OS 60 Scheduer 538/ MemOW X"

7 US 2014/ A1 Apr. 24, 2014 SYSTEMS AND METHODS FOR EXPOSING PROCESSOR TOPOLOGY FOR VIRTUAL MACHINES CROSS REFERENCE TO RELATED APPLICATIONS [0001] This appication is a continuation of U.S. patent appication Ser. No. 11/018,337 fied on Dec. 21, 2004, the entirety which is incorporated herein by reference. FIELD OF THE INVENTION [0002] The present invention generay reates to the fied of virtua machines (aso known as processor virtuaization ) and to operating systems that execute in virtua machine environments. More specificay, the present invention is directed to systems and methods for exposing the processor topoogy of a virtua machine to a guest operating system executing on a virtua machine wherein said topoogy is dynamic based on aocations of host computer system pro cessor and memory resources. BACKGROUND OF THE INVENTION [0003] Computers incude genera purpose centra process ing units (CPUs) or processors that are designed to execute a specific set of system instructions. A group of processors that have simiar architecture or design specifications may be considered to be members of the same processor famiy. Exampes of current processor famiies incude the Motoroa 680X0 processor famiy, manufactured by Motoroa, Inc. of Phoenix, Ariz., the Inte 80X86 processor famiy, manufac tured by Inte Corporation of Sunnyvae, Caif., and the Pow erpc processor famiy, which is manufactured by Motoroa, Inc. and used in computers manufactured by Appe Com puter, Inc. of Cupertino, Caif. Athough a group of proces sors may be in the same famiy because of their simiar architecture and design considerations, processors may vary widey within a famiy according to their cock speed and other performance parameters. [0004] Each famiy of microprocessors executes instruc tions that are unique to the proces sor famiy. The coective set of instructions that a processor or famiy of processors can execute is known as the processor s instruction set. As an exampe, the instruction set used by the Inte 80X86 proces sor famiy is incompatibe with the instruction set used by the PowerPC processor famiy. The Inte 80X86 instruction set is based on the Compex Instruction Set Computer (CISC) for mat. The Motoroa PowerPC instruction set is based on the Reduced Instruction Set Computer (RISC) format. CISC pro cessors use a arge number of instructions, some of which can perform rather compicated functions, but which require gen eray many cock cyces to execute. RISC processors use a smaer number of avaiabe instructions to perform a simper set of functions that are executed at a much higher rate. [0005] The uniqueness ofthe processor famiy among com puter systems aso typicay resuts in incompatibiity among the other eements of hardware architecture of the computer systems. A computer system manufactured with a processor from the Inte 80X86 processor famiy wi have a hardware architecture that is different from the hardware architecture of a computer system manufactured with a processor from the PowerPC processor famiy. Because of the uniqueness of the processor instruction set and a computer system s hardware architecture, appication software programs are typicay written to run on a particuar computer system running a particuar operating system. Virtua Machines [0006] Computer manufacturers want to maximize their market share by having more rather than fewer appications run on the microprocessor famiy associated with the com puter manufacturers product ine. To expand the number of operating systems and appication programs that can run on a computer system, a fied of technoogy has deveoped in which a given computer having one type of CPU, caed a host, wi incude a virtuaizer program that aows the host computer to emuate the instructions of an unreated type of CPU, caed a guest. Thus, the host computer wi execute an appication that wi cause one or more host instructions to be caed in response to a given guest instruction, and in this way the host computer can both run software designed for its own hardware architecture and software written for computers having an unreated hardware architecture. [0007] As a more specific exampe, a computer system manufactured by Appe Computer, for exampe, may run operating systems and program written for PC-based com puter systems. It may aso be possibe to use virtuaizer pro grams to execute concurrenty on a singe CPU mutipe incompatibe operating systems. In this atter arrangement, athough each operating system is incompatibe with the other, virtuaizer programs can host each of the severa oper ating systems and thereby aowing the otherwise incompat ibe operating systems to run concurrenty on the same host computer system. [0008] When a guest computer system is emuated on a host computer system, the guest computer system is said to be a virtua machine as the guest computer system ony exists in the host computer system as a pure software representation of the operation of one specific hardware architecture. The terms virtuaizer, emuator, direct-executor, virtua machine, and processor emuation are sometimes used interchangeaby to denote the abiity to mimic or emuate the hardware architec ture of an entire computer system using one or severa approaches known and appreciated by those of ski in the art. Moreover, a uses of the term emuation in any form is intended to convey this broad meaning and is not intended to distinguish between instruction execution concepts of emu ation versus direct-execution of operating system instruc tions inthe virtua machine. Thus, for exampe, the Vrtua PC software created by Connectix Corporation of San Mateo, Caifornia emuates (by instruction execution emuation and/or direct execution) an entire computer that incudes an Inte 80X86 Pentium processor and various motherboard components and cards, and the operation of these components is emuated in the virtua machine that is being run on the host machine. A virtuaizer program executing on the oper ating system software and hardware architecture of the host computer, such as a computer system having a PowerPC processor, mimics the operation of the entire guest computer system. [0009] The virtuaizer program acts as the interchange between the hardware architecture of the host machine and the instructions transmitted by the software (e.g., operating systems, appications, etc.) running within the emuated envi ronment. This virtuaizer program may be a host operating system (HOS), which is an operating system running directy on the physica computer hardware (and which may comprise

8 US 2014/ A1 Apr. 24, 2014 a hypervisor, discussed in greater detaied ater herein). Ater natey, the emuated environment might aso be a virtua machine monitor (VMM) which is a software ayer that runs directy above the hardware, perhaps running side-by-side and working in conjunction with the host operating system, and which can virtuaize a the resources ofthe host machine (as we as certain virtua resources) by exposing interfaces that are the same as the hardware the VMM is virtuaizing. This virtuaization enabes the virtuaizer (as we as the host computer system itsef) to go unnoticed by operating system ayers running above it. [0010] To summarize, processor emuation enabes a guest operating system to execute on a virtua machine created by a virtuaizer running on a host computer system, said host computer system comprising both physica hardware and a host operating system. Processor and Memory Topoogy [0011] Modern operating system scheduers take into account the processor and memory topoogy of the machine to maximize performance. This is usuay done at startup and, for an operating system executing on physica hardware, this is usuay suiicient because the processor topoogy of physi ca hardware remains constant. The Windows Operating Sys tem (Windows XP, Windows 2003) and other operating sys tems typicay determine the topoogy of the system at boot time in two ways: (a) by examining the memory and proces sor node topoogy information in the BIOS Static Resource Affinity Tabe (SRAT) and (b) by reading sef-contained pro cessor identification data (CPUID in x86/x64 processors) to determine specific Simutaneous Mutithreading (SMT, a.k.a. hyperthreading) and muticore topoogies. [0012] As used herein, the term processor topoogy is broady intended to refer to physica characteristics of the processor and associated memory that, if known by an oper ating system, coud theoreticay enabe an operating system to better utiize the associated processor resources. Processor topoogy may incude, but is not imited to, the foowing: static processor information such as SMT, muticore, and BIOS SRAT data and/or information, static NUMA infor mation such as processor, memory, and I/ O resource arrange ments, and any changes to the foregoing. [0013] In a virtua machine environment, however, whie the physica processor topoogy for the hosting agent (the host operating system, virtua machine monitor, and/or hypervisor) remains constant, the physica resources assigned to a virtuaizer, and thus the virtua machine, may vary rapidy over time, making the topoogy assumptions made by the guest operating system running on the virtua machine inaccurate and hence inefficient. [0014] Whie the dynamic nature of the topoogy can be mitigated by aways using the same physica processor assignments for virtua processors or by imiting the assign ments to a specific node, this woud severey and negativey impact the virtuaizer s abiity to make optima use of a host resources. Therefore, what is needed in the art is means for rectifying the inefficiency of a changing virtua topoogy without negativey impacting the virtuaizers abiity to make optima use of a host resources. SUMMARY OF THE INVENTION [0015] Various embodiments of the present invention are directed to systems and methods for making a guest operating system aware of the topoogy of the subset of host resources currenty assigned to it. For certain of these embodiments, at virtua machine boot time a Static Resource Affinity Tabe (SRAT) wi be used by the virtuaizer to group guest physica memory and guest virtua processors into virtua nodes. Thereafter the host physica memory behind a virtua node can be changed by the virtuaizer as necessary, and the virtu aizer wi provide physica processors appropriate for the virtua processors in that node. BRIEF DESCRIPTION OF THE DRAWINGS [0016] The foregoing summary, as we as the foowing detaied description of preferred embodiments, is better understood when read in conjunction with the appended drawings. For the purpose of iustrating the invention, there is shown in the drawings exempary constructions of the invention, however, the invention is not imited to the specific methods and instrumentaities discosed. In the drawings: [0017] FIG. 1 is a bock diagram representing a computer system in which aspects of the present invention may be incorporated, [0018] FIG. 2 is a bock diagram representing the ogica ayering of the hardware and software architecture for an emuated operating environment in a computer system, [0019] FIG. 3A is a bock diagram representing a virtua ized computing system wherein the emuation is performed by the host operating system (either directy or via a hyper visor), [0020] FIG. 3B is a bock diagram representing an atema tive virtuaized computing system wherein the emuation is performed by a virtua machine monitor running side-by-side with a host operating system, [0021] FIG. 4 is a bock diagram iustrating a muti-core processor and a NUMA two-node system for which severa embodiments of the present invention may be utiized, [0022] FIG. 5 is a process fow diagram iustrating one method by which a virtuaizer provides dynamic processor topoogy information for the guest operating system in virtua machine memory for certain embodiments of the present invention, and [0023] FIG. 6 is a bock diagram that iustrates a two-tier discosing and hinting approach for severa embodiments of the present invention. DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS [0024] The inventive subject matter is described with speci ficity to meet statutory requirements. However, the descrip tion itsef is not intended to imit the scope of this patent. Rather, the inventor has contempated that the caimed sub ject matter might aso be embodied in other ways, to incude different steps or combinations of steps simiar to the ones described in this document, in conjunction with other present or future technoogies. Moreover, athough the term step may be used herein to connote different eements of methods empoyed, the term shoud not be interpreted as impying any particuar order among or between various steps herein dis cosed uness and except when the order of individua steps is expicity described. Computer Environment [0025] Numerous embodiments of the present invention may execute on a computer. FIG. 1 and the foowing discus

9 US 2014/ A1 Apr. 24, 2014 sion is intended to provide a brief genera description of a suitabe computing enviroment in which the invention may be impemented. Athough not required, the invention wi be described in the genera context of computer executabe instructions, such as program modues, being executed by a computer, such as a cient workstation or a server. Generay, program modues incude routines, programs, objects, com ponents, data structures and the ike that perform particuar tasks or impement particuar abstract data types. Moreover, those skied in the art wi appreciate that the invention may be practiced with other computer system configurations, incuding hand hed devices, muti processor systems, micro processor based or programmabe consumer eectronics, net work PCs, minicomputers, mainframe computers and the ike. The invention may aso be practiced in distributed com puting environments where tasks are performed by remote processing devices that are inked through a communications network. In a distributed computing environment, program modues may be ocated in both oca and remote memory storage devices. [0026] As shown in FIG. 1, an exempary genera purpose computing system incudes a conventiona persona com puter 20 or the ike, incuding a processing unit 21, a system memory 22, and a system bus 23 that coupes various system components incuding the system memory to the processing unit 21. The system bus 23 may be any of severa types of bus structures incuding a memory bus or memory controer, a periphera bus, and a oca bus using any of a variety of bus architectures. The system memory incudes read ony memory (ROM) 24 and random access memory (RAM) 25.A basic input/output system 26 (BIOS), containing the basic routines that hep to transfer information between eements within the persona computer 20, such as during start up, is stored in ROM 24. The persona computer 20 may further incude a hard disk drive 27 for reading from and writing to a hard disk, not shown, a magnetic disk drive 28 for reading from or writing to a removabe magnetic disk 29, and an optica disk drive 30 for reading from or writing to a remov abe optica disk 31 such as a CD ROM or other optica media. The hard disk drive 27, magnetic disk drive 28, and optica disk drive 30 are connected to the system bus 23 by a hard disk drive interface 32, a magnetic disk drive interface 33, and an optica drive interface 34, respectivey. The drives and their associated computer readabe media provide non voatie storage of computer readabe instructions, data structures, program modues and other data for the persona computer 20. Athough the exempary environment described herein empoys a hard disk, a removabe magnetic disk 29 and a removabe optica disk 31, it shoud be appreciated by those skied in the art that other types of computer readabe media which can store data that is accessibe by a computer, such as magnetic cassettes, fash memory cards, digita video disks, Bemoui cartridges, random access memories (RAMs), read ony memories (ROMs) and the ike may aso be used in the exempary operating environment. [0027] A number of program modues may be stored on the hard disk, magnetic disk 29, optica disk 3 1, ROM 24 or RAM 25, incuding an operating system 35, one or more appication programs 3 6, other program modues 37 and program data 38. A user may enter commands and information into the per sona computer 20 through input devices such as a keyboard 40 and pointing device 42. Other input devices (not shown) may incude a microphone, joystick, game pad, sateite disk, scanner or the ike. These and other input devices are often connected to the processing unit 21 through a seria port interface 46 that is couped to the system bus, but may be connected by other interfaces, such as a parae port, game port or universa seria bus (USB). A monitor 47 or other type of dispay device is aso connected to the system bus 23 via an interface, such as a video adapter 48. In addition to the moni tor 47, persona computers typicay incude other periphera output devices (not shown), such as speakers and printers. The exempary system of FIG. 1 aso incudes a host adapter 55, Sma Computer System Interface (SCSI) bus 56, and an externa storage device 62 connected to the SCSI bus 56. [0028] The persona computer 20 may operate in a net worked environment using ogica connections to one or more remote computers, such as a remote computer 49. Ihe remote computer 49 may be another persona computer, a server, a router, a network PC, a peer device or other common network node, and typicay incudes many or a of the eements described above reative to the persona computer 20, athough ony a memory storage device 50 has been ius trated in FIG. 1. The ogica connections depicted in FIG. 1 incude a oca area network (LAN) 51 and a wide area net work (WAN) 52. Such networking environments are com monpace in offices, enterprise wide computer networks, intranets and the Internet. [0029] When used in a LAN networking environment, the persona computer 20 is connected to the LAN 51 through a network interface or adapter 53. When used in a WAN net working environment, the persona computer 20 typicay incudes a modem 54 or other means for estabishing com munications over the wide area network 52, such as the Inter net. The modem 54, which may be interna or externa, is connected to the system bus 23 via the seria port interface 46. In a networked environment, program modues depicted rea tive to the persona computer 20, or portions thereof, may be stored in the remote memory storage device. It wi be appre ciated that the network connections shown are exempary and other means of estabishing a communications ink between the computers may be used. Moreover, whie it is envisioned that numerous embodiments of the present invention are par ticuary we-suited for computerized systems, nothing in this document is intended to imit the invention to such embodiments. Virtua Machines [0030] From a conceptua perspective, computer systems generay comprise one or more ayers of software running on a foundationa ayer of hardware. This ayering is done for reasons of abstraction. By defining the interface for a given ayer of software, that ayer can be impemented differenty by other ayers above it. In a we-designed computer system, each ayer ony knows about (and ony reies upon) the imme diate ayer beneath it. This aows a ayer or a stack (mu tipe adjoining ayers) to be repaced without negativey impacting the ayers above said ayer or stack. For exampe, software appications (upper ayers) typicay rey on ower eves of the operating system (ower ayers) to write fies to some form of permanent storage, and these appications do not need to understand the difference between writing data to a foppy disk, a hard drive, or a network foder. If this ower ayer is repaced with new operating system components for writing fies, the operation of the upper ayer software appi cations remains unaffected. [0031] The fexibiity of ayered software aows a virtua machine (VM) to present a virtua hardware ayer that is in

10 US 2014/ A1 Apr. 24, 2014 fact another software ayer. n this way, a VM can create the iusion for the software ayers above it that said software ayers are running on their own private computer system, and thus VMS can aow mutipe guest systems to run concur renty on a singe host system. This eve of abstraction is represented by the iustration of FIG. 2. [0032] FIG. 2 is a diagram representing the ogica ayering of the hardware and software architecture for an emuated operating environment in a computer system. n the figure, an emuation program 94 runs directy or indirecty on the physi ca hardware architecture 92. Emuation program 94 may be (a) a virtua machine monitor that runs aongside a host oper ating system, (b) a speciaized host operating system having native emuation capabiities, or (c) a host operating system with a hypervisor component wherein said hypervisor com ponent performs said emuation. Emuation program 94 emu ates a guest hardware architecture 96 (shown as broken ines to iustrate the fact that this component is the virtua machine, that is, hardware that does not actuay exist but is instead emuated by said emuation program 94). A guest operating system 98 executes on said guest hardware archi tecture 96, and software appication 100 runs on the guest operating system 98. n the emuated operating environment of FIG. 2?and because of the operation of emuation pro gram 94?software appication 100 may run in computer system 90 even if software appication 100 is designed to run on an operating system that is generay incompatibe with the host operating system and hardware architecture 92. [0033] FIG. 3A iustrates a virtuaized computing system comprising a host operating system software ayer 104 run ning directy above physica computer hardware 102 where the host operating system (host OS) 104 provides access to the resources of the physica computer hardware 102 by exposing interfaces that are the same as the hardware the host OS is emuating (or virtuaizing )?which, in turn, enabes the host OS to go unnoticed by operating system ayers run ning above it. Again, to perform the emuation the host oper ating system 102 may be a speciay designed operating sys tem with native emuations capabiities or, aternatey, it may be a standard operating system with an incorporated hyper visor component for performing the emuation (not shown). [0034] Referring again to FIG. 3A, above the host OS 104 are two virtua machine (VM) impementations, VM A 108, which may be, for exampe, a virtuaized nte 386 processor, and VM B 110, which may be, for exampe, a virtuaized version of one of the Motoroa 680X0 famiy of processors. Above each VM 108 and 110 are guest operating systems (guest OSs) A 112 and B 114 respectivey. Running above guest OS A 112 are two appications, appication A1 116 and appication A2 118, and running above guest OS B 114 is appication B [0035] n regard to FIG. 3A, it is important to note that VM A 108 and VM B 110 (which are shown in broken ines) are virtuaized computer hardware representations that exist ony as software constructions and which are made possibe due to the execution of speciaized emuation software(s) that not ony presents VM A 108 and VM B 110 to Guest OS A 112 and Guest OS B 114 respectivey, but which aso performs a ofthe software steps necessary for Guest OS A 112 and Guest OS B 114 to indirecty interact with the rea physica com puter hardware 102. [0036] FIG. 3B iustrates an aternative virtuaized com puting system wherein the emuation is performed by a vir tua machine monitor (VMM) 104' running aongside the host operating system 104". For certain embodiments the VMM may be an appication running above the host operating sys tem 104 and interacting with the computer hardware ony through said host operating system 104. n other embodi ments, and as shown in FIG. 3B, the VMM may instead comprise a partiay independent software system that on some eves interacts indirecty with the computer hardware 102 via the host operating system 104 but on other eves the VMM interacts directy with the computer hardware 102 (simiar to the way the host operating system interacts directy with the computer hardware). And in yet other embodiments, the VMM may comprise a fuy independent software system that on a eves interacts directy with the computer hard ware 102 (simiar to the way the host operating system inter acts directy with the computer hardware) without utiizing the host operating system 104 (athough sti interacting with said host operating system 104 insofar as coordinating use of said computer hardware 102 and avoiding conficts and the ike). [0037] A of these variations for impementing the virtua machine are anticipated to form aternative embodiments of the present invention as described herein, and nothing herein shoud be interpreted as imiting the invention to any particu ar emuation embodiment. n addition, any reference to inter action between appications 116, 118, and 120 via VMA 108 and/or VM B 110 respectivey (presumaby in a hardware emuation scenario) shoud be interpreted to be in fact an interaction between the appications 116, 118, and 120 and the virtuaizer that has created the virtuaization. Likewise, any reference to interaction between appications VM A 108 and/or VM B 110 with the host operating system 104 and/or the computer hardware 102 (presumaby to execute computer instructions directy or indirecty on the computer hardware 102) shoud be interpreted to be in fact an interaction between the virtuaizer that has created the virtuaization and the host operating system 104 and/or the computer hardware 102 as appropriate. Processor Topoogy [0038] n genera, a processor is ogic circuitry that responds to and processes the basic instructions that drive a computer, and is aso the term that is often used as shorthand for the centra processing unit (CPU). The processor in a persona computer or embedded in sma devices is often caed a microprocessor. [0039] With regard to processor topoogy, and as used herein, the term processor specificay refers to a physica processor. A physica processor is an integrated circuit (1C)?sometimes caed a chip or microchip 4comprising a semiconductor wafer ( siicate ) on which numerous tiny resistors, capacitors, and transistors form at east one processor core comprising at east one ogica processor. Each processor core has the capabiity to execute system instruc tions, and each ogica processor represents the hyperthread ing capabiities (aso known as symmetric muti-threading or SMT ) by which a singe processor core seemingy executes two threads in parae (and thus appears to be two cores to the system). [0040] Each physica processor is fixed into a singe socket on a CPU motherboard. A physica processor may have more than one processor core (each having one or more ogica processors). Each processor core wi typicay have its own eve-1 cache but share a eve-2 cache with other processor cores on the physica processor.

11 US 2014/ A1 Apr. 24, 2014 [0041] A muti-core processor is a physica processor having two or more cores for enhanced performance, reduced power consumption, and/or more efficient simutaneous pro cessing of mutipe tasks (e.g., parae processing). For exampe, a dua-core processor? which, as its name sug gests, is a muti-core processor having two processor cores? is somewhat simiar to having two separate processors instaed in the same computer. However, these two cores reside on a singe physica processor and are essentiay pugged into the same socket, and thus the connection between these two processor cores is faster than it woud be for two singe-core processors pugged into separate sockets. [0042] Because of these performance gains, muti-core processing is growing in popuarity as singe-core processors rapidy reach the physica imits of possibe compexity and speed. Companies that have produced or are working on muti-core products incude AMD, ARM, Broadcom, Inte, and VIA. Both AMD and Inte have announced that they wi market dua-core processors by [0043] FIG. 4 is a bock diagram iustrating a muti-core processor and a NUMA two-node system for which severa embodiments of the present invention may be utiized. In this figure, a physica processor 406 comprises two processor cores 404 which each in turn comprise two ogica processors 402. The physica processor 406 is coupe to memory 408, such as an L3-cache, that is shared and utiized by both cores 404 of the physica processor 406. This figure is further described beow. Memory Topoogy [0044] NUMA (non-uniform memory access) is a method of configuring a node of physica processors in a mutipro cessing system so that they can share memory ocay, improving performance and the abiity of the system to be expanded. NUMA is typicay used in a symmetric mutipro cessing (SMP) system that is a tighty-couped, share every thing system in which mutipe processors working under a singe operating system access each other s memory over a common bus or interconnect path. Ordinariy, a imitation of SMP is that as microprocessors are added, the shared bus or data path gets overoaded and becomes a performance botte neck, however, NUMA adds an intermediate eve of memory (node memory) shared among that node s microprocessors so that a data accesses do not have to trave on the main bus. [0045] Referring again to FIG. 4, the two node system 416 comprises two nodes 414, each having four physica proces sors 406, each physica processor 406 having its own L3 cache that is shared by the processor cores 404 of each said cache. In addition, each physica processor 406 and its asso ciated L3 cache memory 408 is couped to each other and to a shared node memory 412. The nodes 414 and their associ ated node memories 412 are aso couped together in this two-node system 416 as shown. [0046] A NUMA node typicay consists of four physica processors interconnected on a oca bus to a shared memory (the L3 cache ) a on a singe motherboard. This unit can be added to simiar units to form a symmetric mutiprocessing system in which a common SMP bus interconnects a of the nodes. Such a system typicay contains from 16 to 256 microprocessors. To an appication program running in an SMP system, a the individua processor memories ook ike a singe memory. [0047] When a processor core ooks for data at a certain memory address, it first ooks to its L1 cache, then on the L2 cache for the physica processor, and then to the L3 cache that the NUMA configuration provides before seeking the data in the remote memory ocated near the other microprocessors. Data is moved on the bus between the custers of a NUMA SMP system using scaabe coherent interface (SCI) techno ogy. SCI coordinates what is caed cache coherence or consistency across the nodes of the mutipe custers. Exposing Processor Topoogy [0048] Various embodiments of the present invention are directed to systems and methods for making a guest operating system aware of the topoogy of the subset of host resources currenty assigned to it. For certain of these embodiments, at virtua machine boot time a Static Resource Affinity Tabe (SRAT) wi be used by the virtuaizer to group guest physica memory and guest virtua processors into virtua nodes. Thereafter the host physica memory behind a virtua node can be changed by the virtuaizer as necessary, and the virtu aizer wi provide physica processors appropriate for the virtua processors in that node. This approach aows NUMA aware operating systems executing on the virtua machine to schedue for optima performance without further modifica tion. [0049] For certain aternative embodiments, the virtuaizer may aso provide dynamic processor topoogy information for the guest operating system in virtua machine memory. This information may be paced directy into the guest oper ating system s interna tabes or, aternatey, the guest oper ating system may execute additiona code to pick this infor mation from a shared memory ocation. The atter approach, referred to as discosing (where the VM discoses informa tion on a reguar basis to the guest operation system, and the guest operating system reguary checks for updated info and adjusts accordingy) requires that the guest operating system be provided with additiona code to cause it to periodicay acquire this dynamic information. [0050] FIG. 5 is a process fow diagram iustrating one method by which a virtuaizer provides dynamic processor topoogy information for the guest operating system in virtua machine memory for certain embodiments of the present invention. In the figure, the virtuaizer, at step 502, becomes aware that the physica hardware resources aocated to it has changed. At step 504, the virtuaizer reconfigures the proces sor topoogy it is virtuaizing. At step 506, the virtuaizer updates the processor topoogy information for the guest operating system directy pacing updated topoogy informa tion directy into the guest operating system s interna tabes. [0051] For certain embodiments of the present invention, the guest operating system woud execute a virtua machine ca (a ca to the virtuaizer) which designates a virtua machine s physica memory page to be shared by both the virtuaizer and the guest OS. This page may contain a contro fied with discosure data to determine, for exampe: (a) whether the virtuaizer shoud send an interrupt to the guest operating system whenever it changes the virtua machine topoogy to match changes in host computer system resource aocations to said virtua machine, (b) the vector to be used for the notification interrupt, (c) a generation counter which is incremented whenever the hypervisor updates the topoogy data, (d) a bit-mask of a virtua processors in the same SMT or hyperthreaded processor core, and/or (e) a bit-mask of a virtua processors in the same physica processor, that is, a ogica processors in a cores in each physica processor. In addition, discosure data may address any of the foowing

12 US 2014/ A1 Apr. 24, 2014 aspects of efficiency: (a) thread priority; (b) /O priority; (c) range of protected memory; (d) NUMA nodes; (e) data per taining to near memory and far memory access; (f) processor speed and processor power consumption; (g) sockets and, for each core, hyperthreading; and/or (h) sharing eve for each physica processor. [0052] The scheduer of an operating system that has access to dynamic processor and NUMA topoogy information, such as when the discosing approach is used, is abe to use this information to optimize its own resource aocation mecha nisms (e. g. processor scheduing, memory aocation, etc.) and resource utiization schemes. For certain additiona embodiments of the present invention, the guest OS (either through virtuaizer/virtua machine cas or through a shared memory page) may provide hints about resource aocation preferences to the virtuaizer in a process caed hinting (which is the ogica converse of discosing ). For exampe, if the guest OS woud prefer to keep two virtua processors assigned to two cores on the same processor or two processors within the same NUMA node for efficiency, it coud provide such a hint to the VM and the virtua machine scheduer coud take this hint into account with regard to the virtuaized pro cessors as they pertain to the underying physica processors assigned to saidvm at any given time. More specificay, such hints may address any of the foowing aspects of efficiency: (a) thread priority; (b) /O priority; and/or (c) atency infor mation. Thus, for embodiments ofthe present invention, both the scheduer for the guest operation system as we as the scheduer for the VM?which independenty manage resources?to utiize and empoy discosing and hinting to work cooperativey to maximize the eñiciency ofthe entire system. [0053] FIG. 6 is a bock diagram that iustrates a two-tier discosing and hinting approach for severa embodiments of the present invention. n the figure, the guest operating system 602 comprises an OS scheduer 604 and the virtua machine 612 comprises a VM scheduer 614. The VM scheduer 614 schedues execution of virtua machine threads on the various ogica processors of the physica hardware as such ogica processors are made avaiabe to the virtua machine (and which are ever-changing) by, for exampe, the host operating system which schedues utiization of said physica hardware resources. A shared memory 622 that has been aocated to the virtua machine is utiized by both the guest OS scheduer 604 to provide hinting information to the VM scheduer 614, and this shared memory 622 is aso utiized by the VM sched uer 614 to provide discosing information to the guest OS scheduer 604. For exampe, aong data fow 632, the VM scheduer 614 writes discosing data 642 to the shard memory 622 and, aong data fow 634, this data is read by the OS scheduer 604 and used to by the OS scheduer 604 to more eñicienty use the current processor resources that are avai abe (and which dynamicay change from time to time). Conversey, aong data fow 636, the OS scheduer 604 writes hinting data 644 to the shared memory 622 and, aong data fow 638, this data is read by the VM scheduer 614 and used by the VM scheduer 614 to more eñicienty assign (and/or request) current processor resources to said guest operation system. CONCLUSION [0054] The various systems, methods, and techniques described herein may be impemented with hardware or soft ware or, where appropriate, with a combination of both. Thus, the methods and apparatus ofthe present invention, or certain aspects or portions thereof, may take the form of program code (i.e., instructions) embodied in tangibe media, such as foppy diskettes, CD-ROMs, hard drives, or any other machine-readabe storage medium, wherein, when the pro gram code is oaded into and executed by a machine, such as a computer, the machine becomes an apparatus for practicing the invention. n the case of program code execution on pro grammabe computers, the computer wi generay incude a processor, a storage medium readabe by the processor (in cuding voatie and non-voatie memory and/ or storage ee ments), at east one input device, and at east one output device. One or more programs are preferaby impemented in a high eve procedura or object oriented programming an guage to communicate with a computer system. However, the program(s) can be impemented in assemby or machine an guage, if desired. n any case, the anguage may be a compied or interpreted anguage, and combined with hardware impe mentations. [0055] The methods and apparatus of the present invention may aso be embodied in the form of program code that is transmitted over some transmission medium, such as over eectrica wiring or cabing, through fiber optics, or via any other form of transmission, wherein, when the program code is received and oaded into and executed by a machine, such as an EPROM, a gate array, a programmabe ogic device (PLD), a cient computer, a video recorder or the ike, the machine becomes an apparatus for practicing the invention. When impemented on a genera-purpose processor; the pro gram code combines with the processor to provide a unique apparatus that operates to perform the indexing functionaity of the present invention. [0056] Whie the present invention has been described in connection with the preferred embodiments of the various figures, it is to be understood that other simiar embodiments may be used or modifications and additions may be made to the described embodiment for performing the same function of the present invention without deviating there from. For exampe, whie exempary embodiments of the invention are described in the context of digita devices emuating the func tionaity of persona computers, one skied in the art wi recognize that the present invention is not imited to such digita devices, as described in the present appication may appy to any number of existing or emerging computing devices or environments, such as a gaming consoe, handhed computer, portabe computer, etc. whether wired or wireess, and may be appied to any number of such computing devices connected via a communications network, and interacting across the network. Furthermore, it shoud be emphasized that a variety of computer patforms, incuding handhed device operating systems and other appication specific hard ware/software interface systems, are herein contempated, especiay as the number of wireess networked devices con tinues to proiferate. Therefore, the present invention shoud not be imited to any singe embodiment, but rather construed in breadth and scope in accordance with the appended caims. [0057] Finay, the discosed embodiments described herein may be adapted for use in other processor architec tures, computer-based systems, or system virtuaizations, and such embodiments are expressy anticipated by the disco sures made herein and, thus, the present invention shoud not be imited to specific embodiments described herein but instead construed mo st broady. Likewise, the use of synthetic instructions for purposes other than processor virtuaization

13 US 2014/ A1 Apr. 24, 2014 are aso anticipated by the discosures made herein, and any such utiization of synthetic instructions in contexts other than proces sor virtuaization shoud be mo st broady read into the discosures made herein. What is caimed: 1. A method for optimizing performance of an operating system executing on a computer system, said computer sys tem having a dynamic virtua processor topoogy, said method comprising updating said operating system after star tup With at east one update to refect at east one change in said virtua processor topoogy. * * * * *

14 US A1 (19) United States (12) Patent Appication Pubication (10) Pub. No.: US 2014/ A1 Traut et a. (43) Pub. Date: Apr. 24, 2014 (54) SYSTEMS AND METHODS FOR EXPOSING Pubication Cassification PROCESSOR TOPOLOGY FOR VIRTUAL MACHINES (5) Int. C. (7) Appicant: Microsoft Corporation, Redmond, WA (52) G06F 9/44 ( ) U_S_ C, (US) CPC..... G06F 9/4406 ( ) (72) Inventors: Eric P. Traut, Beevue, WA (US), Rene USPC /1 (73) Assigneeï Antonio Microsoft Vega, Corporation, RedmOHd WA The present invention is directed to making a guest operating (Us) system aware ofthe topoogy ofthe subset of host resources currenty assigned to it. At virtua machine boot time a Static (21) App' NO': 14/ Resource Affinity Tabe (SRAT) Wi be used by the virtua. ~ izer to group guest physica memory and guest virtua pro (22) Fed' Dec cessors into virtua nodes. Thereafter, in one embodiment, the.. host physica memory behind a virtua node can be changed Reated U'S' Apphcaton Data by the virtuaizer as necessary, and the virtuaizer Wi provide (63) Continuation of appication No. 11/018,337, ñed on physica processors appropriate for the virtua processors in Dec. 2, 2004, now Pat. No. 8,621,458. that node. Computer 20 S Siem Memor Q î M (ROM 24) î Monitor 47 (RAM P V. A H A i 25 rocessmg Unit ideo dapter ost dapter SCSI BUS 56 î Storage Device 1_ :i APPLICATioN T PROGRAMS 36 ' Sii-Siem BUS 23 I OTHER, y I.. PROGRAMS 37, r r y Y Hard Disk Drive /F 32. Magnetic Disk Drive /F 33. Optica Drive / F 34 Seria Pon UF Network UF 53 PROGRAM 1 4 LAN 51 \ Hard Drive 27 Other rogs. 37 Program u Data 38 Foppy Drive 28 È. Removabe îstorage r~~~~ Appications 36' Foppy Drive 50