Classifications

• • H—ELECTRICITY
  • H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
  • H05K—PRINTED CIRCUITS; CASINGS OR CONSTRUCTIONAL DETAILS OF ELECTRIC APPARATUS; MANUFACTURE OF ASSEMBLAGES OF ELECTRICAL COMPONENTS
  • H05K7/00—Constructional details common to different types of electric apparatus
  • H05K7/20—Modifications to facilitate cooling, ventilating, or heating
  • H05K7/20709—Modifications to facilitate cooling, ventilating, or heating for server racks or cabinets; for data centers, e.g. 19-inch computer racks
  • H05K7/20836—Thermal management, e.g. server temperature control

Definitions

• the present invention relates to thermal management of data centers.
• the invention provides a method and system for efficient real time thermal management of a data center using a fast thermal model and an optimizer.
• a data center usually contains a variety of electronic equipment such as servers, telecom equipment, networking equipment, switches and other electronic equipment.
• the heat generated by such electronic components is cooled with the help of cooling units.
• the cooling units are computer room air conditioners (CRAC) or computer room air handlers (CRAH) which supply cold air for cooling.
• CRAC computer room air conditioners
• CRAH computer room air handlers
• Power consumed by cooling equipment contributes to a major portion of the total data center power consumption.
• inefficient operation of cooling units is one of the major factors causing poor cooling efficiency of the data center. Cooling capacity of the data center is designed and typically run for maximum heat load conditions. In practice, data centers rarely operated at maximum conditions.
• cooling units are controlled according to heat loads in a very elementary manner e.g. return air temperature control. All these practices lead to poor cooling efficiency.
• Reliability of computing equipment by ensuring proper thermal management is another concern for data center managers in addition to poor cooling efficiency. Prolonged exposure of such equipment to conditions beyond their recommended and allowable range of environmental conditions, especially approaching the extremes of the allowable operating environment like temperature, may result in decreased equipment reliability and longevity. Exposure of operating equipment to conditions outside its allowable operating environment risks catastrophic equipment failure. Various reasons like hot air recirculation, CRAC failure etc may lead to higher temperatures i.e. hot spots. Due to lack of distributed sensing and continuous monitoring of environmental parameters in the data center, these thermal problems go unobserved leading to thermal failures. This issue is typically circumvented by data center managers by over provisioning of cooling capacity, which in turn, reduces cooling efficiency.
• a data center manager faces dual challenge: is to ensure safety of electronic equipment by ensuring appropriate temperatures and at the same time achieving optimum cooling efficiency of the data center.
• data centers are not equipped with large number of sensors which can give a reasonable understanding of thermal management in the data center.
• the data center managers do not have analysis tools which can pinpoint the reasons behind the thermal problems or cooling inefficiencies and provide appropriate recommendations for mitigation of these thermal problems and improvement in cooling efficiency in real time.
• a data center manager can alter only operational parameters of CRAC in real time for quick and easy mitigation of thermal problems and increasing cooling efficiency.
• optimizing operational parameters of CRAC is typical and easy way for real time cooling optimization of data center.
• US7031870 to Sharma et al. describes a method for controlling CRAC based on air- recirculation index.
• This air-recirculation index is quantified using sensed temperatures at various locations in the data center.
• This method ensures appropriate temperatures at inlets of racks by controlling cooling but does not attempt to fully optimize the cooling parameters. Thus it fails to ensure optimum cooling efficiency.
• This invention does not pinpoint the reasons behind thermal problems. Further, the scope of this invention is limited to the data centers which are equipped with large number of sensors at required locations.
• US7493193B2 to Hyland et al. discloses a method, Apparatus and computer program for monitoring and real-time heat load control based on server and environmental parameters.
• heat load control rules are used to ensure operation of electronic equipment within predefined specification. These predefined control rules are primitive and hence do not ensure optimum cooling efficiency of the data center. Further, the scope of this invention is limited to the data centers which are equipped with large number of sensors at required locations.
• US2010076608A by Nakajima et al. provides a system and method for controlling cooling using correlations namely temperature sensitivity coefficients. These temperature sensitivity coefficients also have limited scope compared to influence indices which are used in present invention. They typically quantify correlation between CRAC and racks only. This invention does not give causal analysis of thermal problems. Further, the scope of this invention is limited to the data centers which are equipped with large number of sensors at required locations.
• the primary objective is to provide a method and system for efficient real time thermal management of a data center using a fast thermal model and an optimizer.
• Another objective of the invention is to provide a method and system for real time monitoring of operational parameters of data center with the help of sensor network and predicting where sensors are not present for finding potential thermal problem and cooling inefficiencies in the data center.
• Another objective of the invention is to provide a method and system for generating quick and precise recommendations regarding optimum operational parameters of CRAC for mitigation of thermal problems and improving cooling efficiency using the fast thermal model and the optimizer capable of running online in quick time.
• Another objective of the invention is to provide a method and system for generating and sending SMS alerts, emails, alarms or notification recommending corrective actions needed to be taken to a user in case of thermal problems.
• the present invention provides a method and system for efficient real time thermal management of a data center using a fast thermal model and an optimizer.
• a method and system for real time monitoring of operational parameters of data center at various locations in the data center with the help of sensor network and predicting where sensors are not present for finding potential thermal problem in the data center and cooling inefficiencies of the data center.
• a method and system for generating quick and precise recommendations regarding optimum operational parameters of CRAC for mitigation of thermal problems and improving cooling efficiency using the fast thermal model and the optimizer capable of running online in quick time.
• a method and system is provided for generating and sending SMS alerts, emails, alarms or notification recommending corrective actions needed to be taken to a user in case of emergency such as an equipment failure.
• the above said method and system are preferably for efficient real time thermal management of a data center but also can be used for many other applications.
• Figure 1 shows a flow diagram of the process for efficient real time thermal management of a data center.
• Figure 2 shows a flow diagram of the process for generating recommendation.
• Figure 3 shows a block diagram of the user interface displayed on a display device.
• the present application uses specific terminologies such as CRAC, rack, etc. " only for simplicity.
• the subject matter of the present application is applicable to any type of electronic equipment like servers, networking equipment, telecommunications equipment etc. arranged in any fashion, any type of air delivery mechanism such as raised floor, overhead ducts etc, any type of air cooling infrastructure and any type of cooling units.
• the data center may contain racks, housing various electronic and electric equipment and the racks are arranged in rows.
• Heat generated by the electronic and electric equipment is cooled by CRACs which are situated near periphery of the data center. These CRACs enable cold air to flow into the under-floor plenum. This cold air is delivered to intended places (e.g. fronts of racks) through tiles or vents.
• the equipment typically has fans for taking in cold air. This air picks up the heat generated and the hot air is exhausted. Some of this hot air is returned back to CRAC and some of this hot air may mix with cold air from tiles and recirculated into inlets of equipment. This recirculation of hot air may cause rising of temperature at inlets of racks above recommended temperature suggested by manufacturer of equipment.
• the heat generation inside racks change with time depending upon amount of workload put onto the equipment inside the racks.
• the CRAC have mechanism to change amount of cooling dynamically according to changing the heat load conditions in the data center. There may be emergency cases of CRAC failure causing the temperatures at some regions in the data center to overshoot.
• various parameters in the data center such as temperatures at inlets of racks, heat generation by equipment inside racks and amount of cooling provided by CRAC are very dynamic. Given these dynamic conditions in the data center, continuous monitoring with alarms and notifications enables data center managers to supervise the data center effectively and reliably. Also, an analysis tool which can pinpoint reasons behind thermal problems and cooling efficiencies give data center manager valuable insights into current status of the data center.
• the present invention fulfils all the above needs. It continuously monitors operational parameters of data center like power consumption of racks and temperatures at various locations, pinpoints reasons behind thermal problems and cooling inefficiencies, generates quick and precise recommendations regarding optimum operating parameters of CRAC.
• the invention also enables data center manager for effective thermal management by displaying the monitored parameters and recommendations on a user interface and generating alarms and notifications.
• FIG. 1 is a flow diagram of the process for efficient real time thermal management of a data center.
• the method is provided for efficient real time thermal management of a data center.
• the procedure starts with step 102 in which measurement or prediction of power consumption of racks and measurement of temperatures at various locations is carried out at specific interval set by timer (116). Temperatures are measured using sensors which are typically thermocouples. These sensors may be placed at locations such as at inlets and outlets of racks, return and supply of CRAC. Typically, data center are equipped with only few number of sensors. Temperatures are predicted using influence indices where sensors are not present.
• the power consumption of racks can be measured using iPDUs which measure the power consumption of devices connected to their outlets and can be polled over the network. Power consumption can also be measured by leveraging Lights On Management (LOM), which is built into the servers that includes remote management of servers including switching them off or on.
• LOM Lights On Management
• a typical LOM architecture has separate management processer with its own system software and network interface. Current LOM installations measure the power consumption of the servers they control. This in turn can be polled through the LOM's network interface.
• Power consumption of servers inside racks can be predicted by using an estimation model too, where an empirical estimation is done by using CPU utilization which can be measured using software. These measured or predicted power consumption of racks and measurement of temperatures is then displayed to the data center manager on a display device through the user interface (112) as explained in Figure 3.
• temperatures at various locations are predicted using a fast thermal model and measurements of temperatures carried out at few locations. This prediction of various temperatures is done at a specified interval e.g. every 1 minute, as specified by timer (116).
• This fast thermal model makes use of method of prediction of temperatures using influence indices.
• temperatures at supply of CRAC and outlet of racks are measured and temperatures at other locations like inlets of racks, returns of CRAC are predicted using the fast thermal model.
• temperatures at supply of CRAC are measured and the temperatures at outlets of the racks along with temperatures at inlets of racks and returns of CRAC are predicted using the fast thermal model.
• the temperatures at outlets of the racks are predicted from temperatures at inlets of racks using equation (1) stated below:
• T (R j ) denotes temperature of air at outlet of a rack.
• T (Rj denotes temperature of air at inlet of a rack
• P(Rj ) denotes the power consumption of rack /? . .
• m (Rj ) denotes mass flow at the inlet of a rack
• C p denotes specific heat of air at constant pressure and room temperature
• Temperatures at various locations in the data center are predicted using the fast thermal model and a temperature map is prepared. These predicted temperatures and temperature map are then displayed to the data center manager on a display device through the user interface (112) as shown in Figure 3. This prediction of temperatures using fast thermal model is also used during generation of recommendations (108).
• various measured and predicted temperatures are analyzed.
• the measured or predicted temperatures at rack inlets are checked for hot spots by comparing with set threshold.
• Measured or predicted temperatures at supply and returns of various CRAC are analyzed to check for overloading of CRAC by calculating the amount of cooling provided by those CRAC.
• These temperatures are also analyzed to identify problems associated with CRAC such as compressor failure etc.
• These thermal problems may be communicated to the data manager using alarms and notifications at step 110 and also displayed to the data center manager on a display device through the user interface in step 112.
• influence indices are used for causal analysis of a thermal problem. For example, if hot spot is determined after analysis of measured or predicted temperature, exact cause of the hot spots like hot air recirculation from a rack or higher supply temperature from CRAC etc can be determined from appropriate influence indices. This causal analysis can be displayed to the data center manager on a display device through the user interface (112). If a thermal problem like hot spot is detected, then the predicted or measured temperatures are observed for a specific interval of time, for example, 5 minutes specified by timer (116). This is carried out to mitigate fluctuating nature of temperatures. If the thermal problem is repeatedly observed during this interval, then recommendations are generated at the step 108.
• the measured or predicted power consumption of racks from step 102 is analyzed for a specified observation interval as specified by the timer (116). This is done to analyze the dynamic nature of these variables. For example, the power consumption of racks is observed for half an hour and maximum and average power consumption levels are calculated. These levels are used for generation of recommendations rather than instantaneous power consumption levels. This is required to mitigate the dynamic and highly fluctuating nature of power consumption of racks.
• various recommendations are generated. These recommendations are aimed towards either mitigation of thermal problems like hot spots or improvement of cooling efficiency of the data center.
• Recommendations regarding exact CRAC supply temperatures are generated using process (200) explained in Figure 2.
• a combination of CRAC supply temperatures is a tuple consisting of ordered list of supply temperatures of the individual CRACs.
• the optimum CRAC supply temperature combination is the one that maintains the rack inlet temperatures just below predefined threshold at the same time ensures minimum cooling power.
• the rack temperatures for a particular combination of CRAC supply temperatures are predicted using fast thermal model used at the step 104.
• the predefined threshold is set for each rack according to the type of equipment housed in the racks.
• the cooling power can be calculated from step 118 using a relation between cooling power i.e. power consumed by the CRACs and CRAC supply temperature and CRAC flow rate. This relation can be determined by using component level models, manufacturer's data or by experimentation.
• exhaustive searching is used to search for the optimum combination of CRAC supply temperatures from all possible combinations.
• the complexity of this exhaustive search and time taken for searching increases rapidly with the number of CRACs but can be reduced using techniques such as Genetic algorithm, Hill climbing, Simulated annealing, etc.
• genetic algorithm is used for searching optimum combination of CRAC temperatures.
• the various recommendations generated are displayed on a display device through the user interface at the step 112 or communicated to the data manager using alarms and notifications at the step 110. At the step 110, alarms and notifications are generated.
• Alarms such as electronic buzzers, visual clues in the user interface screen, SMSes/E-mails to the concerned people enable the data center managers to act immediately upon an emergency (in case of a hot spot, in case of equipment failure etc). These alarms are also displayed to the data center manager on a display device through the user interface shown at step 112.
• Figure 2 shows a flow diagram of the process for generating recommendation.
• the process (200) is provided for generation of recommendation. It calculates new CRAC supply temperatures under two circumstances - when at least one of the racks has been a hot spot for a time Tw (waiting interval) and when there are no any hot spots but time TR (recommendation interval) has elapsed. These time intervals are specified by timer (116). In the former scenario, waiting for a time Tw ensures that recommendation are not generated for short-lived hot spots while in the latter scenario, though there aren't any hot spots, recommendations are generated periodically (every T R ) to optimize the CRAC supply temperatures.
• rack temperatures are predicted using the fast thermal model after specific time interval (e.g. every minute) as specified by timer (202).
• step 206 it is checked if time interval TR has elapsed. If the outcome of the step 206 is Yes then the process proceeds to step 210. If the outcome of the process 206 is No then the process proceeds to step 208. In step 208, every hot spot is checked to see if it has been a hot spot for a time Tw- If the outcome of the step 208 is Yes then the process proceeds to step 210. If the outcome of the step 210 is No then the process returns to step 204.
• the genetic algorithm based optimizer which optimizes the CRAC supply temperatures for a given power consumption levels of the racks is used.
• a genetic algorithm is a search heuristic that mimics the process of natural evolution. This heuristic is routinely used to generate useful solutions to optimization and search problems.
• Genetic algorithms belong to the larger class of evolutionary algorithms (EA), which generate solutions to optimization problems using techniques inspired by natural evolution, such as inheritance, mutation, selection, and crossover. The evolution usually starts from a population of randomly generated individuals and happens in generations. In each generation, the fitness of every individual in the population is evaluated, multiple individuals are stochastically selected from the current population (based on their fitness), and modified (recombined and possibly randomly mutated) to form a new population. The new population is then used in the next iteration of the algorithm. Commonly, the algorithm terminates when either a maximum number of generations has been produced, or a satisfactory fitness level has been reached for the population.
• Step 210 uses the fast thermal model from step 204 to predict rack inlet temperatures for any combination of CRAC supply temperature throughout this evolution process to check whether any rack inlet temperatures is exceeding the threshold.
• the cooling power can be calculated from step 214 using a relation between cooling power i.e. power consumed by the CRACs and CRAC supply temperature and CRAC flow rate.
• the measured power consumption levels of every rack over a time T R specified by timer (202) are analyzed in 212 and the maximum power consumption level for each rack within that observation interval is calculated. These power consumption levels are used for prediction of rack inlet temperatures. This is done in anticipation that the instantaneous power of each rack in the next TR interval would not overshoot the maximum power observed for that rack in the current TR interval.
• recommendations are generated immediately, as explained in step 208 above. Genetic algorithm based optimizer might struggle to evolve towards the optimal solution if the number of supply temperature combinations which don't contribute to any hot spots for the given power levels are very less.
• step 216 it is checked whether the new combination of CRAC supply temperatures correspond to lesser cooling power than the cooling power corresponding to combination of existing CRAC supply temperatures. If the outcome of the step 216 is Yes, the process proceeds to step 220, where the new combination of CRAC supply temperatures are displayed on a display device through user interface 112 as recommendations or communicated to the data manager using alarms and notifications in 110. If the outcome of the step 216 is No, then the process proceeds to step 218 where the recommendation is not implemented and the existing CRAC supply temperatures are kept unchanged.
• the invention may be adapted to optimize CRAC flow rates as well.
• invention makes use of appropriate set of influence indices calculated for that particular combination of CRAC flow rates.
• the invention determines optimum combination of CRAC supply temperatures for many combinations of CRAC flow rates and the combination which gives lower cooling power and keeps rack inlet temperatures below threshold may be chosen as optimum combination of CRAC flow rates along with optimum combination of CRAC supply temperature.
• Figure 3 shows a block diagram of the user interface displayed on a display device.
• the user interface is provided for keeping the data center manager informed of the datacenter operating conditions and alerts the data center manager in case of any thermal problems.
• the entire layout of the datacenter (302) as seen from the top view is shown in the Figure 3. Labels on each rack indicate the rack name and predicted or measured temperatures at rack inlet. It may also include the power consumption of each rack. As a visual clue, the colors of racks change with their predicted temperature. Generated recommendations, alarms and causal analysis of thermal problems are displayed in the top-right corner of the GUI (304) while plots such as the temperature Vs time (306) is shown in the bottom-right corner of the screen.
• the machine may comprise a server computer, a client user computer, a personal computer (PC), a tablet PC, a laptop computer, a desktop computer, a control system, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
• PC personal computer
• tablet PC tablet PC
• laptop computer a laptop computer
• desktop computer a control system
• network router, switch or bridge any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
• the term "machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.
• the machine may include a processor (e.g., a central processing unit (CPU), a graphics processing unit (GPU, or both), a main memory and a static memory, which communicate with each other via a bus.
• the machine may further include a video display unit (e.g., a liquid crystal displays (LCD), a flat panel, a solid state display, or a cathode ray tube (CRT)).
• the machine may include an input device (e.g., a keyboard) or touch-sensitive screen, a cursor control device (e.g., a mouse), a disk drive unit, a signal generation device (e.g., a speaker or remote control) and a network interface device.
• Dedicated hardware implementations including, but not limited to, application specific integrated circuits, programmable logic arrays and other hardware devices can likewise be constructed to implement the methods described herein.
• Applications that may include the apparatus and systems of various embodiments broadly include a variety of electronic and computer systems. Some embodiments implement functions in two or more specific interconnected hardware modules or devices with related control and data signals communicated between and through the modules, or as portions of an application-specific integrated circuit.
• the example system is applicable to software, firmware, and hardware implementations.
• the methods described herein are intended for operation as software programs running on a computer processor.
• software implementations can include, but not limited to, distributed processing or component/object distributed processing, parallel processing, or virtual machine processing can also be constructed to implement the methods described herein.

Landscapes

• Engineering & Computer Science (AREA)
• Computer Hardware Design (AREA)
• General Engineering & Computer Science (AREA)
• Physics & Mathematics (AREA)
• Thermal Sciences (AREA)
• Microelectronics & Electronic Packaging (AREA)
• Cooling Or The Like Of Electrical Apparatus (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=49083402

Family Applications (1)

Country Status (1)

Cited By (4)

• 2013
  • 2013-02-18 WO PCT/IN2013/000103 patent/WO2013128468A2/fr not_active Ceased

Similar Documents

Legal Events