Data Center Energy Saving using 5kV Distribution System Architecture 18 Sep 2009
Power Density Growing www.networkcomputing.com/crash course data center power.php
Increasing Demands 1984 Bank of America Data Center Concord, California 250,000 Square Foot Facility 10 Megawatt 21 kv Service 480V Distribution 2009 Typical Local Facility Santa Clara 40,000 Square Feet 12 Megawatt 12 kv Service 480V Distribution In 25 Years, 16% of the Space; 120% of the Power
Why 480 Volts? Common voltage for industrial and commercial distribution Wide spread usage results in readily available devices/components in equipment/distribution Allows easy integration of conventional static UPS systems Allows for easy conversion to server utilization voltage generally 120 VAC
1879 Thomas Alva Edison invents the carbon filament lamp. It burns for forty hours. 1880 Edison improves the lamp to last over 1200 hours. Voltage selected is 100V to allow for longer life 1882 Edison provides the first 59 customers in New York City with electricity The Beginning
New York City Distribution 1889 Individual distribution to users at 110V DC. At user level voltage is 100V DC. Three wire (+110V, 0, 110V) system Limited to about 1.5 mile radius
The Rival System AC George Westinghouse; ZBD Team, Nikola Tesla First system in Great Barrington, Massachusetts in 1886 At user level voltage is 100V AC. Distribution Voltage is 3000V AC (ZBD team was Hungarians Károly Zipernowsky, Ottó Bláthy, Miksa Déri)
War of the Currents Battle between DC (Edison) and AC (Westinghouse/Tesla) over which system is better. Harold Brown writes: The only excuse for the use of the fatal alternating current is that it saves the company operating it from spending a larger sum of money for the heavier copper wires which are required by the safe incandescent [DC] systems. That is, the public must submit to constant danger from sudden death, in order that a corporation may pay a little larger dividend.
Lessons to be Learned Maintaining status quo is generally in the best (commercial) interests of the market leaders Better methods often require new technology Consideration of methods already used elsewhere may lead to better methods for the local market Safety is always a consideration and it even sometimes can be a selling feature
Typical Distribution System Medium Voltage Service (12/21/35 kv) Expected System Losses 10 to 15% Server Medium Voltage Transformer 12/21/35 kv to 277/480 V PDU 277/480 V 120/208 Output LV Distribution 277/480 V UPS System 277/480 V
Implications of 480V System Data centers with 480 volt service and distribution virtually float on copper * Example: 75 2500 kva Feeder (3000A) Qty 8 4 inch conduits Over 2700 feet of 750 MCM wire For 10 MW system, four such feeders required Depending on layout, longer runs might be required Similar Feeder at 4160V 350 A Qty 1 4 inch conduit Under 275 feet of 750 MCM *
4160V Distribution System Medium Voltage Service (12/21/35 kv) Expected System Losses 5 to 8% (1/2 of 480V System) Server Medium Voltage Transformer 12/21 kv to 4160 V PDU 4160 V 400/230 Output MV Distribution 4160 V UPS System 4160 V
General Held Assumptions Medium Voltage Equipment is expensive; more than equivalent LV equipment Medium Voltage Equipment is much larger than equivalent LV equipment Medium Voltage Equipment requires more maintenance and is less reliable than LV Equipment Medium Voltage Equipment is not as safe as LV Equipment
Actual Considerations Cost of equipped 1200A MV feeder section about the same as a 3000A 480V unit assuming similar application and protection Conventional MV equipment is both larger than equivalent LV equipment and requires greater clearances, but IEC based equipment is equal or smaller Reliability depends on devices selected Safety may be better with MV equipment
Medium Voltage Equipment
IEC Switchgear Typical of most European builders. About 55 to 60 Deep X 24 to 30 Wide (Compares to 85 to 96 Deep x 36 Wide ANSI Equivalent Available with Conventional Vacuum or SF6 Breakers and Magnetically Actuated Vacuum Breakers
IEC MV Equipment Issues Not designed to meet US UL and ANSI standards most significant issue is bus insulation and isolation Designs require the use of bar type CTs limiting number of CTs for a given breaker Wiring techniques differ between US and IEC cable access in IEC generally requires disassembly.
New IEM Solution Designed as Arc Resistant 5 and 15 kv Rated 2000A Initial Maximum 31.5 ka Initial Max ka Bus Isolated and insulated Designed for use of Standard CTs 24 Width per section 60 Depth No Rear Access required for operation Front Accessible Connections Will be ANSI 37.20 UL Listed Metal Clad Switchgear
Construction Costs Computer Data Center with Tilt Up Concrete / Steel Frame Location: US National Average Stories: 1 Story Height (16.5 ) Floor Area (S.F.): 22,500 Labor Type: Union Release: Year 2008 Cost Per Sq Ft: $243.95 Building Cost: $5,488,900 A more accurate estimate of costs to build your particular building is available on: MeansCostWorks.com
Size Implications Equipment Type Size Clearances Costs 2H MV Switchgear ANSI Type 12kV Vacuum Breakers 1H MV Switchgear IEM Type 12 kv Vacuum Breakers (2 Sections) 2H MV Switchgear ANSI Type 12kV Vacuum Breaker w/ PTs 1H MV Switchgear IEM Type 12 kv Vacuum Breakers w/ PTs 36 x 84 48 x 60 36 x 84 24 x 60 72 front 72 rear 72 front 24 rear (recommended but not required) 72 front 72 rear 72 front 24 rear (recommended but not required) $14,250 $13,000 $14,250 $6,500
Reliability of Breakers
Safety Considerations Sample Case compared 4160 and 480 systems connected to identical loads and utility supply (34.5 kv 750 MVA) Results: 480V system Dangerous w/o Instantaneous (1.0 sec); PPE 3 w/normal instantaneous; PPE 1 w/low safety setting 4160V System PPE 1w/normal instantaneous setting; PPE 0 w/low safety setting
Summary 480V Systems 4160V Systems Costs: Space Initial Operating Equal Equal Higher Equal Equal Lower Size Equal Equal Reliability Lower Higher Safety Lower Higher