9 Hidden Data Center Operations Costs

And how to eliminate them

9 Hidden Costs

Table of Contents

Table of Contents
Introduction Hidden Cost #1: Capital Hidden Cost #2: Stagnation Hidden Cost #3: Delayed Benchmarking Hidden Cost #4: Knowledge Walk Hidden Cost #5: Corrective Maintenance Hidden Cost #6: Ignoring Predictive Data Hidden Cost #7: Vendor Maintenance Hidden Cost #8: Airflow & Set Points Hidden Cost #9: Its Not Always Energy Key Takeaways Where to Go From Here?
2

9 Hidden Costs

Introduction

Introduction
Did you know

In a recent survey by IDG Enterprise, 23% of IT Executives listed operating costs as their top concerns of 2012.

Everyone knows its expensive to operate a data center. Between the energy bill, human capital, and routine upgrades, the cost to run a data center is almost as painful as filling up your car. However, you may be unaware of hidden expenses that can drive up cost centers and break the budget. In this ebook, well look at a few of the ways you can eliminate these hidden costs, and get your data center operations program running efficiently.

9 Hidden Costs

Cost #1: Capital

Cost #1: Capital

Did you know that your operations program could be driving up your capital budget? It can be argued that a lower or multi-tier level design can be supplemented with a pristine operations program to achieve the reliability of a tier III or tier IV data center thats money saved before you even have an operating budget to spend! But, the right operations program can do more than that. It could save you from investing capital in a new build before you even need it. Lets look at an example: A Fortune 100 company brought in a 3rd party data center solutions provider to consult on future data center planning. At the time, the company was nearing the end of available space in their data centers and was in the middle of planning the build of another. The solution providers facility operations team immediately got to work, and within the first few months recommended and implemented several best practices which increased efficiency in their current data centers and freed up much needed computing space. This allowed the company to not only dramatically lower their operating costs, but also extend the life of their existing facilities by several years. By eliminating the need to build in the near future, the client preserved millions in capital. Not to mention, they saw an immediate return on their investment. What is your facility hiding that could be saving you money? 4

9 Hidden Costs

Cost #2: Stagnation

Cost #2: Stagnation

Does your operations program have a culture of innovation or stagnation? They say if youre not moving forward, youre falling behind. In this case, falling behind means that youre also losing money as well as losing your competitive advantage. One of the most important aspects of your operations program is a culture of innovation and proactive improvement. Without it, all the processes and procedures in the world could fall victim to the check-in-the-box syndrome (when something is done simply to be able to check it off your list). This type of thinking is flat out dangerous in a data center. It also means youre losing opportunity for improvement, whether that be reliability, process efficiency or energy efficiency. Change Management and Quality Systems (QA, QC & QI) help instill a proactive culture in an operations program. Not to mention, its what separates most good programs from great ones. Here is an example of what a culture of innovation can do: Facility Operations technicians identified an opportunity to enhance the air economizer on site. The technicians did their research, and reached out to the local utility company, which granted $800,000 to complete the project, which covered almost 75% of the cost. Once implemented, the improvement saved $100,000 in utility costs per month and showed a three month return on investment.

9 Hidden Costs

Cost #3: Delayed Benchmarking

Cost #3: Delayed Benchmarking

Almost all energy efficiency projects you implement will be costly upfront. Even if youre certain a project will save you money, you must have a baseline in place to prove ROI and secure funds for the project. Not only does benchmarking provide a way to measure savings, it allows you to make sound decisions backed by data that wont affect reliability and availability. This is why you need to establish a energy usage and energy efficiency baseline. Having a years worth of data is ideal to account for seasonal cost power consumption. The longer you wait to establish and track your baseline, the more time and money is wasted operating a non-optimized facility. Optimizing the efficiency of any system requires the measurement and evaluation of system performance. Energy benchmarking of the data center is an effective means to:

To have a world-class facility operations program, you must instrument and create efficiency models for your data center. By creating a baseline for energy usage and efficiency, you can use modeling and best practices to make and document efficiency improvements of up to 80%.

Establish baseline performance to measure against future performance or for comparison to similar facilities. Identify operational and maintenance issues. Qualify and quantify changes in performance. Assist in management and planning for both current and future facilities.

9 Hidden Costs

Cost #4: Knowledge Walk

Cost #4: Knowledge Walk

What happens if your operations MVPs get injured, retire or leave the company? Does their knowledge of the facility walk out the door with them? There are often operations programs where there is no formal documentation of training and operations procedures. Instead, tribal knowledge exists in that data center. When training is necessary, its almost always on the job, which means there is almost always lost knowledge in the process. In this situation, the worst thing that could possibly happen is a knowledgeable technician leaving the program. This knowledge walk is almost impossible to replace, as it takes years for a new employee to discover everything about the data center facility. Not only does this cost you in terms of employee experience, it puts you at risk of downtime. There is potential for HUGE cost savings that could be realized if a formal training program is implemented. Heres what we mean:

1. In a formal training program, youve got the best people being constantly trained, drilled and tested to 2. 3.
perform better. This does two things: (1) retains good employees by keeping them motivated and giving them a career path; and (2) reduces the risk of human error by putting the classroom into practice so the right actions are second nature. Formal training programs provide a conduit for consistently relaying techniques for lower resource consumption (time, power, water, equipment wear and tear, etc.). Formal training provides a reliable way to start self performing some maintenance activities, and in some cases, significantly reducing vendor costs. More on this later.

9 Hidden Costs

Cost #5: Corrective Maintenance

Cost #5: Corrective Maintenance

As you know, it is far cheaper to change the oil in your car (preventative maintenance) than fixing a blown engine (corrective maintenance). The same goes for preventative maintenance in a data center. Did you know that 30% of all preventative maintenance activities go unperformed?* This is not surprising considering that, on average, it takes 8 man hours to schedule a single maintenance activity. Now consider that in a 50,000 ft2 data center it is typical to see 800 preventative and 500 corrective maintenance activities take place per quarter its easy to see how preventative maintenance gets neglected. You can help combat the time suck, save money and improve reliability by leveraging workflow automation and document management software, such as a CMMS (computerized maintenance management system) and DMS (document management system). For example, one facility operations team recommended workflow software that would organize, simplify and automate the MOP process (Method of Procedure). The result? Saving $30,000+ annually by being able to produce more MOPs, perform the work faster, taking on more work elsewhere and eliminating countless hours sifting through a manual, disorganized filing system.

*Based on data collected through Lee Technologies NOC (National Operations Center).

9 Hidden Costs

Cost #6: Ignoring Predictive Data

Cost #6: Ignoring Predictive Data

What good is data that you dont use? While collecting data is important, the data must be useful to justify the process. Weve already talked about how benchmarking your resource usage, like power and water, can help you identify areas for greater efficiency. The concept also applies to maintenance. Similar to preventative maintenance, predictive maintenance uses your past experiences and data to identify weak points and correct them before they become costly problems. To clarify, predictive maintenance allows you to:

Know your systems weak points Correct weak items before they fail Modify procedures & scope of work to address such items Adjust your data gathering and collecting as necessary

The key to predictive maintenance is documenting your preventive maintenance (PM) and instituting a Lessons Learned program. By analyzing preventative maintenance data, you can locate recurring problems, see when equipment performance is degrading and make changes to increase total system reliability. A Lessons Learned Program enhances the reliability and quality of your work by providing a conduit to make improvements or corrections to processes and procedures (It also feeds back in to the culture of innovation!). The savings will really start to add up!

9 Hidden Costs

Cost #7: Vendor Maintenance

Cost #7: Vendor Maintenance

Could you be self-performing preventative maintenance or renegotiating third party contracts? By now, you see there are many ways to lower operating costs through your maintenance practices. Two final way to lower costs are by performing preventative/predictive maintenance in house and renegotiated third party contracts. Take this advice with a word of caution: you should only take in maintenance activities that:

1. 2.

You will actually do (remember 30% of preventative maintenance activities go unperformed) You can do without reducing reliability. (Is your team qualified? Do they have consistent training? Do you have a operations methodology in place that will reduce the risk of human error?)

The savings could be tremendous. For example, one company hired two technicians to perform more work in-house and reduced vendor costs by $400,000.

10

9 Hidden Costs

Cost #8: Airflow & Set Points

Cost #8: Airflow & Set Points

Alright, so you benchmarkednow what?

A computational fluid dynamics (CFD) study can be performed to better optimize computer room air handler units, under floor dampers and perforated tile placement. One company performed adjustments made based off this study to save the facility up to $2 million.

Once you have benchmarked your energy usage and other data points, you can begin implementing some free or low cost efficiency improvements, such as:

Airflow Management Fine tuning the air delivery system (sealing floor penetrations, proper placement of perforated tiles and the use of blanking panels) Balancing the system and tuning the under floor airflow with diffusers and baffles Optimize data center configurations and set points and various settings on equipment, such as HVAC systems, when they are not fully loaded. Look at utility charges and make sure that client is getting fair price Employing efficient power system components (these components are usually cost-effective when installing new or replacing end-of-life equipment)

You can expect to save 20% of total energy spend on low hanging fruit types of activities, such as these. Expect to save much more on capital projects, such as economizers. 11

9 Hidden Costs

Cost #9: Its Not Only Energy

Cost #9: Its Not Only Energy

While energy efficiency is a hot topic and a huge area for operational savings, dont forget there are other means to cost savings, such as:

Reducing energy consumption all around as opposed to just using it efficiently. Both will positively affect your power bill. Other resource consumption, such as materials, man power and time. Use your culture of innovation to make improvements to cost, energy usage, process and quality. Costs to avoid, such as safety hazards, downtime, knowledge walk, etc. Sometimes the best cost savings is having insurance or assurance against expensive, negative events. If youre building, consider the impact your facility design has on operations. Consider reusing or repurposing materials. For instance, designing a chilled water system that treats and reuses water from the chillers to irrigate site vegetation. This design saves one company $2.6 million gallons of water of year and saves $80,000 annually in water/sewage. It also has a 2 year ROI.

12

9 Hidden Costs

Key Takeaways

Key Takeaways
1. 2. 3. 4. 5.
Your operations program can affect the cost of future builds and visa versa. If you are planning to build, verify with a unbiased third party that you do, in fact, need to build. If so, include your operations team in the design process to help determine the designs impact on future operational costs. Data is one of your most valuable assets, so put it to work to optimize the energy efficiency and reliability of your data center. A quality training program can help your employees perform their best, keeps them happy and helps you get the most bang for your buck. Performing maintenance before problems arise is cheaper than corrective maintenance. Preventative and predictive maintenance can also be performed in-house to lower thirdparty vendor costs. It will also help extend the life of your equipment. Dont forget that increasing energy efficiency is not the only way to lower your operating costs.

13

9 Hidden Costs

Where To Go From Here?

Where To Go From Here?

Still wondering where to go from here? Talk to your data center management team to determine what activities can realistically be performed in-house. Discuss the possibility of performing an audit on your data center to identify inefficiencies. Once the audit has been performed, look at where improvements can be made. Stack rank them in order of ease to perform and cost effectiveness. For more best practices on facility operations, check out the blog, Data Center Insider, from Lee Technologies, a division of Schneider Electric.

14

9 Hidden Costs

2011 Schneider Electric. All rights reserved. All trademarks are property of Schneider Electric or its affiliated companies. address: 132 Fairgrounds Road, West Kingston, RI 02892 USA

15

make the most of your energy