Reality Check: Server Insights : multi-core processor

I received a new HPC Multi-core server today – documenting the configuration

d-field — Thu, 04 Dec 2008 14:36:00 GMT

One of the complexities in benchmarking applications is defining the server configuration adequately. The purpose of a benchmark is to provide guidance - to demonstrate how to obtain a given performance from a specific application workload. This guidance is not useful if the performance cannot be reproduced. To make benchmark results reproducible, it is necessary to define the server configuration, specifying everything that can affect performance. From a hardware point of view, this is not difficult - specify the model numbers of every component in each server, then specify the model numbers of the components in storage and networking components. Some of this information is available on-line, such as processor model number. Other information, such as model or speed of memory DIMMs, is usually not accessible on-line, but this data is important. For example, some current x86 servers have an option of 667MHz or 800MHz DIMMs, and this choice can affect application performance considerably.

Identifying the software components can be difficult, since you need to know which components affect the performance of the workload.

And the most obscure configuration area is firmware - in some cases, versions of firmware have a big impact on workload performance. It is rarely necessary to document the firmware version of the server, but it is a good idea to document firmware versions of networking components.

Next, it is important to know how the quantities of specific components affect performance. Performance varies with the number of disks internal to the servers, the number controllers connecting the server to external disks, the number and topology of network switches, etc.

One important variable is the number of memory DIMMs. The number of DIMMs affects performance in two ways - the total amount of memory on the server, and the memory performance. It is useful to run the workload using the maximum number of DIMMs, then repeat the benchmark using ½ as many DIMMs. Memory is expensive, and it is very useful to know how the workload performance varies with memory configuration.

Virtualization Never Looked So Good

R_Palmer — Fri, 21 Nov 2008 21:33:00 GMT

Virtualization has become a mainstay, even a business imperative, within most data centers today. Today, every server manufacturer is claiming leadership in virtualization platforms, so the real question you need to ask is, "Why HP?" Well, let me give you a few of the key reasons why HP is the right choice for your trusted advisor when it comes to virtualization.

Whether it's for server consolidation, workload balancing or security and disaster recovery purposes, the dramatic growth of virtual machine implementation is mind boggling. This is not necessarily coupled to the comfort level of IT managers though, as there continues to be a disarray of management tools and interfaces required to manage both physical and virtual machines. HP has taken this challenge head on with our new Insight Control Environment (ICE) which gives you a single pane of glass to monitor and manage both your virtual machines and physical machines simultaneously.

On November 17^th, we introduced the new HP ProLiant DL385 G5p server based on the new Quad-Core AMD Opteron 2300 processor (code named "Shanghai"). The HP ProLiant DL385 G5p is designed and optimized specifically for virtualization and consolidation environments. In fact, we looked at every facet of the product in our development effort with a few key principles in mind:

Provide a strong investment protection value for existing AMD Opteron customers
Improve the overall performance in the same, or lower, power envelope
Optimize the virtualization performance
Increase the number of virtual machines on a single platform
Simplify and improve the security of deploying of virtual machines

The new DL385 G5p is now available, having met and exceeded these objectives, arriving ahead of schedule. With the new 45nm "Shanghai" processor, with a 6MB L3 cache, we have achieved higher performance and maintained Socket F compatibility, yet at a substantially lower power envelope. HP, unlike some of our competitors, has designed the DL385 G5p to provide our customers with the flexibility to choose between single processor or dual processor configurations, without sacrificing any of the improvements offered by Shanghai. We have optimized the real estate inside of the 2U chassis to deliver twice the memory, twice the NICs and 6 times the storage capacity as the previous generation DL385. With 16 DDR2 DIMM sockets, you can reach 128GB of memory using 8GB high performance DIMMs. This allows for significantly more virtual machines to run on this single platform.

In addition, with virtual machine deployment, networking capabilities are critical. In the new DL385 G5p, we integrated 4 gigabit Ethernet ports to free up the PCI-Express slots to meet your flexible design requirements and increase the bandwidth for the virtual machines running on the server. The flexibility of the PCI bus slot configuration is another key benefit of the HP ProLiant DL385 G5p. With up to 6 PCI-e slots, additional networking, storage and application specific I/O adapters can be added to improve the virtualization experience. There are many customers who require legacy PCI-X controller support and some that are looking for the latest and greatest x16 PCI-e controller support. In the DL385 G5p, we have given those customers the flexibility to configure the PCI bus slots to meet their unique application and environment needs.

Storage performance is a key differentiator in the market today, even more so in virtualization environments. That's why we increased the internal disk storage on the DL385 G5p to 16 2.5" SAS or SATA disk drives; or 6 3.5" SAS or SATA disk drives. This provides up to 6TB of internal disk storage to be shared between the virtual machines with improved performance when compared to external shared disk solutions. By bringing the disks closer to the processors, the performance bandwidth issues are minimized and the overall performance of the storage is improved.

This server is built with efficiency in mind! The HP ProLiant DL385 G5p helps save on energy costs with the industry's highest efficiency power supplies that currently power the complete portfolio of HP ProLiant servers. Finally, with the internal USB slot, you can utilize an iHypervisor on a USB key that allows you to deploy virtual machines securely and easily.

In closing I'd like to quote Paul Gottsegen, VP of Marketing for HP's Industry Standard Servers division, from a recent interview:

"Customers can drive down costs through new "Shanghai"-based HP ProLiant servers that set new levels of power efficiency and performance. HP has experienced unparalleled success over the past four years working with AMD in bringing AMD Opteron processor-based platforms to customers of all sizes. Early results indicate "Shanghai" is a winner."

HP continues to lead the market in delivering more ProLiant servers featuring AMD Opteron processors than any other system manufacturer. In fact, HP has shipped over four times more than IBM and 1.5 times more than Dell (IDC- Q208 Server Tracker, Oct.08). Have confidence in choosing HP as your trusted advisor as we continue to set the bar for performance, energy efficiency and optimization in virtualization environments.

For more information click here on the DL385 G5P server link, or listen to the replay from Paul Gottsegen at the AMD Shanghai launch event on November 13^th.

I received a new HPC Multi-core server today – Live from SC08

d-field — Tue, 18 Nov 2008 22:59:00 GMT

This week, I am attending the annual SC08 conference. For 2 decades, the tradition in HPC is to announce and demonstrate new products at the SC conference, and there is a lot to absorb.

For me, the event started with a 2-day HP user conference, attracting some of HP’s largest HPC customers. It was a very full 2 days, with lectures on many subjects from customers, Intel, AMD, software development companies, and of course HP. Even though I work here, I learned a lot about projects in other parts of the company.

At the SC08 show, I saw my 1^st HP POD – portable optimized datacenter – a 40-foot-long shipping container on the outside, and a self-contained computer facility on the inside. This excellently-designed mobile facility can contain 3500 servers or 12,000 disk drives, in standard 19” racks. Just provide a flat location, power, chilled water, and a network cable, and you have a new computer room.

To make the week more interesting, AMD announced the new Opteron Shanghai processor – higher clock speed, bigger cache, faster Northbridge, and improved performance:power ratio.

One of the good things about big companies is that they spawn startups. This year, a startup company announced a new HPC architecture at SC08. Nearly all the employees are my friends and previous HP co-workers. Check out www.conveycomputer.com

AMD launches quad-core Shanghai processors

aimeeschoaf — Mon, 17 Nov 2008 01:47:00 GMT

This is an exciting time for our partner, AMD, as they launch their newest family of processors, and HP is pleased that our ProLiant portfolio will offer new capabilities computing to customers of all sizes. The new "Shanghai" products (Quad-Core AMD Opteron 2300 processors) will help HP customers gain greater computing power and business results when they build their infrastructure on HP ProLiant ML, DL and BL servers. Whether they are using virtualization technology to optimize their IT assets or they are consolidating their server inventory, HP ProLiant servers featuring the Quad-Core AMD Opteron 2300 processors will help customers do more business in a smaller footprint.

Around the globe, customers using AMD Opteron-based HP ProLiant servers value investment protection, consistent performance improvement, power efficiency and optimized design elements that enhance their virtualization strategies. The Quad-Core AMD Opteron 2300 processors deliver on these expectations with the larger 6GB L3 cache, Socket F compatibility, decreased power envelope, improved performance/watt and enhanced virtualization indexing. Watch for news on HP ProLiant offerings featuring the Quad-Core AMD Opteron 2300 processors.

- Rich Palmer

Director, HP Technology Strategy

Industry Standard Servers

I received a new HPC Multi-core server today – How to measure the power usage

d-field — Wed, 12 Nov 2008 00:38:00 GMT

Given that it is important to measure power usage and correlate it to application performance, how do you measure the power?

We use 2 different methods - one for rack-mounted servers and another for blade servers. The rack-mounted servers do not provide power meters, so we bought a power meter. We plug the server into the power meter, so we are measuring the total power used. Then, with a simple PC interface, we allow the application user on the server to obtain continuous power data which is easy to correlate with the applications.

This is easy for the users, but it requires planning and logistics and some work by our system managers, to connect the meter to the right server at the right time.

We often want to measure the power of a cluster running one HPC application in parallel, and it is usually sufficient to measure the power of any one server in the cluster running the application.

It is easier to measure power on an HP blade enclosure, since the enclosure contains power measurement capability and provides this data in a usable way. The available data includes the total enclosure power and also the power used by each blade server and each fan in the enclosure. We integrated this information with the Platform Computing LSF job scheduler. Now, users of our blade servers submit their jobs via LSF and automatically receive their power usage data as part of the job.

Next week, I expect to post a message from the SC08 conference.

I received a new HPC Multi-core server today – Intro to power measurement

d-field — Thu, 06 Nov 2008 00:02:00 GMT

Until a couple of years ago, when we referred to performance measurement of an application, we meant the amount of time that it took the job to run vs. the specific resources it used - number of cores, number of servers if you are using a cluster, the specific characteristics of the server cores and memory and other server specs, plus IO/storage resources and specs.

Basically, we only measured one thing, the elapsed time of the job. Then, using the resources and specs, we computed lots of things - throughput efficiency, parallel scalability and efficiency, performance per core or per server, IO metrics, etc, etc.

Now, we make an additional measurement - power utilization, which we correlate in time with the execution of an application. We want to know the average power used during the execution of a single job, and we also look at the variation of power during a job, and the maximum power used.

Of course, lots of people measure power used by computers. But, since most of these people are system managers or system designers, they don't have a reason to correlate power with specific applications and compute jobs. They want to know the average and peak power used to run their overall workload, so they can plan for current and future power requirements. This is important work, but it does not give them the ability to optimize their workload.

If you measure the average power used during the execution of one compute job, and you multiply that power by the elapsed time of the job, you have Application Energy - the electrical energy used to run that specific job. This is a very convenient quantity, since it gives you a single number that relates power usage to compute jobs. You can use Application Energy to optimize your workload, just as you use elapsed time.

A couple of examples:

1. You can measure application energy for a given set of applications on two or more different server models, and then select the more energy-efficient model. You can use this App Energy comparison together with elapsed time comparison, and then make speed vs. energy tradeoffs. If a job runs 30% faster but consumes 50% more Application Energy on Server A than it does on Server B, which is a better choice for your requirements?

2. We are also using Application Energy to determine the most efficient way to run applications which run in parallel on a cluster of servers - a common way to run HPC codes. For one common HPC application, we ran the same job at 3 levels of parallelization and compared the elapsed times and Application Energy. We showed that the job used only 4% more Application Energy running 32-way-parallel (on 32 cores) vs. 16-way parallel. But the job used 20% more Application Energy running 64-way-parallel vs. 32-way-parallel. In other words, there is very little energy cost using 32 cores and returning the results to the user much faster vs. using 16 cores. But there is a substantial energy cost to use 64 cores, which returns the results even faster.

Does anyone find this interesting, or agree (or disagree) with this approach?

I received a new HPC Multi-core server today – Running on a subset of cores

d-field — Wed, 29 Oct 2008 20:57:00 GMT

In some situations, it is useful to not use some of the cores on a server. Since most processors do not have sufficient memory BW to support a memory BW-intensive code running on all cores, such codes do not "scale" perfectly. There are 2 common ways to define scaling - serial-job throughput workload (multiple serial jobs), and a single parallel code workload.

If scaling is perfect, then an 8-core server can run 8 copies of a serial job in the same time as one serial job.

For a highly scalable application: if scaling is perfect, then an 8-core server runs an 8-way-parallel job 8 times faster than a serial job.

Most HPC jobs can not scale perfectly, so this is an issue. But in most cases, the server can run more total work (jobs per day) using all of its cores than it can if some cores are unused. So why would we consider leaving some cores idle? The primary reason is the cost of running licensed applications. Many HPC applications are licensed on a per-core basis, although the cost may not be linear with the number of cores. It is useful to compare the per-core job performance to the per-core license cost to determine the best performance-to-cost operating point.

Given that it may be useful to use a subset of the cores, doing so correctly is difficult. You need to know the architecture of your server. Here is an example of an HP ProLiant server containing two Intel Xeon Harpertown quad-core processors.

-Each processor has a separate connection to the memory system.

-Each processor has 4 cores. Each pair of cores shares a data cache. The 4 cores share the processor's memory BW.

If you draw a picture of this, you will see that not all combinations of cores are equal in terms of cache size and memory BW resources.

Let's say we want to run a workload consisting of one parallel job, using only 4 cores in the server. The best performance is obtained using 2 cores on each processor, and selecting cores which do not share a cache (so that one core has full use of the entire shared cache). The next-best performance uses 2 cores per processor, and selecting cores which share a cache. The 3^rd-best performance uses 3 cores on one processor and one core on the other processor. The worst performance is obtained using 4 cores on one processor.

If we want to run a single parallel job on only 2 cores in the server, there are three possible choices. The best performance uses 1 core per processor. The next-best performance uses 2 cores on one processor, selecting cores which do not share a cache. Third best uses 2 cores on one processor, selecting cores which share a cache.

How big is the performance difference based on these choices? The answer depends on the application, the specific input data, and other factors. But here is an example, for a moderately parallel HPC application. Running the code in parallel using one of its standard performance benchmarks, it runs 4 times faster using all cores (8-way-parallel) than using 1 core (serial).

Running 4-way-parallel using the above choices of 4 cores, the performance varies from 3.4 (best case) to 2.9 (next-best case) to 2.4 (third-best case) to 2.3 (worst case) times faster than serial.

Running 2-way-parallel using the above choices of 2 cores, the performance is either 2.0, 1.8, or 1.5 times faster than serial.

Clearly we can lose a lot of performance if we do not select the cores carefully!

This analysis would be different for different processors or different server configurations.

I received a new HPC Multi-core server today – Measuring memory BW

d-field — Tue, 21 Oct 2008 14:39:00 GMT

The lmbench memory latency benchmark gave us a lot of information about the new system. Next, we ran the STREAM memory BW benchmark suite.

Before running HPC codes, we needed to ensure that hardware multi-threading is disabled, if it exists. This feature allows each core on the server to appear to the OS as if it were 2 cores. For many applications, this technique increases throughput. But for nearly every HPC application, a server has insufficient memory BW to dilute it with multi-threading.

We then measured memory BW - we used the industry standard benchmark, STREAM.

There are four benchmarks in the STREAM suite: COPY, ADD, SCALE, and TRIAD. There are multiple ways to run each of these benchmarks. We started by running each benchmark in serial (1 non-threaded copy of each benchmark), one at a time. This shows us the maximum amount of memory BW that one core can consume.

Next, we measured memory BW consumed by multiples of 2 cores, up to the maximum number of cores. There are 2 obvious ways to do this: run multiple simultaneous copies of the serial benchmark, or run a single benchmark that is multi-threaded using SMP parallelism (via OpenMP). We usually run the SMP-parallel STREAM benchmarks.

We ran each SMP-parallel STREAM benchmark from 2 cores to the maximum, in 2 different arrangements:

-Packed - use all cores on each processor before going to next processor

-Cyclic - alternate cores among the processors

In Packed mode, the memory BW usually increases monotonically with the number of cores.

In Cyclic mode, the memory BW has a zig-zag pattern and often approaches the maximum memory BW of the system when only one core per processor is used.

By observing these measurements, it is possible to determine how the memory BW is allocated to the processors. Each processor may have a unique path to the memory system and therefore an amount of memory BW that is (roughly) independent of the memory usage of the other processors. Or, two or more processors may share a path to memory.

Normally, the maximum measurement for each benchmark occurs when all cores in the server are used. But we have seen strangely architected systems which reach a maximum memory BW using a subset of the cores, after which memory BW declines with additional cores.

One interesting bit of information is obtained by comparing the memory BW measurement used by all the cores on one processor vs. the BW used by 1 core. This tells us the fraction of the processor's memory BW that a single core can consume. In many cases, one core can consume the entire memory BW available to the entire processor; one core can often consume a significant fraction of the total system's memory BW. When applications requiring high memory BW are run on such a server, it is impractical to run the application on more than one core per processor (or even one core per server).

All these measurements may seem excessive, but memory BW is often the performance limiter in HPC applications. If an application developer knows the STREAM information, the application can be run in an optimal way on a server - using a subset of the cores to run the application, without demanding more memory BW per core than the system can provide.

I received a new HPC Multi-core server– Learning from standard benchmarks

d-field — Wed, 15 Oct 2008 22:07:00 GMT

Now that we have configured the hardware components and firmware settings in a known and hopefully optimum way, it is time to run the 1^st performance test. Personally, I like to run the memory bandwidth benchmark lmbench first, since there is a lot to learn from it. This benchmark computes the time required to move different amounts of data from the caches and memory to a core.

A modern server has multiple levels of caches - 2 or 3 levels. The highest level may be shared among several cores. The output of lmbench shows plateaus for each level of cache, showing both the latency to the cache and also the size of the cache. If the system has NUMA memory organization, then lmbench shows the latency for each of the NUMA "hops", as the data traverses the system topology.

Usually, the latency of each cache level is a fixed number of processor clock cycles. It is both interesting and important to know this number. It's interesting, because it allows you to compare different cache architectures, even if the systems you are comparing have different clock speeds. For example, it is interesting to me that the 1^st level cache latency of several modern servers, with very different architectures, is 3 cycles.

And it is important, because you are not always sure what clock speed your system is running, and you can compute it using lmbench. Or, you might encounter the problem we hit yesterday - on a 2-processor pre-production server, the two processors were running at 2 different clock speeds! This is of course not good - a user observed strange performance, but lmbench took the mystery out of the problem and told us exactly what was happening.

You can run lmbench in different ways, and each provides additional understanding.

The "stride" method runs sequentially through memory addresses and provides best-case latency.

The "random" method accesses memory addresses randomly, showing the worst-case latency. It is very useful to know the best-case and worst-case latencies - if their ratio is small, then memory performance is somewhat predictable. If the ratio is large, prediction becomes difficult.

If you set lmbench to access memory in units of the VM page size, then you can observe the impact of TLB misses on latency.

And, if you run lmbench simultaneously on cores which share a cache, you learn about the behavior of the shared cache.

Memory latency is a great tool if you need to unravel the architecture of a new server!

I received a new HPC Multi-core server today - Initial test, part 2

d-field — Thu, 09 Oct 2008 18:36:00 GMT

Today's work needed to be done before we can do any code performance testing: First, we configured the memory. I received half as many memory DIMMs as the system can hold, and I searched for the person who could tell me which memory slots to use for best performance. After this step was completed, we tackled the firmware. Firmware contains optional settings which impact performance. We first determined the optional settings that are available for this server. We looked for options like NUMA/interleave, snoop filter, multi-threading, adjacent-sector prefetch, hardware prefetch, etc.

Eventually, we will test the performance of all values of each of these settings. For most of the options, we will get lucky, and one value will be definitely better than all others.

But for some options, the best setting of the option depends on the applications being run. Memory organization is a good example. For a system with uniform memory architecture (UMA), the time to obtain a cache line from memory is roughly independent of which core accessed it. There are no settings needed for this organization.

For a NUMA system, the memory is "closer" to the cores in one processor than in others.

Some applications run better in NUMA mode, and others in UMA mode.

In some systems with NUMA memory organizations, a firmware option exists to run the system in both ways - use the NUMA organization, or interleave the memory among all processors to create a roughly uniform memory organization. When this option exists, we run many applications in both ways.

We also needed to determine the way that cores are numbered on each processor. Core numbering is not always consistent from one model to the next, and to measure performance on a subset of the cores in a server, it's necessary to know which cores you are using - which cores are on which processor; which cores share a cache.

Having completed this work and established a baseline of configurations and settings, we can start to do some performance measurements.

I received a new HPC Multi-core server today - Initial performance testing

d-field — Wed, 01 Oct 2008 23:15:00 GMT

After the operating system boots on a new server model, it's time to start performance testing. I'm hoping to get some comments on this, since there are many different ways to proceed.

Personally, I am not a fan of industry standard benchmarks, but I think they are the best starting point for new product testing. For an HPC server, I want to check the basics, to ensure that the system meets its design goals - memory BW, memory/cache latency, 64-bit floating point math, and filesystem IO are the first measurements. We use STREAM, lmbench, LINPACK, and IOZONE standard benchmarks.

We compare the measurements to older models and to the goals of the new product. Since our group didn't design the product, we don't know if there are interesting but undocumented features which enhance or limit performance. I have a name for the process of performance testing of a new product - discovery engineering. We study the external behavior of a system and try to understand the design features which affect HPC performance.

Each of these standard benchmarks provides information about the server. In numerous cases, this information has solved performance mysteries in real codes. It is hard to solve a performance problem with a large, complex application. The standard benchmarks are a simpler starting point for problem-solving.

But we don't rely on industry standard benchmarks. By design, each one tests a subset of a server's performance characteristics. I've had little success predicting the performance of HPC ISV applications based on the standard benchmark results, since real applications require a balance of all the performance features in the system.

After using the standard benchmarks to assure ourselves that the system is running correctly, we can move on - measure performance on real ISV applications, and experimenting with multi-core processor configurations!

I received a new HPC Multi-core server today!

d-field — Tue, 23 Sep 2008 19:16:00 GMT

About this blog series - This is the 1^st posting of a series which describes the experiences of engineers who test the performance of HPC servers and server clusters at HP.

My name is Dave Field. I lead an engineering group at HP - we measure the performance of new HP servers. In addition to the common industry-standard benchmarks, we concentrate on the performance of real HPC ISV applications. In the 20+ years we have done this work, we have seen many server architectures. These days, HPC clusters of servers using multi-core processors occupy most of our energy.

We evaluate the performance of new server products, so receiving a new server model is a common occurrence. This has been an especially rich year for new products - this is the 14^th new HP server we've tested this year, with at least one more to go before the year is over. HP servers for HPC span the range of industry-standard processors - Intel Xeon and Itanium2 and AMD Opteron. (In HP terminology, the processor is the physical component which plugs into the system board. A processor contains one or more cores, or CPUs.) And for each processor type, there are specific models with different architectural features.

Since we test pre-production, or prototype, computers, it's not quite true that I received a server - we usually receive new product kits. Testing new products can be very interesting, but to get to the interesting part, there are inevitably a number of problems to solve. We need to turn the kit into a working computer, then ensure that the performance meets the product specs, before we can do meaningful performance evaluation. These initial steps are lessons in patience and expectation-setting, during which I always meet some new people who will help in problem-solving.

The new server kit usually contains the server enclosure, system board, and processors. To turn the kit into a computer, we need to obtain three layers of stuff - supporting hardware (the right DIMMs, network interfaces, and disks), firmware, and operating system.

Firmware is in flux during the pre-production period, and each version of pre-production firmware changes the server's performance. Usually the processors are pre-production versions, tied to specific firmware revs. Most of the performance data collected on these early versions will be discarded. But if we don't get some measurements now, we can't influence the product. Sometimes we identify performance issues which can be fixed before production release - so this is a very satisfying part of the job.

These days, the current versions of the major Linux distributions work out-of-the-box on new server models.

When the operating system boots, we can begin to measure performance!