One of the complexities in benchmarking applications is defining the server configuration adequately.  The purpose of a benchmark is to provide guidance - to demonstrate how to obtain a given performance from a specific application workload.  This guidance is not useful if the performance cannot be reproduced.  To make benchmark results reproducible, it is necessary to define the server configuration, specifying everything that can affect performance.  From a hardware point of view, this is not difficult - specify the model numbers of every component in each server, then specify the model numbers of the components in storage and networking components.  Some of this information is available on-line, such as processor model number.  Other information, such as model or speed of memory DIMMs, is usually not accessible on-line, but this data is important.  For example, some current x86 servers have an option of 667MHz or 800MHz DIMMs, and this choice can affect application performance considerably.

Identifying the software components can be difficult, since you need to know which components affect the performance of the workload.

And the most obscure configuration area is firmware - in some cases, versions of firmware have a big impact on workload performance.  It is rarely necessary to document the firmware version of the server, but it is a good idea to document firmware versions of networking components.

Next, it is important to know how the quantities of specific components affect performance.  Performance varies with the number of disks internal to the servers, the number controllers connecting the server to external disks, the number and topology of network switches, etc. 

One important variable is the number of memory DIMMs.  The number of DIMMs affects performance in two ways - the total amount of memory on the server, and the memory performance.  It is useful to run the workload using the maximum number of DIMMs, then repeat the benchmark using ½ as many DIMMs.  Memory is expensive, and it is very useful to know how the workload performance varies with memory configuration.