HP LoadRunner and Performance Center Blog : Performance Center

Where does Performance Testing fit in Agile SDLC?

HMakhija — Thu, 12 Nov 2009 18:55:00 GMT

Is agile software development lifecycle (SDLC) all about sprinting i.e. moving stories from product backlog to sprint backlog and then executing iterative cycles of development and testing? IMHO, not really! We all know that certain changes in an application...(read more)

Testing is Back on the Charts

mark.tomlinson — Fri, 02 Oct 2009 23:47:00 GMT

...and Quality sounds better than ever! The latest release entitled Here Comes Science from the Grammy-winning duo They Might Be Giants (John Linnell and John Flansburgh) includes a track called "Put It to the Test" which celebrates the enthusiasm...(read more)

Update: 15-hours into the Non-stop HP Virtual Conference

mark.tomlinson — Wed, 30 Sep 2009 06:29:00 GMT

*original post was at 11:30pm on 9/29/2009* Okay - so I've been working the Worldwide HP Virtual Conference: Functional, Performance and Security Testing for about 15 hours now - nearly non-stop and I've had some great chat sessions with customers...(read more)

The LoadRunner Non-stop HP Virtual Conference

mark.tomlinson — Mon, 28 Sep 2009 17:55:00 GMT

THE TIME HAS COME!! I really don't want you to miss the Worldwide HP Virtual Conference: Functional, Performance and Security Testing in Today’s Application Reality. And just to show you how committed I am to you and to the conference, I'm...(read more)

Performance Testing Needs a Seat at the Table

sfeloney — Fri, 25 Sep 2009 18:49:00 GMT

It is time Performance Testing gets a seat at the table. Architects and developers like to make all the decisions about products without ever consulting the testing organization. Why should they? All testers have to do is test what's created. If testers...(read more)

What are the Goals of Performance Testing?

sfeloney — Fri, 25 Sep 2009 18:44:00 GMT

So what is the point of performance testing? I get this question often. And depending on who you talk to, you get different answers. First let me begin by telling you what are NOT the goals of performance testing / validation. Writing a great script Creating...(read more)

HP Performance Engineering Best Practices Series

mark.tomlinson — Fri, 11 Sep 2009 21:41:00 GMT

Just to let you know that we've been putting together some published practices for LoadRunner and Performance Testing...and the electronic version of the book(s) are available free of charge!

This one is "Performance Monitoring Best Practices" authored by Leo Borisov from our LoadRunner R&D team. Leo sent along this description and instructions for downloading the book:

"We have always recognized the fact that having best practices and methodology would greatly simplify the life of performance engineers and managers, and now we are beginning to fill this need. The book is available with the SP1 installation. Access it from the product documentation library, or from the help menu.

To download a copy from the hp software support site:

go to http://h20230.www2.hp.com/selfsolve/manuals
log in using an hp passport or register first at and then log in
in the list of hp software product manuals choose either LoadRunner or Performance Center – the book is listed under both
select product version: 9.51 and operating system: windows
click search

Since this is the first book in the series covering various aspects of methodology, we would really appreciate your feedback. Please send your feedback directly to me or lt_cust_feedback @ hp.com."

Congratulations Leo - thanks for your efforts!

ROI: You Get What You Pay For

sfeloney — Tue, 12 May 2009 20:45:00 GMT

We've all heard that saying. But how many times do we really follow it? We have bought, ok I have bought, cheap drills, exercise machines, furniture, only to be sorry about when they break prematurely. Or you find a great deal on shoes only to have...(read more)

Managing Changes – Web 2.0, where's your shame

mark.tomlinson — Wed, 06 May 2009 20:08:00 GMT

Oh, this strange fascination started in 2004 when they coined this new generation of ‘web development’ called Web 2.0. I witnessed this evolution of technology from my seat in steerage at Microsoft as customers switched from the old Active Architecture (remember Windows DNA?) to the warm impermanence .NET and J2EE architectures for web applications. Out with the old and in with the new, but the performance problems were generally the same – memory management, caching, compression, heap fragmentation, connection pooling, and so on. It might have had a new name, but it was the same people making the same mistakes. Back then we dismissed some of these new architecture as unproven or non-standard. But that didn’t last long. Now almost 5 years later with Web 2.0, any major player in the software industry that hasn’t adopted the latest web architectures is being spit on as being plainly outdated or stuck with the label of being traditional.

When it comes to testing tools and Web 2.0, I think that “traditional” does not equate to obsolete – no matter how some of the “youngsters” in the testing tool market may like to imply. The software industry is competitive, certainly and I think new companies and software should just evangelize the positive innovations they have and then the facts can speak for themselves. If the ‘old guys’ can’t support new Web 2.0 stuff…then it will be obvious soon enough. For instance, if a new testing tool company doesn’t fully support AJAX clients it’s just unacceptable at this point.

However, I do believe it is fair game to evaluate existing software solutions (pre-Web 2.0) on how well they can be adapted to support newer innovations in technology. As for LoadRunner, I think we have a long history of adapting and embracing every new technology that has come along. I started using LoadRunner with X-Motif systems running on Solaris. That era and generation of technology is long since died (no offense intended to Motif or Sun). Today, the same concepts for record, replay , execution, scripting, and analysis are still innovative and very relevant. As long as the idea for the product is still valid, you can still deliver a valid product.

Adapting to changes here in LoadRunner we usually start with overcoming the technical hurdles for creating a new virtual user, or updating an existing one. And as I stated above, we have a long and rich history of doing this – probably more than any other testing tool. As an example, in versions 9.0, 9.1 and 9.5 we have continued to improve our support for AJAX, Flex and Asynchronous applications. We respond to change quite well and even if we take some extra time to evaluate every aspect of what this ‘new web’ change means to our customers. It’s worth getting right and not being swayed by the hype of the ‘Web 2.0’ label.

Let me finish by stating that these new web technologies as challenge to testing tools, but it’s even more of a change to testers. I’ve heard that many-a-tester gets a surprise by the next version of the AUT which secretively has implemented new Web 2.0 architecture or even started using web services calls to a SOA. Change is a surprise only if you’re unaware or unconscious. Sure, it would be a failure to not communicate to QA that there were some significant technology changes coming, right? To some, this would sound too familiar. Like an institutionalized version of “throw it over the wall” behavior, but honestly these new technologies (like AJAX) have been around for nearly 5 years now.

As for most testers, here’s a thanks to Web 2.0 – “You've left us up to our necks in it!”

Email Questions about Think Times

mark.tomlinson — Thu, 07 Aug 2008 03:06:00 GMT

From: Prasant
Sent: Monday, August 04, 2008 7:55 AM
To: Tomlinson, Mark
Subject: Some questions about LR

Hi Mark,

I am Prasant . I got your mail id from yahoo LR group. I have just started my career in Performance testing. I got a chance to work on LR . Currently I am working with LR 8.1. I have one doubt regarding think time. While recoding one script automatically think time got recorded in the script. While executing the script I am ignoring the think time. Is it required to ignore the think time or we have to consider the think time while executing the script.

I have questions in mind like, when think time is considerd as [time] the user is taking before giving input to the server . In that case while recording any script for a particular transaction I may take 50 seconds as think time and my friend who is recording the same script will take less than 50 seconds (let's say 20 seconds). So, in his script and in my script the think time will vary for same transaction. If I will execute both the scripts considering the think time the transaction response time will vary . It may create confusion for the result analysis. Can you please give some of your view points about this.

Thanks,
Prasant

**********************************************
From: Tomlinson, Mark
Sent: Thursday, August 07, 2008 2:59 AM
To: Prasant
Subject: RE: Some questions about LR

Hi Prasant,

Yes – it is good that think time gets recorded, so the script will be able to replay exactly like the real application – with delays for when you are messing around in the UI. But you must be careful, if you are recording your script and you get interrupted…or perhaps you go to the bathroom, or take a phone call…you will see VERY LONG THINK TIMES getting recorded. You should keep track of this, and then manually go edit those long delays – make them shorter in the script. Make them more realistic, like a real end user.

Also, as a general rule of thumb you should try *not* to include think time statements in between your start and end transactions. You are right that it will skew the response time measurements. But for longer business processes where you have a wrapper transaction around many statements…it might be impossible to clean every transaction.

Here are 3 other tips for you:

First – in the run time settings, you have options to limit or adjust the think time settings for replay…you can set a maximum limit, or multiply the amount. The combinations are very flexible. You can also choose to ignore think times and run a stress test, although I typically will include even 1 second iteration pacing for most stress tests I run.

Second – you can write some advanced functions in the script to randomize the think times programmatically. This could be used to dynamically adjust the think time from a parameter value, in the middle of the test.

Third – even if you do have think times inside your start and end transactions, there is an option in the Analysis tool to include or exclude the think time overhead in the measurements displayed in the Analysis graphs and summary.

I hope you’ll find that with those 3 tips, you can get all the flexibility you need to adjust think times in your scripts – try to make the most realistic load scenario you can.

Best wishes,
Mark

Flash compared to Silverlight

mark.tomlinson — Wed, 06 Aug 2008 16:36:00 GMT

Today I read Brandon's article which compares Adobe Flash and Microsoft Silverlight. He makes some excellent points about the strengths of Flash's market penetration compared to Silverlight's latest enhancements. For rich internet application, I think we still see Flash as the primary Ux platform out there…and it is a challenge for any testing tools to keep up with the fast pace of Adobe's innovations.

Brandon points out one of the main advantages that Silverlight has is the "Speed to Production" - getting the app delivered, out to production quickly. The advantage is better responsiveness and agility for the business. Unfortunately this usually equates to less time for proper testing, and especially performance testing.

It's also interesting how he points out performance comparison at the Ux layer - which I think could be described as the "perceived performance" for the entire application system. In Enterprise RIA or mashed-up application users might not perceive on-par performance from either Flash or Silverlight, depending on the backend systems. In a big RIA, you have multiple points of exposure to latency risk introduced in the data services calls behind the application - so even if the UI is responsive to the user, the data retrieval might be slow. Check out James Ward's blog on "Flex data loading benchmarks" - showing the combination of AMF and BlazeDS, which is proving to be a very scalable and responsive combination.

Welcome to Loadrunner at hp.com

mark.tomlinson — Wed, 06 Aug 2008 09:52:00 GMT

Greetings fellow LoadRunner Gurus! This is the introductory entry to the new LoadRunner @ hp.com blog, written by yours truly - Mark Tomlinson, the Product Manager for LoadRunner here at HP Software.

As you might be expecting a lengthy personal introduction for the first blog entry, I've decided to not deliver on that foolishly stereotypical initiation. Instead I'd like to start off here with a few opportunities to engage with you directly for the betterment of LoadRunner's future.

First, we are looking for a few good and challenging applications for LoadRunner to go after - as a our New Horizon research team are developing some very new exciting solutions for advanced client record and replay. If you got an application with any extreme architecture including:

Web v2.0 or Rich Internet Applications on Java, .NET or Adobe
Multi-multi-multi-protocol…like a mashed-up app with several backend systems
Encoded/Serialized or Secured communication protocols
Asynchronous, multi-threaded client(s) or data-push technologies
Any combination or all of the above.

If you have a challenge for LoadRunner, we'd love to hear from you.

Second, we have a new release of LoadRunner coming soon and we are just getting our plans for the early-access beta program put together. If you're an existing customer and you're interested in getting into our formal beta program for LoadRunner drop us an email. We have an early-access program that does require your feedback, production usage and reference for the new release. We'd love to have your support for all that - but I certainly understand some folks just want to share some feedback on the new stuff. We need that also, if that's all you can do.

Lastly, I'd love to hear from you - so drop me an email (loadrunner at hp . com). What do you love about the product, what do you not like so much? What kinds of testing are you doing? What new applications are you being asked to test? How do you get your testing done? How do you generate meaning from the load test results? What is your favorite BBQ restaurant? Let me know your thoughts and feedback - the good, the bad, the ugly. I have been using LoadRunner for nearly 15 years - so I plan include your input into our strategy for moving forward with innovating our solutions. I will post back weekly with some Q&A, if you'd like to share the conversation with our community.

Again - all of these initiatives are really important to the future of LoadRunner. Your participation is encouraged and greatly appreciated!

Thanks - and again, welcome to the blog!