Operationalizing SOA

If Al Gore can win a Nobel Peace Prize, can HP and SOA bring peace to IT?

2008-07-16T20:40:00Z

In case you missed the news from last October (2007), former U.S. Vice President Al Gore was a co-recipient of the Nobel Peace Prize for his work to raise awareness of global warming. Now, Al Gore took some good natured ribbing a few years ago for his misstatement about being the guy who “invented” the Internet. But, somewhere in the dark corners of my brain I started thinking about the peace prize, the Web, enterprise IT and the adoption of SOA. I realize this seems a bit disconnected. Let me explain.

If you’ve ever worked in an IT organization with more than 1 employee, you know that people don’t always agree on things like:

· How to problem solve (this include identifying the problems and then fixing them)?

· What and where to invest (either from a strategic perspective or in terms of problem mitigation)?

· What kinds of technologies should be used (this includes everything from operating system choices to programming languages and tools)?

Even if everyone agreed on these things within the organization, vendors continue to deliver differented products and new approaches to solving problems which then creates new things to argue about. Even if consensus is restored, different choices are being made across corporate boundaries and then mergers and acquisitions take place throwing the whole organization into disarray again. The introduction of SOA and the underlying sets of implementation technologies was an attempt to see if there are some common principles, patterns, and specifications upon which the various stakeholders could agree to which would allow for the dust to settle a bit faster every time some kind of change occurred.

We are at War

- Enterprise Architects and Operations people typically don’t like each other

- Why?

o Differing approaches to the same problems

§ EAs – want to work on new technologies to solve existing problems

§ Operations – demands stability in the environment so that things can be automated

§ These approaches are at odds with one another.

One glimmer of hope in the War

While the SOA adoption trend has largely been led by Enteprise Architects, the Operations folks have been leading their own trend; ITIL adoption. ITIL v3 is largely service-centric and while the service catalog established as part of ITIL adoption is a superset of all services that IT offers (some of which may be based on SOA-based services or processes orchestrated via SOA), the motivation, language and underlying approaches that each of these groups are now adopting might have them spending a little more time at the water cooler together...

Can HP end the war by helping align the approaches through SOA and ITIL?

Isn’t this worthy of an IT Peace Prize? Or perhaps an "IT Peace Prize" is something that HP should sponsor?

Mixing Concept and Implementation

2008-07-12T00:45:00Z

Techies are hearing about a concept and then running out and attempting to build something that represents that concept. Seems reasonable, right? Well, the issue arises when they explain what they have done to someone else and they say, "I've built that concept." Too abstract....

Try this SOA = Web services (and the set of standards that underpin them: WSDL, UDDI, SOAP, etc.). Is the picture getting clearer?

Service-oriented architecture is an architectural pattern that cannot be packaged, sold or delivered in a product. It is as much a philosophy about the way in which next generation business processes will be designed, described, tested, evolved and deployed as it about the underlying technologies that support it. The adoption of SOA as a means for delivering business processes made up of loosely-coupled integrations which are supposed to be rapid to deploy, more flexible to change, and ultimately delivering more with less from IT has some significant implications both for people delivering software-based solutions as it does the company adopting SOA as an approach.

As the pace of technological complexity grows within IT, there are a number of compounding factors that are weighing heavily upon today's CIOs and with good reason their tenures are getting shorter and shorter:

the business is demanding more and giving IT less to work with.
the rapid pace of innovation is continuing and there are so many different technologies still in use that just keeping the lights on (KTLO) represents a significant part of the IT budget; IT is not retiring legacy technologies fast enough.
IT must evolve out of this problem by using their limited innovation spend to invest in new, more flexible approaches to deliver what the business wants AND, at the same time, retire (or at least start the retirement of) enough legacy technologies to (at a minimum) not raise the KTLO spend.

What SOA proposes from a pattern or philosophical approach is the following: IT must transform from a resource-centric function to a service-centric line of business, so that it becomes a service provider (and only a service provider, nothing else). In so doing, IT can transform itself, its budgets, its way of doing business and better supporting the organizations they work within.

In terms of organizational impact, there are four things that are critical for companies which adopt SOA to understand in order to optimize their outcomes with SOA. If they do not understand the following, they are simply not going to realize the benefits of service-orientation (we are starting to hear about some of these companies already and we will hear more in 2009):

Organize IT deliverables around the notion of a service as the economic unit of measure for supply, demand, quality and value
Fund IT around a structure and operating model that supports the business in the context of their structure and operating models as Service Providers (enterprise architecture for IT)
Apply automation to IT in the context of the enterprise architecture for IT, and exploit the resulting information to drive business direction
Initiate transformation methodology to institute service-orientation addressing SOA Governance, Quality, and Management

Now, I haven't mentioned web services, CORBA, JMS, DCOM, WCF, HTTP, REST, SOAP, WSDL, UDDI, IIOP or any number of other implementation, middleware, or standards that can be used in some way shape or form to deliver upon all of this. But, what I have outlined (I hope), says something significant about what our customers are faced with and, subsequently, the solutions that HP Software MUST provide if we are going to continue to be relevant and successful as a software company.

In the short run, we delivering solutions focused on SOA Governance, SOA Quality and SOA Management. That being said, these solutions are pretty flexible and can be used to support a number of SOA-styles that we are seeing customers implement; everything from WS-* to REST, to XML over JMS, and more. One thing I will share with you is that we see A LOT of customers approaching their existing Application Modernization projects (in an attempt to retire CORBA-based solutions or mainframe deployments, monolithic applications, and even client/server apps) by taking a service-centric approach and looking for ways to catalog existing application artifacts (assets like Copybooks, IDL interfaces, etc.) using our SOA Governance solution.

In the longer run, you should see the connection being made to the financial decisions through integration of our existing solutions to the Portfolio and Project Management solutions. You should see the alignment between Operations personnel and Enterprise Architects at our customer sites when the Ops folks adopt ITIL v3 and Enterprise Architects adopt SOA. HP can facilitate this via integrations between our SOA-based solutions with our ITSM solutions. Finally, the more structured information is being captured which describes the applications, processes, and services the more HP Software should be able to do in terms of driving automated deployment of these solutions. The earlier in the lifecycle that we capture this "model" of what is being created (i.e. through SOA Governance), HP Software should be able to automate the deployment of the appropriate parts in a test lab as well as correctly instantiate a working system in the production environment including automated population of Configuration Management Databases (CMDBs) [btw, this is another example of a concept and implementation issue].

So, in terms of solutions that we may be building for our customers, SOA is a critical trend that essentially is behind the scenes on everything from Cloud computing, to virtualization, Web 2.0 (specifically enterprise mashups), and Software-as-a-Service). Oracle and SAP have bet their companies on delivering integrated suites of their applications based on this approach. Yes, there are a lot of technologies out there that are focused on integration (I actually liked CORBA a lot!), but what are we doing to HELP customers understand what I've written above, what our solutions can (and cannot/will not) do, and how we can help them address the problems that we face?

This is the collective challenge of our industry.

Couldn't stop thinking about yesterday's email: Web Services does not a SOA make...

2008-07-01T17:14:00Z

Yesterday, I received an email from a company doing a survey that stated the following:

“We are now collecting input for our upcomingWeb Sevices Development Survey. The deadline to provide your information is Friday July 11th, 2008.

The Web Services Survey examines the usage and future expectations of developers working with and/or creating Web Services applications. Also known as Services Oriented Architecture, many engineering professionals think that Web Services is the computing platform of the future and will revolutionize the way applications interact.”

I had to stop and think, did I just read that?.......

It seems there is still a lot of learning needed to realize that Service Oriented Architecture does not equal Web Services. IMO this thinking can lead an IT organization into a great deal of trouble as it fails to take into account the “A” of SOA. SOA is about planning and building out an architectural foundation for agility and re-use.

Embracing a SOA approach can have a major payoff in IT agility but it requires planning and creating a foundation for service visibility (how can a distributed organization find services they need instead of creating service duplication?), trust (services are single points of value or failure and must be trustworthy) and control (SOA introduces new dependencies, faster rate of change and more requirements on operations).

Finally, SOA does not dictate a particular technology. Anything exposed as a consume-able, trusted, shared service can be considered part of SOA: Web Services and REST services and EJB services and….you get my point.

Equating Web Services to SOA is like equating aluminum siding to a house, it can be a useful component but it won’t put a roof over your head.

Recent SOA best practice gems from HP SOA Customers – or more than lost wages comes from attending HP Software Universe in Las Vegas

2008-06-30T21:34:00Z

I drew the lucky straw to be a track manager at HP Software Universe. This meant I was tagged to organize all the sessions at HP Software Universe for our SOA track . The positive side of the administrivia was that in the process, I had the opportunity to review and hear some great presentations from our SOA customers who are making the journey towards effective SOA transformation. Here are some of the quips and gems that stood out for me: 1. One customer mentioned how for their organization, SOA is a tool within an Enterprise Architecture: The key message here is that you never do SOA for “SOA’s sake”. It’s a tool to help you achieve an architectural objective: agility, re-use, efficiency.

If you want to dig into this topic more, there is some interesting writing about SOA and EA here

2. One needs clear SOA transformation metrics to measure success: What is your SOA measured against? For one key customer, the clear metrics are 1) standards conformance and 2) business agility. Business agility is defined as the ability to see new business intentions, realize business changes and make improvements quickly.

To hear more about measureable benefits of SOA check out Tim Hall’s podcast

3. Don’t be surprised that one of the largest adoption hurdles is that SOA is fraught with “charge back challenges” -- In order for a group to adopt SOA it must be cheaper for the potential service consumer than to do it themselves. (This should be obvious) 4. How to manage versions effectively in a SOA? A best practice is to establish a time-based contract between service consumer and provider – allows the service provider to get consumers onto new versions, otherwise they will never move. 5. My favorite -- How to entice project teams to adopt SOA? Show them how SOA will grow their kingdom and make them a bigger king – appeal to ego, create a buzz – the more groups that use the service, the more successful the service provider, or the bigger the "kingdom". 6. Sometimes silos are not a dirty word. One customer started building out SOA in silos and is now connecting them when the organization has matured enough to handle it – the goal, build initial focused success, gain people buy-in, then expand scope 7. And since there are two sides to every story, that said, some silos must be broken from the get-go. For example, to have success in attaining SOA quality testing that works—you must have collaboration between development and QA – services change continuously during design, requirements definition, development and testing and dev and QA must have a dynamic feedback loop to get a positive result. 8. Finally a good quote ”you need SOA governance but it can’t be bureaucratic” Governance should make SOA easier not put up roadblocks. As the cliché goes..for many of these customers the carrot is proving more effective than the stick.

GIF and you shall receive

2008-03-18T20:05:00Z

The one thing that I find consistent in any customer IT environment working with SOA is heterogeneity. Whether it pertains to runtime platforms, intermediaries – ESBs and appliances, development tooling or back-end applications, there are almost always more than one.

SOA Governance offers the promise of visibility and control but only if it can work with and enforce polices across heterogeneous infrastructure. To that end, the industry needs interoperability efforts with teeth. Not just press release-only relationships (otherwise known as Barney relationships) but real collaboration where vendors work together to achieve integration and interoperability based on standards that can be shared and replicated across a variety of infrastructure.

HP is driving one such effort called the Governance Interoperability Framework. We invite all industry vendors working on pieces of the SOA puzzle who want to participate in this domain called governance to take a look at the GIF spec and participate in the GIF community.

Just don’t take my word for it, check out the recent blog by industry expert Anne Thomas Manes at http://apsblog.burtongroup.com/2008/03/hpsystinet-fina.html

You can access the GIF spec at HP.com at www.hp.com/go/soa

and the Wikipedia posting here: http://en.wikipedia.org/wiki/Governance_Interoperability_Framework

Quick thoughts – SOA, Web 2.0 and Wall Street – a legitimate mashup?

2008-02-15T16:47:00Z

This week, I was sloshing through the sleet and snow of New York City (still a fabulous city no matter what the weather) to attend and present on a panel at SOA on Wall Street (also so named Web Services on Wall Street). I participated on the opening panel which took a look at Web 2.0 and SOA and its implications and relevance for Wall Street organizations.

In a nutshell, Wall Street needs SOA and situations like Societe Generale are making it more urgent. I also believe Wall Street can benefit from the “wild and wooly” world of Web 2.0. Perhaps not for their mission critical infrastructure that requires the highest levels of performance, operational integrity and security, but definitely for collaboration, encouraging innovation between department and teams and non-critical information syndication. However, I disagreed with one of the panelist members who made a statement that SOA capacity is hardly realized so we should not worry about the governance or operational concerns and focus our energy on fostering innovation. I believe the time is now for all organizations, financial services and otherwise, to plan for and work with SOA governance, as well as SOA quality and management, to build out the infrastructure and best practices, so that when they find that “killer composite application” or that viral Web 2.0 service, they are prepared for the onslaught of new consumers, the load and performance implications, the version management challenges and can support innovation at the speed of business with the rigor of operational IT.

For the last two years, it seems organizations have focused around the need to modernize the middleware infrastructure to support SOA. Firms have been investing in ESB, BPM and data services and building out integration, data and some business-level services for use in composite applications. Now banking and FSI is primed for the embrace of Business-Technology Optimization for SOA -- governance as well as quality and management. These companies are going to run into all the visibility and trust issues the industry analysts discuss about SOA Governance by just focusing their architecture, development and delivery teams on the ESB and integration layer without thinking about how they will foster service visibility, consistency, policy management and service consumer management.

That said, I think the Wall street front office is probably the most challenged in embracing SOA right now as the focus has been on performance, data integrity, security, operational resilience and performance (did I already say that?). But that doesn't mean a Wall Street firm is not ready for SOA. During the conference we discussed a lot about the need for SOA in the middle office to help with innovation, research, new customer offerings, and to avoid situations like Societe Generale through visibility, trust and control.

It doesn’t matter where SOA is happening (front, back or middle) it will need governance.

For companies that don't have appetite yet for full-scale governance, something like HP’s newly announced Governance Validation service is a great entrée… Think big and start small -- take a domain or single project and implement a focused governance implementation for a contained set of services and then scale when ready. With this approach you can immediately see measure able benefits without a large initial cost (assuming you have a clear organizational owner and roles defined for governance – worth another blog)

There is a place for SOA and Web 2.0 on Wall Street – in the middle office or anywhere where agility and collaboration are paramount. To hear more about the insights shared at the event, see the event page at: http://www.webservicesonwallstreet.com.

Emo's SOA World -- All I really need to know about SOA, I learned in Kindergarten

2008-01-11T20:23:00Z

[Source: "ALL I REALLY NEED TO KNOW I LEARNED IN KINDERGARTEN" by Robert Fulghum. See his web site at http://www.robertfulghum.com/ ]

Metaphors are magic. IMO, they help people grok (wikipedia: http://en.wikipedia.org/wiki/Grok) complex subjects and make them understandable. This year, I’m experiencing a right-of-passage as my youngest has entered Kindergarten. That being top of mind, it brought back to me the well-known prose from Robert Fulghum’s “All I Really Need to Know I Learned in Kindergarten”. He made the statement about the message in his book:, “Take any one of those items, (from his list of everything I really need to know I learned in Kindergarten) and extrapolate it into sophisticated adult terms and apply it to your family life or your work or government or your world and it holds true and clear and firm”.

I thought to myself how would this concept apply to the domain of SOA Best Practices? Here’s my attempt to apply the metaphor -- “All I’ll need to know about how to successfully implement SOA is there to be learned in kindergarten. SOA Success is not at the top of the Enterprise Architecture Mountain, but there in the sand pile at school”.

These are the things I learned:

Share everything – I’ll take a slight derivative of this, “learn to share”. This statement doesn’t mean re-use everything but it does imply that you need to ensure the infrastructure, processes and corporate culture is built to encourage sharing, not penalize those for it. This maps to adopting SOA processes and technologies that deliver visibility, trust, a culture of shared benefit and shared costs and favorable cost models (avoiding “not in my budget”)
Play fair—Allocate costs accordingly, put in measures to encourage good SOA practices (reporting and dashboards help) and understand the implications of change and how people and culture must change as well to accommodate SOA.
Don't hit people—Corollary to the above, do not penalize the first SOA project for being over budget. SOA will initially cost more but then the rewards come with subsequent projects when the benefits of agility, trust and potentially re-use kick in. This first project should not be penalized for investing in “good, clean SOA”
Put things back where you found them – This is a biggie. SOA governance is critical here. We need to avoid rogue services. Services and their associated artifacts should be effectively catalogued and changes should be managed and reflected as such. If you “move” a service (such as redeploy it to a more scalable platform), update the catalog so consumers can find the change. This simple rule will avoid SOA chaos and web sprawl.
Clean up your own mess—Key to SOA success is measurement, reporting and accountability through a service lifecycle. A successful SOA includes having the capabilities to visualize the SOA in action, proactively monitor and measure service interactions and performance and feed that information back into the SOA catalog or system of record so that changes can be made to “clean up any messes” and ensure SOA health, stability and availability on an on-going basis.
Don't take things that aren't yours—This comment may first seem to contradict the concept of re-use but I looked at it a different way. A successful SOA is a Governed SOA. Governance allows services to go through a process of understanding how they should be re-used, and who has authorization to re-use services. A successful SOA supports this concept and enables policies to ensure that services are accessed and used in ways that support the business objectives and not endangered by rogue consumption.
Say you're sorry when you hurt somebody—I interpret this one from a SOA perspective as building into the SOA process the ability to apply governance at all stages of the SOA lifecycle and to be able to proactively manage when services fall out of compliance. You cannot determine whether a SOA action is helpful or harmful without comprehensive governance across the lifecycle. .
When you go out in the world, watch out for traffic, hold hands and stick together—And when you go out into the SOA world, set yourself up for success. Ensure that core to your SOA effort is management and visibility (watch for traffic), Ensure that your processes and organization are designed to “hold SOA hands”, in other words, encourage the creation of shared services and all the implications of that (shared costs, upfront complexity for downstream benefit, participation in the governance process). And finally, align business and IT across the SOA goals (stick together) and this is worth another blog…. .

Successful SOA is more than just creating and integrating standards-based services. It requires organizational and process changes to support sharing of code, costs, and information. It requires effective governance to ensure that services are compliant to corporate and IT objectives, services are visible for re-use and services can be trusted. It requires management to ensure that the in-production SOA results map to customer SLA and SLO expectations regardless of how the landscape changes, and it requires alignment of the business and IT towards common goals and expectations so that the SOA effort pays off in business terms now and in the future to become “not just another IT experiment” but truly an element of strategic business transformation. More on that in another “Emo’s SOA World”.

Kelly Emo's World on SOA

2008-01-11T20:16:00Z

Welcome to Emo’s World on SOA. I’ve been on a professional journey for the last eight years with SOA, and some would argue even before that time while working on technologies such as CORBA and DCE (does anyone remember the Distributed Computing Environment?) .

Through the years, I’ve been getting my hands dirty with SOA from the chaotic “new economy” days of early 2000 while attempting to build innovative ASP (another legacy buzzword—Application Service Provider) solutions through harnessing Web Services (can we say point-to-point?), to attempting to creating a more agile IT infrastructure for SOA by leveraging the concept of an intermediary, getting on the “bus” to implement SOA if you will. During the last few years, I’ve seen first-hand the short-term benefits of investing in infrastructure but also the sticky longer-term challenges – where projects get stalled or fail due to a singular focus on the foundation without the planning and investing in the governance processes, operational management and other key success factors to realize a true working SOA that delivers rewards to the business.

After working with a wide array of customers and partners in this space over the last several years, I plan to share real-world examples and anecdotes of best practices for taking the next steps to SOA success, not just through infrastructure but true SOA deployment and operations with tangible value.

Through my blog, I’ll be taking you on a journey to explore and debate what capabilities and best practices are needed to realize measurable business value from SOA efforts while leveraging investments in SOA enablement such as integration suites, application servers and ESBs. I also plan to tackle some controversial subjects such as “an ESB does not a SOA make” or “When is an org not ready for SOA?” Come along, it will be an interesting ride…

Title: Accelerating SOA Success through Domain Models

2007-12-03T20:58:00Z

Content:

Our user conference (HP Software Universe) was last week in Barcelona.

There were many discussions of SOA, as you would expect. Pawel Maszczyk presented his experience with SOA. The Carphone Warehouse, who has grown 500% over the past five years to have approximately £4B in revenue, has been deploying SOA over the past two and a half years. The business believes that SOA has been a successful initiative because they have seen two business results: a dramatic increase in IT’s ability to respond rapidly to change; and a decrease in cost for IT projects.

One thing that struck me about Pawel’s presentation is that he believes successful SOA adoption can be accelerated significantly. Carphone Warehouse has built their SOA essentially on their own – no extensive involvement of any consulting company or vendor. As they started building and deploying SOA services and composite applications, they addressed issues as they arose. They have now organized all the issues into a seven-part domain model, including projects, building blocks, architecture, business processes, organization, governance, and financial model.

By taking each of the first six domains into consideration at the beginning of a SOA initiative, companies should be able to reduce the challenges that occur during service and application development.

Carphone Warehouse built their domain model as they went through the process of deploying their architecture, essentially putting the model together as they went. HP’s Consulting and Integration practice has an eight-part domain model (business, people, program management, governance, architecture, enabling technologies, operations and management, and supply & demand); the high degree of overlap with Carphone Warehouse is a good thing.

What domain model do you use? How many parts are there?

What are the practical challenges of reusing services?

2007-11-09T17:11:00Z

What are the practical challenges of reusing services? Well, if you’ve tried to deploy SOA, the following questions might sound familiar.

To reuse a service, you need to know it exists. So, how do you even find a service? Then, how can you know whether it's designed to do what you need? And the big one: how can you trust that the service will deliver what's promised? Since you’re the one on the line to deliver your application, it’s hard to trust someone that you’ve never met.

Once the service is in production, how can you be sure that it is available and performing when you need it? Since it isn’t under your control, isn’t it difficult to commit to using it?

And that’s just the start. Certainly you need to be sure that whenever it gets changed, you are aware of the change when you need to be. And it must continue to work when there is a totally internal change. How do you get an enhancement request in to the provider? How do you report a bug? How do you prioritize a new service to be created?

Now, if you’re the person responsible for SOA overall at your org, don’t forget the people portion of the equation. How do you motivate people to reuse services rather than building new ones? Is there a best way to train the different players involved in SOA? Who’s responsible for saying that a service is ready to go into production? What organization structures should you use?

Our customers have told us that these are the challenges that they face as soon as they get a few services under their belts – as soon as they get some services that they really care about. And once they begin to address these challenges, they really start seeing the business benefits that SOA is supposed to provide.

And that is what it all comes down to – getting the business results. SOA isn’t meant to be technology for its own sake. It really can make a difference to the company’s bottom line – and to your paycheck. But only if you address these challenges – getting SOA to be successful in the ongoing operations of your IT. And that’s why we call this Operationalizing SOA.

The rules of the blog

2007-11-09T17:10:00Z

The rules of the blog are simple and pretty standard. Please adhere to these policies.
The purpose of the blog is to share viewpoints and opinions on how to move SOA into the main operational processes of IT shops all over the world. We encourage all reasonable points of view and all posts that are positively trying to improve the chance for SOA success. It’s not meant to be a sales or marketing channel – it’s meant to be a communication device for SOA professionals.
No belittling other points of view. No rudeness. No flaming.

Welcome!

2007-11-09T17:09:00Z

This is a special blog that combines the input from many of the SOA product and services experts at HP Software. Our team here is dedicated to helping you get the most out of SOA. We concentrate on a specific set of challenges that SOA presents.

We think that the most difficult parts of SOA are figuring out approaches to the practical challenges of how to re-use services. If we’re ever going to be successful with SOA, we need to be good enough at it that it’s a normal way of doing business. Sounds easy, but the challenges are everywhere, and they run deep.

So, let’s hear your comments on our points of view as they come up. Get engaged and tell us when we’re wrong or when our experiences don’t match yours. Or refer us to other posts that might eloquently illustrate a point of view. Working together, we really can get SOA to be the new IT architecture.