4 Steps To Better SOA

In NASA's effort to integrate data from many different sources, the space agency wanted to take data collected during a feild expedition, including sample data, photos, voice recordings, and GPS information and integrate it with satellite imagery, GIS data, and information about the characterstics of the minerals found in the samples. Not only were the types of information heterogeneous, but the formats ranged widely from spreadsheet files to SQL databases to WebPages.

It's a common programming problem. Lets say, you have feild in two seperate databases, that are both named 'temp', is it legitimate to assume they represent the same quantity and can be integrated together? One might represent temperature and other temporary value. To properly combine these 2 fields, you really need to understand what the data represents.

At NASA and other large entities - both in government and business - integrating heterogeneous data is a challenge, but it's one that must be faced in order to easily share information internally and with outside partners. The integration challenge is one reason why NASA and many organizations are turning to SOA combined with semantic integration. As we know SOA consists of services that offer interoperability capabilities built into the network. While SOA's business-centric approach has sparked enthusism, the challenge is now to build in inference capabilities to make intelligent and dynamic selection of Web services possible. This is where semantic technology - the modelling of an area of knowledge like biology or economics as close to natural language as possible - comes into play.

Semantic technology
A SOA platform with built-in inference capabilities can let companies make runtime decisions based on service capabilites and invocation context where tasks are executed based in pre-defined conditions. On its own, SOA does'nt have this capability, but with the addition of inference, SOA gets the semantic richness it needs to make runtime decisions. Semantic integration techniques build context and awareness of the underlying services and data in a metalayer.

Ontology-Enabled SOA
Semantic integration techiniques are based on the idea that it's possible to find patterns in the origin of various pieces of heterogeneous data. Once done, then it's possible to define these patterns using a metamodel, and then connect several metamodels by defining how they relate to one another. One emerging approach to semantic integration is ontology. Ontology is a structured representation of a domain- by which we mean an area of knowledge like biology or economics - in terms of classes and interrelationships between the classes that can be defined using the XML-based Web Ontology Language.

There are 4 steps in building an ontology-enabled SOA:
Step 1 : Analyze the Business Process: In execution of business process, conditions may represent intelligent routing decisions requiring semantic power. We define these as "points of variability", or areas where inference capability is required. Future business needs may mandate additional variables or new values for existing variables. Identify them.

Step 2: Construct the Ontology Model: Ontology is defined in terms of concepts and relationships. Ontology concepts are implemeted in classes. Relationships are implemeted in terms of "object properties" and "data types properties".

Step 3: Create Context-Sensitive Invoker: Creating the runtime component that consults the ontology and makes dynamic context-sensitive service invocations in the next step. The business process flow can be encapsulated as a linear process in a BPEL, block.

Step 4: Web Services: provide the actual business service. They're the "consuming" point, invoked to acheive business functionality. The policy that operates on the Web Services is part of task required to incoporate a new service end point.

Final Word:
Ontology can be used to enable semantic integration by modeling the business domain process as a workflow, and constructing an ontology model along-side the business process modeling. The classes and relationships of the ontology model are created based on the domain knowledge and the business process.

Case study : Novell - "Zero Day Start"

A slightly old case study (2003)...however it shows the challenges the corporate world used to face then....and even now....and spends millions of dollars to get it sorted.....the case described by Art Vandenberg was similar to this one....based on a "authoritative" employee directory (in our case it was the PeopleSoft HR database) as the starting point for other employee attribute synchronization across the corporate....


Given above is how the old Novell IS&T infrastructure looked like prior to the one Net solution (a classic Systems Integration Solution Implementation). This was a nightmare and very costly to maintain the system. However, Novell is not alone. Many companies are suffering with even more complex infrastructure than Novell. The goal for CIO is to streamline and simplify this complexity.


Novell implemented the "Zero Day Start" infrastructure which resulted in dramatic changes across a variety of processes. The figure above shows the automated distribution of employee information between different directories and applications today. By using open standards, it simplified a very cumbersome process in the past and a more manageable infrastructure.

Wide adoption of important new standards are helping make one Net concepts such as Zero Day Start a reality. They include:
1. LDAP: the lightweight directory access protocol.
2. XML: for data Interchange between dissimilar applications and directories.
3. SSL: as a security standard,


The central pieces for Zero Day Start are using eDirectory and DirXML to automate the synchronization of information between the relative systems and provide access to services. eDirectory and DirXML connects the enterprise data systems, like PeopleSoft, to the corporate directory as shown above. This allows automatic synchronization and authorization between user information and disparate systems.

I am proud that I was a part of this implementation team. As typical to any project the initial hiccups was challenging.....but was very satisfying once the project went "live"....

Why do we integrate ?

To deliver better and faster customer service.....accurately....no hiccups in process handshakes...

How about having a business process with 99.999999 % right deliveries without any computers involved in the process at all.....??

And that too in Mumbai, possibly the most densely populated city in the world - 19 million...

The Mumbai Dabbawala (=Tiffin Box Carrier) is a unique service industry whose primary business is collecting the freshly cooked food in lunch boxes from the residences of the office workers (mostly in the suburbs), delivering it to their respective workplaces and returning back the empty boxes by using various modes of public transport. Watch this .... http://www.youtube.com/watch?v=sxW9sUnodM8&feature=related

According to Forbes Magazine..... Today the 5,000 Dabbawalas make about 200,000 lunch deliveries in the city and have become famous for their clockwork precision and efficiency. Reportedly their mistake rate is just 1 in 16 million deliveries, which caused the Forbes Global magazine to award its Six Sigma certification in 2001. According to Forbes the Dabbawalas work with 99.999999 percent accuracy.

Started in 1890, the monthly fee max is 11 dollar per month....Forbes magazine marks this to be one of the best business processes in the world....

They are so dedicated to their work that it is said....that Prince Charles had to adjust his schedule to have a meeting with them to talk to them....

No frills....

http://en.wikipedia.org/wiki/Dabbawala

You cannot think of Mumbai without them....I miss Mumbai.....

SAML in commercial use....

Hi,
I found this interesting link (given below)where Lockheed Martin (the people who make the F16) are hiring....SAML experts....hence I think SAML has taken the test of time and graduated to become a commercially viable solution.....not for academic institutions only....

http://www.careerbuilder.com/JobSeeker/Jobs/JobDetails.aspx?job_did=J3F6BS6KZ1L1XH3MPGQ

Qualifications:
Background in security and networking, familiar with protocols, knowledge of requirement process, good communications skills.
Demonstrated successful experience in a customer-facing role
Demonstrated successful experience in DMZ design
Expertise in PKI, SAML, XACML, WS-Secur*.

Lockheed Martin Aeronautics Company
Join in designing, building and supporting the world's most sophisticated military aircraft. Your involvement may include: The multiservice F-35 Joint Strike Fighter (JSF) * The proven multirole fighter, the F-16 *

Jim Ownby touched every point discussed in this class

The semester is quickly moving to its end, and only 2 presentations are left now. I was thinking to INTEGRATE our discussion topics and prepare a summary of the learnings of this class. But then we got this auspicious opportunity to hear our guest lecturer Mr. Jim Ownby. I observed an interesting thing in our guest lecturer's presentation, and it is that he almost touched upon every topic/aspect which we have discussed in class till date. I thought to revise you how?

1. Jim talked about the Integrated Lines of Business of BCD Travel which are "Traditional Travel Management", " Online Travel Management", "Traveler Security", "Crisis and Data management", "Incentives" and "Consulting Services". System Integration Team(Prachi, Andrew, Ben) talked about Integration Models and Styles of Integration (Data, Presentation & functional). We can relate it here that BCD had to integrate its business lines with all or some types of integration. CIS 8020 Achievers group (Asha, Jimmy and Rahul) had a presentation on ERP Systems on the same day, talking about how ERP systems are implemented to integrate these types of operating tenets and gave examples of Microsoft Dynamics and SAP. We can think that BCD Travel might also have implemented some similar kind of ERP system to integrate all the above stated business lines.

2. Then, Jim presented us with a slide where he showed us that BCD have travel partners like Sabre, Amadeus, Worldspan, Tripit etc. It is obvious that all these partners and BCD has to communicate (may be via sending messages). It reminds us of the third presentation by Mercury Group (Bill, Juhi, Gautam) where they discussed aboutMessaging Patterns and presentation by MegaHotPot (Mike, Vivian, Bing) onDatabase Oriented Middleware. If we assume that there might be message communication between BCD &its Travel Partners, certainly BCD Travel would have came across problem of integrating heterogeneous data. Its most probable that they might have thought of using Database Middle ware.

3. Jim then elaborated about BCD's IT platform selection and the technology portfolio they posses. Jim declared that all their development work is on Microsoft .NET development platform. Then they deploy SOA / Virtualized Services for solving System Integration problems. Mercury Group discussed about Service Oriented Architecture and presented some case studies. If you remember, they took example of Car Renting companies, and the same analogy applies here, where the challenge is to integrate systems of different organization. Here the scale is large and hence using SOA can safely be said "Appropriate" in this particular business situation.

BCD uses ETL (Extract, Transform, Load) tool for its data center management. Recall that Mike in his presenation told us about ETL tools (I remember he said some open source tools/ hand coded scripts could be executed to solve the purpose), inMegaHotSpot presentation on "Data Warehousing".

4. Our guest speaker discussed about the technical challenges which his team faces regularly. These challenges happens to be the challenges which are either discussed in every Tuesday class or are implications of System Integration.

•Reduced Maintenance Costs (TCO)

•Greater System Flexibility

•Reduced Time-to-Market

•Better interoperability

•Improved Client/Vendor accessibility/extensibility

•Improved Governance

5. On Gautam's question about "The technologies used by BCD's competitors", Jim expressed confidence that the top Travel Management Giants must be leveraging benefits of Web Services, which was the topic of Team Analyst (Ken, Veronica, David) in which they cleanly told us about how to create and publish Web Services.

6. On Sumantra's question about integrating a new partner into BCD's network, Jim threw some light on Process Integration which was our topic, "The Team CIS INTEGRATEM" where we presented our research on Process Modelling and Notations, and Team Analyst extending the discussion to BPEL (Business Process Execution Language). Jim told us that its their responsibility to align the interests and come up with integration solution in the event of adding a new company tgo their business partneship network.

7. On question from Vivek Menon about the role of "Cloud Computing", which happen to be Topic of Nucleus Group (Vivek, Sangeetha, Tarun) where they presented concepts, types and pros & cons of Cloud. Jim exposed the inherent problems in this emerging technology. He complained that using Cloud Computing is like sticking yourself to a proprietary technology, because you have to develop your application around the service provider's proprietary platform. Use there language specifics / data formats to insert data into the cloud. Once you enter your data into the cloud, and if you are not satisfied and want to migrate, you do not know how to pull your whole data out in a viable format. He audaciously named it "Cloud of uncertainty" and obviously raised question from the corporate side. I feel it was enlightening, and gave me a new vision of seeing this technology.

8. In the question and answer session, Jim briefly touched upon creating RESTful services and Mash ups. My team discussed about REST architecture and its notations and its relevance with Web and Internet. Mash up was talk topic of Silk Group (Jason, Max, Dmitriy), where they presented the fundamentals of mash-ups. They also discussed about technology behind mash-ups, and presented a great demo on yahoo pipes.

So, this is how Jim Ownby, Sr. Architect of BCD Travel "integrated" all the topics of CIS 8020 and made my job easy. He really gave a practical and industrial view of seeing these technologies. Prof. Zheng has done a commendable job by inviting such a knowledgeable person for talk. I appreciate it Professor. Now, I think this Blog has become very long, so I should stop.

Thank you. Your comments are highly solicited.

Articles on Systems Integration....a host of them....50 to be precise - I counted them......

Dr Zheng,
I think you had a post where you were looking out for a comprehensive book on Systems Integration....I think the challenge is that by the time a book is published we have 3 more technologies to be integrated.....and the book is outdated....

However, I found this link which has a summary of good articles on Systems Integration Course which looks useful....

http://www.enterpriseintegrationpatterns.com/bib.html

We covered most of the topics ...but there are some new ones too......:

1. Alexander
C. Alexander, S. Ishikawa, M Silverstein, A Pattern Language - Towns Buildings Construction, Oxford University Press, 1977, ISBN: 0195019199

2. Alpert
S. Alpert, K. Brown, B. Woolf, The Design Patterns Smalltalk Companion, Addison-Wesley, 1998, ISBN: 0201184621

3. Box
Don Box, Essential .NET, Volume 1: the Common Language Runtime, Addison-Wesley, 2002, ISBN: 0201734117

4. BPEL4WS
Business Process Execution Language for Web Services, Version 1.0, http://www.ibm.com/developerworks/webservices/library/ws-bpel

5. CSP
C. A. R. Hoare, Communicating Sequential Processes, Communications of the ACM , 1978, http://doi.acm.org/10.1145/359576.359585

6. CoreJ2EE
Alur, Crupi, Malks, Core J2EE Patterns: Best Practices and Design Strategies (2nd Edition), Prentice Hall PTR, 2003, ISBN: 0131422464

7. Dickman
Alan Dickman, Designing Applications with MSMQ, Addison-Wesley, 1998, ISBN: 0201325810

8. Douglass
Bruce Powel Douglass, Real-Time Design Patterns, Addison-Wesley, 2003, ISBN: 0201699567

9. EAA
Martin Fowler, Patterns of Enterprise Application Architecture, Addison-Wesley, 2003, ISBN: 0321127420

10. EJB20
Enterprise JavaBeans Specification, Version 2.0, http://java.sun.com/products/ejb/docs.html

11. Garlan
Mary Shaw, Garlan David, Software Architecture: Perspectives on an Emerging Discipline, Prentice Hall, 1996, ISBN: 0131829572

12. GoF
Erich Gamma, Richard Helm, Ralph Johnson, John Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software, Addison-Wesley, 1995, ISBN: 0201633612

13. Graham02
Steve Graham, Simon Simeonov, Toufic Boubez, Glen Daniels, Doug Davis, Yuichi Nakamura, Ryo Nyeama, Building Web Services with Java: Making Sense of XML, SOAP and UDDI, SAMS Publishing, 2002, ISBN: 0672321815

14. Hohmann
Luke Hohmann, Beyond Software Architecture: Creating and Sustaining Winning Solutions, Addison-Wesley, 2003, ISBN: 0201775948

15. Hapner
Mark Hapner, Rich Burridge, Rahul Sharma, Joseph Fialli, Kim Haase, Java Messaging Service API Tutorial and Reference, Addison-Wesley, 2002, ISBN: 0201784726

16. JMS
Java Message Service (JMS), Sun Microsystems, http://java.sun.com/products/jms/

17. JMS11
Java Message Service (the Sun Java Message Service (JMS) 1.1 Specification), Sun Microsystems, http://java.sun.com/products/jms/docs.html

18. JTA
Transactions and Java Technology, http://java.sun.com/j2ee/transactions.html

19. Kahn
G. Kahn, The Semantics of a Simple Language for Parallel Programming, Information Processing 74: Proc. IFIP Congress 74, North-Holland Publishing Co., 1974,

20. Kaye
Doug Kaye, Loosely Coupled: The Missing Pieces of Web Services, RDS Press, 2003, ISBN: 1881378241

21. Kent
William Kent, Data and Reality, 1stBooks, 2000, ISBN: 1585009709

22. Lewis
Rhys Lewis, Advanced Messaging Applications with MSMQ and MQSeries, Que, 2000, ISBN: 078972023X

23. Leyman
Frank Leyman, et al., Production Workflow: Concepts and Techniques, Prentice-Hall PTR, 1999, ISBN: 0130217530

24. MDMSG
Multiple-Destination Messaging, http://msdn.microsoft.com/library/

25. Monson-Haefel
Richard Monson-Haefel and David A. Chappell, Java Message Service, O’Reilly, 2001, ISBN: 0596000685

26. MicroWorkflow
Dragos Manolescu, Micro-Workflow: A Workflow Architecture Supporting Compositional Object-Oriented Software Development, http://micro-workflow.com/PhDThesis/phdthesis.pdf

27. Monroe
Robert T. Monroe, Drew Kompanek, Ralph Melton, David Garlan, Stylized Architecture, Design Patterns, and Objects, 1996, http://www-2.cs.cmu.edu/afs/cs/project/compose/ftp/pdf/ObjPatternsArch-ieee97.pdf

28. MQSeries
MQSeries from IBM, http://www.ibm.com/software/mqseries

29. MSMQ01
Microsoft Message Queuing (MSMQ), http://www.microsoft.com/msmq

30. POSA
Buschmann et al, Pattern-Oriented Software Architecture, Wiley, 1996, ISBN: 0471958697

31. POSA2
Schmidt et al, Pattern-Oriented Software Architecture Vol.2, Wiley, 2000, ISBN: 0471606952

32. PatternForms
Pattern Forms, Wiki-Wiki-Web, http://c2.com/cgi-bin/wiki?PatternForms

33. PLOPD1
James Coplien, Douglas Schmidt (Editor), Pattern Languages of Program Design, Addision-Wesley, 1995, ISBN: 0201607344

34. Sharp
Alec Sharp, Patrick McDermott , Workflow Modeling: Tools for Process Improvement and Application Development, Artech House, 2001, ISBN: 1580530214

35. SOAP 1.1
W3C Simple Object Access Protocol (SOAP) 1.1 Specification, http://www.w3.org/TR/SOAP/

36. SOAP 1.2 Part 2
SOAP Version 1.2 Part 2: Adjuncts, http://www.w3.org/TR/soap12-part2/

37. Stevens
W. Richard Stevens, TCP/IP Illustrated, Volume 1: The Protocols, Addison-Wesley, 1994, ISBN: 0201633469

38. SysMsg
System.Messaging Namespace in the .NET Framework SDK, Microsoft, http://msdn.microsoft.com/library/

39. Tennison
Jeni Tennison, XSLT and XPath On The Edge, John Wiley & Sons, 2001, ISBN: 0764547763

40. UML
UML Distilled: A Brief Guide to the Standard Object Modeling Language, Third Edition, Addison-Wesley, 2003, ISBN: 0321193687

41. UMLEAI
UML Profile for Enterprise Application Integration, Object Management Group , 2002, http://cgi.omg.org/cgi-bin/doc?ptc/02-02-02

42. Wright
Gary R. Wright, W. Richard Stevens, TCP/IP Illustrated, Volume 2: The Implementation, Addison-Wesley, 1995, ISBN: 020163354X

43. WSAUS
Web Services Architecture Usage Scenarios, W3C, http://www.w3.org/TR/ws-arch-scenarios/

44. WSDL 1.1
Web Services Description Language (WSDL) 1.1, http://www.w3.org/TR/wsdl

45. WSFL
Web Services Flow Language, IBM, http://www-3.ibm.com/software/solutions/webservices/pdf/WSFL.pdf

46. WSMQ
WebSphere MQ Using Java, IBM, http://publibfp.boulder.ibm.com/epubs/pdf/csqzaw11.pdf

47. XML 1.0
Extensible Markup Language (XML) 1.0 (Second Edition), http://www.w3.org/TR/REC-xml

48. XSLT 1.0
XSL Transformations (XSLT) Version 1.0 -- W3C Recommendation, http://www.w3.org/TR/xslt

49. Waldo
Waldo, J., Wyant, G., Wollrath, A., and Kendall, S., A Note on Distributed Computing, Technical Report SMLI TR-94-29, Sun Microsystems Laboratories, November 1994, http://citeseer.nj.nec.com/waldo94note.html

50. Zahavi
Ron Zahavi, Enterprise Application Integration with CORBA , John Wiley & Sons, 1999, ISBN: 0471327204

CIS 8020 Assignment 2 [PS] [Google Health]

Assume a company provides health care services/health insurance to its employees as incentives(perks) in-house, i.e the company has its own team of HealthCare Professionals and covers health care of its employees. Company must consider using the Google Health API to keep Electronic Medical Record (EMR) of employees.

Company may want to conduct regular health check-ups of employees and would like keep records of the inspection & diagnosis results to analyze their physical and mental health. If company is multi-national, it would like to have all these records in one common format, which is possible with Google Health.

Google Health is a web service so you are able to use any language that can perform the four HTTP methods (GET/POST/PUT/DELTE). Google Health leverages the Google Data APIs, which have client libraries in Java, .NET, PHP, Python, Objective-C, and JavaScript. At this time, only the Java and .NET libraries support Google Health. In Google Health, you can add medical information to a user's profile by posting notices containing CCR data. The API supports a subset of the CCR ("Continuity of Care Record"), a standard format for transferring snapshots of a patient's medical history. Notices are messages that can contain certain XHTML elements, CCR data, or both. Notices might be sent to remind users to pick up a prescription, or they might contain lab results. The Google Health Data API uses the same underlying protocol as the other Google Data APIs. Google Data is the preferred way for applications to interface programmatically with the Google Health platform. Google Data eases application integration by encapsulating and hiding the AtomPub specification and implementation. In the application, doctors with right permissions, read user's profile or query for medical health records that match particular criteria. Company may use the results to provide personalized functionality based on the data. Here is an example of how to code this application. The example shows how to code the application in .NET, how to design Health Profiles (signed & unsigned) and how to build it.



Google Health API provides flexibility to use number of programming languages (such as Java, .NET etc). The biggest advantage is that Google Health can be integrated seamlessly into an existing application. In this case, company could leverage the possibility of integrating the Google Health application into its Finance Module of ERP implementation to keep record of the expenses on this venture.

Google Health API would be an appropriate and effective solution in this case.