Siebel moves to the SOA world

Siebel Web Services (SOA enabled) provide platform independent integration and rely on Internet development standards and protocols (XML, SOAP, WSDL etc) for communication. There are two type of Siebel Web Service known as Inbound Web Service and Outbound Web Service and Siebel provides host of pre-built ASI (Application Service Interface) which can be used for two way integration between external and Siebel Applications. Inbound Web Services are used to expose existing Siebel Business Service and Workflows to Web Service which will be consumed by external system. Outbound Web Service relies on a WSDL file provided by the external system to define the external service definitions to enable Siebel to invoke the Web Service.

An existing Siebel business service or workflow process can be exposed as an Inbound Web Service to be consumed by an external application. Siebel Outbound Web Services framework can also consume a Web Service Definition Language (WSDL) file provided by an external application and have the Siebel application invoke that Web Service. The diagram below depicts Siebel-External system integration using Siebel EAI/Web Service.

The OracleAS Adapter for Siebel uses the same Siebel Web Services Framework (introduced in Siebel 7.5).

Options
• ORACLE offered pre-built (release independent) ASI to functions such as Account and Opportunity
• Custom built ASI (release dependent) from the Siebel exposed business services
• ORACLE AS adapter
• Exposing Pl/SQL stored procedures as Web Services
• The iWay Adapter for to integrate Siebel application system with any other legacy system, DBMS, ERP, SCM, or CRM application on any platform
• Oracle Fusion middleware which support SOA and BPM for platform independent integration.

Methods
• ASI Exposes Siebel business services and workflow processes as Web Services which can be consumed by external system for further processing.
• Inbound ASI: The Siebel application allows enterprises to publish any Siebel business service or business process as a Web Service. This process is also known as creating an Inbound Web Service for external application invoking Siebel exposed ASI as a web service (ORACLE financials synching ORACLE Order status to Seibel Sales Order).
• Outbound ASI: Siebel apps invoking external web service. (Siebel sales order applet invoking ORACLE ERP to query quantity available.)

System Requirements for ASIs
• Siebel server environment, with the Siebel Web Server Extension (SWSE) set up.
• Application server
• Siebel Enterprise Application Integration (EAI) component group must be enabled.
• Access to the Web Services Administration view for this application, like using Siebel Client.

Inbound Web Service Strategy: The published Siebel Web Service will be consumed by External. The above diagram illustrates steps involved:
• The WSDL document for the active Siebel Web Service will be published as Inbound Web Service.
• WDSL will be generated for Siebel business service/Workflow and will be published on net. The published WSDL will be consumed by external system for further processing.
• The external system will send SOAP request to Siebel Application.
• The Siebel Inbound Web Service dispatcher will convert SOAP request into property sets and pass it to Siebel Business Service or Workflow for further processing.
• The property sets will be returned from business service or business process to the Siebel Web Service inbound dispatcher.
• The response will be converted into SOAP message and sent to the external system Web Service as a response.

Siebel Outbound web service integration strategy: External systems can publish Web Services/SOA WSDL file which will be consumed by Siebel. The above diagram illustrates steps involved:
• The WSDL document for the active Web Service will be published as outbound Web Service.
• The generated WDSL from outer system will be consumed by Siebel Outbound Web Services for further processing. Siebel Tools will be used to mapping and processing of external request.
• The Siebel will send SOAP request to external system.
• The Siebel Outbound Web Service dispatcher will convert SOAP response into XML and pass it to external system for further processing.
• The property sets will be returned from business service or business process to the Siebel Web Service outbound dispatcher.
• The response/request will be converted into SOAP message and sent to the foreign system Web Service as a response/request.

OracleAS: The OracleAS Adapter for Siebel supports the following integration access methods:
• Siebel Java Data Bean for services involving Siebel Business Components or Siebel Business Services. • Siebel COM Data Interface for services involving Siebel Business Components or Siebel Business Services.
• Siebel XML for events and services involving Siebel Integration Objects.

Actuate iServer exposes its functions as Web Services; which can be accessed by Actuate Web Services API or through WSDL published by Actuate.
PL / SQL Stored Procedures – to be exposed as web services.

Service Enabling Reference Architecture tools
• Workflow and business services are exposed as web services.
• Web service dispatcher handles inbound web service calls from consumers.
• Web services outbound proxy handles invocation of external web services to be consumed by Siebel.

Siebel 7.5.3
• SOA support
• SOAP 1.1
• WSDL 1.1
• XSD support
• ASI (application service interface) introduced.

Siebel 7.7
• SOA support
• SOAP 1.1
• WSDL 1.1
• XSD support
• WS-I
• Workflow and Business service based ASI
• WS-security

Siebel 7.8.2
• SOA support
• SOAP 1.1
• WSDL 1.1
• XSD support
• Interoperability validation
• Access control

Siebel 8.0
• WS-I attachment
• Customer order management exposed as web service
• SOA support
• SOAP 1.1
• WSDL 1.1
• XSD support
• Enhanced ASI (application service interface)
• Fusion Middleware (SOA and BPEL Support)

Inputs from Sanjeev Patel

Read More about Siebel 

What is Informatica?

Informatica is a data integration platform, some of the key characteristics of Informatica that stand out are the

1. ETL process components and the user interface
2. Metadata
3. Wider data handling

ETL process components and the user interface

Datastage was the first GUI based data integration platform with an engine at its core, Informatica was probably designed & built to overcome some of the issues faced in using the Datastage. Informatica’s initial ETL product PoweMart’s Visual C++ based user interface was much appealing and easier to use than the Visual Basic based interface of Datastage. The port level view of the transformations/widgets made things much easier to view. The simple transformations like Filter, Update Strategy helped ETL developers to easily convert their design ideas into mappings. For every release there are definitive enhancements being done to improve the developer experience.

Metadata

Informatica has set standards into terms of either collecting or exposing the Metadata of an ETL process. The metadata of Informatica is stored in relational structure and is easily accessible directly or through its MX Views or through the SDK kit. The detail in capturing the ETL process into multiple components has enabled the product to introduce much better features or integrate newer solutions like data lineage or data type propagation or reusability that are not easily possible in other competitor products. The level of metadata capture right from mapping to transformation to the port level ensured that data about the process is captured at all levels and that no separate documentation is to be maintained outside the system.

Wider data handling

Unquestionably Informatica has made some great products available at the right time, something like mainframe access component which it acquired through Striva Inc still remains as the best mainframe access, and if you would have used the MVS edition of Datastage you would have seen its code generation based mainframe support. Informatica’s successful take on the ‘ELT’ based database engine products by introducing the ‘Push Down Optimization’ are great examples on how the platform has been flexible and agile to get itself extended in regular product updates.

Now Informatica is also a complete ‘data governance’ platform it provides the tools like data profiling or data quality management or the recent one identity systems solution integration. ‘Informatica Way’ has been to always make complex things look simpler and easily manageable.

We would discuss further on many more topics related to the Informatica platform right from simple tips to solution options, keep reading…Thank You.

Read More about Informatica

Prune days and Change Capture in DAC

Here is a question I often get from customers regarding change capture by DAC when it is used for extracting Siebel data. The problem is equally valid for non-Siebel data sources if the default change capture mechanism provided in DAC is used for identifying changes in source data.

How can we ensure that data committed with an update timestamp between the start of an ETL run and the end of the extract is not excluded by the next ETL?

• DAC stores a value called LAST_REFRESH_DATE at the end of an ETL extract.
• During the next run, DAC captures changes to the source data by comparing this LAST_REFRESH_DATE with the LAST_UPD date of records in the source.
• So if the LAST_REFRESH_DATE is less than LAST_UPD of a source record, then the record being processed has either been updated or created between the time of last execution and the current run
• But what will happen if a record in the source is committed between the time of starting the data extract and completing the extract with a commit date that is less than the LAST_REFRESH_DATE

Prune days to the rescue!!! The solution to the above problem is to set the PRUNE_DAYS parameter.

If the prune days parameter is set to 1 for example, the change capture process will use LAST_REFRESH_DATE – Prune days as the cutoff timestamp for change capture

Here is an example of such a scenario

• Let us assume that there is a table called S_CUSTOMER in transaction database with 2 records created during business hours on the 1st of January.
• On the January 1st, Extraction of Customer Data begins at 1700 hours and completes at January 1st, 1800 hours into a customer Dimension called WC_CUSTOMER.
• LAST_REFRESH_DATE stored is January 1, 1800 hours


• Now, a new customer is created between 1700 and 1800. But it is committed to the database AFTER the extract is completed with a timestamp of January 1st, 17.30



• More customers are added to the transaction database during the next working day. As a result the following is the condition of the source table at the time of the next ETL Extract i.e. Jan 2, 12:00 am





• Now the change capture process in DAC will compare the stored LAST_REFRESH_DATE with the update timestamp of the records in the source data and pull only the records updated after this LAST_REFRESH_DATE i.e. Jan 1, 1800 hours into the target table. Unfortunately for the 3rd record depicted below, the LAST_UPD is Jan 1, 1730 i.e. it is before the LAST_REFRESH_DATE. So, it does not qualify for the incremental extract and we have a missing record in the target table.



• g) But if we had set the PRUNE DAYS parameter to 1, the cutoff date used to identify changes in source data would be .Dec 31, 1800 hours i.e LAST_REFRESH_DATE – 1 day . If this ploy is used, all the records with LAST_UPD > Dec 31, 1800 hours will qualify for the incremental extract and we will have a complete set of data in the target table

The Prune days parameter can be used to ensure that DAC Change Capture does not miss out data created around the time of data extraction.

Set it with care to a reasonable value to avoid increasing the volume of data selected by incremental load process.

New Feature in DAC 7.9 for non-Siebel Sources

For source systems other than Siebel, the Prune Days setting is used in the same way except that the DAC subtracts the number of prune days from the LAST_REFRESH_DATE of a given source and supplies this as the value for the $$LAST_EXTRACT_DATE parameter.

Inputs by: Raghunatha Yadav & Sanjay Rao

Read More about DAC

Part I – Peoplesoft and Microsoft Active Directory Single sign on Setup

Hi Friends, Its been a long time that I had posted in our Pitstop

May be because I have been kept busy with the work load and also the 2008 Formula One Championship,.. Its was too good to watch the thundering drive to the chequered flag by the team ferrari and hardly was concentrating on the opponents pitstops,…

Here we go,.. I would like to bring in the ‘technical collaboration’ (term taken out of air) between the Microsoft Active Directory and Oracle-Peoplesoft. You may find a lot of information on this in the Internet and other related material of interest to you. So I thought of listing down some of the basic aspects that needs to be concentrated while we are into a single sign on process.

• Gathering the correct ‘Connect String’ Information from the Active Directory (LDAP)
• Selecting a Network Analyzer Tool to be used to trace the data packet movement – Just in case if we head into connectivity problem, and
• Configuring the Schema Cache on the Peoplesoft Environment.

Do post in queries if you do happen to run into any issues when you are performing single sign on or when you need more information on the procedure.

I’m happy that i had a very quick pitstop here back on track and will be back soon

Read more About: Peoplesoft

A test drive with the Siebel 8 ST engine

A new script engine i.e. the Siebel ST eScript engine has been made the default scripting engine in Siebel 8.0.

In case you did not know, Siebel eScript is a scripting or programming language that application developers use to write simple scripts to extend Siebel applications. Till recently, the default script engine for Siebel Tools and Client Applications was the T eScript engine. And ST for the uninitiated stands for Strongly Typed.

If you stayed awake through your programming classes, you might remember that strongly typed languages bind variables to a particular data type. So if you declare a variable to be of type Dog, you cannot abruptly decide to assign a Road-Roller to the variable midway through the program. The good thing about this is you cannot call invalid methods like Road-Roller.bark() or Dog.BlockTrafficAndAnnoyCommuters() by mistake.

The bad thing is that you need to declare more variables and therefore need more runtime memory when using a strongly typed language instead of a weakly typed languages. There are proponents for both types of programming languages. But that is not the issue that interests us here.

What did interest me is that Siebel claims that the ST eScript engine provides the following enhancements:

• Improved performance and self documentation due to strong typing of variables

• ScriptAssist utility and

• Fix and Go feature

I decided to take the ST engine out for a test drive to see if the above claims are indeed true.

In my little experiment, I decided to use a COMCreateObject to pull the data from an external source, in Excel and update the Siebel database. The excel file contained some new records that and some changes to existing records. The update would be triggered on the click of a button which would in turn invoke a Business Service which comprises of many functions.

Here are my findings:

ScriptAssist

I loved the new Script Assist Feature which makes the life of the programmer much easier providing auto-complete, auto-indentation, method listing, and method signature capabilities.

There is no longer a need to remember all the methods related to an object. Just press Ctrl+Space or “.” and ScriptAssist will display a dropdown of all methods and properties available for that particular object.

This is a big timesaver and is highly appreciated by us.

Auto-complete is another nifty feature that helped us avoid a lot recompilation. Just enter the first 2-3 alphabets of a method and it shows the available method name. For example, when I started off with the letters “BusC”, the editor offered to save me the effort of typing in the remaining 5 letters in “BusComp()”. This is similar to IntelliSense technology from Microsoft.

Fix and Go

With Fix and Go enabled, there is no need to compile the .srf to restart the debugger after making a change.

This was a big pain in the bad old days. With fix and go, I hope to have a lot more time to spend on “strategic meetings” near the coffee machine.

Performance improvement because of Strong typing:

I tried to see if I would get any performance improvements by using ST Script by putting an equivalent piece of code created in ST-Engine with another piece of code in Siebel 7.8 T-Engine

Although we ran the code through some 10,000 iterations, we did not see a major difference in the execution time.

However, there should in theory be a performance improvement on account of saving time on type-checking at run-time and compile time.

Also, the code should be more robust as it would not be possible to call invalid methods as it would be in a weakly typed language

So, should I throw away all my T-script Engine code and embrace the new ST-Script Engine ?

It depends on the amount of scripting you have done in your Siebel Installation. If you have heavily customized your objects with scripting, it is going to be a major exercise to verify that the code is working correctly in the new environment.

But once, the upgrade is done, there should be a performance improvement and your application will be more robust because there is a lower likelihood of getting those errors because of calls to invalid methods.

Inputs from Sajid Attar and Abhinav Anand.

Read More about Siebel 8.0

Siebel Smartscript

Business Scenario
Call center/helpdesk agents have customer interactions, in which they address customer issues, ask questions and resolve problems. The chances of the customer coming back become higher if the interaction has been satisfying. Thus, in order to serve the customer better, capturing the flow of interaction becomes vital. In such real time scenarios, Siebel Smartscript helps call center agents to define application workflow and capture customer interactions.

Siebel Smartscript and its usability
Siebel SmartScript guides agents through each customer interaction, suggesting solutions based on the customer’s profile, environment, current requirements, and buying patterns. Siebel Smartscipt GUI includes questions in the right pane and tree structure summarizing the questions and answers in the left pane. In order to build this UI developer needs to include the script, page(s), question(s) and answer(s).

The answers of Smartscript can be stored in a BC, few values can be updated, and some business logic or functionality can be extended based on these answers. For such customizations script can be written at question or script object level. Smartscript programs can be coded in VBScript or escript. Some additional syntax has been provided by Siebel only specific to Smartscript usage.

Why Siebel Smartscript?
It is user-friendly having a very intuitive GUI. Programmers having basic knowledge of coding can graphically create Smartscript. The scripts written for pages, questions and script objects can be re-used and can have multiple language translations. Using SmartScripts the customer interactions become more personalized and effective in nature.

Alternatives to Siebel Smartscript
Using customized applets and embedding scripts is an alternative to Siebel Smartscript. This alternative has demerits of increased development time and cost, weaker GUI and upgrade difficulties.

New features in Siebel Smartscipt 8.0
In Siebel 8.0, Smart script can be invoked using a hyperlink. Also in the newer version Business services and Siebel Assignment manager can be invoked from Siebel Smartscript. Siebel Smartscript 8.0 has an improved GUI too.

Inputs from Shardul Trivedi

Read More about Siebel Smartscript

Let’s talk EPM – Part 2 on Metrics Profiling

In my earlier post on Enterprise Performance Management (EPM), I had enumerated the six steps of a practical EPM strategy in an organization. They were:

1. Business Process Maps – Understand the business process
2. Metrics Identification – Get hold of the metrics
3. Metrics Profiling – Understand the metrics in depth
4. Metrics Maps – Understand the cause and effect relationships between metrics
5. Metrics Visualization – Implementation of Metric Maps on BI Tools
6. Watch and Improve – Monitor Metrics and Improve business process as required

It is important to realize that building a data warehouse (enterprise wide) or data mart (functional area wise) or simply an integrated, subject-oriented data repository (without getting lost in semantics!) is implicit in the set of steps outlined above.

Steps 1 and 2 (Business Process and Metrics identification) are self-explanatory. Though getting hold of the right metrics is easier said than done, it is fairly well understood that the measures/metrics selected for analysis should align itself with the organization’s mission, business model and value creation aspects.

Step 3 – Metrics Profiling, in my opinion, is the step often missed out in EPM implementations and arguably is a major cause of failures in such programs. Metrics Profiling stated simply is a way of understanding your metrics in depth. Given below is a sample template for profiling your metrics and can be customized for each organization.

Profiling Parameters:

1. Metric Name – Name of the metric

2.Metric Definition – Brief definition of the metric

3.Metric Type – Is it a ratio, absolute number, trended value, etc.

4.Sources of data – Identify the source of data for the metric and the owners

5.Application – Brief description of how the metric helps in managing the business better

6.Potentially Affected Metrics – Identify the other metrics that are impacted (positive or negative) by this metric.

7.Example – Provide an example of metrics usage. (For example: ABC Computers released three new product lines during the last 12 months, generating $15 million in new revenue out of total annual revenue of $125 million. New Products Index = 15 ÷ 125 = 12%)

Metrics Profiling is a very important step in the implementation of enterprise wide performance

management system. I will discuss the other aspects of EPM in my subsequent posts.

Thanks for reading .

Data Integration Challenge – Storing Timestamps

Storing timestamps along with a record indicating its new arrival or a change in its value is a must in a data warehouse. We always take it for granted, adding timestamp fields to table structures and tending to miss that the amount of storage space a timestamp field can occupy is huge, the storage occupied by timestamp is almost double against a integer data type in many databases like SQL Server, Oracle and if we have two fields one as insert timestamp and other field as update timestamp then the storage spaced required gets doubled. There are many instances where we could avoid using timestamps especially when the timestamps are being used for primarily for determining the incremental records or being stored just for audit purpose.

How to effectively manage the data storage and also leverage the benefit of a timestamp field?

One way of managing the storage of timestamp field is by introducing a process id field and a process table. Following are the steps involved in applying this method in table structures and as well as part of the ETL process.Data Structure

1. Consider a table name PAYMENT with two fields with timestamp data type like INSERT_TIMESTAMP and UPDATE_TIEMSTAMP used for capturing the changes for every present in the table
2. Create a table named PROCESS_TABLE with columns PROCESS_NAME Char(25), PROCESS_ID Integer and PROCESS_TIMESTAMP Timestamp
3. Now drop the fields of the TIMESTAMP data type from table PAYMENT
4. Create two fields of integer data type in the table PAYMENT like INSERT_PROCESS_ID and UPDATE_PROCESS_ID
5. These newly created id fields INSERT_PROCESS_ID and UPDATE_PROCESS_ID would be logically linked with the table PROCESS_NAME and its field PROCESS_ID

ETL Process

1. Let us consider an ETL process called ‘Payment Process’ that loads data into the table PAYMENT
2. Now create a pre-process which would run before the ‘payment process’, in the pre-process build the logic by which a record is inserted with the values like (‘payment process’, SEQUNCE Number, current timestamp) into the PAYMENT table. The PROCESS_ID in the payment table could be defined as a database sequence function.
3. Pass the current_prcoess_id from pre-process step to the ‘payment process’ ETL process
4. In the ‘payment process’ if a record is to inserted into the PAYMENT table then the current_prcoess_id value is set to both the columns INSERT_PROCESS_ID and UPDATE_PROCESS_ID else if a record is getting updated in the PAYMENT table then the current_process_id value is set to only the column UPDATE_PROCESS_ID
5. So now the timestamp values for the records inserted or updated in the table PAYMENT can be picked from the PROCESS_TABLE by joining by the PROCESS_ID with the INSERT_PROCESS_ID and UPDATE_PROCESS_ID columns of the PAYMENT tableBenefits
6.  

• The fields INSERT_PROCESS_ID and UPDATE_PROCESS_ID occupy less space when compared to the timestamp fields
• Both the columns INSERT_PROCESS_ID and UPDATE_PROCESS_ID are Index friendly
• Its easier to handle these process id fields in terms picking the records for determining the incremental changes or for any audit reporting.

Read More about Data Integration