Prune days and Change Capture in DAC

Here is a question I often get from customers regarding change capture by DAC when it is used for extracting Siebel data. The problem is equally valid for non-Siebel data sources if the default change capture mechanism provided in DAC is used for identifying changes in source data.

How can we ensure that data committed with an update timestamp between the start of an ETL run and the end of the extract is not excluded by the next ETL?

• DAC stores a value called LAST_REFRESH_DATE at the end of an ETL extract.
• During the next run, DAC captures changes to the source data by comparing this LAST_REFRESH_DATE with the LAST_UPD date of records in the source.
• So if the LAST_REFRESH_DATE is less than LAST_UPD of a source record, then the record being processed has either been updated or created between the time of last execution and the current run
• But what will happen if a record in the source is committed between the time of starting the data extract and completing the extract with a commit date that is less than the LAST_REFRESH_DATE

Prune days to the rescue!!! The solution to the above problem is to set the PRUNE_DAYS parameter.

If the prune days parameter is set to 1 for example, the change capture process will use LAST_REFRESH_DATE – Prune days as the cutoff timestamp for change capture

Here is an example of such a scenario

• Let us assume that there is a table called S_CUSTOMER in transaction database with 2 records created during business hours on the 1st of January.
• On the January 1st, Extraction of Customer Data begins at 1700 hours and completes at January 1st, 1800 hours into a customer Dimension called WC_CUSTOMER.
• LAST_REFRESH_DATE stored is January 1, 1800 hours


• Now, a new customer is created between 1700 and 1800. But it is committed to the database AFTER the extract is completed with a timestamp of January 1st, 17.30



• More customers are added to the transaction database during the next working day. As a result the following is the condition of the source table at the time of the next ETL Extract i.e. Jan 2, 12:00 am





• Now the change capture process in DAC will compare the stored LAST_REFRESH_DATE with the update timestamp of the records in the source data and pull only the records updated after this LAST_REFRESH_DATE i.e. Jan 1, 1800 hours into the target table. Unfortunately for the 3rd record depicted below, the LAST_UPD is Jan 1, 1730 i.e. it is before the LAST_REFRESH_DATE. So, it does not qualify for the incremental extract and we have a missing record in the target table.



• g) But if we had set the PRUNE DAYS parameter to 1, the cutoff date used to identify changes in source data would be .Dec 31, 1800 hours i.e LAST_REFRESH_DATE – 1 day . If this ploy is used, all the records with LAST_UPD > Dec 31, 1800 hours will qualify for the incremental extract and we will have a complete set of data in the target table

The Prune days parameter can be used to ensure that DAC Change Capture does not miss out data created around the time of data extraction.

Set it with care to a reasonable value to avoid increasing the volume of data selected by incremental load process.

New Feature in DAC 7.9 for non-Siebel Sources

For source systems other than Siebel, the Prune Days setting is used in the same way except that the DAC subtracts the number of prune days from the LAST_REFRESH_DATE of a given source and supplies this as the value for the $$LAST_EXTRACT_DATE parameter.

Inputs by: Raghunatha Yadav & Sanjay Rao

Read More about DAC

A test drive with the Siebel 8 ST engine

A new script engine i.e. the Siebel ST eScript engine has been made the default scripting engine in Siebel 8.0.

In case you did not know, Siebel eScript is a scripting or programming language that application developers use to write simple scripts to extend Siebel applications. Till recently, the default script engine for Siebel Tools and Client Applications was the T eScript engine. And ST for the uninitiated stands for Strongly Typed.

If you stayed awake through your programming classes, you might remember that strongly typed languages bind variables to a particular data type. So if you declare a variable to be of type Dog, you cannot abruptly decide to assign a Road-Roller to the variable midway through the program. The good thing about this is you cannot call invalid methods like Road-Roller.bark() or Dog.BlockTrafficAndAnnoyCommuters() by mistake.

The bad thing is that you need to declare more variables and therefore need more runtime memory when using a strongly typed language instead of a weakly typed languages. There are proponents for both types of programming languages. But that is not the issue that interests us here.

What did interest me is that Siebel claims that the ST eScript engine provides the following enhancements:

• Improved performance and self documentation due to strong typing of variables

• ScriptAssist utility and

• Fix and Go feature

I decided to take the ST engine out for a test drive to see if the above claims are indeed true.

In my little experiment, I decided to use a COMCreateObject to pull the data from an external source, in Excel and update the Siebel database. The excel file contained some new records that and some changes to existing records. The update would be triggered on the click of a button which would in turn invoke a Business Service which comprises of many functions.

Here are my findings:

ScriptAssist

I loved the new Script Assist Feature which makes the life of the programmer much easier providing auto-complete, auto-indentation, method listing, and method signature capabilities.

There is no longer a need to remember all the methods related to an object. Just press Ctrl+Space or “.” and ScriptAssist will display a dropdown of all methods and properties available for that particular object.

This is a big timesaver and is highly appreciated by us.

Auto-complete is another nifty feature that helped us avoid a lot recompilation. Just enter the first 2-3 alphabets of a method and it shows the available method name. For example, when I started off with the letters “BusC”, the editor offered to save me the effort of typing in the remaining 5 letters in “BusComp()”. This is similar to IntelliSense technology from Microsoft.

Fix and Go

With Fix and Go enabled, there is no need to compile the .srf to restart the debugger after making a change.

This was a big pain in the bad old days. With fix and go, I hope to have a lot more time to spend on “strategic meetings” near the coffee machine.

Performance improvement because of Strong typing:

I tried to see if I would get any performance improvements by using ST Script by putting an equivalent piece of code created in ST-Engine with another piece of code in Siebel 7.8 T-Engine

Although we ran the code through some 10,000 iterations, we did not see a major difference in the execution time.

However, there should in theory be a performance improvement on account of saving time on type-checking at run-time and compile time.

Also, the code should be more robust as it would not be possible to call invalid methods as it would be in a weakly typed language

So, should I throw away all my T-script Engine code and embrace the new ST-Script Engine ?

It depends on the amount of scripting you have done in your Siebel Installation. If you have heavily customized your objects with scripting, it is going to be a major exercise to verify that the code is working correctly in the new environment.

But once, the upgrade is done, there should be a performance improvement and your application will be more robust because there is a lower likelihood of getting those errors because of calls to invalid methods.

Inputs from Sajid Attar and Abhinav Anand.

Read More about Siebel 8.0

Siebel Smartscript

Business Scenario
Call center/helpdesk agents have customer interactions, in which they address customer issues, ask questions and resolve problems. The chances of the customer coming back become higher if the interaction has been satisfying. Thus, in order to serve the customer better, capturing the flow of interaction becomes vital. In such real time scenarios, Siebel Smartscript helps call center agents to define application workflow and capture customer interactions.

Siebel Smartscript and its usability
Siebel SmartScript guides agents through each customer interaction, suggesting solutions based on the customer’s profile, environment, current requirements, and buying patterns. Siebel Smartscipt GUI includes questions in the right pane and tree structure summarizing the questions and answers in the left pane. In order to build this UI developer needs to include the script, page(s), question(s) and answer(s).

The answers of Smartscript can be stored in a BC, few values can be updated, and some business logic or functionality can be extended based on these answers. For such customizations script can be written at question or script object level. Smartscript programs can be coded in VBScript or escript. Some additional syntax has been provided by Siebel only specific to Smartscript usage.

Why Siebel Smartscript?
It is user-friendly having a very intuitive GUI. Programmers having basic knowledge of coding can graphically create Smartscript. The scripts written for pages, questions and script objects can be re-used and can have multiple language translations. Using SmartScripts the customer interactions become more personalized and effective in nature.

Alternatives to Siebel Smartscript
Using customized applets and embedding scripts is an alternative to Siebel Smartscript. This alternative has demerits of increased development time and cost, weaker GUI and upgrade difficulties.

New features in Siebel Smartscipt 8.0
In Siebel 8.0, Smart script can be invoked using a hyperlink. Also in the newer version Business services and Siebel Assignment manager can be invoked from Siebel Smartscript. Siebel Smartscript 8.0 has an improved GUI too.

Inputs from Shardul Trivedi

Read More about Siebel Smartscript

Single Sign-On in Siebel

Being a software developer in Siebel, I had numerous opportunities to create something using this tool. Apart from acknowledging client requests which is everyone’s number one priority, like any other developer I also wanted to first get my fundamentals strong in this tool. There were times when I tried 2-3 different ways to deal with one of the old demands and was successful in discovering many new and simpler methods. Since then I always experiment to meet the requirements and never hesitated to play with Siebel.

This led me into using these concepts readily and implementing them in real time. While using Siebel for many servers in my project, the traditional practice to Sign-In every time you open a new Siebel application server was not only exasperating at times but also a shortcoming in terms of Siebel’s user friendly behavior. Searching an answer for this problem I came across the Single Sign-On Process.

For starters, what is Single Sign-In (SSO)?

It is a session/user authentication process that allows a user to enter name and password for only a single time in entire session in order to access multiple applications. The process authenticates the user for all the applications they have been given rights to and eliminates further prompts when they switch applications during a particular session.

Single Sign-On (SSO) is becoming the standard for advanced security in IT sector. This equips us in streamlined, consistent and secure password management policy for every user. Cutting down on the number of passwords issued to users and reducing time spent on administration, support and training, SSO directly impacts the daily user ability.

The following steps need to be followed for configuring Web Single Sign On and the Active Directory Security Adapter.

1. An ADSI service account will be needed.This service account is one used to bind to the Active Directory and will also hold the shared database credentials.
2. The service account will also be the web anonymous user for Single Sign On and must have a Siebel login with Web Anonymous User responsibility. This anonymous user does not need a database account.
3. Create the shared database account. All users will connect to the database with this database account instead of an individual database account.
4. Create a custom application object manager that will be configured for ADSI security adapter authentication instead of database authentication.
5. Add the SSO Sales Object Manager to the Siebel Web Server Extension Update the Siebel Web Server Extension eapps.cfg file to include the SSO Sales Object Manager.
6. Add a new virtual directory to IIS for SSO.
7. Configure Siebel Web Server for Web SSO. Open the eapps.cfg file and set parameter IntegratedDomainAuth = True in [swe] section and add following parameter in [SSO] section.

- SingleSignOn=TRUE
- TrustToken=token name
- userspecsource=server
- userspec=REMOTE_USER
- StartCommand= SWECmd=GotoView&SWEView=Sales+Home+Page+View&SWERF=1

This will enable the Single Sign On and it will be authenticated by Windows Integrated Authentication.

Configure ADSI Security Adapter Named Subsystem in Server Administration – Configuration; Enterprises; profile configuration tab and set the parameters for ADSI.

If the authentication is stolen then all resources becomes vulnerable. In addition, you need to be aware of whether there are backups to the SSO in the event the system is down; if the Server is down then the network resources will remain unavailable.

Post by Suman Kumar

Read More about Siebel

Introducing Siebel Street

I would like to welcome you to Hexaware’s Siebel blog– “Siebel Street”. Hexaware’s Siebel practice has been doing wonderful things and we would like to share some of them with you through Siebel Street. In this first post I will introduce the Siebel CRM and describe its capabilities. I will end with giving a sneek peek at some of the forthcoming posts in this blog.

Siebel has been the market leader in the CRM space for more than a decade. It started with a sales-force automation software and then expanded into marketing and customer service applications. It was taken over by Oracle in 2005. Today, Siebel CRM is at the forefront of Oracle’s CRM product strategy. Siebel’s CRM Suite caters to clients in the BFSI, Telecom, Pharmaceuticals , Retail , HI-tech Manufacturing and Transportation verticals among others. Siebel’s Technical architecture provides a top-notch user interface, strong integration, diversified industry-wide applications, easy upgrades and reliable customer support.

Most customers need to change the Siebel application by modifying delivered functionality or by implementing new modules. The customizations in the Siebel applications are implemented using Configuration, Scripting and Workflows. Siebel also provides excellent data management capabilities. Siebel EAI integrates third party applications/objects with Siebel and Siebel EIM populates external data in interface tables and user data in base tables. Siebel Analytics is the business Intelligence application from Siebel Systems. It has been rebranded as OBIEE and will form the centerpiece of Oracle’s BI Strategy.

In Siebel Street, contributors from Hexaware’s Siebel practice will blog about their experience with Siebel CRM and Siebel Analytics/ OBI-EE. We will describe the business and technical challenges, their resolutions and the benefits of our approach. We are working on the entire range of tools available on Siebel. Our team consists of experts in EIM, Config, Integration, ETL, Analytics, Administration and Testing. We will share our project experiences in these areas in the forthcoming posts.

In this month we will post on DAC, escripting etc. Watch this space..

Read More  about  Siebel Street

Improvement in DAC 7.9 – Subject Area Assembly

One of the nice new features in DAC 7.9 is the facility to assemble all the ETLs required to populate a fact using the Assemble button

In the Pre DAC 7.9 days, the process for building a star schema was as below

1. Think up all the tasks required to populate a star schemas (Basically the Informatica workflows)
2. Create the tasks and Identify Dependent and Depending on tables manually when creating a task.
3. Construct a Table Group where one had to figure out all tables that belong to the star schema i.e. the fact and related dimensions
4. And then at last build the Subject Area
5. Run the subject area and then curse yourself for not remembering some tables.
6. Start all over again.

Now

1. You don’t have to identify Dependent and Depending on tables. There is a sync feature which automatically identifies all the sources and targets which are affected/dependent on a task
2. The concept of a Table Group is dead. You just have to name the fact to be populated and click on the Assemble button
3. Click option: selected task only (unless you want to assemble for all subject areas delivered out of the box. Trust me : you don’t want to do this)
4. The DAC assembles the selected subject area by determining what dimensions and other related tables are required, and what tasks are needed to load these tables.
5. You will receive an informational message when the assemble process is comp
6. Hey presto, the Subject Area is ready

But wait a minute,

1. You still have to do some cleanup and remove unwanted tasks
2. Click the Tasks tab to view which tasks the DAC has determined are required for this subject area
3. Tasks that are automatically assigned to the subject area by the DAC are indicated with the Auto generated check mark.
4. You can inactivate a task from participating in the subject area by selecting the Inactive check box.
5. When the Inactive check box is selected, the task remains inactive even if you reassemble the subject area.
6. You can also remove a task from the subject area using the Add/Remove command, but when you remove a task it is only removed from the subject area until you reassemble the subject area
7. You still have to put in the implicit dependencies i.e. dependencies on tables that are not featured in the select statements or the workflow
8. Upgrade does not work for custom workflows setup in DAC

There’s still more

1. Unless you have a lot of time on your hands, don’t try clicking on the Assemble tab when DAC server has been up for some time
2. Stop the server, restart the client and then click on Assemble
3. It is much quicker. But don’t know why it works this way

So in conclusion Oracle DAC 7.9 is great and has some nice new features. But upgrading is going to be no fun. More on this later.

–Posted by Raghunatha Yadav/Sanjay Rao

Read More about  DAC 7.9