Tuesday, October 16, 2007

All About Decision Management System

There can be several works in a end to end decisioning system. Decision Management System has to include all those steps. I have tried to summarize them here:
1. Collect and Explore data
2. Understand and Organize Data
3. Design Data Repository and Implement it
4. Build Reporting System (MIS)
5. Analyze Data for hidden insights
6. Perform Predictive Modeling and Segmentation of Portfolio
7. Design Strategy Based on Analysis, Modeling and Segmentation
8. Build and Implement Decision Optimization System
9. Monitor the performance of DO System
10. Fine tune the DO System with feedback from Monitoring System

I am a strong believer of Good Data. Analytics all depends on data, so before performing any analysis on data it has to be ensured that data is Analysis Ready.
Most of the resource intensive work is around data handling. So from points 1 through 4 are related to it.

If some company wants to start using DM system, then its best to follow steps one by one.

#1. Collect and Explore Data: Collect all relevant data either from internal source or from external sources. External sources may be getting information from Data Bureaus or Market Research firms or from partner companies, more the merrier. Then comes exploration of these data collected and finding the necessary ones. All data cannot be analyzed due to resource crunch and it does not make any sense to analyze useless ones.


# 2. Understand and Organize Data: The data in raw form is useless. It needs to be transformed into usable form. So a thorough data study has to be done to understand the business interpretation. Then the knowledge has to be used to organize data for Analytic use. This may include creating derived variables, converting files to usable forms. The usable form may be creating csv files from SQL server in order to convert to SAS (or whichever Analytic Package is used). The Quality Assurance test will need to be transformed in each step, either it is reorganizing data or converting formats.
Also, there might be need to collect some data for future use. Identify such shortage of data.

# 3. Design Data Repository and Implement it: Every time need for Analytics arises performing data works becomes very costly and those data works are mostly common between those functions. So maintaining a good data repository system pays.
Data Repository System should be designed in such a way that all the problems in the first two steps are phased out. The learnings from data exploration and organization can be implemented into an automated system.
If some new data needs to be collected, then build a module in DDR to do that function.


# 4. Build Reporting System (MIS): Data Collection and Maintenance is almost done with the earlier three steps, and now comes reporting. Data in raw form, however good, is hardly interpretable in business. So build a system to report basic insights from data.
Banks may like to know daily cash flow, number of customers visiting daily, rate of increase in credit, number of accounts going charge-off daily etc. There may be hundreds of such metrics that can be directly found out on daily basis and its all repetitive. Such insights when handy can help make many business decisions fairly easy.


# 5. Analyze Data for hidden insights: Not all information data can say is readily available. There might be some hidden reason when some outcome change and it needs to performed on need basis. This might be done routinally in order to avoid sudden accident, or can be done after a alarm.
This might also be needed when a new portfolio program is to be started, or some redesigning is required vis-a-vis competition.


# 6. Perform Predictive Modeling and Segmentation of Portfolio: To take strategic decisions, predictions are always important. It can be best done by segmentation and predictive modeling.

# 7. Design Strategy Based on Analysis, Modeling and Segmentation: Performing the statistical functions does not directly relate to business. It has to be put into business understanding and a strategy needs to be designed. It should include not the Analysis output and Model scores but Business Rules based on those outputs. Each and every decision points need to be evaluated and decisions need to be optimized w.r.t the desired outcome(s).
The desired outcome may be income, revenue, risk, attrition, response etc. based on the business need.

# 8. Build and Implement Decision Optimization System: After the strategy designed, its the time for it to be put in action. In the competitive environment, there might be need to change the strategy quite frequently following the change in market conditions and competition. So each time a different strategy is adopted, putting a new decision system makes it almost impractical in terms of both time and resources. This calls for the need of a Decision Optimization System that allows to change decisions often.
Two of the features of good decision management system can be:
a. creating, changing and removing business rules
b. fine-tuning the operating points for predictive model scores

# 9. Monitor the performance of DO System: Continuous monitoring of DO system is as important as building it. There might be various reasons to do it. The change in market conditions is one, while few business rules and models may have error in them or there might be error in their implementation. But most commonly the need is of change of business need than anything else. There may be many operational issues (resources crunch, lack of expertise etc) while trying new strategy.


# 10. Fine tune and upgrade the DO System with feedback from Monitoring System: The information collected from monitoring system can be used to fine-tune decision points as well as to upgrade the system. No system is perfect and it is hard to find the drawbacks prior to actual implementation, so it is important to follow the path of continuous improvement.

Now, its your turn to say. I have said all I had, and I expect comments here. Please share if you find that I missed to include something or I missed the actual path.

5 comments:

Datasmith said...

Well done, having said that the title was "All about decision Management sys" ...you have covered all about the data processing for decision support,i think there is a lot more...;) Any company that knows its customers & business is already well aware of many important high pay off patterns that it has observed over the years. Given that every business is dynamic, constantly evolving plus the ever changing market forces plus internal forces etc all this multiplied by uncertainity, the BI team (decision support team as u may want to call)needs to go beyond seeing unusual patterns in data & find out what it means to business? with goal in mind does it make sense to have all the strategy in place or is there a need for change if so, or if any further action is needed the BI team should very well cordinate with the stakeholders(board), to make itself heard loud and clear with all its sense.........& induce growth/sustainability etc etc which i think a lot of BI teams in most companies lack......

Datasmith said...

whats up ???? guess you busy ?

Bhupendra said...

Datasmith:
Sorry for late reply. I was out of net for some days.

You are correct in calling for a greater role and coordination between the BI and Business Teams. I too feel that it is lacking in many BI Teams, but these things is changing now thanks to great corporate disturbances globally. The fall of many companies having not so good intelligence system have forced others to be tight. This phenomenon is already prevailing in Banking space and getting into other fields like Retail, Insurance and others.

I have actually written this post for giving overall picture of DM System to people who are still unaware. So its quite trivial for power readers.

Thanks for stopping and commenting.

Bhupendra

James Taylor said...

Nice list Bhupendra. I think there needs to be an equal focus on the IT design of a decision service where you are focused on the analytics/data side of the equation but useful.
Thanks
JT

James Taylor
The Smart (Enough) Systems blog
My ebizQ blog
Author of Smart (Enough) Systems

Bhupendra said...

James:
Thanks for stopping by my blog.

You are right in calling equal focus on IT Design. It is actually the implemented software that deliver everything. Without proper software platform, Analysis and Modeling, however good, end up being concepts alone.

Bhupendra

Page Views from May 2007

 

© New Blogger Templates | Webtalks