Monday, March 23, 2009
Why Infosys Should Buy FICO
Wednesday, January 21, 2009
Business Intelligence 2.0 : Decision Management
Introduction
Decision Management System refers to the integrated IT System which handles Data Management, Analytics and their Implementation for Business Use.Decision Management System which are built by DM Companies in enterprise level are even referred to asEnterprise Decision Management or EDM. EDM is equivalent to an ERP System, where ERP is for operations and EDM is for Decisions.
Capabilities
1. Collect and Explore data2. Understand and Organize Data
3. Design Data Repository and Implement it
4. Build Reporting System (MIS)
5. Analyze Data for hidden insights
6. Perform Predictive Modeling and Segmentation of Portfolio
7. Design Strategy Based on Analysis, Modeling and Segmentation
8. Build and Implement Decision Optimization System
9. Monitor the performance of DO System
10. Fine tune the DO System with feedback from Monitoring System
#1. Collect and Explore Data: Collect all relevant data either from internal source or from external sources. External sources may be getting information from Data Bureaus or Market Research firms or from partner companies, more the merrier. Then comes exploration of these data collected and finding the necessary ones. All data cannot be analyzed due to resource crunch and it does not make any sense to analyze useless ones.
# 2. Understand and Organize Data: The data in raw form is useless. It needs to be transformed into usable form. So a thorough data study has to be done to understand the business interpretation. Then the knowledge has to be used to organize data for Analytic use. This may include creating derived variables, converting files to usable forms. The usable form may be creating CSV files from SQL server in order to convert to SAS (or whichever Analytic Package is used). The Quality Assurance test will need to be transformed in each step, either it is reorganizing data or converting formats.
Also, there might be need to collect some data for future use. Identify such shortage of data.
# 3. Design Data Repository and Implement it: Every time need for Analytics arises performing data works becomes very costly and those data works are mostly common between those functions. So maintaining a good data repository system pays.
Data Repository System should be designed in such a way that all the problems in the first two steps are phased out. The learnings from data exploration and organization can be implemented into an automated system.
If some new data needs to be collected, then build a module in DDR to do that function.
# 4. Build Reporting System (MIS): Data Collection and Maintenance is almost done with the earlier three steps, and now comes reporting. Data in raw form, however good, is hardly interpretable in business. So build a system to report basic insights from data.
Banks may like to know daily cash flow, number of customers visiting daily, rate of increase in credit, number of accounts going charge-off daily etc. There may be hundreds of such metrics that can be directly found out on daily basis and its all repetitive. Such insights when handy can help make many business decisions fairly easy.
# 5. Analyze Data for hidden insights: Not all information data can say is readily available. There might be some hidden reason when some outcome change and it needs to performed on need basis. This might be done routinally in order to avoid sudden accident, or can be done after a alarm.
This might also be needed when a new portfolio program is to be started, or some redesigning is required vis-a-vis competition.
# 6. Perform Predictive Modeling and Segmentation of Portfolio: To take strategic decisions, predictions are always important. It can be best done by segmentation and predictive modeling.
# 7. Design Strategy Based on Analysis, Modeling and Segmentation: Performing the statistical functions does not directly relate to business. It has to be put into business understanding and a strategy needs to be designed. It should include not the Analysis output and Model scores but Business Rules based on those outputs. Each and every decision points need to be evaluated and decisions need to be optimized w.r.t the desired outcome(s).
The desired outcome may be income, revenue, risk, attrition, response etc. based on the business need.
# 8. Build and Implement Decision Optimization System: After the strategy designed, its the time for it to be put in action. In the competitive environment, there might be need to change the strategy quite frequently following the change in market conditions and competition. So each time a different strategy is adopted, putting a new decision system makes it almost impractical in terms of both time and resources. This calls for the need of a Decision Optimization System that allows to change decisions often.
Two of the features of good decision management system can be:
a. creating, changing and removing business rules
b. fine-tuning the operating points for predictive model scores
# 9. Monitor the performance of DO System: Continuous monitoring of DO system is as important as building it. There might be various reasons to do it. The change in market conditions is one, while few business rules and models may have error in them or there might be error in their implementation. But most commonly the need is of change of business need than anything else. There may be many operational issues (resources crunch, lack of expertise etc) while trying new strategy.
# 10. Fine tune and upgrade the DO System with feedback from Monitoring System: The information collected from monitoring system can be used to fine-tune decision points as well as to upgrade the system. No system is perfect and it is hard to find the drawbacks prior to actual implementation, so it is important to follow the path of continuous improvement.
Wednesday, September 10, 2008
Metamorphosis of Decision Management Process
- Analytics Consulting
- Market Research Consulting
- Management Consulting
But Consulting was not that alone. Fair Isaac has been providing Analytics based Consulting services for the last 50 years and they have helped companies grow in their own way. Analytics Consultant helped business managers to understand the existing operations, the pros and cons of the decisions made etc.
The third too existed, Market Research Consulting. This is somewhere between the first two. This is about knowing Market and competitors, and planning ahead to catch the opportunity that Market throws. TNS and AC Nielson are the major players in this area.
All three have existed and done well. They did not compete but worked together to help companies grow. But they never talk with each other. This has created a huge problem for the long term prospects of the company.
Every company waits for the issue to come and then call for the decision management service. There is however no sustainable solution. And each time one issue gets over, another problem is awaited.
Next comes knowing the operations in a better way and identifying the major decision points. This follows replicating the profitable decisions and eliminating the unprofitable ones.
Once this phase is overcome, the stage is ready to understand the market and the competitors to come up with highly optimized decisions.
At the end, when everything is streamlined and understood. The forward looking thought needs to be formed with all the necessary information ready. This is the stage for calling the Management Consulting services.
Until this process is not implemented by the companies today. Every year they will face a new problem in their business and will have to call Management Consultants to find the escape route. And the Management Consultants go away with a huge fee showing companies a quick solution, which in itself is based on lack of actual information of the existing business.
Wednesday, February 13, 2008
Decision Management is all about People
ERP by its name itself suggest the planning of existing resource. It is Resource Management Business while EDM is about using the resource smartly by taking optimal decision. So DM is a Knowledge Business.
Here I am trying to challenge his hypothesis that exist in the market.
"EDM can be implemented like ERP"
This will be a great mistake. Better Software can be chosen for ERP but for EDM to be successful that is not enough. You need better people.
Many of the companies tend to buy Enterprise Analytics Software and EDM Tool, and establish a in-house Analytics division straightaway with out experienced people. This is creating two type of problems in the industry -
1. Lack of Technical Know-how
The teams are not at all capable of Analyzing Data and fail to prove the significance of the team. A good analytics team should have people with business knowledge, statistical knowledge and the relevant analytics functional experience. If one lacks, the result is unlikely to be positive.
2. Over Expectation from Board
When people don't have enough knowledge about what can be done and what cannot be done, they tend to over expect.
Most of the time, people in the Management board decide to go with EDM system to get some advantage of the system and implement it. But they are unaware of the unavailability of the good data collection system. This creates huge tension to the Analyst, if he is not well equipped with proper knowledge of Data necessity and the Analytical functions. So he fails and the team too.
So be careful while forming a BI Team, have people first and software second. Else ready for the negative ROI from the BI Team.
Tuesday, October 16, 2007
All About Decision Management System
1. Collect and Explore data
2. Understand and Organize Data
3. Design Data Repository and Implement it
4. Build Reporting System (MIS)
5. Analyze Data for hidden insights
6. Perform Predictive Modeling and Segmentation of Portfolio
7. Design Strategy Based on Analysis, Modeling and Segmentation
8. Build and Implement Decision Optimization System
9. Monitor the performance of DO System
10. Fine tune the DO System with feedback from Monitoring System
I am a strong believer of Good Data. Analytics all depends on data, so before performing any analysis on data it has to be ensured that data is Analysis Ready.
Most of the resource intensive work is around data handling. So from points 1 through 4 are related to it.
If some company wants to start using DM system, then its best to follow steps one by one.
#1. Collect and Explore Data: Collect all relevant data either from internal source or from external sources. External sources may be getting information from Data Bureaus or Market Research firms or from partner companies, more the merrier. Then comes exploration of these data collected and finding the necessary ones. All data cannot be analyzed due to resource crunch and it does not make any sense to analyze useless ones.
# 2. Understand and Organize Data: The data in raw form is useless. It needs to be transformed into usable form. So a thorough data study has to be done to understand the business interpretation. Then the knowledge has to be used to organize data for Analytic use. This may include creating derived variables, converting files to usable forms. The usable form may be creating csv files from SQL server in order to convert to SAS (or whichever Analytic Package is used). The Quality Assurance test will need to be transformed in each step, either it is reorganizing data or converting formats.
Also, there might be need to collect some data for future use. Identify such shortage of data.
# 3. Design Data Repository and Implement it: Every time need for Analytics arises performing data works becomes very costly and those data works are mostly common between those functions. So maintaining a good data repository system pays.
Data Repository System should be designed in such a way that all the problems in the first two steps are phased out. The learnings from data exploration and organization can be implemented into an automated system.
If some new data needs to be collected, then build a module in DDR to do that function.
# 4. Build Reporting System (MIS): Data Collection and Maintenance is almost done with the earlier three steps, and now comes reporting. Data in raw form, however good, is hardly interpretable in business. So build a system to report basic insights from data.
Banks may like to know daily cash flow, number of customers visiting daily, rate of increase in credit, number of accounts going charge-off daily etc. There may be hundreds of such metrics that can be directly found out on daily basis and its all repetitive. Such insights when handy can help make many business decisions fairly easy.
# 5. Analyze Data for hidden insights: Not all information data can say is readily available. There might be some hidden reason when some outcome change and it needs to performed on need basis. This might be done routinally in order to avoid sudden accident, or can be done after a alarm.
This might also be needed when a new portfolio program is to be started, or some redesigning is required vis-a-vis competition.
# 6. Perform Predictive Modeling and Segmentation of Portfolio: To take strategic decisions, predictions are always important. It can be best done by segmentation and predictive modeling.
# 7. Design Strategy Based on Analysis, Modeling and Segmentation: Performing the statistical functions does not directly relate to business. It has to be put into business understanding and a strategy needs to be designed. It should include not the Analysis output and Model scores but Business Rules based on those outputs. Each and every decision points need to be evaluated and decisions need to be optimized w.r.t the desired outcome(s).
The desired outcome may be income, revenue, risk, attrition, response etc. based on the business need.
# 8. Build and Implement Decision Optimization System: After the strategy designed, its the time for it to be put in action. In the competitive environment, there might be need to change the strategy quite frequently following the change in market conditions and competition. So each time a different strategy is adopted, putting a new decision system makes it almost impractical in terms of both time and resources. This calls for the need of a Decision Optimization System that allows to change decisions often.
Two of the features of good decision management system can be:
a. creating, changing and removing business rules
b. fine-tuning the operating points for predictive model scores
# 9. Monitor the performance of DO System: Continuous monitoring of DO system is as important as building it. There might be various reasons to do it. The change in market conditions is one, while few business rules and models may have error in them or there might be error in their implementation. But most commonly the need is of change of business need than anything else. There may be many operational issues (resources crunch, lack of expertise etc) while trying new strategy.
# 10. Fine tune and upgrade the DO System with feedback from Monitoring System: The information collected from monitoring system can be used to fine-tune decision points as well as to upgrade the system. No system is perfect and it is hard to find the drawbacks prior to actual implementation, so it is important to follow the path of continuous improvement.
Now, its your turn to say. I have said all I had, and I expect comments here. Please share if you find that I missed to include something or I missed the actual path.
Thursday, August 2, 2007
Behavioral Targeting -- follow up from Avinash
Premature use of BT Software may lead to various problems as a consequence of poor or no data, and no proper plan or skill to use the results.
The action based on the BT results can be the change in marketing source change (mode of advertising) or actualy change in website (content change, content location change etc). Each is a outcome of focussing something and discarding something. These strategic move may come by the dominance of customer segment by volume or quality.
You need to be aware for not doing one of these mistakes.
1. Loss of important but unprofitable segment: The segment that is found to be unprofitable (in direct calculation) may be profitable indirectly. They may be strong in spreading the news or good views about the company or product etc. They may even be a segment whom the other mass follow. Like for my own blog, major source of traffic is due to the inclusion of web analytics discussions while I am not a web analyst (but I learned it later from blogger friends and their books). There is huge chance that if I remove web analytics topics, I will lose much traffic. And this is not because that majority of the readers come for web analytics topics in my blog but the source from where they come are web analyst friends blogs referal links. (I could find this in the Google Analytics report.) In this scenerio, BT study may suggest something where the business benefit lies somewhere else, and the problem is not with the BT software but with the user (weaknesses listed above).
2. Over-reacting to profitable segment: This is as worse as above. If I like pink rose, it not necessarily mean I like a pink car. To find out accurately what customers want and what not, proper data is not enough but sufficient data is. This justifies Avinash's point 1.
3. Portfolio Moves ok, but how about market move: I am not sure how sophisticated are today's BT tools, but I expect them to fairly show the portfolio movement and give good segmentation picture. Testing of software and the method of using the BT software outcome can be done as Avinash has pointed out in his point 2. But the question lies, is that enough?
My take is, it is not. Both the methodology and the software need regular testing and upgrades. Market is moving faster than ever before, the technology changes, the macro-economic fundamentals change, and new products and process evolve almost everyday. This asks for us to be more prone to change than ever. We need a more advanced system in the likes of Enterprise Decision Management (the link follows James Taylor's blog where he writes on EDM), but I am not sure whether EDM is available for website management. If its not available, I am sure it will come sooner than later. In the absence of such system, we will need very skilled people and frequent testing infrastructure. Atleast once a month testing of software and the methodology is required (which I found practical and relevant while working for sub-prime loan market earlier). This is more important if the software vendor is not very proactive. I beg to differ with Avinash here.
This is all I have to say. And now its your turn to say. Please share your views in comments.