Science & Technology > Managing information & knowledge >
|
Data Mining for Effective Decision Making |
| Paper ID: |
517 |
Last updated: 31/01/2012 09:08:31
|
Criteria:  |
Impact:  |
Likelihood:  |
Controversy:  |
Where: Global |
When: 3-10yrs |
How Fast: Years |
0
people thought this paper expanded their thinking
|
| Keywords: |
Information and Knowledge - mathematics, economics |
Summary  |
|
Discussion |
Mathematical tools for discovering patterns in large databases, along with stochastic modeling, could contribute to better decision making in a range of fields.
Technologies are increasingly available for collecting and storing massive amounts of data, whether about the movement of the equity markets, drug treatment outcomes, or changing air quality. Mathematical techniques will increasingly enable mining of this data to provide a basis for making effective decisions.
In the financial world, for example, large quantities of high-frequency data need to be processed effectively and efficiently so that opportunities can be quickly identified and acted upon 24 hours a day, 7 days a week. Rapid implementation of new mathematical models and algorithms, along with an expanding computing infrastructure, has the potential to meet this need. Statistical pattern recognition could be used to identify arbitrage opportunities and improve speculative trading. Integration of techniques from mathematical finance and insurance could lead to new products: catastrophe bonds, futures, and options; equity-linked life insurance; credit risk derivatives; and mortgage-backed securities. More financial decision making could be automated and made more effective by superior models, analytical tools, and processing systems.
Advances in data mining are also likely to be made in the areas such as national security and crime detection. Advances will possibly come from the development of human-machine-synergistic methods, designed to maximize the different abilities of people and machines. This will require collaborative work between mathematicians and computer scientists and psychologists, in many ways parallel to the existing collaborations between mathematicians and computers scientists and biologists and chemists on projects such as genome mapping and pharmacological research.
The highly visible collapses of Enron and LTCM have brought mathematics-based business into disrepute. In the case of Enron, however, the problems seem to have been associated with people and culture rather than with the underlying equations. |
Implications |
More effective decision making based on analysis of large quantities of data
Expansion of the array of financial products on the market |
Early indicators |
Current application of transaction analysis for fraud detection and product recommendation systems over large data sets
What To Watch:
Data-mining techniques are applied to massive collections of environmental sensor data to find causal relationships between industrial pollutants and global warming.
Data mining is applied increasingly to the massive amount of data associated with health care, concerning patients, procedures, and drug treatment outcomes.
Mathematicians and physicists are hired at an increasing rate by investment banks and traders.
Leaders:
Institutions:
Department of Homeland Security (the proposed Analysis, Dissemination, Visualization, Insight, and Semantic Enhancement system)
National Security Agency
KDNuggets, a leading site and newsletter on Data Mining and Knowledge Discovery [<http://www.kdnuggets.com/>link]
Autonomy, world-leading data mining software spun out from Cambridge University
University of Namur, Belgium
University of Technology, Sydney, Australia
Wessex Institute of Technology
European Bioinformatics Institute
University of Kent
University of Manchester, School of Informatics
UK e-Science Data Mining Special Interest Group |
Drivers & Inhibitors |
Effective testing of mathematical models in real-world environments
Growth of grid computing
Development of massive distributed storage devices
Continued development of stochastic algorithms, data-mining and knowledge discovery algorithms, and modeling languages |
Parallels & Precedents |
| Enron's and Long Term Capital Management's attempts to combine academic modeling of financial markets with trading expertise, despite the ignominious end of both enterprises.[1][2][3][4][5][6][7][8][9][10] |
Social Bookmarking |
|
|
Comments |
|
People who viewed this paper also viewed
|
|
|
Sources |
| Ref. | Publisher | Date | Title | Category |
| 1 | Other | | The Data Mine  | Tech |
| 2 | Other | | Case Study: LTCM - Long-Term Capital Management." ERisk.com. [<http://www.erisk.com/Learning/CaseStudies/ref_case_ltcm.asp>link]" | Tech |
| 3 | Other | | NOVA: Trillion Dollar Bet." PBS. Aired 8 February 2000. [<http://www.pbs.org/wgbh/nova/transcripts/2704stockmarket.html>link]" | Tech |
| 4 | Other | | Data Mining Techniques (statistical explanation) | Tech |
| 5 | Other | | Basterfield, David. Directions in Computational Finance." OGI. [<http://www.cse.ogi.edu/2020vision/Basterfield.pdf>link]" | Tech |
| 6 | Other | | Mathematics: Giving Industry the Edge." Faraday Partnership for Industrial Mathematics (April 2002). [<http://www.smithinst.ac.uk/News/Roadmap/Roadmap/>link]" | Tech |
| 7 | Other | | McLean, B. and P. Elkind. 2003. Partners in Crime." <i>Fortune.com</i> 27 Oct. [<http://www.fortune.com/fortune/investing/articles/0,15114,517415,00.html>link]" | Tech |
| 8 | Other | | Data Mining and Discovery." American Association for Artificial Intelligence. [<http://www.aaai.org/AITopics/html/mining.html>link]" | Tech |
| 9 | Other | | Kargupta, H., A. Joshi, K. Sivakumar, and Y. Yesha. Abstract of <i>Data Mining: Next Generation Challenges and Future Directions</i>" UMBC [<http://www.cs.umbc.edu/~hillol/Kargupta/ngdmbook.html>link]" | Tech |
| 10 | Other | | Agosta, Lou. The Future of Data Mining -- Predictive Analysis." <i>DMReview magazine</i> (August 2004). [<http://www.dmreview.com/article_sub.cfm?articleId=1007209>link]" | Tech |
|
Other Sources
|
|
|
|
|
|
|
| The contents of this paper were supplied by the Institute for the Future, and have been reviewed by the Outsights-Ipsos MORI Partnership. Any views expressed are independent of government and do not constitute government policy. |
|