Business Intelligence (BI) for the Masses Comes Alive

Microsoft is now marketing new business intelligence (BI) integrations between Excel 2010 (with PowerPivot), SharePoint Server 2010, and SQL Server 2008 R2. By tightening the integration between Excel, SharePoint, and SQL, Microsoft believes that “self-service” BI will finally become a reality resulting in a dramatic increase in the adoption rates of BI technologies.

Microsoft believes the future of enterprise BI is about making everyone a BI practitioner with familiar and affordable tools…. Microsoft hopes to push BI further into the enterprise… by giving end users easier access to the information they need and connecting it to the decision-making process through collaboration tools. The company seeks to do this by integrating SQL Server 2008 R2 with two tools that most users already feel comfortable using: Microsoft Excel 2010 and Microsoft SharePoint Server 2010…. Upgrading to SQL Server 2008 R2 will enable the company to eliminate some third-party BI tools it had previously been trying to use, saving hundreds of thousands of dollars in licensing fees per year, as well as the cost of managing complicated packages that were less efficient…. Today’s release is a step toward bringing BI to the masses. Some 500 million people currently use Microsoft Office, which means 500 million potential BI users. If Microsoft is able to penetrate just 5 percent of this market, that is 25 million new BI practitioners. With SQL Server 2008 R2, PowerPivot for Excel, and PowerPivot for SharePoint Server, Microsoft is making businesses more agile and productive, ultimately allowing end users to drive better business decisions.

Given the ubiquity of Excel, the ingenuity of SharePoint Services, and the arrival of a more versatile version of SQL Server, it seems that Microsoft is now a real contender for leadership in the BI marketplace. The truth is that most people already use one or more of these application already. Moreover, the layered simplicity of Microsoft's upgraded offerings holds strong appeal for CIO's eager to control costs.

Nevertheless, Microsoft's goals are ambitious, particularly if it intends to confront and reverse the current low-adoption rates for BI technologies to date. I'll be keeping an eye on the comments of new BI users to see if Microsoft's solutions are making headway. More to follow...

Source: Microsoft Brings Business Intelligence Deep into the Enterprise with SQL Server 2008 R2 (2010, April 21), Microsoft News Center.

Securitization versus Market Making

Prof Emanuel Derman proposes that "the creation of content and its delivery are better off separated."

The world gets uncomfortable when dominating amounts of creation and flow are controlled by the same person or company.

Thus, Amazon can dominate the selling of books but it will be bad if they take over publishing, as rumor has it they would like to do. Similarly, Hollywood shouldn't control the TV channels, to take a very outdated example. And securitization and market making should be done by different companies too.

There is perhaps a sort of anti-Heisenberg principal (anti because it has an upper bound rather than lower bound on the right hand side) that says that if the product of creation and flow controlled by one entity (person or company) gets too large, it becomes dangerous from the point of view of both that one entity and from the point of view of the entities it serves:

[Principal] [Agency] < h

[Creation] [Distribution] < h

where h is a constant, say (30%)^2 that shouldn't be exceeded. You can have a lot of one and a little of the other, or vice versa, but not a lion's share of both.

The notion of separating securitization from market making is intriguing from a regulatory standpoint...

Source: Derman, E (2010, April 24), Lessons of the Fall, Emanuel Derman's Blog on Wilmott.

Chauffeur or Ferrari?

I commend this pair of brilliant articles by Dr John D Cook of The Endeavor entitled, Would You Rather Have a Chauffeur or a Ferrari? and Chauffeurs and Ferraris Revisited. Be certain to read the comments that follow each article, including those by Dan Bricklin of VisiCalc (Electronic Arts) fame.

As for me, I choose the Ferrari…

Too Big...

From my perspective, financial institutions that are "too big to fail" are likewise "too big to manage," "too big to regulate," and thus "too big to keep around." The US should proceed with comprehensive financial reforms and not look back!

Statistics by Amateurs

According to Ray Fisman of Slate, "It's a sad statistical reality: Half of us are below average." Well, I regret to inform Fisman and the editors at Slate that this claim is emphatically not "reality." Recall that "average" is also known as the "mean," which is the value derived by dividing the sum of a set of quantities by the number of quantities in the set. The "median" is the numeric value separating the higher half of a sample, population, or a probability distribution, from the lower half. The "mode" is the value that occurs the most frequently in a data set or a probability distribution. Fisman is confusing the "average" with "median" in his assertion.

Source: Shor (2010)

Consider the average per capita incomes in the US and The Netherlands as depicted above. The distributions show that the average income in the US is $36,092, while the average income in The Netherlands is $32,972. In other words, the average income in the US is higher than in The Netherlands. However, the median income in the US is $22,960, while the median in The Netherlands is $28,032. In other words, workers in The Netherlands generally earn more than workers in the US. Finally, the mode or most common annual income in the US appears to be around $20,000, while the mode in The Netherlands is clearly higher at an amount above that of the US.

The error that Fisman makes in his assertion that "half of us are below average" is a banal misconception, and demonstrates why analytical "amateurs" (in every field of endeavor) should take greater care with their use and interpretation of statistics.

Sources:

Fisman, R (2010, April 23), Nudges Gone Wrong, Slate.

Shor, D (2010, January 16), A Quick Trick for Approximating Median Income, Stochastic Democracy.

Related Posts:

The Flaw of Averages

Exotic Investments: Want a Taste...?

For those interested in exotic investments, here's a copy of the original offering perspectus for ABACUS 2007-AC1, LTD (incorporated with limited liability in the Cayman Islands) from Goldman Sachs. Want a taste...?

ABACUS Offer Document

The Inside Story at Goldman Sachs...?

Is this the inside story about what really happened at Goldman Sachs...?

Tetsuya Ishikawa worked for Goldman Sachs until 2007 when he joined Morgan Stanley. Ishikawa’s name appears on the preliminary term sheet for the Abacus 2007-AC1 deal, a collateralized debt obligation (CDO) the US Securities and Exchange Commission accuses Goldman Sachs of using to commit fraud. I found this quote from Ishikawa’s book intriguing:

Well aware of the orgy of scape-goating that the world had embarked upon, I didn’t want to blame anyone, let alone run the risk of over-burdening some and under-burdening others with responsibility for the crunch. After all, I’m not one to say who’s guilty or not, and ultimately it doesn’t help us to move on anyway. (2009, pp. 2-3)

Follow the link below to learn more.

How I Caused the Credit Crunch

Let's Dance

by Joy Broe

In her mixed-media paintings, Joy Broe alternates layers of rich textures, metals, paints and hand-made finishes on wood panels or watercolor paper. Many of her custom works use several layers of oils and acrylics combined with copper and brass to achieve the desired texture and depth—the wood panels are then finished in gallery wrap style designed to hang without frames. Her strong points are design and color, and rather than attempt to capture reality, she strives to impart a sense of tranquility and optimism through her unconventional and distinctive abstract imagery.

"Let's Dance" (48" x 60") by Joy Broe hangs in my home...

Training New Physicians in the US

According to the Association of American Medical Colleges (AAMC), the number of doctors graduating from our nation's medical schools has been essentially flat since 1981, with 15,632 medical doctors graduating in 1981, and 16,167 graduating in 2008. The average annual production of new physicians for the period 1981-2008 was 15,819 per year.

The economist in me looks at this information and concludes that given the scarcity of physicians' services today, it is no wonder that the cost of medical care has risen sharply. The law of supply and demand makes this so -- more demand with no more supply results in higher prices -- the invisible hand of economics ensures this.

Over the past year, Americans have been exposed to almost constant debate regarding how to divide up and pay for the increasingly scarce supply of medical services. However, there has been little to no discussion about how America might expand its capacity to deliver medical care to the nation. Again, the economist in me is attracted to the idea of increasing the supply of medical services not only to meet current demand, but also to increase competition and eventually lower the cost of delivering healthcare to citizens.

America needs more good doctors to care for our sick and ill. If society must choose between building more medical schools versus sending more astronauts to the moon, then I for one would vote for more medical schools. In the mean time, our nation would benefit from a public debate about how to expand the healthcare industry in order to meet future demand while reducing costs. Part of that debate should focus on how to train more physicians.

Financial Services and Banking are in Desperate Need of Reform at the Top

On April 16th, the US Securities and Exchange Commission (SEC) filed a complaint against Goldman Sachs in US District Court (linked below) alleging that the firm defrauded customers by selling investments in subprime mortgages while surreptitiously betting against the same investments in separate transactions. The SEC's complaint additionally alleges that Goldman’s conduct “contributed to the recent financial crisis by magnifying losses associated with the downturn in the United States housing market…”

Goldman Sachs is arguably the most powerful and visible investment bank in the world, and so the ramifications of the SEC’s charges will likely tarnish not only the image of Goldman Sach’s, but that of the entire US banking industry. The SEC’s complaint also raises serious questions and concerns about the nature and character of banking as a profession.

Goldman Sachs Tower, Jersey City, NJ

I remain dumbfounded by the apparent current state of financial services and banking in the US. Clearly, the banking industry is in desperate need of reform at the top.

SEC Complaint

From My Yard...

From My Yard Today... (click to enlarge)

The Flaw of Averages

For those who seek to "solve" the problem of risk through the calculation of a single numerical value (as in the mean, median, mode, variance, standard deviation, alpha, beta, power, value at risk, etc), I commend this video by Dr Sam L Savage, author of The Flaw of Averages (2009, John Wiley):



To understand and evaluate risk, decision-makers must consider a multitude of fundamental, descriptive, and inferential measures and indicators in comcomitance. The potential for misspecification (as in model risk) and misinterpretation (as in naivety) are great. Thus, risk management (including analysis) should be viewed and treated as a polyvalent task that defies commoditization and delegation in enterprise.

From Ledgers to Electronic Spreadsheets

There was a time through the 1970’s when spreadsheets were synonymous with ledger books and paper as shown below. Of course, you can still purchase ledger paper at any office supply store, and many accountants still use ledgers for light bookkeeping and budgeting tasks.

Ledger Paper

However, by the early 1980's the world marveled as the first electronic spreadsheets found their way onto the personal computers of the day. My introduction to electronic spreadsheets began with VisiCalc (Electronic Arts). Below is a screenshot of VisiCalc as it would have appeared on my Apple //e computer back in 1983. At the time, VisiCalc amazed me with its capabilities and potential. Remarkably, VisiCalc still runs on today’s computers (follow the link below for instructions and a free download of the program).

VisiCalc

VisiCalc (Electronic Arts)

VisiCalc was soon followed by the introduction of Lotus 1-2-3 (Lotus Development), which by the mid-1980's was the best selling electronic spreadsheet program on the market. Lotus 1-2-3 (shown below) boasted improved speed and functionality over VisiCalc, as well as the addition of integrated graphing capabilities.

Lotus 1-2-3 (Lotus Development)

However, the emergence of Windows (Microsoft) in 1987 was quickly joined by the arrival of Excel (also Microsoft) as shown below. By the mid-1990's, Windows and Excel had become industry standards in the fast-growing technology sector.

Excel 2.1p (Microsoft)

The spreadsheet technology available today has certainly come a long way since VisiCalc, Lotus 1-2-3, and the early editions of Excel. The latest version of Excel (shown below) now reigns as the the most ubiquitous software application in the world, and is widely regarded to be an essential business intelligence and personal productivity tool with unparalleled capacities to support problem-solving and decision-making.

Excel 2010 (Microsoft)

I have omitted a number of other less popular spreadsheet applications from this short history. However, all spreadsheet programs (including Excel) trace their lineage to the paper ledgers of years past. Some say that spreadsheets have seen their day and will eventually be replaced by something new. Perhaps, but I suspect that whatever comes next will still somehow resemble plain old ledger paper.

Business Intelligence Requires Thinkers

The emerging turf war between information technology (IT) departments and business intelligence (BI) analysts is reaching a crescendo as the vanguards of these interest groups meet in the void between. Clearly, the topological space between technologists and analysts has become blurred as the commoditization of knowledge assets into technology encroaches into the vital knowledge domains of subject matter experts and professionals. At stake is the future of vast human and structural capital formations, and while IT departments continue to herald BI as a technology function, the voices and concerns of business analysts are starting to be heard in the executive suite, especially as BI becomes a form of competitive advantage between firms in the new millennium. Arthur Ritchie of SAND Technology concludes that analysts will require greater access to controlled data resources in order for BI to fulfill its potential:

In my view, unless talented analysts are given unfettered access to whatever corporate data they require and the ability to analyze it as they see fit in the context of the many external data sources that are available, we will continue to find ourselves unprepared to deal with the unexpected. Implemented correctly, corporate Business Intelligence (BI) systems can support an organization’s best analysts as they challenge traditional business dogmas and develop a practicable way forward based on the facts, as recorded in detailed corporate data. To achieve this, IT departments need to stop acting as “data jailors” who strictly control which data will be accessible, in what form, and start empowering creative thinkers to realize their maximum potential, be it in marketing, manufacturing, distribution or some other field. In order to do this, however, IT departments need to start acting more like a power utility service: enabling “decision support” (to revive an older term for BI) by providing corporate information or raw data as required, in the right amounts at the right time, while also serving as “consultants” who help end users access the data they require, when and how they need it.

Ritchie’s central argument is that BI systems must work to support BI production as defined by analysts’ requirements. The fact is that BI is not only a production process that requires systems, but also a thinking process that requires both ad hoc and post hoc analysis and testing by subject matter experts. The future of BI requires restoration of the decision support function. Moreover, analysts rather than technologists must assume greater responsibility and leadership over the overall BI effort.

Source: Arthur’s Blog

Evidence of Indeterminism

As a graduate student during the 1980's, I became fascinated by the colorful, but seemingly chaotic wave patterns exhibited by electricity in plasma bulbs. I soon concluded that the eratic wave patterns provided intuitive evidence of indeterminism in nature (though not proof). Thirty years later, I am still persuaded that indeterminism trumps determinism as a guiding philosophical proposition. This conclusion eventually shaped my views and approach to financial economics leading to an advocacy of stochastic modeling methodologies in research and practice.

I recorded the video that follows using a plasma lamp from my study and post it here for others to view and ponder. Notice how the waves respond as my finger touches the globe.



Follow the link below for primers on determinism and indeterminism, as well as metaphysics.

Dialogos of Eide