Showing posts sorted by relevance for query data science. Sort by date Show all posts
Showing posts sorted by relevance for query data science. Sort by date Show all posts

Friday, August 12, 2011

Business Intelligence (BI) versus Data Science (DS)

Steve Miller of Information Management offers this summary of the key distinctions between business intellience (BI) and data science (DS):
Below is a summarization of what I currently see as some of the differences between DS and BI. I consider the observations preliminary, more gray than black and white. My hope is that in time the “best practices” of BI will weigh positively on DS – and vice-verse. Just as I think BI will benefit from a deeper focus on approximate answers and ubiquitous machine learning, data science should appreciate what's been learned over the years in BI on methodology and governance.
Business Intelligence (BI) versus Data Science (DS)

Source: Miller, S (2011, May 3), Data Science, Part II, Information Management.

Related Posts

Saturday, May 18, 2013

On Philosophy, Science, and Data

According to Jim Harris (2013) of the Obsessive-Compulsive Data Quality (OCDQ) Blog:
Some might argue that philosophy only reigns in the absence of data, while science reigns in the analysis of data. Although in the era of big data there seems to be fewer areas truly absent of data, a conceptual bridge still remains between analysis and insight, the crossing of which is itself a philosophical exercise. So, an endless oscillation persists between science and philosophy, which is why science without philosophy is blind, and philosophy without science is empty. Data needs both science and philosophy.
Read More

Jim Harris

Let's face it, the historical relationship between science and philosophy has not always been friendly. Nevertheless, one cannot separate philosophy from science and still make sense of the world.

Source: Harris, J (2013, March 14), On Philosophy, Science, and Data, OCDQ Blog.

Related Posts

Saturday, March 13, 2010

Why Policymakers Need to Take Note of High-Frequency Finance

by Richard Olsen © VoxEU.org

Why should high-frequency finance be of any interest to policymakers interested in long-term economic issues? This column argues that the discipline can revolutionise economics and finance by turning accepted assumptions on their head and offering novel solutions to today’s issues.

I believe high-frequency finance is turning aspects of economics and finance into a hard science. The discipline was officially inaugurated at a conference in Zurich in 1995 that was attended by over 200 of the world’s top researchers. Since then, there have been a large number of publications including a book with the title Introduction to High-frequency Finance. “High-frequency data” is a term used for tick-by-tick price information that is collected from financial markets. The tick data is valuable, because they represent transaction prices at which assets are bought and sold. The price changes are a footprint of the changing balance of buyers and sellers.

The term “high-frequency finance” has a deeper meaning and is a statement of intent indicating that research is data-driven and agnostic. There are no ex ante theories or hypotheses. We let the data speak for itself. In natural sciences this is how research is often conducted. The first step towards discovery is pure observation and coming up with a description of what has been observed – this may sound easy but is not at all the case. Only in a second step, when the facts are clearly established, do natural scientists start formulating hypotheses that are then verified with experiments.

In high-frequency finance:
  • The first step involves the collecting and scrubbing of data.
  • The second step is to analyse the data and identify its statistical properties.
Here one looks for stylised facts which are significant and not just spurious. Due to the masses of data points available for analysis (for many financial instruments one can collect more than 100,000 data points per day), identification of structures is straightforward, either there is a regularity or there is none.
  • The third step is to formalise observations of specific patterns and seek tentative explanations, theories to explain them.
The abundance of data in high-frequency finance has profound implications for the statistical relevance of its results. Unlike in other fields of economics and finance, where there is not sufficient data to back up the inferences, this is not an issue in high-frequency finance. The results are unambiguous and turn economics and finance into a hard science, just as is the case for natural sciences. This is not a bad thing.

High-frequency data as an answer to singularity of macro events

Today we are all grappling with the global financial crisis and have to make hard decisions. In living memory, we have not seen a crisis of a similar scale, so policymakers are in a vacuum and do not have any comparable historical precedents to validate their policy decisions.

If the global economy had been in existence for 100,000 years, this would be a different matter. We would have had many crises of a similar scale, and we could use these previous events as a benchmark to evaluate the current crisis. The modern economy with financial markets linked together through high speed communication networks trading trillions of dollars on a daily basis is a new phenomenon that did not exist even 20 years ago. People refer to the events of 1929 and subsequent years, but while these events can be used as one possible point of reference, they are not meaningful in the statistical sense. On a macro level, we can make observations but no inferences because we do not have the historical data. There is a void that researchers and policymakers need to acknowledge.

Fractals: Understanding macro structure from micro data

High-frequency finance can fill the void with its huge amounts of data – if we embrace fractal theory that explains how phenomena are the same even if they occur at different scales. Fractal theory suggests that we can search for explanations of the big crisis by moving to another time scale, the short term.

At a second-by-second level, there are an abundance of crises and systemic shocks; just imagine the occurrence of the many price jumps due to unexpected news releases and political events or large market orders. Albeit on a short-term time scale, we study how regime shifts occur and how human beings react. The large number of occurrences allows for meaningful analysis. We study all facets of a crisis, how traders behave prior to the crisis, how they react to the first onslaught, how they panic, when the going gets hard and finally, how their frame of reference which previously was a kind of anchor and gave them a degree of security breaks down and how later, when the shock has passed, the excitement dies down, there is the aftershock depression and then eventually how gradual recovery to a new state of normality begins.

The everyday events sum up and shape the tomorrow

High-frequency finance has another big selling point, one which policymakers should take note of: the study of market events on a tick-by-tick basis brings to the surface the detailed flows of buying and selling that occur in the market. From this information, it is possible to build maps of how market participants build up positions and how asset bubbles develop over time. By tracking price action on a tick-by-tick basis, it is possible to infer the composition of those bubbles similar to the work of geologists studying rock formations. Researchers can identify, who has been buying and selling, on what time horizons they trade, how resilient they are to price shocks, what makes them turn their position and become net sellers as buyers. Based on this information we can make inferences of the likely collapse of those bubbles.

High-frequency finance opens the way to develop "economic weather maps". Just as in meteorology, where the large scale models rely on the most detailed information of precipitation, air pressure and wind, the same is true for the economic weather map. We have to start collecting data on a tick-by-tick level and then iteratively build large scale models. Today, the development of such a global economic weather map has barely started. The "scale of market quake" (a free Internet service) is a first instalment, but the start of an exciting development.

High-frequency finance holds out the hope of turning aspects economics and finance into a hard science by the sheer volume of data and its ability to set events into their appropriate context by mapping rare events into a short-term time scale with a near infinity of events, albeit at a shorter-term time scale. Second, the tracking of events on a tick-by-tick basis opens the door to identify underlying flows and develop economic weather maps. Surely that’s not a bad thing?

References

Bisig T, Dupuis, A, Impagliazzo, V, and Olsen, R (2009), “The scale of market quakes”, working paper, September.

Gençay, R, Dacorogna, M, Müller, U, Olsen, R, and Pictet, O (2001), An Introduction to High Frequency Finance, Academic Press.

Mandelbrot, B (1997), Fractals and Scaling in Finance, Springer.

Mandelbrot, B, Hudson, R (2004), The (Mis)behavior of Markets, Basic Books.

Republished with permission of VoxEU.org

Wednesday, May 15, 2013

Business Intelligence (BI) versus Data Science

David Smith at Revolutions (2013) compares business intelligence (BI) with data science as follows:


Read More

The discipline of analytics is constantly evolving, or so it seems...

Source: Smith, D (2013, May 15), Statistics vs Data Science vs BI, Revolutions.

Related Posts

Friday, August 07, 2009

Statisticians in Demand

We live in a world where data is the raw material that analysts use to produce information, also known as knowledge. However, without analysis, data remains data in raw form. This might explain the recent upsurge in hiring for statisticians. Steve Lohr (“For Today’s Graduate, Just One Word: Statistics,” NYT, Aug 5, 2009) argues that statistics may be the password for hiring in the coming years:
The rising stature of statisticians, who can earn $125,000 at top companies in their first year after getting a doctorate, is a byproduct of the recent explosion of digital data. In field after field, computing and the Web are creating new realms of data to explore — sensor signals, surveillance tapes, social network chatter, public records and more. And the digital data surge only promises to accelerate, rising fivefold by 2012, according to a projection by IDC, a research firm.

The demand for statisticians is consistent with a larger trend toward competing on analytics in enterprise. This trend has also given impetus to the need for other experts, especially in computer programming.

Though at the fore, statisticians are only a small part of an army of experts using modern statistical techniques for data analysis. Computing and numerical skills, experts say, matter far more than degrees. So the new data sleuths come from backgrounds like economics, computer science and mathematics.

Over the past several decades, firms have invested heavily into data management technology, including server and data-warehousing systems. These investments have created massive amounts of raw data that are begging to be analyzed by people trained and skilled in descriptive and inferential statistics, stochastic modeling, linear and non-linear forecasting, and so forth. The creation of so much raw data in recent years makes statistical analysis of that data a vital value-adding activity that enables competing on analytics.

“I keep saying that the sexy job in the next 10 years will be statisticians,” said Hal Varian, chief economist at Google. “And I’m not kidding.”

Saturday, January 16, 2010

Toward Norms for the Development of Models

Today, modeling and data issues pervade social science research. Likewise, the discipline of finance is now confronting issues of model risk. Assuming the nature of financial economics is similar to that of the social sciences, then perhaps the former might take a lesson from the latter. Prof W James Bradley and Prof Kurt C Schaefer offer the following “norms” in order that researchers might avoid the misue of models and data (1998, pp. 145-151):
1. For each situation under analysis, practitioners benefit from a detailed understanding of the situation, from different disciplinary perspectives when possible, before the problem is modeled.

2. Practitioners should learn what is important enough to measure before trying to measure it, then decide which of the five measurement scales [i.e., nominal, ordinal, interval, ratio, absolute] is appropriate. One must then live within the bounds imposed by the characteristics of that measurement scale.

3. Random error terms convey information about the abstractions, approximations, ignorance, and measurement problems that are involved in the model we have constructed. The drawing of inferences should therefore involve a careful inspection of the residual errors between our data and our model.

4. “Probabilities” in the social and human sciences are degrees of warranted belief, not relative frequencies. But this means that the classical ratio scale of probabilities is not appropriate in the situations we are discussing. Therefore, the level of confidence in a result cannot be stated as a single number; significance involves a judgment about the reasonableness of the entire model and its data.

5. Levels of statistical significance are always somewhat arbitrary, but we should be especially skeptical in cases when (a) the social processes under study are extremely complex, with many auxiliary hypotheses complicating the primary hypotheses; (b) the entity being measured is not clearly definable, or there is a poorly developed theory of the entity and its relationship to the measureable variables, or the measurement instrument is not precise and reliable; (c) inappropriate measurement scales are used; (d) the statistical methods (and, when present, functional forms) employed are not consistent with the measurement scale; (e) the specification of the model and its functional form are not clearly justified by reference to the actual situation being modeled; (f) the error residuals are not observed and analyzed (e.g., some ANOVA and correlation studies); and (g) the quality of the data and reliability of the source are questionable. We should be particularly skeptical when the analyst does not fully disclose the relevant information on these topics. In fact, it should be a professional norm that the statement of one’s results must, as a matter of habit, discuss these details.

Reference: Bradley, W J & Schaefer, K C (1998), The Uses and Misuses of Data and Models: The Mathematization of the Human Sciences, Thousand Oaks, CA: Sage.

Wednesday, May 15, 2013

Statisticians versus Data Scientist

David Smith at Revolutions (2013) compares statisticians with data scientists as follows:


Read More

The discipline of analytics is constantly evolving, or so it seems...

Source: Smith, D (2013, May 15), Statistics vs Data Science vs BI, Revolutions.

Related Posts

Monday, August 30, 2010

The Art and Science of Data Enjoined

Check out this fascinating talk by data journalist David McCandless. The art and science of data are increasingly being enjoined by technology.

Sunday, September 05, 2010

Business Intelligence = Data + Analytics

We might all find inspiration from IBM's recent advertising regarding the future of business intelligence, data, and analytics.



For more information, visit:

A Smarter Planet

Related Posts:

The Art and Science of Data Enjoined

Nature by Numbers

The Fourth Paradigm: Data-Intensive Scientific Discovery

Monday, November 21, 2011

Parsing Business Intelligence (BI)

Business intelligence (BI) is now a vital imperative of 21st century enterprise. Yet, a unified view about just what BI is and does for enterprise is only now emerging. Prof Ronald K Klimberg and Prof Virginia Miori (2010) address the challenge by parsing BI as follows:
Organizations, both corporate and academic, have been rushing to the table with their own BI groups and programs. Though these groups share a common title, BI, they do not share a common understanding of all that BI comprises. The establishment of their functions follows from the specific strengths and expertise within all of these organizations. This shared limitation was not based on a lack of inclusiveness, but merely a lack of cohesive vision. Further, consider that industry’s definition of business intelligence is by and large quite different from academia’s definition. More so, within industries and within academia, these definitions also vary. The definition of BI seems to depend heavily upon your particular perspective or training. What then is business intelligence...?

Despite the appearance of BI in both academia and industry, until now the field has lacked a clear definition. Not all aspects of BI will be exploited in every situation, but it is still important to know what the future holds. Within this structure, BI was broken down into three significant areas: business information intelligence (BII), business statistical intelligence (BSI) and business modeling intelligence (BMI). Specialists exist in all of these areas, but the importance of the intersection and unions of these areas needs to be emphasized. True intelligence results from the melding of all of these technologies and tools....

Figure 1 presents a cohesive vision of business intelligence as a melding of technologies, models, techniques and practices. The three circles of the Venn diagram each represent areas of study and application that had previously been considered quite distinct: 1. information systems and technology, 2. statistics, and 3. OR/MS [Operations Research/Management Science]. It serves to encapsulate the broadening definition of BI. With this new vision, we may now characterize BI from each of three viewpoints as: business information intelligence (BII), business statistical intelligence (BSI) and business modeling intelligence (BMI). Each of the viewpoints has particular business aspects, and academically speaking, courses that are independent of the other viewpoints. Conversely, each viewpoint can work together or utilize techniques/skills from one or possibly two of the other disciplines. For example, data mining, which requires a high level of statistical knowledge as well as the availably of necessary data, may require significant IT skills and/or knowledge. Further, if data mining analysis demands a systematic process of analysis, modeling skills may be required.

Business analytics (BA), within our framework, is classified as a combination of business statistics intelligence (BSI) and business modeling intelligence (BMI): BA = BSI + BMI. BI is the union of the areas of BA, BI and business information intelligence (BII): BI = BII + BA or more specifically BII + (BSI + BMI). As evidenced in the data-mining example, black and white distinctions between disciplines can quickly become gray.
Figure 1: Business intelligence/business analytics breakdown

Note that modeling is a central practice in BI. Follow the link below to read the entire article.

Read More

Source: Klimberg, R K & Miori, V (2010, October), Back in Business, INFORMS, 37(5).

Related Posts

Thursday, September 01, 2011

Imagining Mathematics

Read below the pedagogical proposition by Dr Sol Garfunkel and Prof David Mumford (2011) regarding pragmatic approaches for teaching and learning mathematics:
Imagine replacing the sequence of algebra, geometry and calculus with a sequence of finance, data and basic engineering. In the finance course, students would learn the exponential function, use formulas in spreadsheets and study the budgets of people, companies and governments. In the data course, students would gather their own data sets and learn how, in fields as diverse as sports and medicine, larger samples give better estimates of averages. In the basic engineering course, students would learn the workings of engines, sound waves, TV signals and computers. Science and math were originally discovered together, and they are best learned together now.
Mathematics is at once, symbolic, real, and conceptual for some -- I tend to agree. Dr Garfunkel is the executive director of the Consortium for Mathematics and its Applications (COMAP). Follow the link below to learn more about COMAP's activities.

COMAP

Source: Garfunkel, S & Mumford, D (2011, August 21), How to Fix Our Math Education, New York Times Online.

Related Posts

Thursday, January 17, 2013

Good Data Does Not Guarantee Good Decisions

According to Shvetank Shah, Andrew Horne, and Jaime Capellá in Harvard Business Review (2012):
Analytic skills are concentrated in too few employees. When a new form of analytics enters the workplace, companies typically start by hiring experts versed in using it, reasoning that the skills will trickle down to all. But too many companies are stuck in the “expert” phase. They have a handful of highly skilled analytics professionals but have not begun to train everyone else to make use of their analytics methodology.
Read More


Let's face it, the world needs more fully qualified data analysts in all fields. Those entering college should not hesitate to undertake degrees in business-related analytical disciplines, including actuarial science, finance, accounting, statistics, and risk analysis. Go for it!

Source: Shah, S; Horne, A; & Capellá, J (2012, April), Good Data Won't Guarantee Good Decisions, Harvard Business Review.

Related Posts

Tuesday, April 21, 2009

Valuing Business Intelligence

I have a theory I am pondering based on my reading of the emerging business intelligence literature. My hypothesis is that the more efficient and simple the analytical framework between the data and the decision, the more valuable the intelligence becomes. Said another way, if intelligence is what connects data to decisions, then the value of the intelligence increases as the analytical framework that supports and generates the intelligence is simplified. A related research question would be whether decision-makers who do their own analytical work make better decisions than those who rely on intermediaries. If I ever test the theory, I suppose the results could have implications for existing and emerging enterprise resource and risk management regimes, as well as management science in general. If you are contemplating a future research topic, this might provide you with a starting point.

Thursday, July 30, 2009

The Spreadsheet Reinvented

My father was an accountant, and I vividly recall his working on spreadsheets (also known as ledger sheets) on the dining room table late at night. He once gave me a short lesson about how to make proper entries onto a spreadsheet. I recall his emphasis on neatness and penmanship, selecting the proper pencil (my father preferred the Ticonderoga No. 3), and having a serviceable gum eraser nearby for making clean erasures. In the early 1980’s, he introduced me to electronic computing. At the time, VisiCalc was all the rage. Later, Lotus 1-2-3 came into vogue. Today, Excel is the most widely-used spreadsheet program in the world, and has almost completely replaced ledger paper in professional practice (though some small businesses still rely on paper ledgers to this day).

Of course, the essential features of the electronic spreadsheet are inherited from its paper lineage, and the ontological purpose of all spreadsheets is the same, regardless of whether one is using an electronic or paper version of the tool. Nevertheless, it's interesting to compare the paper spreadsheet to its electronic successor in an effort to better understand why the spreadsheet remains essential to the practice of finance and accounting.

Below, you will find a short definition of the term “spreadsheet,” together with two download files – one is a facsimile of a paper spreadsheet, the other an electronic spreadsheet. Once you have downloaded both files, place them side-by-side on your monitor screen and then ask yourself the following question: Can one replace the spreadsheet without reinventing the spreadsheet? I look forward to your comments.
spread·sheet n. 1. A piece of paper with rows and columns for recording financial data for use in comparative analysis. 2. Computer Science An accounting or bookkeeping program that displays data in rows and columns on a screen.
Paper Spreadsheet

Electronic Spreadsheet

Thursday, March 25, 2010

Financial Management Requires Polyvalence

A frequent problem in advanced financial management is the task of interpreting (and understanding) probabilistic forecasts. The following exercise illustrates the essence of the problem.

Suppose that the following (very simplistic) financial projection for the “next” period arrives in your hands:

The financial analyst who handed you the report then comments “…the projection is based on historical data…” You focus your attention on the positive net profit number and conclude that the projection looks reasonable based on your “gut.” Having reached your conclusion, you head home to enjoy your life.

Unfortunately, a forecasted numerical value is rarely accurate, because the projected “number” is inevitably “off” by some (often significant) amount.

Contrast the above with this second story. In this instance, the analyst hands you a financial projection for the same next period; the report begins again with the worksheet displayed above. However, the report continues with additional information as shown in the tables and graphs that follow:

After looking over the report, you listen as the analyst interjects that a lognormal distribution was used to model gross revenues based on a fitting of available historical data, and the analyst confides that your feedback regarding that decision would be helpful. The analyst also expresses concern about net profitability for the coming period by predicting “…there appears to be a significant chance that we won't be profitable…” (see the last histrogram projecting a 30% chance of losses). With this information, you call your spouse with a message that you will be working late in the office to “deal with some problems…”


The moral of the story is that a “number” in itself is rarely sufficient information and justification for financial decision-making. In many ways, financial managers must be able to “hear” and “feel” the meaning of probabilistic reporting much like the conductor of a symphony orchestra must hear and feel the meaning of a musical score. Effective financial managers maintain a polyvalent sensitivity toward a complexity of factors and dependencies in order to transcend numerics as guidance for decisions. In this manner, the art of financial economics embraces its science.

Financial managers might take comfort in these words by Nobel Laureate Prof Robert J Aumann (2000, p. 141):
The best art is something that strikes a chord with the viewer or listener. It expresses something that the viewer or listener has experienced himself and it expresses it in a way that enables him to focus his feelings or ideas about it. You read a novel and it expresses some kind of idea with which you can empathize, or perhaps something that you yourself have thought about or experienced. Take a sculpture or a cubist painting. It expresses some reality, some insight, in an ideal way. That is what the best mathematical economics does. It is a way of expressing ideas, perhaps in an ideal way.
By the way, probabilistic (stochastic) reasoning is now integral to finance, so if any of the reporting above is a challenge (for you or your staff), please consider my training and software offerings here.

Source: Aumann, R J (2000). Economic Theory and Mathematical Method: An Interview. In Collected Papers (Vol I, pp. 135-144). Cambridge, MA: MIT Press.

Related Posts:

Polyvalence Defies Commoditization

The Topological Landscape between Automation and Expertise

Sunday, February 07, 2010

Subjectivism is a Fact of Life

Many scientists, and especially classically trained statisticians believe that science should be objective and tend to reject methodologies that are based on subjectivism. However, effective risk analysis requires that risk analysts make subjective judgments throughout the analytical process. As observed by David Vose (2008, p. 215):
For the risk analyst, subjectivism is a fact of life. Each model one builds is only an approximation of the real world. Decisions about the structure and acceptable accuracy of the risk analyst's model are very subjective. Added to all this, the risk analyst must very often rely on subjective estimates for many model inputs, frequently without any data to back them up.
Bottomline, subjective judgments matter, even in quantitative risk analysis.

Reference: Vose, D (2008), Risk Analysis: A Quantitative Guide (3rd ed), Hoboken, NJ: John Wiley.

Wednesday, April 14, 2010

What Do Professors Want?

by Thomas C Reeves © MercatorNet.com

The shady groves of academe have cachet as a home address, but the pay is lousy, the prestige is negligible, and the power is derisory.

Polls and studies have shown consistently that professors, especially in the humanities and social sciences, side with the Left in political and cultural matters. So do public schoolteachers, whose unions are major contributors to the Democratic Party. This bias contrasts sharply, of course, with the dispassionate search for truth that scholars and teachers claim to revere. There are many reasons, no doubt, for the bent shown by professors in the humanities and social sciences, but the most obvious, it seems to me, is envy. A history professor for 40 years, I have felt this prominent member of the Seven Deadly Sins myself, many times. Let us consider three aspects of this thesis.

Take the issue of money -- always a good place to begin with things American. Academics outside business and the sciences often labor for many long years in college and graduate school in order to obtain a doctorate. More than a few collect their diplomas sporting some gray in their hair along with a briefcase full of debts. If we are lucky enough to land a tenure-track position in higher education, a large "if" over the last four decades, we frequently start at a salary that a skilled blue collar worker might expect a few years out of high school. Don't think about salaries at Harvard; consult the data on most academics published in the Chronicle of Higher Education. A friend's son, a brand new pharmacist, recently started work at a local drug store with a salary that exceeded my University of Wisconsin System salary when I retired as a full professor.

Serious economic problems face the glowing, self-confident scholar with little money. How, for example, is he able to find adequate housing? Even US$300,000, well beyond the reach of most young and many senior professors, won't buy much in Boston, New York, Los Angeles, New Orleans, Atlanta or Chicago, not to mention Madison, Sarasota, Ann Arbor, Palo Alto or Santa Barbara. The affluent suburbs, where the successful in other fields gather, are out of the question, of course. And so many of us move into older, deteriorating, often dangerous areas, telling all who listen that we made the choice deliberately and that we, being humanists, have a natural desire to live among the poor and oppressed. In my experience, some English and anthropology professors actually believe this nonsense, and enjoy dressing as factory workers and displaying furniture obviously purchased at a rummage sale.

Many academic families have two incomes, and some have other sources of private income. These professors can and often do enter the less exclusive suburbs, only to find that they have very little in common with their neighbors. They aren't invited to join the country club, as everyone understands that professors lack the necessary funds. They aren't invited to join the yacht club for the same reason. It's difficult to join a cocktail party discussion on the joys of owning a Lexus when you've just driven up in an older Corolla.

At public gatherings of all sorts, the professor might receive many awkward occupational questions. I was once asked how much professors are paid by the hour. I once gave a talk before a group of Rotarians as a favor for a dentist friend, and was introduced as a writer. The businessman sitting next to me during lunch asked, "What do you do all day beside write?"

Neighbors often assume that professors spend their summers in indolence and revelry. Thus they conclude that such people are not actually professionals and shouldn't make much money. Tell them you're writing a book and you might be asked what its chances are of being approved by Oprah. If it's a university press sort of topic, you might face such questions as "Who would read that?" and "How much could that make?" These inquiries are often followed by a wan smile or patronizing chuckle.

The education of the professor's children is another sticky point. Good private schools are out of reach financially, and religious schools are, well, religious. That leaves the public schools, which all good humanists officially champion. Those who know better feel obligated to remind colleagues and neighbors that young people learn a lot about "real life" while evading bullies, drug dealers, and gangs, and being instructed by teachers whose true calling in life was employment at Wal-Mart.

As for higher education, the low income professor faces an even greater obstacle to happiness. Tuition and expenses in even the mediocre private institutions are absurdly high, and public colleges and universities have been steadily raising their tuition for years. Few if any want to send their young people to the open-admissions College for Dummies across town, even if that would save some money. One wants to boast to a sniffy neighbor at a cocktail party that junior attends Brown, not Damp Valley State. Scholarships, grants, and federal student jobs are hoped for. Large loans increase the frustration.

Many academics not only envy people with money, but also those who enjoy political authority. Professors are more confident than most that they have the truth and are convinced that, if given the opportunity, they would rule with intelligence, justice, and compassion. The trouble is that few Americans, at least since the time of Andrew Jackson, will vote for intellectuals. (The widespread assumption that Presidents who have Ivy League degrees are intellectuals is highly debatable. The Left declared consistently that George W. Bush, who had diplomas from Yale and Harvard, was mentally challenged. Barak Obama, who was not really a professor, has sealed his academic records.) How many professors run City Hall anywhere? How many would like to? How many humanities and social science professors are consulted when great civic issues are discussed and decided? Who would even invite them to join the Elks?

Instead of steering the machinery of local, state, and national politics, academics are relegated to writing angry articles in journals and websites read by the already converted and pouring their well-considered opinions into the ears of young people who are mostly eager to get drunk, listen to rap, watch ESPN, and find a suitable, or at least willing, bed partner for the night.

On the Left and Right money means power, and we "pointy heads" and "eggheads" are on the outside looking in. One thinks of Arthur Schlesinger Jr swooning over the Kennedys for the rest of his life because they gave him a title and a silent seat in some White House deliberations. Those making as much money as, say, an experienced furnace repairman account for little in this world, despite the PhD. How many academics even sit on the governing board that sets policies for their campus? It is all most humiliating. (To see how intelligently and objectively academics use the authority they have, examine the political correctness the suffocates the employment practices and intellectual lives of almost all American campuses. Aberlour's Fifth Law: "Political correctness is totalitarianism with a diploma.")

Thirdly, there is the issue of occupational mobility and professional advancement. High income neighborhoods have constant turnover because of promotions and advancement. Professors, on the other hand, are more often than not (especially the white males) stuck on a campus for many years without a prayer of moving up or out. They have little or no control over their annual salary increases, if any, and having attained the rank of full professor have only "more of the same" and retirement to look forward to. Watching their former students scale the heights of prosperity and power can cause considerable chagrin.

A few professors will attempt to become campus administrators. Chancellors and top level bureaucrats often have very high incomes and command real authority. But most faculty choose not to become politicians. Many lack the necessary cynicism.

One way to compensate for this bleak and futureless existence is to become involved in left-wing causes. They give us a sense of identity in a world seemingly owned and operated by Rotarians. And they provide us with hope. In big government we trust, for with the election of sufficiently enlightened officials, we might gain full medical coverage, employment for our children, and good pensions. These same leftist leaders might redistribute income "fairly," by taking wealth from the "greedy" and giving it to those of us who want more of everything. A "just" world might be created in which sociologists, political scientists, botanists, and romance language professors would achieve the greatness that should be theirs. It's all a matter of educating the public. And hurling anathemas at people of position and affluence we deeply envy.

Thomas C Reeves writes from Wisconsin. Among his dozen books are Twentieth Century America: A Brief History, and biographies of John F Kennedy, Joseph R McCarthy, Fulton Sheen, Walter J Kohler, Jr and Chester A Arthur.

Republished with permission of MercatorNet.com