Showing posts sorted by relevance for query r project. Sort by date Show all posts
Showing posts sorted by relevance for query r project. Sort by date Show all posts

Tuesday, August 16, 2011

R Project is Real

R Project (or simply, R) has become the analytical software of choice for many data scientists around the world. According to the R website:
R is an integrated suite of software facilities for data manipulation, calculation and graphical display. It includes:
  • an effective data handling and storage facility,
  • a suite of operators for calculations on arrays, in particular matrices,
  • a large, coherent, integrated collection of intermediate tools for data analysis,
  • graphical facilities for data analysis and display either on-screen or on hardcopy, and
  • a well-developed, simple and effective programming language which includes conditionals, loops, user-defined recursive functions and input and output facilities.
For Excel (Microsoft) users, an R interface/add-in called RExcel is available and offers an integration path for analysts who use Excel. I tested RExcel on my personal workstation and found the add-in extensions to be reliable and useful.


R is available as "Free Software" under the terms of the Free Software Foundation's GNU General Public License in source code form. R compiles and runs on a wide variety of UNIX platforms and similar systems (including FreeBSD and Linux), Windows and MacOS.

Analysts and data scientists in all fields who have no yet evaluated R should do so at their earliest convenience. Follow the link below to learn more.

R Project

Thursday, July 19, 2012

The Benefits of R

Prof John Verzani (2001) summarizes some of the major benefits of using R (Project R) for learning and teaching statistics as follows:
  1. R is free.
  2. R is open-source and runs on Linux, Windows, and Apple.
  3. R has an excellent built-in help system.
  4. R has excellent graphing capabilities.
  5. R is a computer programming language.
I will be talking more about each of the above benefits in future posts. For now, let's focus on the first benefit, which is that R is free software. Follow the link below to download and install a free copy of R.


Source: Verzani, J (2001), SimpleR: Using R for Introductory Statistics.

Related Posts

Tuesday, September 11, 2012

Discovering Statistics Using R

Open source enthusiasts seeking to become statisticians will want to check out, Discovering Statistics Using R (2012) by Prof Andy Field, Prof Jeremy Miles, and Zoë Field. According to the publisher:
Hot on the heels of the award-winning and best selling Discovering Statistics Using SPSS (3rd ed), Andy Field has teamed up with Jeremy Miles (co-author of Discovering Statistics Using SAS) and Zoë Field to write Discovering Statistics Using R. Keeping the uniquely humouous and self-deprecating style that has made students across the world fall in love with Andy Field's books, Discovering Statistics Using R takes students on a journey of statistical discovery using R, a free, flexible, and dynamically changing software tool for data analysis that is becoming increasingly popular across the social and behavioral sciences throughout the world.
Those seeking to practice statistics while learning to use R (Project R) will find Discovering Statistics Using R to be a useful and instructive guide for learning statistics while also conquering the R programming environment. Follow the link below to learn more about this terrific resource.


Source: Field, A, Miles, J, and Field Z (2012), Discovering Statistics Using R, Thousand Oaks, CA: Sage.

Related Resources

Wednesday, May 08, 2013

Well Said...

"R [Project R] has become the lingua franca [i.e., working language, bridge language, vehicular language, unifying language] for statistical computing."

~ Bernhard Pfaff

Prof Bernhard Pfaff

Follow the link below to learn more about R for statistical computing.

Project R

Related Posts

Wednesday, September 12, 2012

Andy Field on Discovering Statistics Using R

Well, I simply can't resist reposting Prof Andy Field's entertaining promotional video for his new book, Discovering Statistics Using R (2012) by Prof Andy Field, and his compatriots, Prof Jeremy Miles, and Zoë Field.



By the way, R (Project R) is free software!

Follow the link below to learn more about Discovering Statistics Using R (2012).


Source: Field, A, Miles, J, and Field Z (2012), Discovering Statistics Using R, Thousand Oaks, CA: Sage.

Related Resources

Saturday, March 30, 2013

Regarding Analytics Software

According to Dr Abhijit Dasgupta of R-bloggers (2013, March 26):
The truth is, very few of us data geeks (data scientists, data analysts, statisticians, or what ever we call ourselves use only a single tool for all of our work. We will often extract data from a SQL database, munge it using Perl or Python, and then do statistical analysis using R or SAS, reporting the results using Word or, increasingly, the web. Specially for data analysis, there is often no single tool that can do the end-to-end workflow well, however much we would like to believe that there is. Each tool has its strengths and weaknesses, and often a mixture works best. The trick is in finding the right “glue” that can string our workflow together.
Read More

RStudio console on my computer

The analytics applications I personally use most include Excel (Microsoft), Decision Tools Suite (Palisade), ModelRisk (Vose), Crystal Ball (Oracle), XLStat (Addinsoft), R (Project R), RStudio (RStudio), Visual Basic for Applications (Microsoft), and Python (Python Software Foundation).

Source: Dasgupta, A (2013, March 2013), Python vs R vs SPSS -- Can’t All Programmers Just Get Along? R-bloggers.

Related Posts

Thursday, June 28, 2012

RStudio in Action

The following short video provides an instructive demonstration of RStudio's many features and capabilities.



Users of R (Project R) will find the capabilties that RStudio adds to R, useful, if not essential. In fact, I cannot imagine using R without RStudio as the user interface. I have also experimented with Revolution R (Revolution Analytics), which is a powerful commercial alternative to RStudio. However, RStudio's free price tag is hard to pass up.

Follow the link below to learn more about RStudio.

RStudio

RStudio is "alive and well" on my desktop...

Related Posts

Thursday, February 14, 2013

Happy Valentine's Day in R

Here's a quick Happy Valentine's Day plot coded in R (Project R):



# Source: http://statistics.berkeley.edu/classes/s133/heart.r
heart = function(name){
t = seq(-1,1,len=5000)
x = sin(t) * cos(t) * log(abs(t))
y = sqrt(abs(t)) * cos(t)
plot(x,y,type='l',axes=FALSE,xlab='',ylab='')
text(0,.38,"Happy Valentine's Day",col='red',cex=2.5)
text(0,.32,name,col='red',cex=2.5)
}

heart1 = function(name){
t = seq(0,60,len=100)
plot(c(-8,8),c(0,20),type='n',axes=FALSE,xlab='',ylab='')
x = -.01*(-t^2+40*t+1200)*sin(pi*t/180)
y = .01*(-t^2+40*t+1200)*cos(pi*t/180)
lines(x,y)
lines(-x,y)
text(0,7,"Happy Valentine's Day",col='red',cex=2.5)
text(0,5.5,name,col='red',cex=2.5)
}

cat("Type (1 or 2)? ")
type = readLines(n=1)
cat("Name? ")
name = readLines(n=1)

if(type == 1)heart(name) else heart1(name)

Happy Valentine's Day!

Related Posts

Saturday, June 09, 2012

The R Imperative

Prof Courtney Brown makes a succinct case for learning the R programming language.



I concur with Prof Brown. The R programming language is real. Much more to follow...

Project R

Related Posts

Thursday, June 07, 2012

Business Intelligence Process Integration via Knime

The business intelligence (BI) production path integrates data access, transformation, analysis, visualization, and exploitation into a unified process. Knime (Konstanz Information Miner) is a professional open-source software package that integrates all of these functions onto a single platform. According to the Knime website:
Knime, pronounced [naim], is a modern data analytics platform that allows you to perform sophisticated statistics and data mining on your data to analyze trends and predict potential results. Its visual workbench combines data access, data transformation, initial investigation, powerful predictive analytics and visualization. Knime also provides the ability to develop reports based on your information or automate the application of new insight back into production systems.
Knime is supported by an expanding number of third-party extensions that enable interfacing with Excel (Microsoft), R (Project R), BIRT (Eclipse), WEKA (University of Waikato), and more. Expand the graphic below to see how Knime integrates various functions and processes via its discrete process simulation features.


Knime Function Nodes [click to enlarge]

Follow the link below to learn more about Knime and its powerful BI production features for enterprise.

Knime

Related Posts

Thursday, May 17, 2012

Knime for Business Intelligence

For business analysts and firms seeking to expand their skills and capabilities from localized analytics into the broader realm of business intelligence processes and solutions, check out Knime. According to Knime's website:
Knime (Konstanz Information Miner) is a user-friendly and comprehensive open-source data integration, processing, analysis, and exploration platform. From day one, Knime has been developed using rigorous software engineering practices and is used by professionals in both industry and academia in over 60 countries.
Knime delivers robust features that encompass the full spectrum of business intelligence production requirements, including tools for: a) integrating multi-source data via open database connectivity (ODBC) and real-time processes; b) diverse analytic tools for data mining such as clustering, decision trees, rule induction, neural networks, association rules, scoring, meta-analysis, and more; and c) state-of-the-art presentation tools that easily integrate with existing ad-hoc reporting, automated dashboard, and systems actuation platforms. Knime is also actively supported by third-party extensions that integate Knime with R (Project R), Excel (Microsoft), and other widely-used integration, analytics, and presentation platforms.


Knime is the missing application that analysts have long-sought to enable self-service production of business intelligence. Anyone seeking to understand and manage the entire business intelligence production process will find Knime to be didactically useful.

Follow the link below to learn more.

Knime

Related Posts

Monday, August 13, 2012

Project R Training Videos

Anthony Damico has created a series of ninety (90) training "twotorials" for Project R users. Each training video is approximately two minutes in length. Follow the link below to learn more.

Twotorials

Anthony Joseph Damico

I have watched all ninety of the videos, and each is insightful. The training videos are free to use.

Regards to Anthony Damico for a job well done!

Related Posts

Thursday, April 08, 2010

Demographics and Stock Market Fluctuations

by Carlo Favero, Arie Gozluklu, and Andrea Tomoni © VoxEU.org

Are long-run stock market returns predictable? This column shows that a forecasting model that uses a demographic variable – the ratio of middle-aged to young adults – as well as the dividend price ratio, performs “very well” in forecasting long-horizon stock market returns.

Figure 1 shows 1-year and 20-year annualised US stock market returns (S&P 500 index) over the course of nearly the last century. Returns are determined by a “slow-moving” information component and by a “noise” component. The noise component dominates the data at high frequencies, while the information component emerges when high-frequency observations are aggregated over time to construct long-horizon returns.

Figure 1

As the information component is naturally related to “fundamentals”, Figure 1 helps understand the empirical evidence that fundamentals perform better in predicting returns as the predictive horizon gets longer. In particular, the dynamic dividend growth model (Campbell-Shiller 1988) suggests that the relevant fundamental to capture the information component in stock market returns is the dividend-price ratio. This variable regularly plays an important role in recent empirical literature that has replaced the long tradition of the efficient market hypothesis with a view of predictability of returns (see for example, Cochrane 2007). But there is an ongoing debate on the robustness of return predictability and its potential use from a portfolio allocation perspective (Boudoukh et al. 2008). The essence of this debate is captured by Figure 2 that reports the US dividend/price ratio along with the 20-Year annualised stock market returns.

Figure 2

The Figure shows the presence of some co-movement between the two variables. This is somewhat limited by the fact that the dividend-price shows a very high degree of persistence that does match the mean reversion of the returns. This high degree of persistence contradicts one of the crucial hypotheses of the dynamic dividend growth model that is based on the assumption that the dividend-price is a stationary variable. This degree of persistence is at the heart of the debate on the robustness of the statistical evidence on the predictability of stock market returns.

Is there a role for demographics?

Intuitive reasoning hints at demography as an important variable to determine the long-run behaviour of the stock market, while it is difficult to imagine a relationship between high-frequency fluctuations in stock market prices and a slow-moving trend determined by demographic factors.

In fact, a theoretical model by Geanakopoulos et al. (2004) predicts that a specific demographic variable – the ratio of middle-aged to young population – explains fluctuations in the dividend yield.

Geanakopoulos and his co-authors consider an overlapping generation model in which the demographic structure mimics the pattern of live births in the US, that have featured alternating twenty-year periods of boom and busts. They conjecture that the life-cycle portfolio behaviour – which suggests that agents should borrow when young, invest for retirement when middle-aged, and live off their investment once they are retired – plays an important role in determining equilibrium asset prices. Consumption smoothing by the agents, given the assumed demographic structure, requires that when the middle-aged to young population ratio is small, there will be excess demand for consumption by a large cohort of retirees and for the market to clear, equilibrium prices of financial assets should adjust, i.e. decrease. The result is that saving is encouraged for the middle-aged. As the dividend/price ratio is negatively related to fluctuations in prices, he model predicts a negative relation between this variable and the middle-aged-to-young ratio.

In a recent CEPR Discussion Paper (Favero et al. 2010), we take the Geanakopoulos et al. model to the data via the conjecture that fluctuations in the middle-aged-to-young ratio could capture a slowly evolving mean in the dividend price ratio within the dynamic dividend growth model. We find strong evidence in favour of using this variable together with the dividend/price ratio in long-run forecasting regressions for stock market returns, as Figure 3 and Figure 4 illustrate.

Figure 3

Figure 3 reports the dividend price ratio and the middle-aged-to-young ratio to show how, in line with the predictions of Geanakopoulos et al., a negative relation between the middle-aged-to-young ratio and the dividend price is present in the data. The demographic variable captures the slowly evolving information component in the fundamental. Does this help to predict long-horizon stock market returns? Yes. In fact, the econometric based yes, contained in our CEPR paper, is visually illustrated by Figure 4, which reports the middle-aged-to-young ratio, 10-year stock market returns, and the deviation of the dividend-price ratio from its slowly time varying mean captured by the middle-aged-to-young ratio.

Figure 4

Figure 4 also illustrates an additional interesting feature of the middle-aged-to-young ratio. Long-run forecasts for this (exogenous) variable are readily available. In fact, the Bureau of Census provides projections up to 2050 for the middle-aged-to-young ratio. In our paper we exploit the exogeneity and the predictability of the demographic ratio to project the equity risk premia up to 2050. Our simulations point to an average equity risk premium of about 5% for the next forty years.

The research agenda

The empirical evidence of a stable relation between a demographic variable and long-horizon US stock market returns naturally generates a number of interesting research questions.
  • First, the fact that a slow moving variable determined by demographics has very little impact on predictability of stock market returns at high frequency but a sizeable and strongly significant impact at low frequency has some obvious consequences on the slope of stock market risk, defined as the conditional variance and covariance per period of asset returns. Demographics should then become a natural input into the optimal asset allocation decision of a long-horizon investor.
  • Second, what about the bond market? If the middle-aged-to-young ratio plays an important role in capturing an information component that helps to predict long-horizon stock market returns it should also have a role in capturing a persistence components also in bond-yields.
  • Third, what is the international evidence? Our empirical results are so far limited to the US case only, but it is important to assess their robustness when the model is extended to other countries.
References:

Boudoukh, J, Richardson, M, and Whitelaw, R F (2008), “The Myth of Long-Horizon Predictability,” The Review of Financial Studies, 21(4):1577-1605.

Campbell, J Y Robert Shiller, R (1988), “Stock Prices, Earnings, and Expected Dividends,” Journal of Finance, 43:661-676.

Cochrane, J H (2007),”The Dog that Did Not Bark: A Defence of Return Predictability,” Review of Financial Studies, 20, 5.

Favero, C A, Gozluklu, A E, and Tamoni, A (2009) “Demographic Trends, the Dividend-Price Ratio and the Predictability of Long-Run Stock Market Returns,” CEPR working paper 7734, forthcoming in the Journal of Financial and Quantitative Analysis.

Geanakoplos, John, Michael Magill and Martine Quinzii (2004), “Demography and the Long Run Behaviour of the Stock Market,” Brookings Papers on Economic Activities, 1: 241-325.

Republished with permission of VoxEU.org

Saturday, November 13, 2010

How to Save the Euro? Lessons from the US

by Jacques Melitz © VoxEU.org

Earlier this year, the fiscal situation in Greece caused turmoil across Europe. This column examines why the financial difficulties of several state governments in the US are not having similar impacts on its economy.

The problems of the Eurozone this year brought to light some failures of the system. Nevertheless, the resulting drop in confidence in the system has gone further than we might have expected. Questions have even arisen about the survival of the system (see Baldwin 2010 and Blejer and Levy-Yeyati 2010 for discussions). Yet monetary systems do not tend to dissolve simply because of faulty performance. On the contrary, as a rule they endure even when they function very badly. It takes a political force majeure to bring about the break-up of a single currency area, typically without connection to monetary performance. Why, then, has the possible default of a country engaged in irresponsible fiscal policy and accounting for only 3% of the Eurozone’s GDP raised questions about ‘saving the euro’ and the survival of the Eurozone?

The issue has not received the attention it deserves. It is often simply taken for granted that the departures from the Stability and Growth Pact provide a sufficient reason for the earthquake that has shaken the whole currency area. Yet if we look around the world past and present, the mismanagement of finances by regional governments has no particular tendency to bring down entire monetary systems, far from it. In line with the usual – I think superficial – diagnosis of the ailment, proposed remedies for the Eurozone centre on strengthening the Pact, increasing joint political control over fiscal policy, and providing joint insurance against government default, or some mixture of the three. But what if a vital element of the problem is really the official doctrine that sovereign default is incompatible with the euro? What if the scale of the crisis that took place this year has resulted from financial markets’ conviction, based on this doctrine, that the future of the euro was at stake? What if assuring the long-run sustainability of the euro means convincing those markets, quite differently, that nothing as manageable as a Greek default can upset the Eurozone?

Lessons from the US

That is precisely what the US example would suggest and what I will defend. With this idea governing beliefs, the right road ahead looks quite different. It means shifting the emphasis away from avoiding government defaults toward assuring the stability and the solvency of the banking system at all times, regardless of the financial difficulties of some member governments.

In the US, default on state and municipal contractual obligations is very much a possibility whenever lower-level governments are in financial trouble; bailout cannot be taken for granted. New York City defaulted in 1975, the biggest default of all by a lower-level government unit since World War II took place in 1983 when the Washington Public Power Supply System went into bankruptcy, and Orange County defaulted in 1995. Various municipal governments have been on the verge of default at times in the last few decades, including Philadelphia and Cleveland. There is also no Stability and Growth Pact in the US. Yet financial discipline is considerably higher in the US at the state government level than in the Eurozone at the national level. All states except Vermont have balanced-budget rules; but these rules are self-imposed. It is easy to argue that this difference in fiscal discipline on the two sides of the Atlantic is related to the fact that when push comes to shove in the US and a lower-level government unit cannot or will not meet its debt obligations, the lenders can expect to take a big part of the hit.

Some rudimentary analysis is relevant. Consider any government unit unable to print money and without any prospect of a bailout. Theory tells us that credit rationing is very much a possibility. As the interest rate that such a government offers on its debt goes up, extra lending dries up completely at some point as the expected rate of return on the government debt falls. This must happen because higher nominal interest rates impair the government’s solvability and bring default nearer. Risk aversion simply lowers the interest rate at which credit rationing begins.

Suppose we compare the situation in the US and the Eurozone since the 2007-2009 financial crisis in this light. The crisis brought about dire financing problems for many lower-level government units in the US and some national governments in the Eurozone. According to the spreads on credit default swaps, California and Illinois now have a higher probability of non-performance on public debt than Portugal and Spain. This has been true for months. Consider next the difference in response in the States and Europe. Recently Illinois simply stopped paying $5 billion of bills. In June of last year California issued vouchers for wage payments. In addition, savage cuts in public services have begun and are now threatened in various states in difficulty, not only these two. Nevada has made startling reductions in spending on higher education and welfare. In the case of Portugal and Spain, nothing so drastic has happened thus far. There have been occasional spikes in interest rate spreads over German bunds of 100 to 200 percentage-points above usual levels. Both Spanish and Portuguese governments have also been forced to plan greater austerity and reduced government deficit spending. Meanwhile, they have been able and willing to keep borrowing.

Why the difference between the Eurozone and the US?

Part of the explanation may be that Portugal and Spain are more able to raise tax revenues than US states. But another part is the higher probability of a bailout in Europe. The example of Greece is to the point. Greece has been able to continue borrowing this year at interest rates typically around 200 percentage-points above Portugal and Spain on 10-year government bonds (and since May more than 500 percentage-points higher than German bunds). If you do the math, it is clear that this could never have happened without a high probability of a bailout. In fact, you do not need to do the math: there have been occasions in February/March and particularly May when some Greek issues would clearly have failed without the assurance of public lending and ECB support. If Greece can borrow on the probability of a bailout, so could Portugal and Spain.

Based on this evidence, the current Eurozone strategy of treating government default as anathema permits member governments to sink into deeper waters, weakens the forces that would otherwise exist toward self-imposed budget restraints, and thereby raises the probability of a bailout. But an actual bailout is perhaps the most likely setting for the breakdown of the Eurozone. If taxes ever need to rise all over the Eurozone in order to bail out a member government, one can easily imagine a pullout by Germany, followed by the Netherlands and Austria (if no others), in order to form a separate monetary union.[1]

What are the dangers of the opposite strategy of mimicking the US instead and moving toward heavier reliance on markets to discipline member governments and to price sovereign risk? The answer lies in the external effects of government default on the payment system and the banks, and this problem would be aggravated by contagion. But those dangers exist in the US as well. If the US federal government were to allow Illinois or California to default on state government debt in today’s circumstances of widespread financial difficulties across the states, there is a serious threat that interest premia would go up on the debt of most state governments and a wave of state defaults would follow. For this reason, the federal government might well step in. But if we look at the institutional manner in which the US deals with the problem, we find the answer to lie in country-wide prudential rules for banks and central bank powers of lender of last resort. There is no general announcement that state government default is incompatible with the dollar. Instead there is a strict separation of the issue of joint support of the financial system and joint support of financing by the sub-government units in the country. Would Europe not be wise to adopt the same strategy and to cease to conflate the two issues?

Tweaking the Pact

What this would mean, of course, is adopting Eurozone-wide prudential rules on banks, providing the ECB full powers of lender of last resort, and, very significantly, dismissing the idea that the Stability and Growth Pact is the pillar on which the whole Eurozone project stands. This idea is highly perilous.[2] Markets believe it, and at times of financial precariousness, what markets believe is extremely important. According to my proposal, the Pact could still be upheld as a code of good behaviour which improves public finances in Europe and facilitates the task of the ECB. But the basic philosophy would be that if any individual member government in the Eurozone engages in irresponsible fiscal conduct, contrary to the Pact, the creditors and its taxpayers would bear the brunt of the consequences. Everything would be done to assure the stability of the financial sector in the Eurozone and the lack of repercussions on the risk premiums that the more financially responsible member governments need to pay. Banks might be bailed out but not governments. Any aid to member governments, if it came, would not concern the euro system but the IMF or if any aid did come from the EU it would be part of a programme that could as well have existed had the euro never appeared and would be clearly sealed off.

VoxEU Editors' note: This article will appear as a roundtable discussion in Miroslav Beblavy, David Cobham and Ludovit Odor, eds., The euro area and the financial crisis, Cambridge University Press, forthcoming.

References:

Baldwin, R (2010), A re-cap of Vox columns on the Eurozone crisis” VoxEU.org, 13 May.

Blejer, M and Levy-Yeyati, E (2010), Leaving the euro: What’s in the box, VoxEU.org, 21 July.

Economist (2010), Can pay, won’t pay, June 19.

Poterba, J (1996), Do budget rules work? NBER Working Paper 5550, April.

Public Bonds (2010), Municipal bonds and defaults, downloaded 23 August.

Reinhart, C M and Rogoff, K (2009), This time is different: eight centuries of financial folly, Princeton: Princeton University Press.

Sinn, H-W (2010), Rescuing Europe, CESifo Forum, special issue, August.

Notes:

[1] Many would say that Greece has already been bailed out. But so far no holder of Greek debt has yet suffered a credit event. Further, no one outside of Greece has yet paid any taxes to fulfil a claim on Greek debt. Thus, according to my usage, no bailout has happened. However, none of the argument hinges on this choice of words.

[2] If we really think that a government default would bring the euro under, we must conclude that the euro has no long-run future ahead – that it is doomed. A reading of Reinhart and Rogoff (2009) should convince anyone.

Republished with permission of VoxEU.org