Speculation By Commodity Index Funds: The Impact on Food and Energy Prices by Scott H. Irwin

1 
Intersections 

We met in 1993 at the University of Illinois, 
where one of us (Scott) was on a sabbatical leave 
from Ohio State and the other (Dwight) was a 
PhD student in agricultural economics. We had 
the good fortune of intersecting in several 
graduate classes, but two stand out. The first 
was a graduate seminar in the business college 
on the newly emerging field of behavioral 
finance taught by Professor Jay Ritter. The se-
cond was a graduate course on time-series 
econometrics taught by Professor Paul Newbold. 
In addition to being a brilliant ‘near’ Nobel
Prize-winning econometrician, Professor New-
bold’s mannerisms and dry British humor were 
seemingly pulled from a Monty Python skit. This 
delighted us no end, even though few others in 
the class seemed to appreciate the daily enter-
tainment. I think it is safe to say that this is 
where our personal friendship and professional 
partnership began. 
It was clear from the outset that both of us 
were fascinated by commodity futures markets, 
which provide both price discovery and risk man-
agement opportunities for commodity producers 
and consumers. These markets are central to the 
operation of much of the global commodity sys-
tem. Naturally, being agricultural economists, 
our main interest was in agricultural futures 
markets, but we were also interested in other 
commodity futures markets, such as the relatively 
new crude oil futures market. 

The next highly fortuitous intersection 
occurred in July 2005, when the roles were 
reversed, and Dwight contacted Scott about 
spending his sabbatical leave from Southern Illi-
nois University back at the University of Illinois. 
Dwight was by then a faculty member at South-
ern Illinois and Scott had moved to Illinois from 
Ohio State in 1997. Commodity prices were just 
starting to take off and we thought this might be 
a good opportunity to dig into issues surround-
ing speculation in commodity futures markets. 
We would not be starting at ground zero because 
Dwight examined ‘noise trader’ issues in futures 
markets for his dissertation research and Scott 
had done a couple of papers on speculation and 
price volatility in futures. So we were fortunate 
to have that base to build upon. 
What’s the old saying, ‘It’s better to be lucky 
than good’? It was incredibly good fortune that 
Dwight ended up spending much of spring semes-
ter 2006 on the Champaign-Urbana campus for his 
sabbatical leave. Scott was advising an MS student, 
Robert Merrin, who was also interested in specula-
tion issues. So an idea was born. We would jointly 
supervise Robert’s thesis on the impact of hedging 
and speculative positions on agricultural futures 
prices. We would use the time-series statistical tests 
that Dwight employed in his dissertation. Who 
could have imagined what followed? 
Commodity futures prices exploded in 
2007–2008 just as we were finishing our work 

with Robert. At the same time, concerns about a 
new type of participant in commodity futures 
markets began to emerge. In particular, market 
participants, regulators, and civic organizations 
began raising concerns that inflows from new 
‘commodity index’ funds were driving the in-
creases in commodity prices instead of economic 
fundamentals. The main argument was that un-
precedented buying pressure from these specu-
lative long-only futures traders created massive 
bubbles that resulted in prices substantially ex-
ceeding fundamental value, as much as 80% by 
some accounts. If true, this would raise major 
questions about the efficiency of price discovery 
in commodity futures markets and the useful-
ness of the markets for managing risk. Numer-
ous proposals were offered to restrict speculation 
in commodity futures markets around the globe, 
including the creation of a ‘virtual reserve’ 
whereby a public agency would take futures 
positions opposite speculators during periods of 
high market volatility, a tax on futures transac-
tions, and tighter limits on speculative positions. 
During this period, it was not uncommon to link 
concerns about speculation to world hunger, 
food crises, and civil unrest. 
The initial empirical analysis presented by 
those raising concerns about commodity specu-
lation consisted of simple graphs that showed a 
concurrent increase in long-only index futures 
positions and price levels. These analyses were 
quite effective in catching the eye of politicians 
and the public. But they clearly failed to establish 
a rigorous statistical link between actual trader 
positions and futures prices. 
We realized right away that the problem 
had to be well defined from an empirical perspec-
tive. This led us to argue for the importance of  
establishing a causal link strictly between futures  
positions and futures prices. Once the relevant 
empirical problem was defined, the proper com-
modity futures position data had to be utilized. 
We then used exhaustive empirical tests across 
numerous markets, time frames, and data sets to 
show that there was no consistent evidence that 
positions held by index investors caused large 
changes in commodity futures prices. Batteries 
of time-series and cross-sectional tests failed to 
find consistent temporal causality between
 index positions and futures prices. This body of  
work conclusively demonstrated that index
 speculation was not the main driver of the great 

commodity price spikes that occurred between 
2007 and 2013. 
While we and others have written review 
articles on the role of index funds in commodity 
futures markets, there is no single resource that 
provides a comprehensive and in-depth treat-
ment of this important subject. In our own case, 
we have written more than two dozen articles 
and reports on this controversy since 2008. 
These publications have appeared in various 
journals over a more than 15-year span of time. 
We believe that there is value in collecting the 
most important of these articles in a single vol-
ume and organizing the articles in a manner 
that reflects how and why our work evolved as it 
did. 
Hence, the purpose of this book is to present 
a curated selection of articles from our body of 
work on the impact of index funds on commod-
ity futures prices. It is important to note at the 
outset that the selected articles do not simply 
represent a ‘greatest hits’ list based on citation 
totals. Instead, the selections roughly follow the 
chronology of our involvement in the worldwide 
debate about commodity speculation as it 
evolved after 2007. The 11 articles selected for 
inclusion in this volume highlight key issues 
that we addressed as the debate evolved. Some of 
the articles ended up being highly cited and 
some did not. 
In addition to the articles in their original 
published form, we include new author fore-
words for each article that provide context and 
interesting backstories about the development 
of the research. The finished product functions 
as a guided tour through more than 15 years of 
work on index funds and the behavior of com-
modity futures prices. 
A synopsis of each chapter in the book 
follows. 
Chapter 2. Devil or Angel? The Role of 
Speculation in the Recent Commodity 
Price Boom (and Bust). This is the first paper 
that we wrote on the speculation controversy 
that erupted in 2007–2008. The article itself 
was largely a synthesis of the arguments we had 
been making about the role of index funds in the 
commodity price spike of 2007–2008 in presen-
tations and other reports. We argued in this 
2009 article that the charge of index funds cre-
ating a massive bubble simply did not stand up to 
close scrutiny. The charges were inconsistent

Speculation By Commodity Index Funds: The Impact on Food and Energy Prices by Scott H. Irwin