Crude Oil, Hurricanes and Oranges. The winding path to becoming a founder of a software company.

Customers and investors often ask this question first when they meet me. What motivated me to start this software company? The short answer is "To help energy traders mine data to validate or challenge their decisions and to provide peripheral vision to opportunities". The longer story revolves around two seminal events that happened decades ago, that unbeknownst to me at the time, were preparation for what I am doing now. 

One thing I promised myself to do when we started 2DA was to take the time to write. Create a scrap book of essays about what I was learning and how events were shaping my views and beliefs. And once I formed the habit of writing, I came to realize how cathartic it is. Publishing your thoughts for others to read - forces confrontation with the truth. It is cheap therapy and a visit to the confessional all rolled into one. Without this confrontation, it is a chore, a mechanical exercise and makes for a boring read. Here is a true story, full of 1980’s movie references, a forgotten Russian Economist and the Nile river. 

Friday August 26th, 2005. I was trading Crude Oil futures, making a living as trader. The coming weekend would be one that would go down in infamy as far as weather events were concerned. Three days earlier Tropical Depression 12 had formed over The Bahamas and the day before it had emerged off the west coast of Florida as Tropical Storm Katrina downgraded from a category 1 hurricane. We all know the subsequent story of Hurricane Katrina, the loss of life, $125 Billion in property damage. Let us stop for a second. For those who feel it inappropriate to associate trading with human tragedy, that is not the intent. I will invoke Marcus Aurelius to help so you don't miss out on the rest of the story.

You are not compelled to form any opinion about this matter before you, nor disturb your peace of mind at all. Things in themselves have no power to extort a verdict from you.

The storm is coincidental to the story, which is in itself is about events that shaped a destiny. I suppose I could have associated something else to the story, but then it would not be a true story and I would be making things up.

One part of my crude oil trading system was purely driven by algorithms that could signal profitably trades at the open of the day session of NYMEX futures. Experienced crude oil traders will remember, the two trading sessions – the day session and the ACCESS overnight session. My algorithms would signal trades and I would manually execute them on the broker platform and adjust my risk levels. This system consistently made 1-3 handles (1 handle = $1) profit on mini-crude futures a day via dozens of short duration trades. I called this my ALGO book. The other part of my trading system was based on convictions around events (in retrospect, it was mostly driven by boredom – the most prolific midwife of weak opinions and impatience) and these trades were consistently unprofitable. They constantly served to remind me that the hardest thing to do, as a trader, is nothing. This was my conviction book. The ALGO book essentially subsidized the conviction book. Looking back, if I had stuck to only the ALGO book, I would probably be dispensing my wisdom from my owners suite at an NFL stadium. But the world would be poorer for this story, so you’re welcome.

Back to the 26th. The market traded in a very tight range all day. Every trader was glued to the weather channel and cable news outlets for any updates on a storm barreling towards the oil production zone in the Gulf of Mexico. Stories of rig evacuations, storm preparations filled the news chyrons. My ALGO trading window came and went, & with no range and low volumes, the algorithms didn't trigger. Buying crude futures as a storm approached was driven by a conviction that higher prices would follow. The problem was being a weekend, if the storm turned out to be weaker than expected, I risked a lower price open on Monday and loss of capital. I decided to play it safe and put in limit buy orders below the low price from Thursday. For a change, what really animated this conviction was research that I had done almost 10 years prior, while working at Coca-Cola.

Being like most of my conviction trades, I got impatient and decided to start buying contracts every time the market dipped a few cents. In the last 5 minutes of the trading session, a forecast saying the storm would dissipate quickly, hit the wires. Futures prices collapsed - closing lower than Thursday’s low price. “Ugh!” I looked at my broker screen and saw that I was carrying a larger position than I thought I had. My limit buy orders - that were set to expire at market close were also executed as price fell below Thursday’s low price.

The last bar on the right is August 26th, 2005. This is a Japanese candlestick chart, when the close is lower than the open the bar is red and green if opposite. The whiskers represent the high and low price of the day and overnight trading. The majority of the long red bar on the 26th happened in the last 5 minutes of trading that day. The price action into the close really bothered me, based on work I had done using data to explore the uncertainty of crop forecasts a decade prior.

Rollback to 1995. A year out of graduate school and hired as a commodities analyst at Coca-Cola and assigned to juice procurement in Houston. On my first day of work, my boss and his boss walked into my office (a converted broom closet) and ceremoniously handed me a file and a floppy disk. They had an air of solemnity that was lost on me at the time. "This", they announced, "is our supply and demand forecast for orange juice". "I hear you are a statistician" Dan, the Head of Procurement growled, the toothpick in his mouth wagging as he spoke, "You’re in charge of it now, hope you were worth hiring." With that they left. The OJ S&D was a key forecast that was used to set global supplier negotiation strategy, procurement budget, cost estimates for every brand that used OJ in the entire Coca-Cola system globally, hedging strategy and many other things that were sent upstairs to the executives, in obscure reports. The other thing it achieved, as a side effect, was pretty much to make me miserable.

My prime responsibility was to own the forecast and update it every month. It was a spreadsheet that carried 12 years of history and 4 years of projections. The first year of forecast broken into monthly intervals to update actual supply and demand as they came in. Supply was based on harvest reports and any forecast revisions. For oranges there are only two regions in the world that matter, the USA for the Northern Hemisphere and Brazil for the Southern. We got two forecast checkpoints a year with the US Department of Agriculture (USDA) forecast in August (oranges are a winter fruit) and Brazil in February. Every other production region in the world was a rounding error. In the US, 90% of the crop was in Florida, with California and Texas making up the remainder. My other duties consisted of taking over every activity my boss, George, didn't want to do. From writing reports he didn't want to write to meeting with people he didn't want to talk to.

The benefits were that I was allowed to email outside the Coca-Cola Intra-Net system – this privilege was limited to very few people in 1995 and the reason on the permission form was stated darkly as “Misc Research”. I also had a Nielsen machine that allowed me to track brand or category sales at any retail outlet in the US and Canada, a terminal for futures prices and full external internet access. The latter also required special dispensation from the CIO's office in Atlanta (again, for confused millennials, the year was 1995, and believe it or not, the internet was a stranger place back then). When George and Dan were busy, I would execute futures and options trades to adjust our hedge position. Frozen Concentrate Orange Juice (FCOJ) futures traded on The New York Board of Trade (NYBOT).  The exchange is long gone now, swallowed up by one of the global behemoth exchanges.  

The FCOJ trading pit was like the dusty western town typically depicted in a Clint Eastwood Western. Frightened women peeking from behind lace curtains and only the town drunk sitting outside. Nothing ever happened in that trading pit, except on one day - the day the USDA released its crop forecast. That day was like the Rose Bowl and Mardi Gras rolled into one. It was wild. The movie “Trading Places” has the most accurate depiction of crop forecast trading day. For the those who remember the movie, I did take my fair share of taunting from my (cooler) MBA peers at Coca-Cola. "Hey Clarence Beeks, how's your boyfriend the Gorilla?" For the record, I would have been the guy, Clarence (the villain) stole the crop report from.

I threw myself into the challenge of scientific forecasting the OJ S&D. Demand was relatively easy, there was growth or decline by category that you could reliably construct with Nielsen data in the US & Canada and the rest of the word was a swag – usually as a % of GDP. Supply was way more interesting. The official crop forecast was done via sampling of orange blossoms in the summer and then extrapolated into a forecast by the USDA. It was so wildly off from actual production that it seemed only to serve one purpose - to keep the NYBOT busy for a day in August. Later, Coca-Cola would commission a private forecast from a respected Brazilian Agricultural Economist for the US and Brazil crop. Her forecast was just as wildly off from reality. Which I guess was quite respectable coming from an Economist.

I tried bounded regression, cycle based forecasts of crop estimates and other techniques to extrapolate historical data. The resulting output would always led to interesting encounters with Dan and George. Dan would lean back in his chair and look at the ceiling, then scratch his belly and say something to the effect of "Drop demand by 2% and increase crop by 10%. I heard from a grower in Florida that this season’s crop is huge.” He just wasn't going to be convinced by any mathematics. George, on the other hand, was obsessive compulsive and came up through accounting. He would brush off my forecast and then type furiously into his book-keepers calculator (the ones with the roll of paper that would print every line like a cash register), tear it off like a receipt and hand it to me - "See that’s what it’s going to be." I would dutifully nod while thinking to myself, "Dude, you just typed a bunch of gibberish into your adding contraption." Reality was, the forecast was just a way for people to justify their view. What we call “confirmation bias”. No one seemed really interested in really forecasting with precision and statistical rigor. And it turns out, my approach of using regression or mean reversion techniques produced results just as inaccurate as the official forecasts. Making up a number was just as viable as a forecast.

I evolved to a different view after 3 years of trial and error. Sampling implicitly assumes that the cause and effect of any time dependent effect is captured in the sample. But sampling is a point in time forecast and does not provide an understanding of the dynamics that lead to the result. For me, to forecast is not just to produce a value, but to understand the uncertainty around the forecast and the variance of all the inputs that go into the forecast. Properly done, it should inform the confidence around the estimate. Simply using history to forecast the future was bundling so many assumptions into the forecast to make the probability of it being close to reality very small (and keep economists employed). The Florida orange crop forecast was multivariate problem – there were so many variables that could contribute to the crop yield. What were those and when did they matter? How much did temperature, location, rainfall, the age of the orange grove contribute to the change of the forecast? The goal was not to forecast an absolute number but the probability of how much it would change, up or down year on year and how much confidence we could assign to this.

I had to get data to test my hypotheses and this kicked off a very intense nine month period in late 1998. I pinned a large map of Florida on the wall of my office and mapped out every commercial orange growing operation. Then mapped every weather station in the growing regions. Then went back in history to remap the planting of commercial groves over time, We had big dusty binders full of USDA forecasts and actuals. I transcribed the data into spreadsheets. I called the Florida Department of Citrus and had them bulk mail me their reports and transcribed those as well for cross referencing. (As a side note, the FDOC economists were based on the campus of the University of Florida. Not only was Gainesville campus a fun place to visit, the staff Economists at that time were literally the only people under the age of 35 in this industry.) I got information from the largest growers cooperative, the Florida Citrus Processors Association (FCPA). They had stacks of weekly reports printed on orange paper (of course), all transcribed into spreadsheets with the help of Dan's executive assistant.

Weather data was a bit more challenging. Not to obtain, but to make useable. Data was in long text files, by weather station by county. In some cases it would be in microfiche. The lady who answered the phone at the National Weather Service field office in Miami (and I would call about once a month) would chuckle after I finished asking for a particular weather station's data. Perhaps she was happy someone was calling and actually using the data she managed, although I suspect more than likely she was amused by a caller who seemed to be a bit mad. She would kindly convert long rolls of microfiche into text files that we could then import into excel and parse. All in, I had collated 40 years of production, planting, crop yield, and a 100 years of daily temperature and precipitation data. Purchasing a statistics textbook, got me a floppy disk with a trial version of SPSS and with that the analysis could begin.

Long story short, it took additional months of analysis, going back and getting more data, investigating data sources, rejecting data, finding new ones (at one point, I got a bit lazy and called the USDA and asked them if they had done any correlation between weather, planting and crop forecasting. There was a muffled cough, maybe a stifled chuckle, and they hung up the phone after a very quick "No"). My analysis showed the secular trends in crop yields very nicely. As commercial groves were planted further and further south in Florida to avoid the winter freeze lines that started to regularly affect older northern groves, as planting density increased and as time to full yield maturity was shortened due to the adoption of grafting, the orange crop steadily grew. We could now see how groves reached the limit of how far south they could be planted before hitting the Everglades and at what level planting densities caused incremental production to plateau. This mapping exercise had the side benefit of detailing crop yields by county. So now in the event of localized freeze, fungal or insect outbreaks, we could very quickly see impact to the total crop.

The analysis also showed the consistent leading predictor of material decline in the crop forecast was greater than normal rainfall. Running a time series, rainfall exceeding more than x standard deviations the from the running mean resulted in a material drop in the crop yield. This made sense, as orange trees are very water averse, which is why they thrive in the sandy alluvial soil of Florida. Sitting in water logged soil for hours will stress a tree, even kill it. Stressed out trees drop blossoms. The data showed deviations in rainfall both greater and lower than the mean. There were very large outliers that had to be considered. Usually heavier than normal rainfall was associated with hurricane activity. So here is where the story starts to converge to 2005.

Hurricanes, like that unwelcome guest at the kegger you threw at your parent’s house, regularly pass through Florida on their way to a bigger party in the Gulf of Mexico. Leaving in their trail destruction, havoc and a lot of rain.

Analyzing data on tropical storms to account for their effect on rainfall patterns led to an insight that fueled my skepticism regarding the price action of crude oil in August, 2005 and is something that I have had opportunity to lean back on up to this day. 

Hurricane activity follows a clustering pattern with periods of heavy activity followed by periods of diminished activity. These cycles roughly follow a 16 year span from peak to peak or trough to trough. This does not mean that there cannot be years with heavy or diminished hurricane activity in either cycle, it is just that the odds are lower. On average by a magnitude of 20-30%.  

Being in an increased hurricane activity cycle would enable us to increase the weighting on the forecast model that had rainfall as a key input (I had several models under consideration at this time). We could dynamically adjust the odds of a greater than forecast decline in crop yields in the model. The model was not precise, more of a heuristic. But, and this is important, it provided a definitive data driven decision boundary that we could use to set our outlook. Instead of selectively using or ignoring data to confirm our bias we would be using data to set our bias.

Just because more storms form into hurricanes does not mean that a particular location will be affected, however if you live in Florida or Texas, chances are you will have a higher likelihood of impact. Since 1851 (updating the data to 2018), a 120 Atlantic basin hurricanes (Category 1 and above) have hit Florida, the most of all states, followed by 64 in Texas, 56 in North Carolina and 54 in Louisiana. Today this is a simple Google search, back in 1998 these statistics had to be manually collated and assembled. That these events happen in clusters and not equally over years is insight.

Before rolling back to 2005, let me close the story of orange crop forecasting. I put together all the data and analysis and set up a 2 hour presentation of my findings. I invited all the supply planners, brand P&L managers and my bosses. Of the dozen or so people I invited, only my boss George and three others showed up. I went through the presentation, showed the data, the forecast model back tested over historical data. Concluding with how we should forecast going forward. At the end of the session, George got up and remarked that this was the biggest bunch of nonsense he had seen and all this effort to inform him that hurricanes led to lower crop yields was like proving water was wet. The rest of the group shuffled out without a word. I remember walking back to my office, throwing the presentation and floppy disk into a desk drawer and sitting in silence, utterly dejected. I would leave the company in less than a year.

So back to the main story. On Sunday the 28th, the noon forecast update from the National Weather Service showed this image on the screen.

Winds of 257 kilometers per hour (160 miles per hour) and stronger gusts. The air pressure, another indicator of hurricane strength, at the center of this Category 5 storm measured 902 millibars, the fourth lowest air pressure on record for an Atlantic storm. (Picture courtesy of Jeff Schmaltz, MODIS Rapid Response Team, NASA/GSFC)

The totality of the havoc that this storm was about to inflict would take weeks, if not months, to determine. My work ten years prior had marked the bottom of low cycle in hurricane activity somewhere about 1994/5. That meant that we were well into the increased activity cycle. With the exception of 1997 and 2002, we were seeing close to double digits in hurricanes formed every year. 2005 turned out to be an incredible outlier. 28 named storms with 15 becoming hurricanes. Katrina was already the 9th hurricane of the year. 

Crude oil futures markets trading opened at 6pm central time on Sunday the 28th. I placed myself in front of the screen at 5.45 and waited for the market to open. The first tick (executed trade) was $69.96 and the market climbed up rapidly. I had decided to liquidate by entire position at the open and had placed limit sell orders at and above $70. My fills came back. $70.00, $70.02, $70.06.  The first ever $70 print in crude oil futures had me on one side of the trade.

The last bar on the right was the trading day of the 29th including the overnight session from the 28th. A $3+ gain on a single trade. Each futures contract represented 500 barrels of crude oil, so $1500+ gain per contract and I was carrying over a 100 contracts. The conviction book never looked healthier than on that day. A conviction trade that was backed by some serious statistical analysis, albeit from 10 years prior.

Hurricanes cycles have since been further informed by studying sea temperature data and understanding effects of oceanic current cycles like “El-Nino”.  All nature is cyclical and thus a lot of human activities are driven by these cycles. And a lot of human hubris is humbled by ignoring it. This connection back to natural cycles was very fulfilling to me because of two personal heroes who inspired me to pursue mathematics and statistics in graduate school. First, Nikolai Kondratiev. A Soviet era economist who proposed a theory of economic inflation cycles that started with his work forecasting crop cycles. A forecast method so good that it was used by the Soviet planning agencies to time their purchases of grain in the international commodity markets. Something the Soviets had to do with regularity when their grandiose five year production plans resulted in failures. (For all his brilliance, Kondratiev was branded a “Kulak” by Stalin and executed in a gulag at age 46.) And H.E Hurst, the British hydrologist, who derived a formula to accurately predict the flood cycles of the Nile river. Called the "Hurst Exponent", his formula was used to design and size the Aswan dam and has subsequently been used in medicine & finance. We stand on the shoulders of these quiet giants who diligently collected data (often by hand over many years) and were relentless about bringing the scientific method to explaining nature’s supposed randomness.

There are many points to this story and it is not about a $3 trade that happened almost 20 years ago. First, is that data and the tools to analyze it are much more readily accessible today - to anyone. The barrier to discovery is so ridiculously low. All that is needed is curiosity and willingness. 

Second is there are those personalities out there in every organization. My co-founder and I call them “data-hunters” – relentlessly pursuing hidden value. If you happen to have one in your organization, hold them close, nurture and protect them from the administrative types, shallow corporate climbers and MBAs. Giving them tools and access to data is a force multiplier. The one thing I always loved about trading is that it disproportionately rewards diligent data hunters and punishes weak convictions and cheap opinions.

And finally to bring the story back to the realm of 2DA and the software we have created. As many readers appreciate, riskless profits are quickly crowded out. And in today’s energy markets, with excess supply, low prices, lower margins and cost pressures, success is a function of using data to find narrow opportunities and exploit them. Software allows the data hunter a larger surface area to find and exploit opportunities with speed. It is the combination of both that is required. This is what further separates the really good from the rest. Those that complain of no profitable opportunities, no arbitrages and low prices are simply invested in the status quo and are well on their way to extinction.

That is why I co-founded 2DA. I hope you enjoyed reading this as much as I did writing it.