Blog

Letter from a Grateful Hobbyist Who’s Predicting the Financial Markets with Eureqa

Posted by Jon Millis

22.06.2015 02:00 PM

Nutonian users aren’t just large corporations. They’re also hobbyist data modelers leveraging Eureqa to predict the popularity of songs, analyze home science experiments, and even determine what makes some Porsches faster than others. The letter below was sent to us by a former real estate investor and manager named Bill Russell, who’s been using Eureqa to anticipate relatively short-term movements in stock prices. Hopefully Bill’s note will not only shed light on Eureqa’s potential, but will encourage our non-commercial fans to start thinking how they might apply Eureqa to some cool personal projects outside the office.

 

To Michael Schmidt and the team at Nutonian:

Michael, I want to express my deep appreciation for what you have created and shared. I first started following Eureqa in early 2010 when my mathematician brother alerted me to your double pendulum demo and beta download when you were at Cornell.

By way of background, I’m 70 years old, retired from a career in real estate finance and management. My degree was in economics, but I always loved numbers and the numerical analysis side of that business. My serious hobby over the years has been an attempt to predict short-term moves in the financial markets. I never had an impressive level of success, but always a lot of enjoyment with the puzzle of it all. In retrospect, I am sobered by how much time and the many resources I’ve previously put into this hobby.

My attempts in market prediction began with Fourier analysis (thanks to my brother’s programming and math skills) on an HP-85 desktop computer that had 16kB of ram. Next, things got more serious with the IBM XT, Lotus, and very large worksheets of pre-processed data obtained from Pinnacle Data and TradeStation. Over the years, I went to seminars given by John Hill, Larry Williams, Bill Williams, Tom DiMark and others. I purchased the Market Logic course for Peter Steidlmayer’s market profile approach and the trading course Relevance III from Maynard Holt in Nashville. There were many ideas here and there for indicators and inter-market relationships, but choosing which to use, and how to use them together, was daunting. Eureqa has changed that. Along the way, I used some impressive programs at the time. Brainmaker Professional, a neural network program, took plenty of my time in searching for useable predictions. HNET Professional, a holographic neural network program was fast and impressive. AbTech’s Statnet was excellent as was Neuralware’s Neuralsim. Yet despite the prolonged, multi-year and serious approach, I could never find an integrated, consistent pathway to success.

Because Eureqa incorporates so much analytical power in one place and finds relationships that were simply impossible to find previously, I am encouraged as never before. With the opportunity to utilize Eureqa, so much of my past approach is obsolete and elementary. I have left most of my previous analytical programs behind and many of my technical market books have now been donated to the public library. Of great significance for individual traders is that you have diminished the gap between the professional and nonprofessional in approaching the markets. Each group can utilize Eureqa, and Eureqa is equally powerful for each.

In the past, my best insights into what data might be useful came from hundreds of tedious runs of Pearson correlations and trial-and-error runs in the neural networks. I looked for ways to recast and understand the data in S-Plus and now the R language, but I am not a programmer. Trying to smooth data with splines in R was almost an insurmountable task for me. Eureqa is enabling me now to pursue options that were previously impossible. Here are some of the reasons:

1) Power and Speed: I’m able to pursue so many more alternatives than were previously within my reach. Because Eureqa is so fast, I am now able to compare runs with a) raw data; b) the same data recast to binary form; c) the data uniformly redistributed; d) the data in a de-noised wave-shrunk form. There was simply not enough time to do this before I found Eureqa.

2) Fast Data Processing and Visualization in Eureqa: I had previously done smoothing, normalizing, and rescaling in S-Plus or R. Here Eureqa saves significant time and I have complete confidence that it is being done correctly. I was often uncertain if I was getting it right on my own with the R language.

3) Tighter Selection of Input Variables: I had previously looked for any correlated relationships among a bar’s open, high, low, close, and volume, and relationships with each of those inputs delayed four periods back. I likewise did this for inter-market correlations. There was lots of manual work with Excel. All this has become moot since Eureqa does this in a flash. I have been able to substantially reduce the number input variables.

4) Most importantly, Eureqa is finding predictive relationships that had simply been impossible to find.

Michael, it is a delight to be alive at 70, and see the breathtaking leaps in technology. I programmed a little in college, utilizing punched cards; I bought a cutting-edge four-function electronic calculator before finals in 1971 for $345 (a Sharp EL-8) and thought it was a bargain. And now there is Eureqa…….Wow!!! I can appreciate some of the incredible differences this product will continue to make in so many areas. Thank you so much for what you and your team have created, for sharing it in beta form in the past, and for still keeping it within reach for individuals.

With much appreciation,

Bill Russell

Topics: Big data, Eureqa, Financial Services

Trading Necessitates Speed Along Every Step of the Data Pipeline

Posted by Jon Millis

10.06.2015 01:43 PM

We just returned from Terrapinn’s The Trading Show, a data-driven financial services conference that brings together thought leadership in quant, automated trading, exchange technology, big data and derivatives. With more than 1,000 attendees and 60 exhibitors gathering at the Navy Pier in Chicago, this year’s event was an excellent way not only for us to educate the market about using AI to scale data science initiatives, but for us to learn about the most pressing needs faced by financial services companies.

The first day, Jay Schuren, our Field CTO, presented to an audience of 50 executives. His demo used publicly available data from Yahoo Finance – such as cash flow, valuation metrics and stock prices – to predict which NYSE companies were the most over- and undervalued compared to the rest of the market. To say the least, Jay’s discoveries, as well as the seamless and automated way in which he created his financial models, spurred heavy booth traffic for the rest of our trip.*

Finance is an interesting animal. Many industries have relatively straight-forward applications for machine intelligence. Utilities companies are often interested in daily demand forecasting. Manufacturing companies look to optimize processes and design new materials. Retailers want to determine the best locations to build new stores, while healthcare providers want to preemptively detect and treat diseases. But finance is a bit different.

Let’s take a timely analogy. As I was walking home last Friday, I saw probably half a dozen limos of Boston high-schoolers posing for photos and heading to prom. Most of our customers purchase Eureqa and just can’t help but gush to us how excited they are to go to prom with us. Leading up to the big day, we show off our dance moves (give them a live demo), and take them out for a few dates (send them a free two-week trial), and by the end of our brief tryout, they’re bursting with energy and telling us all about their plans for the big dance with us. Trading firms, on the other hand, are the stunning mystery girls.** They’re smart, they’re confident, and you don’t think they should be shy, but when you ask them to prom, they shrug their shoulders and indifferently and say, yeah, I guess that sounds cool. You raise an eyebrow unsure if you just got a date or got slapped in the face with a frozen ham. But then she sees you drag racing around the neighborhood, and all of a sudden, you’re the biggest heartthrob on the planet. What in the world just happened?

In the trading world, everything is about speed. It’s not only about the speed at which a company can execute a trade (though there were plenty of vendors there offering to shave off fractions of a second to do this), but it’s also about the time it takes for a firm to arrive at an answer about how their market works, whether that’s determining when a currency is undervalued, an asset is likely to significantly appreciate, or a large loan is too risky. Everything in the trading game revolves around timing. And everyone. Loves. Speed. Where Eureqa instantly became interesting to attendees was the automation from raw data to accurate analytical/predictive model, a process that Eureqa consolidates – and accelerates discovery – by orders of magnitude.

A majority of trading technology on display was new hardware and software that incrementally improves time-to-execution. Milliseconds are important, but implementing a trading strategy that no one else has thought of or discovered could be game-changing. Nutonian will never compete with these other products and services directly. But we’re bringing more than one date to prom.

 

* Email us at contact@nutonian.com for a live demo of this particular application. We’d love to share our current use cases in financial services and explore how we might be a fit for others. 

** We’ll ignore the fact that, in reality, it seems like a “trading” prom would be about 95% guys. Woof.

Topics: Big data, Eureqa, Financial Services, Machine Intelligence, The Trading Show

Follow Me

Posts by Topic

see all