How to Classify Evergreen Content with Machine Learning and Eureqa

Posted by Matt Fleming

1/16/14 9:18 AM

Watch the Tutorial »

As a part of a recent Kaggle competition, StumbleUpon challenged users to build a model using machine learning that would classify whether a webpage should be considered evergreen or ephemeral.  The ability to better classify and understand evergreen content would allow StumbleUpon to greatly improve the performance of its recommendation engine. 

Evergreen content, for those not familiar with the term, signifies content that remains relevant, valuable, and authoritative year after year.  Evergreen content is of immense value to marketers, as it continually generates traffic and leads season after season.  Hubspot has a great introductory article on the subject.

Read More

Topics: Eureqa, Tutorial

Predicting a Bond’s Next Trade Price with Eureqa: Part 2

Posted by Jess Lin

11/26/13 10:00 AM

In my previous post, we walked through the process of using Eureqa to predict the next price a bond will trade at. Starting with a massive spreadsheet with >760,000 rows and 61 columns, we were able to generate 11 equations to describe the data in 20 minutes. While I focused on just one of the equations, there is still more we can learn from Eureqa.

Read More

Topics: Advanced Techniques, Eureqa, Tutorial, Making predictions

Predicting a Bond’s Next Trade Price with Eureqa

Posted by Jess Lin

11/21/13 1:30 PM

Watch the Tutorial »

Investors use predicted bond trade prices to inform their trading decisions throughout the day. While bond yield curves can be used to help make decisions, there are many other factors that could help predict a bond’s next trading price. 

Read More

Topics: Eureqa, Tutorial, Making predictions

Predicting Financial Delinquency Using Credit Scoring Data

Posted by Matt Fleming

11/14/13 8:10 AM

Predicting future financial distress and understanding the factors that cause it are critical to how banks decide who can get financing and on what terms.  Credit scoring algorithms, which predict the probability of default, are the primary method banks use to determine whether or not a loan should be granted to a given applicant.

Read More

Topics: Eureqa, Tutorial

Predicting Liability Insurance Claim Payments with Vehicle Data

Posted by Matt Fleming

11/7/13 8:47 AM

Understanding risk is intrinsic to running a successful insurance business.  Developing a deep understanding of the relationships between various risk factors and the likelihood and cost of associated claims enables insurance companies to charge customers appropriately for the risk they represent.

Read More

Topics: Eureqa, Tutorial

Using Eureqa to Predict Sales Next Quarter

Posted by Hod Lipson

10/8/13 9:36 AM

Forecasting sales with Eureqa
Read More

Topics: hod lipson, Tutorial

Setting and using validation data with Eureqa

Posted by Michael Schmidt

6/28/13 4:02 PM

Eureqa automatically splits your data into groups: training and validation data sets. The training data is used to optimize models, whereas validation data is used to test how well models generalize to new data. Eureqa also uses the validation data to filter out the best models to display in the Eureqa user interface. This post describes how to use and control these data sets in Eureqa.

Default Splitting
By default, Eureqa will randomly shuffle your data and then split it into train and validation data sets based on the total size of your data. Eureqa will color these points differently in the user interface, and also provide statistics for each when displaying stats, for example:

All other error metrics shown in Eureqa, like the "Fit" column and "Error" shown in the Accuracy/Complexity plot, use the metric calculated with the validation data set.

Validation Data Settings

You can modify how Eureqa chooses the training and validation data sets in the Options | Advanced Genetic Program Settings menu, shown below:

Here you can change the portion of the data that is used for the training data, and the portion that goes into the validation data. The two sets are allowed to overlap, but can also be set to be mutually exclusive as shown above.

For very small data sets (under a few hundred points) it is usually best to use almost all of this data for both training and validation. Model selection can be done using the model complexity alone in these cases.

For very large data sets (over 1,000 rows) it is usually best to use a smaller fraction of data for training. It is recommended to choose a fraction such that the size of the training data is approximately 10,000 rows or less. Then, use all the remaining data for validation.

Finally, you can also tell Eureqa to randomly shuffle the data before splitting or not. One reason to disable the shuffling is if you want to choose specific rows at the end of the data set to use for validation.

Using Validation Data to Test Extrapolating Future Values

If you are using time-series data and are trying to predict future time-series values, you may want to create a validation data split that emphasizes the ability of the models to predict future values that were not used for optimizing the model directly.

To do this, you need to disable the random shuffling in the Options | Advanced Genetic Programming Options menu, and optionally make the training and validation data sets mutually exclusive (as shown in the options above). For example, you could set the first 75% of the data to be used for training, and the last 25% to be used for validation. After starting the search, you will see your data split like below:

Now, the list of best solutions will be filtered by their ability to predict only future values - the last rows in the data set which were not used to optimize the models directly. 

Read More

Topics: Preparing Data, Advanced Techniques, Eureqa, Tutorial

Working with discontinuous data in Eureqa

Posted by Michael Schmidt

6/28/13 4:02 PM

Several features in Eureqa assume that your data is one continuous series of points by default, such as the smoothing features and numerical derivative operators. This post shows how to tell Eureqa that there are breaks in the data.

Entering discontinuity with a blank row:

Read More

Topics: Preparing Data, Advanced Techniques, Eureqa, Tutorial

Custom error metrics and special Search Relations in Eureqa

Posted by Michael Schmidt

6/28/13 4:01 PM

Eureqa's Search Relation setting provides quite a bit of flexibility to search for different types of models. This post describes some advanced techniques of using the Search Relation setting to specify custom error metrics for the search to optimize; or more specifically, arbitrary custom loss functions for the fitness calculation.

Custom Fitness Using Minimize Difference

Eureqa has a built-in fitness metric named "Minimize difference". This fitness metric minimizes the signed difference between the left- and right-hand sides of the search relationship. For example, specifying:

y = f(x)

with the minimize difference fitness metric selected tells Eureqa to find an f(x) to minimize y - f(x). A trivial solution to this relationship would be f(x) = negative infinity. However, you can enter other relations that are more useful. Consider the follow search relation:

(y - f(x))^4 = 0

Here, the minimize difference fitness would minimize the 4th-power error. In Eureqa this setting looks like:

In fact, you can enter any such expression and the f(x) can appear multiple times. For example:

max( abs(y - f(x)), (y - f(x))^2 ) = 0

would minimize the maximum of the absolute error and squared error, at each data point in the data set.

Other Methods

There are many other possible ways to alter the fitness metric using the search relationship setting. For example, you could use a normal fitness metric (e.g. absolute or squared error) but scale both sides of the relation. For example, you could wrap each side of the search relation with a sigmoid function like tanh:

tanh(y) = tanh( f(x) )

Now, both the left and right sides get squashed down to a tanh function (an s-shaped curve that ranges -1 to 1) before being compared. This effectively caps large errors, reducing their impact on the fitness.

Even More Tricks

You can also use the search relationship to forbid certain values by exploiting NaN values (NaN = Not a Number). For example, consider the following search relation, which forbids models with negative values:

y = f(x) + 0*log( f(x) )

Notice the unusual 0*log(f(x)) term. Whenever f(x) is positive, the log is real-valued, and the multiplication with zero reduces the expression to y = f(x). However, whenever f(x) is negative, log(f(x)) is undefined, and produces a NaN value. Whenever a NaN appears in the fitness calculation, Eureqa automatically assigns the solution infinite error. Therefore, this search relationship tells Eureqa to find an f(x) that models y, but f(x) must be positive on each point in the data set.

This behavior can be used in other ways as well. Any operation that would produce an IEEE floating point NaN, undefined, or infinity will trigger Eureqa to assign infinite error. You can also add multiple terms like this to place multiple different constraints on solutions.
Read More

Topics: Advanced Techniques, Eureqa, Tutorial, Custom Error Metrics

Use time-delays or time-lags of a variable in Eureqa

Posted by Michael Schmidt

6/28/13 4:01 PM

A time delay retrieves the value of a variable or expression at a fixed offset in the past, according to the time ordering or index of each data point in the data set. This post describes the time-delay building-blocks available in Eureqa and different modeling techniques with delayed values.

Time Delay Building Blocks:

Eureqa provides the delay(x, c) building block to represent an arbitrary time-delay, where x could be any expression. The expression delay(x, c) returns the value of x at c time units in the past. When used as a building-block, Eureqa can automatically optimize expressions or variables to be delayed and the time-delay amount  c.

The figure above plots an arbitrary variable x and a delayed value delay(x, 1.0), where the values are ordered by some time variable t. The delayed version is equal to x at 1.0 time units into the past.

To use time-delay building-blocks, your data must have some notion of time or ordering. You also need to tell Eureqa which variable in your data represents the time or ordering value:

If you don't specify a time variable, Eureqa will use the row number in the spreadsheet as the time value of each data point.

If a particular delayed time value falls between two points in the data set, the value is linearly interpolated between the two data points using the time value.

Eureqa also provides the delay_var(x, c) building-block which is identical to delay(x, c), except that it only accepts a variable as input. It's provided as a special case of the delay(x, c) building-block to allow you to constrain the types delays used in the solutions. But in the end they are effectively identical.

Control the Fraction of Data Used for History

Notice that the delayed output plotted above does not have values on the left side of the graph for the first few time points. This is because these points request previous values of x that lie before the first point in our data set. Eureqa will automatically ignore these data points when calculating errors.

However, there is a way to control how much of the data set Eureqa is allowed to ignore - or effectively, specify a maximum delay offset. You can limit the fraction of data used for time-delay history values in the Advanced Solutions Options menu:

The default maximum fraction is 50% of the data. If you find that Eureqa is identifying solutions with very large time delays, perhaps just to avoid modeling difficult features in the first half of the data set, you may want reduce this fraction

Additionally, you can control the number of delayed values per variable (including a zero delay of an ordinary variable use) in this dialog.

Fixed Time-delays:

Another way to model a value as a function of its previous values is with fixed delays. You can enter in fixed time-delays, or "lags" of the variable, directly into the Search Relationship option. For example:

    x = f( delay(x, 2.1), delay(x, 5.6) )

This search relationship tells Eureqa to find an equation to model the value of x as a function of it's value at 2.1 and 5.6 time units in the past.

Minimum Time-delays:

You may also want to specify a minimum time-delay offset. If you entered a search relationship such as x = f(x), Eureqa would find a trivial answer f(x) = x. More likely, you wanted to find a model of x, but as a function of x at least some amount of time in the past. The way to do this is to again use a fixed delay, such as:

    x = f(delay(x, 3.21))

Here, 3.21 is the minimum time-delay. Now, if the time-delay building-blocks are enabled, Eureqa can delay this delayed input further if necessary.

Delay Differential Equations:

Another common use for time-delays in for modeling using Delay Differential Equations. Finding delay differential equations is just like searching for ordinary differential equations. For example, entering a search relationship like:

    D(y,t) = f(y)

but also enabling time-delay building blocks. This relationship has a trivial solution however: Eureqa will return the slope formula such as

    f(y) = ( y - delay(y, 0.1) )/0.1

Therefore, you most-likely want to limit the total number of delays per variable to one (which includes the zero delay of the normal variable use). You can set this in the Advanced Solution Settings menu. The default is unlimited.

Implementing Delays Outside of Eureqa

In Matlab, you can implement a time delay using the interp1 function. For example, the expression delay(x, 1.23) would be implemented as:

    interp1(t, x, t - 1.23, 'linear')

Implementing delays in Excel is a littler harder. You need to download an Excel add-on that adds an interpolate function. For example, the package XlXtrFun adds a function "Interpolate" that is just like Matlab's interp1. There are also other guides for Linear Interpolation with Excel.
Read More

Topics: Advanced Techniques, Eureqa, Tutorial, Techniques, Time Series

Subscribe to Our Blog!

Follow Me

Posts by Topic

see all