Blog

While normalizing your data variables (rescaling the numeric values) is completely optional, it can greatly improve the performance of Eureqa, and numerical stability of solutions. This post discusses when and how to normalize variables in your data.

When to Normalize:

Eureqa works best when all variables in your data have small to medium magnitudes, on the order of 1 to 100. For example, if you have any variable that ranges over a million, it would be best to rescale the values to larger units.

Additionally, the magnitudes of the variable should be similar to the mean or offset of the variable. For example, if you have a variable that only varies between 100.0 and 100.5, it would be best to subtract off 100 so that it ranges between 0 and 0.5.

For example, consider the following two variables in some data set:
Notice that both variables look rather flat. You can't see any interesting variation because the variable a has such a large offset. Do variables in your data look like this? Let's try subtracting off an offset of 10,000 from a:
Now, we can see some interesting variation in the variable a, but the variable b still looks flat because the variable still has a large magnitude relative to b. Next, let's try dividing the values of a by 50:
Now we can see the interesting variation in both variables, as they now have the same relative scale and magnitudes. This is ideally how we want our data to look before entering it into Eureqa. When the variables are reasonably scaled, Eureqa is most likely to utilize their variation to build accurate solutions.

How to Normalize a Variable:

First, consider changing the units of the data you enter into Eureqa. Could you measure values in meters instead of centimeters? Could you measure currency in millions-of-dollars instead of dollars? Pick units such that the numeric values have a range of approximately 1 to 100.

Second, consider measuring values from an offset. Could you measure time since the time of your first data point, instead of since the beginning of the year or century?

Third, check over your data; look for outliers. Are there any values that are drastically out of proportion with the rest of the values? If so, consider removing this entire row in your data set or giving it a very low weight.

The general formula for normalizing a variable y is:

y_normalized = (y - offset)/scale

where offset and scale are the normalization parameters. It's recommended that you pick offset and scale manually, so that the numeric values still have an intuitive meaning. However, if you truly don't care what the numeric values mean, a common approach is to set offset equal to the mean of the variable and scale to the standard deviation of the variable.

It's also recommended that you apply normalization before entering your data into Eureqa. However, you can specify the normalization in the Eureqa Search Relation. For example, consider the search relation:

y = f( x/1000 )

This tells Eureqa to find a model of y as a function of values of x that are divided by 1000.

Automatic Normalization Checks:

By default, Eureqa will check your data for extreme cases of that variables that need to be normalized. When entering or modifying values in the Eureqa data view, you may encounter a message like this:

Here, Eureqa is telling you that the variable y has a large offset. It has a mean value of about 1000, but it only varies by +/- 1.38. Eureqa suggests subtracting 996 from each y value in your data set, but leaving the scale unchanged.

You can also modify this and specify what values to apply. Pick a scale and offset that makes sense and preserves meaning.