Creating a MQL4 Indicator with Encog

From Encog Machine Learning Framework
Jump to: navigation, search

This article demonstrates how to use Encog with MetaTrader 4. Encog’s support for MetaTrader 4 works through Code Generation. Using Encog, a MetaTrader indicator is created that performs two primary functions.

  • Export of data from MetaTrader 4
  • Actual implementation of the Indicator

It is important that this indicator be able to export data from MetaTrader 4. A primary goal is that you are able to use any custom indicators already present in MetaTrader. We do not attempt to replicate these indicators outside of MetaTrader 4.

For this example we will use the Encog Workbench. The general flow of creating a MetaTrader 4 indicator is as follows:

  • Step 1: Create a new Encog Workbench project
  • Step 2: Create a new Real-Time EGA File using the wizard
  • Step 3: Enter all of your desired indicators
  • Step 4: Generate an initial indicator
  • Step 5: Load this indicator into MetaTrader 4
  • Step 6: Capture training data using MetaTrader 4
  • Step 7: Copy this training data back into the Workbench
  • Step 8: Generate Objectives
  • Step 9: Determine ranges (for normalization)
  • Step 10: Run the analyst script
  • Step 11: Load the new indicator into MetaTrader 4
  • Step 12: Test your indicator
  • Step 13 (and beyond): Refine your indicator

The next sections will expand on each of these.

Step 1: Create a New Encog Workbench Project

The first step is to launch the Encog Workbench and create an empty project directory that will hold your Workbench Project. To do this launch the Encog Workbench and choose "New Project Folder...". Name your project something such as "MT4Example". This will open the Workbench with an empty project.

Step 2: Create a New Real-Time EGA File using the wizard

Now you must create the Encog Analyst file that will contain the specifics of the indicator that you wish to create. Choose "Tools->Wizards...", then choose "Realtime Analyst Wizard" from the dialog box. You will then be prompted for an EGA file. Enter something such as "mt4.ega". If there is already a file named this, the wizard will attempt to load as much data as it can before the file is regenerated. If you do regenerate an EGA File, be aware that only the indicators you previously entered will be saved. This will be covered in Step 13.

Step 3: Enter All of your Desired Indicators

Step 2 should have left you with a dialog that looks similar to the following.


For this example we will make use of three indicators. We will use Stochastics K & D, as well as Williams. This is just for the example, and not to imply that these might be a good set of indicators for prediction. Add these indicators as follows.

Simple Field Name: stoch_k
NinjaTrader/MT Source: iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, ##)

Simple Field Name: stoch_d
NinjaTrader/MT Source: iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, ##)

Simple Field Name: williams_r
NinjaTrader/MT Source: iWPR(Symbol(), Period(), 21, 0)

The above indicators are MetaTrader 4 specific. Also, they cannot be tested by Encog, as they are MetaTrader 4 code. If you enter any of the above sources in error, you will not get an error until you actually load the generated indicator into MetaTrader 4. The ## token above is not MetaTrader 4 code. It instructs Encog where to place the bar number.

You should also modify the following options:

Prediction Type: Max PIPs
Forward Window: 15
Backward Window: 10

The above three settings specify the goal, as well as how far back to look to accomplish this goal. Max PIP's means that you would like to maximize the PIP upward movement over the next 15 bars (forward window). There is nothing to prevent you from using ticks (as opposed to bars). However, the wizard will not currently generate code for this. I will probably create a ticks based article once I have experimented with ticks more. You can define other goals, other than PIP maximization, by directly modifying the EGA File that the wizard generates. The goal is what you are teaching the machine to predict for you. The backward window is how many bars to examine looking for a pattern.

The larger you make the backward window, the less effective it will be. For example, if you had an "very large", maybe 500 bar backward window, training would go very well. But the resulting network would not be that useful. Why? Because with that large of a window, each training set item may well be a unique pattern. The network would simply memorize all of these patterns and achieve a very low error. But this would not do much for finding small patterns that allow the network to predict.

Now that you have filled out the dialog box, you can generate the code. You can open the EGA File that was generated. You can see these indicators in the following lines.

"stoch_k",0,1,0,1,100000,-100000,0,0,"iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, ##)"
"stoch_d",0,1,0,1,100000,-100000,0,0,"iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, ##)"
"williams_r",0,1,0,1,100000,-100000,0,0,"iWPR(Symbol(), Period(), 21, 0)"

There is nothing special about the above indicators, and I am by no means advocating this as a strategy. This is merely an example of how to get data between Encog and MetaTrader 4.

Step 4: Generate an Initial Indicator

Ultimately we want to train a Machine Learning Method to recognize patterns in the indicators we obtained above. To do this we need data. Encog does not have built in indicators, it uses indicators from MetaTrader 4. Because of this we need to generate a basic indicator, that uses the sources we created above. You will then run this indicator in MetaTrader 4 and export data to Encog for training.

To create this indicator open the EGA File. Click the task drop-down, and choose task-code. With this selected click Execute, and choose Start on the next window. This will generate a file named mt4.cs. This file contains the generated indicator. The indicator is named EncogExample. You can see the source code here.

//|                                                 EncogExample.mq4 |
//|                                                  Heaton Research |
//|                     |
#property copyright "Heaton Research"
#property link      ""
#property indicator_separate_window
#property  indicator_buffers 1
#property  indicator_color1  Silver
//--- input parameters
extern bool      Export=false;
//--- buffers
double ExtMapBuffer1[];
int iHandle = -1;
int iErrorCode;
// begin Encog main config
			string EXPORT_FILENAME = "mt4.csv";
			int _neuronCount = 0;
			int _layerCount = 0;
			int _contextTargetOffset[] = {-1};
			int _contextTargetSize[] = {-1};
			bool _hasContext = false;
			int _inputCount = 0;
			int _layerContextCount[] = {-1};
			int _layerCounts[] = {-1};
			int _layerFeedCounts[] = {-1};
			int _layerIndex[] = {-1};
			double _layerOutput[] = {-1};
			double _layerSums[] = {-1};
			int _outputCount = 0;
			int _weightIndex[] = {-1};
			double _weights[] = {-1};
			int _activation[] = {-1};
			double _p[] = {-1};
// end Encog main config
//| Custom indicator initialization function                         |
int init()
   IndicatorShortName("Encog Generated Indicator" );
      if( Export )
         iHandle = FileOpen(EXPORT_FILENAME,FILE_CSV|FILE_WRITE,',');
         if(iHandle < 1)
            iErrorCode = GetLastError();
            Print("Error updating file: ",iErrorCode);
         iHandle = -1;
void ActivationTANH(double& x[], int start, int size)
   for (int i = start; i < start + size; i++)
      x[i] = 2.0 / (1.0 + MathExp(-2.0 * x[i])) - 1.0;
void ActivationSigmoid(double& x[], int start, int size)
   for (int i = start; i < start + size; i++)
      x[i] = 1.0/(1.0 + MathExp(-1*x[i]));
void ActivationElliottSymmetric(double& x[], int start, int size)
   for (int i = start; i < start + size; i++)
      double s = _p[0];
      x[i] = (x[i] * s) / (1 + MathAbs(x[i] * s));
void ActivationElliott(double& x[], int start, int size)
   for (int i = start; i < start + size; i++)
      double s = _p[0];
      x[i] = ((x[i]*s)/2)/(1 + MathAbs(x[i]*s)) + 0.5;
void ComputeLayer(int currentLayer)
   int x,y;
   int inputIndex = _layerIndex[currentLayer];
   int outputIndex = _layerIndex[currentLayer - 1];
   int inputSize = _layerCounts[currentLayer];
   int outputSize = _layerFeedCounts[currentLayer - 1];
   int index = _weightIndex[currentLayer - 1];
   int limitX = outputIndex + outputSize;
   int limitY = inputIndex + inputSize;
   // weight values
   for (x = outputIndex; x < limitX; x++)
      double sum = 0;
      for (y = inputIndex; y < limitY; y++)
         sum += _weights[index] *_layerOutput[y];
      _layerOutput[x] = sum;
      _layerSums[x] = sum;
   switch(_activation[currentLayer - 1] )
      case 0: // linear
      case 1:
         ActivationTANH(_layerOutput, outputIndex, outputSize);
      case 2:
         ActivationSigmoid(_layerOutput, outputIndex, outputSize);
      case 3:
         ActivationElliottSymmetric(_layerOutput, outputIndex, outputSize);
      case 4:
         ActivationElliott(_layerOutput, outputIndex, outputSize);
   // update context values
   int offset = _contextTargetOffset[currentLayer];
   for (x = 0; x < _contextTargetSize[currentLayer]; x++)
      _layerOutput[offset + x] = _layerOutput[outputIndex + x];
void Compute(double input[], double& output[])
   int i,x;
   int sourceIndex = _neuronCount
      - _layerCounts[_layerCount - 1];
   for(i = _layerCount - 1; i > 0; i--)
   // update context values
	int offset = _contextTargetOffset[0];
   for(x = 0; x < _contextTargetSize[0]; x++)
	  _layerOutput[offset + x] = _layerOutput[x];
//| Custom indicator deinitialization function                       |
int deinit()
      if( iHandle>0 ) 
  string PadInt(int num, int digits)
   string result = num;
   while( StringLen(result)<digits )
      result="0" + result;
   return (result);
void WriteExportLine(int pos) 
   datetime dt = Time[pos];
   string when = 
      PadInt(TimeYear(dt),4) + 
      PadInt(TimeMonth(dt),2) + 
      PadInt(TimeDay(dt),2) + 
      PadInt(TimeHour(dt),2) + 
      PadInt(TimeMinute(dt),2) + 
			FileWrite(iHandle, when,
			iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos),
			iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos),
			iWPR(Symbol(), Period(), 21, 0)
double Norm(double x,double normalizedHigh, double normalizedLow, double dataHigh, double dataLow)
	return (((x - dataLow) 
		/ (dataHigh - dataLow))
		* (normalizedHigh - normalizedLow) + normalizedLow);
double DeNorm(double x,double normalizedHigh, double normalizedLow, double dataHigh, double dataLow) {
	return (((dataLow - dataHigh) * x - normalizedHigh
		* dataLow + dataHigh * normalizedLow)
		/ (normalizedLow - normalizedHigh));
//| Custom indicator iteration function                              |
int start()
   int countedBars = IndicatorCounted();
   //---- check for possible errors
   if (countedBars<0) return(-1);
   //---- last counted bar will be recounted
   if (countedBars>0) countedBars--;
   int pos=Bars-countedBars-1;
   static datetime Close_Time;
   // only do this on a new bar
   if ( Close_Time != Time[0])
      Close_Time = Time[0];
		if( _inputCount>0 && Bars>=9 )
			double input[40];
			double output[1];
			input[10]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos+0),1.0,-1.0,100000.0,-100000.0);
			input[11]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos+1),1.0,-1.0,100000.0,-100000.0);
			input[12]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos+2),1.0,-1.0,100000.0,-100000.0);
			input[13]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos+3),1.0,-1.0,100000.0,-100000.0);
			input[14]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos+4),1.0,-1.0,100000.0,-100000.0);
			input[15]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos+5),1.0,-1.0,100000.0,-100000.0);
			input[16]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos+6),1.0,-1.0,100000.0,-100000.0);
			input[17]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos+7),1.0,-1.0,100000.0,-100000.0);
			input[18]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos+8),1.0,-1.0,100000.0,-100000.0);
			input[19]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, pos+9),1.0,-1.0,100000.0,-100000.0);
			input[20]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos+0),1.0,-1.0,100000.0,-100000.0);
			input[21]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos+1),1.0,-1.0,100000.0,-100000.0);
			input[22]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos+2),1.0,-1.0,100000.0,-100000.0);
			input[23]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos+3),1.0,-1.0,100000.0,-100000.0);
			input[24]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos+4),1.0,-1.0,100000.0,-100000.0);
			input[25]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos+5),1.0,-1.0,100000.0,-100000.0);
			input[26]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos+6),1.0,-1.0,100000.0,-100000.0);
			input[27]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos+7),1.0,-1.0,100000.0,-100000.0);
			input[28]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos+8),1.0,-1.0,100000.0,-100000.0);
			input[29]=Norm(iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, pos+9),1.0,-1.0,100000.0,-100000.0);
			input[30]=Norm(iWPR(Symbol(), Period(), 21, 0),1.0,-1.0,100000.0,-100000.0);
			input[31]=Norm(iWPR(Symbol(), Period(), 21, 0),1.0,-1.0,100000.0,-100000.0);
			input[32]=Norm(iWPR(Symbol(), Period(), 21, 0),1.0,-1.0,100000.0,-100000.0);
			input[33]=Norm(iWPR(Symbol(), Period(), 21, 0),1.0,-1.0,100000.0,-100000.0);
			input[34]=Norm(iWPR(Symbol(), Period(), 21, 0),1.0,-1.0,100000.0,-100000.0);
			input[35]=Norm(iWPR(Symbol(), Period(), 21, 0),1.0,-1.0,100000.0,-100000.0);
			input[36]=Norm(iWPR(Symbol(), Period(), 21, 0),1.0,-1.0,100000.0,-100000.0);
			input[37]=Norm(iWPR(Symbol(), Period(), 21, 0),1.0,-1.0,100000.0,-100000.0);
			input[38]=Norm(iWPR(Symbol(), Period(), 21, 0),1.0,-1.0,100000.0,-100000.0);
			input[39]=Norm(iWPR(Symbol(), Period(), 21, 0),1.0,-1.0,100000.0,-100000.0);
			ExtMapBuffer1[pos] = DeNorm(output[0],1.0,-1.0,100000.0,-100000.0);
         if( Export )

You can see the indicators that we chose embedded in the code above.

Step 5: Load this Indicator Into MetaTrader 4

You are now ready to load this indicator into MetaTrader. Start up MetaTrader and look in the Navigator pane. Right-click the Custom Indicators node and choose Create. This will open a window that lets you choose several types of MQL4 code to create. Choose to create a Custom Indicator. On the next window name the indicator something such as EncogExample. You can now generate the code. It does not matter what code is generated, because you will replace all of this code with the code generate by Encog. You should name the indicator "EncogExample" to match the generated indicator. If you would like to name the indicator something other than this, simply replace every instance of EncogExample (in the generated code) with the desired indicator name. You can see this in the following image.


Once you have entered the name of your indicator you can click the Next button. This will open a source code window that shows the indicator that you just created. Copy and paste the indicator that Encog generated into this window. You can now save and compile you indicator. You should also note the location of the file that the indicator will export data to. This location is specified in the above script.

string EXPORT_FILENAME = "mt4.csv";

This file will be stored in a directory similar to the following.

C:\Program Files\MetaTrader 4\experts\files

You are now ready to use this indicator to capture data. The captured data will be written to the above file.

Step 6: Capture Training Data Using MetaTrader

You must now attach this indicator to a chart that covers the timeframe that you would like train on. Choose File->New Chart, and create a new EURUSD chart. Make sure this chart is open. Now right-click the EncogExample indicator, which is in the Navigation pane and choose Add to Chart. Make sure that you choose to export your data, as seen here.


Once you see the chart, you can close it. The file will be captured at this point.

Step 7: Copy this Training Data Back Into the Workbench

Once the above indicator has been run you should now have an output file named mt4.csv or similar. Most likely the file was generated right into your Encog workbench project directory, as seen by the path above. If not, copy this file to your workbench project. The base name must match the EGA File, i.e. you will have mt4.csv for mt4.ega.

Step 8: Generate Objectives

You are now ready to generate the objectives. This will generate mt4_process.csv, which contains all of the PIP profits for each of the training elements. To do this, open the mt4.ega file and execute the script. It will actually try to train the neural network and complete the process. If it does train successfully, that is fine. If training goes any longer than a few steps, abort the process. We mainly want the files generated at this point. We need to know the ranges for normalization.

Step 9: Determine Ranges

We must now determine the ranges, so that proper Normalization can occur. If you inspect the following section of your EGA File you will see that there are not valid ranges. Now that the mt4_process file has been generated, these ranges can be analyzed.

"stoch_k",0,1,0,1,100000,-100000,0,0,"iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, ##)"
"stoch_d",0,1,0,1,100000,-100000,0,0,"iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, ##)"
"williams_r",0,1,0,1,100000,-100000,0,0,"iWPR(Symbol(), Period(), 21, 0)"

Open the EGA File and click Analyze Ranges. You will now see the ranges fill in, as follows.

"stoch_k",0,1,0,1,95.59636651,0,49.8783544272,24.1926876809,"iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 0, ##)"
"stoch_d",0,1,0,1,91.82240887,0,49.8921367674,21.1561919697,"iStochastic(Symbol(), Period(), 8, 5, 5, MODE_EMA, 0, 1, ##)"
"williams_r",0,1,0,1,0,-92.61186264,-92.61186264,0,"iWPR(Symbol(), Period(), 21, 0)"

Step 10: Run the Analyst Script

You are now ready to run the Encog Analyst script. To do this open the EGA File and choose task-full from the drop down list and click the Execute button. This will bring up a window that will show you progress as the entire script is executed. This command will process the data and train the Machine Learning Method. Currently Feedforward Neural Networks are the only supported methods.Once training reaches below 5%, training will stop and the indicator will be generated. If training does not complete in a reasonable amount of time you may need to adjust your parameters. Once the entire process is complete, you will be left with a mt4_code.mql file.

Step 11: Load the New Indicator into MetaTrader

Now that the new indicator has been created you can load it into Workbench. To do this open the EncogExample indicator and paste the code from mt4_code.cs into the EncogExample indicator. Save the new indicator and compile it. The indicator is now ready for use.

Step 12: Test Your Indicator

To test the indicator we must create a new chart. We will attach the indicator to this chart. Here you can see an Encog indicator attached to a chart.


The above indicator is trying to predict the max PIP positive performance in the forward window, using the given indicators.

Step 13 (and beyond): Refine your indicator

Now that you have seen the results from your indicator, you may wish to adjust it. This can be performed by regenerating the indicator. Regenerating the indicator saves you from having to re-enter all of your input indicators each time you want to change the window sizes, or other wizard inputs. To refine the indicator choose Tools->Wizards.... Choose realtime wizard. The default filename you are given should be your project EGA File. Once you choose this name, you will be promoted for if you wish to recreate from source data. If you choose "Yes", then all of your indicators will be present in the dialog box already.


This example demonstrated how to use Encog to generate MetaTrader code for an indicator. Everything needed for this indicator is self-contained. There is no linkage to an external DLL.

The indicators used in this example were arbitrary, and for education only. To create more advanced indicators you will want to experiment with other indicators.

External Links