Walk Forward Analyzer for MetaTrader 4

Transcription

1 Walk Forward Analyzer for MetaTrader 4 Version 1.2 Table of Contents Walk Forward Analyzer for MetaTrader 4 Quick Start Guide Requirements Preparation Settings Walk Forward Walk Forward Analysis Report Quick Optimize What is Walk Forward Analysis? An Example of a Walk Forward Analysis How Does the Walk Forward Analyzer for MetaTrader Work? Detailed Instructions Preparing MetaTrader History Data Optimization Settings File Walk Forward Analysis Resuming Testing Quick Optimize Creating a Language File Troubleshooting Error Messages "No Valid Results in Optimization Report" "Could Not Open/Create File" "Optimization Report Not Found" "Invalid Testing Report Data" Testing Tips Support 1

2 Quick Start Guide Requirements Microsoft.NET version 4 (the full version, not the Client Profile) is required for the Walk Forward Analyzer to run. Most computers should already have this installed. If the Walk Forward Analyzer does not install or run, download and install.net 4. Windows Vista & 7 Users: You must disable User Account Control to allow MetaTrader to write log files and reports to it's home directory. Otherwise, the Walk Forward Analyzer will not work correctly. Learn how to disable UAC on Vista & Windows 7. Preparation In MetaTrader's Strategy Tester, load your expert advisor from the Expert Advisor dropdown list. Press the Expert Properties button to open the expert advisor settings. Put a checkmark next to each setting you wish to optimize, and fill out the Start, Step and Stop fields. Limit your optimization to no more than 3 or 4 settings at a time. Press the Save button at the bottom of the Expert Properties window, and save your settings as a.set file to the \tester directory. If desired, set the options under the Testing and Optimization tabs as well, including the starting balance, positions, and any optimization preferences. Make sure you have sufficient history data for the symbol and period that you're testing on in MetaTrader's History Center. 2

3 Download M1 data for the symbol(s) you'd like to test on from Open MetaTrader's History Center (Tools menu History Center, or press F2). Select the symbol and the M1 period from the list on the left hand side. Press the Import button to bring up the Import dialog. Use the Browse button to select the history data file you downloaded above. Press OK to import the history data. (This may take several minutes, and MetaTrader may be non-responsive. Just wait for it to finish.) Use the period_converter script in MetaTrader to convert the M1 data to higher chart periods. Drag and drop the script onto an M1 chart, and enter the number of minutes to convert to in the ExtPeriodMultiplier setting. The example below will convert the M1 data to H1 data. To convert M1 data to H4, use 240, for D1 use 1440, and so on. 3

4 Settings On first launch of the Walk Forward Analyzer, the Settings window will appear. Press the Add button to set the path of your MetaTrader 4 installation folder. If you have multiple copies of MetaTrader installed, you can add each one to the Testing Directories list. The MetaTrader 4 Live Directory field is where your live trading copy of MetaTrader is located. This setting is optional, and is only used for the Quick Optimize feature. If your broker adds a suffix to the symbol name (such as "EURUSDfx", where "fx" is the suffix), put a check mark in Add Suffix to Symbol auto complete, and fill out the suffix in the box below. Restart the program for the change to take effect. By default, the software will automatically adjust the Optimization and Testing periods so that the Testing period is approximately 25% of the Optimization period length. You can edit the percentage, or uncheck Auto suggest Optimization and Testing period lengths to disable. If you're using a non-english language version of MetaTrader, select your language from the Language drop-down box. If your language is not present, please see Creating a Language File. 4

5 Walk Forward To perform a walk forward analysis, make sure the Walk Forward tab is selected. Select your MetaTrader installation folder from the drop-down box. You can launch multiple copies of the Walk Forward Analyzer and perform testing on multiple terminals at once, up to the number of MetaTrader installations in the drop-down box. If you have Birt's Tick Data Suite installed, and you wish to use FXT tick data files for 99% modeling quality, check Use Birt's Tick Data Suite Loader. The file tds.exe must be present in the selected MetaTrader install folder. Make sure you select "Every tick" in the Testing Model drop-down box. Press the Select button next to the Expert Advisor field to browse for the expert advisor that you wish to test. The expert advisor must be in the \experts folder of your MetaTrader installation directory. Press the Select button next to the Optimization Settings File field to select the.set file that contains your optimization settings. This file must be prepared in MetaTrader's Strategy Tester beforehand. The.set file must be located in the \tester folder. In the Symbol field, type the name of the symbol that you wish to perform a walk forward analysis on. A list of suggestions will appear as you type. Select the testing period from the Period drop-down list. 5

6 Select "Open prices only" (fastest), "Every tick (slow)" (slowest), or "Control points" from the Testing Model drop-down box. In the Sort By drop-down box, select "Balance" to sort optimization results by maximum profit. Select "Profit Factor" to sort optimization results by maximum profit factor, or select "Drawdown" to sort by lowest relative drawdown percentage. Enter the length of the optimization period in days in the Optimization field. The testing period in the Testing field will be automatically suggested for you when you select this field, based on the value entered in the Optimization field. The reverse is also true. Make sure the testing period is long enough to produce at least trades on average. Enter the number of optimization/testing rounds in the Passes field. Perform as many rounds of testing as you feel is necessary. Make sure you have the complete symbol history in MetaTrader for the date range displayed to the right in the Information box. Select the testing end date in the Testing End Date field. The default value is today's date. If Interactive Optimization Result Selection is checked, a window with the results of the last optimization will appear before each test. You'll manually select the result you wish to test with. If this setting is not selected, the program will automatically select the top result, depending on the Sort Results By setting. If Save Settings for Next Time is checked, the walk forward analysis settings you've just entered will be saved for the next program run. Press the Start Walk Forward Analysis button. The information box to the right will inform you of the progress. At the end of the walk forward analysis, a report will appear in a browser window. 6

7 Walk Forward Analysis Report A report will be created and displayed after each walk forward analysis. The report displays the walk forward analysis settings, the results of each optimization and testing pass, and the total profit and walk forward efficiency. The Settings section summarizes the walk forward analysis settings. The Results section shows the top (or selected) result for each optimization pass, and the results of the testing using those settings. Links to the report files and the settings file for each testing pass are also included. Dates The start and end date of the testing or optimization period. Trades The number of trades placed. 7

8 Total Profit The total profit or loss. Profit Factor The ratio of profit to loss. Drawdown The relative drawdown in percent. Link View the report or settings file. Settings The settings that were used for the optimization and testing. The Profit section shows the total profit for the optimization and testing rounds, as well as the average daily profit, and the walk forward efficiency. Total Optimized Profit The total combined profit of all of the optimization passes. Total Walk Forward Profit The total combined profit of all of the testing passes. Avg. Daily Optimized Profit The average daily profit of the optimization passes. Avg. Daily Walk Forward Profit The average daily profit of the testing passes. Walk Forward Efficiency Ratio The ratio of the average daily walk forward profit and the average daily optimized profit. A higher value (> 0.50) means that the trading system is robust. A lower or negative value means that the trading system or the optimization parameters needs work. The walk forward report is saved to the \tester\walkforward folder. The testing and optimization reports for each pass are also located in this folder, as well as the settings files for each testing pass. Use the Save Report Files As button to save a copy of the current walk forward report and all of the report and settings files to another folder. 8

9 Quick Optimize When live trading with your expert advisor, you'll need to periodically reoptimize your trading strategy over recent data. The Quick Optimize tab runs an optimization in the Strategy Tester, using today's date as the end date of the optimization period. The start date is determined by the number of days in the Optimization (Days) setting. Once the optimization is complete, the program stores a settings file with the selected result in the \experts\presets folder of your live MetaTrader installation. You can then open a chart in MetaTrader with the expert advisor and settings file attached. The live installation folder is set in the Settings dialog, under the Menu. If you did not set a live installation folder, then the generated settings file will be saved to your demo installation folder. The settings in the Quick Optimize tab are identical to those in the Walk Forward tab. The Suggested Trade Period listed in the Information box is approximately 25% of the optimization period, and can be used as a guideline as to how long to trade with the current settings before reoptimization is necessary. Press the Start Quick Optimization button to run the optimization. After the optimization is run, a dialog box will display the save location of the generated settings file. Press the Open Settings On Live Chart button to open a chart in your live copy of MetaTrader with the selected expert advisor and the generated settings file already attached. 9

10 What is Walk Forward Analysis? Walk forward analysis is the process of optimizing a trading system using a limited set of parameters, and then testing the best optimized parameter set on out-of-sample data. This is similar to how you would use your expert advisor in live trading. The principles of walk forward analysis were first described in the book The Evaluation and Optimization of Trading Strategies by Robert Pardo. The walk forward analysis is the quickest and most reliable method to evaluate the long-term performance of a trading system. To perform a walk forward analysis in MetaTrader, first optimize the expert advisor in the Strategy Tester. Then, choose the most profitable result in the Optimization Results tab, and perform a backtest over a time period immediately following the optimization period. The start date of the testing period is the same as the end date of the optimization period. The length of the testing period should be approximately 25% of the optimization period. The results of the optimization and testing are recorded, and the process is repeated until a satisfactory sample size is achieved. If the expert advisor performs well in testing, relative to the optimization, then one can conclude that the expert advisor will likely be profitable in live trading. If, on the other hand, the expert advisor performs poorly in testing, then the walk forward parameters will need to be adjusted. If after many attempts, the expert advisor still does not perform well in testing, then it can be concluded that the trading system is unprofitable. The illustration to the right demonstrates the walk forward analysis procedure. An optimization is performed over a longer period (the in-sample data), and then the optimized parameter set is tested over a subsequent shorter period (the out-of-sample data). The optimization and testing periods are shifted forward, and the process is repeated until a suitable sample size is achieved. [Source] 10

11 An Example of a Walk Forward Analysis Let's provide a real-life example: We're going to do a walk forward analysis on an expert advisor, using EURUSD M30. We'll optimize this expert advisor over a period of 120 days. We've chosen the 3 or 4 most important parameters to optimize, so as not to over-optimize or "curve fit" the results. Also, fewer parameters means a quicker test. We'll select the most profitable optimization result, and backtest those parameters over a 30 day period immediately following the optimization period. It is recommended to use a testing period of approximately 25% of the length of the optimization period. Once we've recorded our results, we'll move the next optimization and testing period forward by 30 days. After 12 consecutive rounds of optimization and testing, we'll have a year's worth of walk forward analysis data. We compare the average daily profit for the optimization periods to the average daily profit for the testing periods. This will give us a calculation called the walk forward efficiency ratio. A walk forward efficiency ratio greater than 0.5 is considered a very good result. This is what we call a robust trading system. However, an expert advisor can be considered tradeable as long as it is consistently profitable over multiple testing periods. If the walk forward efficiency ratio is negative, then that means that the expert advisor did not perform well relative to it's optimized results. How Does the Walk Forward Analyzer for MetaTrader Work? The Walk Forward Analyzer for MetaTrader 4 works by launching MetaTrader with the settings specified in the Walk Forward Analyzer program. We do this by creating an.ini file in the MetaTrader installation directory with the desired settings. First, an optimization is run with the settings specified in the.ini file. The start and end dates for each optimization period is determined by the Optimization, Testing and Testing End Date settings. After the optimization is finished, an optimization report is saved to disk. MetaTrader shuts down, and the Walk Forward Analyzer scrapes the optimization report for the results. If Interactive Optimization Result Selection is enabled, a window pops up with the sorted optimization results. The user can manually sort the results by column, and select the result they'd like to test with. Otherwise, the software chooses the top result (either the result with the highest profit, or the highest profit factor, depending on the Sort By setting). Another.ini file is created for the testing, and MetaTrader is launched. After the testing is completed, a testing report is saved to disk. MetaTrader shuts down, and the Walk Forward Analyzer scrapes the testing report and stores the results. This process repeats for the number of times specified in the Passes setting. Once the walk forward analysis is done, a report is displayed in your browser window. The walk forward analysis report, as well as the optimization and testing reports and the testing settings files, are located in the \tester\walkforward folder in your MetaTrader installation directory. 11

12 Detailed Instructions Preparing MetaTrader History Data It is very important that you have complete historical data for the symbol and chart period that you wish to test on. Your broker will provide recent data, but this may not be sufficient for a comprehensive long-term walk forward analysis. One method to retrieve history data is to open MetaTrader's History Center, select the symbol and period from the list on the left side, and press the Download button to retrieve the history data from the MetaTrader server. However, the MetaTrader data sometimes has large gaps, and suffers from the same historical limitations as broker data. Another method is to import M1 data from a reliable source. A good source of free M1 data is Use the Import button in the History Center to import data from a file to the selected symbol and period. Once you have complete M1 data for a symbol, use the period_converter script that comes with MetaTrader to convert the data for higher timeframes. Simply drag the script on the chart, and enter the number of minutes to convert to in the ExtPeriodMultiplier setting. For example, to convert M1 data to M15, attach the script to an M1 chart and enter 15 for the ExtPeriodMultiplier setting. For an H1 chart, enter 60 in the ExtPeriodMultiplier setting. The generated data should now appear in the History Center. Double check by opening a chart with the desired symbol and period and scroll backward. If you are using a second MetaTrader installation for testing, you'll need to do these steps in your second MetaTrader install as well. Optimization Settings File You'll need to prepare an optimization settings file for the Walk Forward Analyzer to use. To do this, open MetaTrader and the Strategy Tester window. Load the expert advisor in the Expert Advisor drop-down box. Press the Expert Properties button to open the expert advisor settings dialog. To select the parameters for optimization, place a checkmark next to the settings you wish to optimize. Adjust the Start, Step and Stop settings for each parameter. If you're unfamiliar with optimization in MetaTrader, the Start is the starting value for the range of parameters, the Stop is the ending value, and the Step is the amount to increment the parameter by. For example, a parameter with a Start of 10, a Step of 10, and a Stop of 100 means that the optimizer will start at 10, and increment the parameter by 10, up to a maximum of 100. Thus, the range of values is 10, 20, up to

13 Limit the number of parameters for optimization to as few as possible, no more than 3 or 4 at a time. Optimizing a large number of parameters will take much longer, and is likely to create a condition known as "curve fitting". A curve fitted result will look excellent in optimization, but will test poorly during the walk forward. Use the Save button at the bottom of the Expert Properties dialog to save the current settings to a file. The default save location will be the \tester folder. Your optimization settings file must be located in this folder for the Walk Forward Analyzer to work. Under the Testing tab in the Expert Properties dialog, you can adjust the starting balance and whether to limit trades to long or short only. Ignore the Optimized parameter setting the Walk Forward Analyzer will sort optimization results internally. If you uncheck Genetic Algorithm in the Testing tab, the optimizer will attempt to perform a full backtest for each parameter set to be optimized, which can take much longer, but will be more accurate than the quicker genetic algorithm. Under the Optimization tab in the Expert Properties dialog, you can optionally adjust these settings to limit the optimization results. All of the adjustments in the Testing and Optimization tabs are saved in the expert advisor's.ini file in the \tester folder, and will be loaded on each subsequent test. One last thing to check Put a checkmark next to the Optimization setting, and press Start to run the Strategy Tester. Load the Optimization Results tab. Uncheck Skip Useless Results in the rightclick menu. In the event that an optimization produces no profitable results, the optimization report will be blank if this is enabled. The software will still select the least unprofitable result and continue the test. Walk Forward Analysis To perform a walk forward analysis, make sure the Walk Forward tab is selected at the top of the interface. First, select the MetaTrader installation folder using the drop-down box. You can run multiple instances of the Walk Forward Analyzer and perform testing using multiple terminals at once. You'll need a separate MetaTrader installation for each instance you plan on running. For example, if you'd like to run six walk forward tests simultaneously, you'll need six copies of MetaTrader installed in separate folders. If you have Birt's Tick Data Suite installed, you can use it to launch MetaTrader with support for FXT tick data files. This will give you 99% modeling quality when testing using Every tick as the testing model. Support for the Tick Data Suite is provided by the developer. Press the Select button next to the Expert Advisor field to select the expert advisor to analyze. The expert advisor must be located in the \experts folder of your MetaTrader installation directory. 13

14 The Select button next to the Optimization Settings File field will select the optimization settings file. This file must be located in the \tester folder, and have a.set extension. The settings file must be prepared beforehand in MetaTrader's Strategy Tester. See the previous section for details. Type the name of the symbol to perform the analysis on in the Symbol field. A list of suggestions will appear as you type. The 26 most common forex pairs will appear in the suggestion list. If you have selected Add Suffix to Symbol Autocomplete in the Settings dialog, the suffix you entered will be automatically appended to the suggested symbol names. Select the chart period you'd like to analyze with the Period drop-down box. All of the default chart periods available in the Strategy Tester are listed here. The Testing Model drop-down box allows you to select between "Open prices only", "Control points" and "Every tick" as the testing model. "Open prices only" will be faster and will provide accurate results if your expert advisor is set to open orders only at the close of each bar. "Every tick" will attempt to replicate intra-bar trading results, but it is necessary to have complete M1 or tick data for the symbol you're testing on. Optimizing using "Every tick" will also take longer. The Tick Data Suite will give you 99% modeling quality when using FXT tick data files. "Control points" is a compromise between the two. The Sort By drop-down box allows you to choose between sorting optimization results by maximum balance, maximum profit factor, or by lowest relative drawdown percentage. In the event that there is more than one top result, the top results will be sub-sorted by max profit factor or max balance. The optimization and testing periods are specified in days. The Optimization value should be larger than your desired testing period. Make sure your optimization period is large enough to get a suitable number of results. This will depend on your trading system and the chart period that you're testing on. When the Optimization field is filled out, a suggested value of approximately 25% of the optimization period is added to the Testing field. Make sure that your testing period is large enough to get a suitable number of results. For an intraday trading system, for example, 10 trades is probably too few. However, on a daily chart, 10 trades might be sufficient. Set the Passes field large enough to give you a decent sample size. Take into consideration the amount of symbol data you have. For example, if your testing period is 60 days long, and you set Passes to 25, you'll need at least 4-5 years of complete symbol data, including the initial optimization period. The Testing End Date is the end date for the last testing period in your walk forward analysis. By default, this is set to today's date. Unless you do not have complete symbol data up to the current date, it is suggested that you leave this as is. Otherwise, set it to a date that corresponds to the end of your complete symbol data period. If Interactive Optimization Result Selection is checked, an optimization results grid will pop up after each optimization. The results can be sorted by column, and the desired result is selected by 14

15 the user. This result will be used for the subsequent walk forward testing period. Otherwise, the top optimization result will be selected instead, based on the Sort By setting. Resuming Testing You can resume your most recent walk forward analysis in the event that your testing is interrupted. You must be using the same settings as the previous test for this to work. To resume a previous test, simple press the Start Walk Forward Analysis button. If the program detects that a previous test was interrupted, it will prompt you to resume the last test. Selecting Yes will resume the previous test, and selecting No will start a new test. Quick Optimize The Quick Optimize tab runs an optimization over recent data, and stores a settings file with the top result in the \experts\presets folder of your live MetaTrader installation. You can then open a chart in MetaTrader with the expert advisor and settings file attached. This is useful for periodic re-optimization of your trading strategy. The settings are identical to those in the Walk Forward tab. The Suggested Trade Period is approximately 25% of the Optimization Period, and can be used as a guideline as to how long to trade with the current settings before re-optimization is necessary. The end date of the optimization period is today's date. Press the Start Quick Optimization button to run the optimization. After the optimization is run, a dialog box will display the save location of the generated settings file. The generated settings file name uses the naming scheme {ExpertName} {Symbol} {Period} {Date}.set. After the optimization is completed, the Open Settings On Chart button will be enabled. When pressed, this button opens a chart in your live copy of MetaTrader (as selected in the Settings dialog) with the selected expert advisor and the generated settings file already attached. If no live installation folder is set, the demo installation will be used instead. If the chart does not appear in MetaTrader, double check that the expert advisor exists in the \experts directory of your live MetaTrader folder, and that the value in the Symbol field matches your broker's name for that symbol. Note that the Expert Advisors button on the toolbar will be deactivated for your protection. Prepare your expert advisor for trading, close any old charts, and re-enable the Expert Advisors button to commence live trading. Creating a Language File If you're using a non-english language version of MetaTrader, the Walk Forward Analyzer needs to be configured for your language. If your language does not appear in the drop-down menu under Menu -> Settings, then it will be necessary to prepare a language file. 15

16 First, generate a testing report in MetaTrader's Strategy Tester (not an optimization report!) Next, locate the Total Net Profit, Profit Factor, Relative Drawdown and Total Trades results. Copy and paste the local language strings into a text file, using Notepad or another text editor. The illustration below shows the testing report in German. The highlighted text is copied and pasted into a text file. Here is the final result. This file is saved to the \lang folder in the Walk Forward Analyzer installation folder as German.lang. If you do successfully create a local language file, we'd appreciate it if you send it to us. Alternately, you can send us a copy of the testing report in your local language, and we will create a language file for you. The support address is advisor@easyexpertforex.com 16

17 Troubleshooting Error Messages "No Valid Results in Optimization Report" This error means that the optimization has finished, but there are either no results (the report is empty), or all of the results have zero trades. The most common causes for this are missing symbol history data, invalid optimization settings, or "Skip Useless Results" is enabled in MetaTrader. Open MetaTrader and press Start in the Strategy Tester. The most recently used settings should already be loaded. Check to see that the the Tester is actually producing optimization results. If not, uncheck the Optimization box and run a backtest. Check the Journal tab for errors. Check that the symbol history data is complete for the dates specified. If you see the error "TestGenerator: no history data" under the Journal tab in the Strategy Tester, then the history data is missing for the optimization dates. If the optimization is producing only negative results (i.e. no profitable result for the current testing dates), uncheck "Skip Useless Results" from the right-click menu under the Optimization Results tab. This ensures that all of the optimization results will appear in the optimization report. "Could Not Open/Create File" The program was unable to read or write to the MetaTrader folder or the Walk Forward Analyzer program folder. If you are using Windows Vista or Windows 7 and are getting this error, turn off User Account Control so that MetaTrader and other programs can write to their program folders. This link shows you how to easily disable UAC: "Optimization Report Not Found" This means that the optimization report was not written to disk, either because the optimization did not complete, or MetaTrader was unable to save the report to the \tester\walkforward folder. Check the following: If you have a Windows Explorer window open with the \tester\walkforward folder loaded, this can prevent MetaTrader from writing to the folder. Close any Explorer windows when testing. 17

18 Check today's testing log in the \tester\logs folder for any errors. The logs have an.log extension. Scroll down to the bottom of the file and see if there are any errors listed for your expert advisor. If your expert advisor is limited to use on live or demo accounts only, it may not work correctly with this software. Often, the optimization takes place before MetaTrader can connect to the server. This may result in the EA thinking that it is not logged into a proper account. Programmers: Check the output of the IsTesting() function. If it returns true, then allow the EA to operate normally. The IsDemo() or IsLive() function may return false if MT4 has not yet connected to a server. "Invalid Testing Report Data" If you're using a non-english language version of MetaTrader, the Walk Forward Analyzer needs to be configured for your language. Select your language from the Menu > Settings dialog. If your language is not present in the Menu > Settings > Language drop-down box, you will need to create a language file. Please see the section Creating a Language File above. Testing Tips Q: My expert advisor is not profitable (i.e. low or negative walk forward efficiency). Try longer or shorter optimization and testing periods. Try a different chart period. Most expert advisors will work better on a longer period, such as H1 and above. Adjust the optimization parameters in your testing file. Try adjusting stop losses, for example, or try a wider start-stop range and a smaller step value. Make sure you're using a sufficient number of Passes. A test with 5 rounds may not be enough to give a good idea of the system's long-term performance. Try for at least 1-2 years of data at first, and lengthen the number of Passes to verify your initial results. For best results, make sure that all positions are the exact same lot size, and that you are not using a money management system that dynamically changes the lot size. Otherwise your optimization profits will be skewed, and the walk forward efficiency will be inaccurate. Q: The walk forward test takes too long (many hours or even days). Adjust the Start, Step and Stop parameters in your.set file. Choose a logical Start and Stop value, and increase the size of your Step parameter. 18

19 Change the Testing Model to Open prices only. Using Every tick as your testing model takes much longer, and because of the inherent shortcomings in MetaTrader's Strategy Tester, the results are not guaranteed to be accurate. Support If you are having an issue with the software, or if you believe you have come across a bug, please include the following information when ing for support: The error message you've received, if any. You can attach a screenshot, if necessary. The walkforward.log file from the program installation folder. The default installation folder is C:\\Program Files\Walk Forward Analyzer\. Today's log from MetaTrader's \tester\logs folder. The contents of MetaTrader's \tester\walkforward folder, if available. Your optimization settings file from MetaTrader's \tester folder. The expert advisor you're testing with (optional, but recommended). The support address is advisor@easyexpertforex.com. We also welcome any suggestions regarding improvements or enhancements to this software. 19