Rabbit Web-Based Program Downloader For the Softools Compiler SHDesigns

Rabbit Web-Based Program Downloader For the Softools Compiler Monday, June 05, 2006 2003-2006 SHDesigns Current Version: 1.2

1.0 GENERAL... 1 2.0 HTTP PROTOCOL and PROXIES... 1 2.1 Basic Web server request.... 1 2.2 Request Through Proxy... 2 2.3 Virtual Servers... 2 2.4 Using either Proxy or Web server... 2 3.0 Using the Library... 3 3.1 Controlling web page... 3 3.2 Including the Library in your code.... 4 3.3 Checking for an update.... 5 3.4 Downloading new code.... 6 3.5 RAM Loaders, Download to Main Flash... 6 3.6 Download to reserved Flash option... 7 3.7 Download to Serial Flash on RCM3300 or RCM3700... 8 3.8 Download to External Storage or User-supplied Flash.... 8 3.9 Debugging The Library... 10 4.0 Samples... 10

1.0 GENERAL The Rabbit Webloader uses a standard web server to manage and download new firmware to Z-World-based Rabbit Processor boards. Any public web server may be used and the library will even work through most routers or proxies. The software consists of a library to be included in the project and a binary file that is a small RAM-Based program that does the actual download. One feature of the library is it may be use a reserved area in flash to store a copy of the bin file or use external storage. This is useful for ppp connections. The user program calls a single function to check the version of the files on the web site. This version is compared to the version compiled into the program. A user program can then call the download function to get the update. 2.0 HTTP PROTOCOL and PROXIES To describe how the library works, a description of how the HTTP protocol works and how it is used through proxies is needed. This helps with debugging and understanding the output of the program. 2.1 Basic Web server request. To open a web page a browser connects a standard TCP/IP socket to the web server; usually on port 80. It then requests the page and reads the data. A typical sequence is as follows: 1. Resolve web server IP address (shdesigns.org in this example.) 2. Connect to web server IP port 80 (80 is default http port.) 3. Send a request to read the page. In my sample, I requested the following page: GET /test/test.html HTTP 1.0\n\n The "GET" command is followed by the URL and the protocol. A blank line defines the end of the request. 4. The web server reads the page and extracts information about the data. It then reports information about the page. Here's a sample output: Web Header: HTTP/1.1 200 OK Web Header: Date: Thu, 11 Sep 2003 04:17:02 GMT Web Header: Server: Apache/2.0.44 (Unix) mod_gzip/2.0.26.1a Web Header: Last-Modified: Thu, 11 Sep 2003 02:57:43 GMT Web Header: ETag: "2a838d-ea-f6b537c0" Web Header: Accept-Ranges: bytes Web Header: Content-Length: 234 Web Header: Connection: close Web Header: Content-Type: text/html; charset=iso-8859-1 1

The first line gives the status of the request. 200 is OK The Content-Length specifies the size of the file (234 bytes.) 5 The Web server sends a blank line to signify the end of the header. 6 The Web server then sends the data from the web page. 2.2 Request Through Proxy Working through a proxy is almost the same as directly connecting to a regular web server. The port used is usually different. The "GET" request is slightly different. The above example would be formatted as follows: GET http://shdesigns.org/test/test.html HTTP 1.0 The only difference is that the domain is included in the request. Also, the http:// protocol descriptor may also be required. The Proxy then connects to the URL and passes the data back to the requester. The data is the same as if the server was connected directly. 2.3 Virtual Servers Many web hosting sites use virtual servers. A simple client would use: GET /page.html HTTP/1.0 To get page.html. This will fail if multiple domains are hosted on one virtual sever. Using the full domain/page (as described in proxy above) in the GET command will tell the host the proper domain. 2.4 Using either Proxy or Web server Since the protocol is almost the same, a common GET command can be used. A web server will accept the proxy format of the command. The library uses the same for both. This makes the only difference, the IP address that the TCP/IP socket and port it connects to. The domain name in the URL is not resolved to an IP address. The IP address to connect to is a separate parameter. This simplifies the code, the same GET command is used for regular domains, proxy servers and virtual servers. 2

3.0 Using the Library 3.1 Controlling web page The library reads a web page to determine the current version and the program bin file to download. Within this page is embedded the following information:  Where: <-- - Beginning of comment ROM_FILE - Keyword for the bin file to download. http:// - required for URL server.tld - domain name, i.e. www.xyz.com /path - directory (if needed) /file.bin - file name to download VERSION - File version keyword "nnnn" - an integer version in ascii. Range is any 32-bit value. RAM_FILE - URL for the RAM loader binary COPY_FILE - URL for the flashcopy RAM loader binary --> end of comment This information must be included within a comment block. Each parameter should be on one line. te: the URL for the bin file should me on the same server if the library does not use a proxy. The library assumes all URLs refer to the same IP address when connected to a server. These parameters are hidden in comments. This will make them invisible to users unless they view the page source. Here's the text from my sample (http://shdesigns.org/test/test.html): <HTML> <HEAD> <META NAME="GENERATOR" CONTENT="Adobe PageMill 3.0 Win"> </HEAD> <BODY> SHDesigns Web Downloader Test Area. There is nothing to see here, move along. <!-- ROM_FILE="http://shdesigns.org/test/WEBTEST.bin" VERSION="12348" 3

COPY_FILE="http://shdesigns.org/test/flashcopy.bin" RAM_FILE="http://shdesigns.org/test/WL-All.bin" --> End of web page. </BODY> </HTML> tice the parameters are contained in comments ( ""). 3.2 Including the Library in your code. There are 4 modes of operation: Flash Copy Serial Flash External - Reboot to RAM program that downloads the.bin to main flash - Download to unused Flash and then copy to main. - Download to serial flash and run one of the predefined serial Flash copy loaders. - Download to user-supplied storage routines, then call user RAM program There are several versions of the library: Name Mode Web_DL.lib Web_DL-D.lib Web_DL-All.Lib Web_DL-All-D.Lib Web_DL-Asix.lib Web_DL-Asix-D.lib Web_DL-RT.lib Web_DL-RT-D.lib Web_DL-SMSC.lib Web_DL-SMSC-D.lib Web_DL-FC.lib Flash Copy Web_DL-FC-D.lib Flash Copy Web_DL-SFlash.lib External Web_DL-SFlash-D.lib External Web_DL-SFlash-37K Serial Flash Web_DL-SFlash-37K Serial Flash Web_DL-SFlash-33K Serial Flash Web_DL-SFlash-33K Serial Flash RAM Loader ne ne WL-All.bin WL-All-D.bin WL-Asix.bin WL-Asix-D.bin WL-Realtek.bin WL-Realtek-D.bin WL-SMSC.bin WL-SMSC-D.bin flashcopy.bin flashcopy-d.bin ne ne sflashcopy-37k.bin sflashcopy-37k.bin sflashcopy-33k.bin sflashcopy-33k.bin Debug In the later sections, they will contain Link: line at the start of each section. This indicates the appropriate library to use. For direct download, if the RAM_FILE parameter is defined in the web page, the library will download the RAM loader included in the library. Using the RAM_FILE parameter on the web page will save about 55k of flash. However, It will require this amount of free xmem. 4

For all the copy modes, if the COPY_FILE is defined in the web page, it will override the RAM loader included in the library. The RAM file specified is defined in a section 3.5. The library will add from 2k to 4k of code to your application, depending on options. If the library includes a RAM loader, it will be stored in a segment called RAM_LOADER. Make sure to locate this segment after FARCODE. It will be added by default to the end of STACK and that will not work. See section 3.6 for description of using the copy to Flash option. Add #include web_dl.h to your main code. Then add the appropriate library to the link. 3.3 Checking for an update. Link: (all) The following function will check the web server for the current version: int CheckWebVersion(char * url,char * server_ip,unsigned port,long * version) Where: url - A character string with the URL to pass to the web server to request the data. server_ip - A hostname or dotted IP address, i.e "www.xyz.com" or "192.168.1.2" This is the address of the web server or proxy. port - The port to connect to. Either the web server http port (usually 80) or the proxy server port (8000 and 8080 are common proxy ports.) version - A pointer to a long integer that will get the version (specified in the VERSION string on the web page.) Return value: -1 - response -2 - version info found -3 - bin file found -4 - Can not resolve server or proxy ip. -5 - failed to open socket -6 - RAM loader not found -7 - t enough free xmem() for RAM program -8 - failed to get size of RAM loader (Content-Length:) from server -9 - Bin file does not start with 0xc3 1 - Information downloaded successfully 5

The function saves the needed information for later use. The parameters saved are the IP address of the server or proxy, the binary file URL, the RAM or copy loader loader URL, and the port. These are used later in the download process. The user program should compare the version information with one compiled into the current code. If the version reported by the function is newer, the user code should then request a download. The URL will be used unchanged in the GET request to the server or proxy. For best results, it should always contain the http://, the domain name and the path and file name. The user program should compare the version number returned with one compiled into the program. If the version returned by the web page is newer, the user code should proceed to download the update. 3.4 Downloading new code. Once it has been determined that an new version is available, downloading the new code just requires the following call: int GetWebUpdate() There are no parameters. The function normally does not return. Otherwise it will return the same values as for CheckWebVersion(). The function will return if valid information was not received by a previous CheckWebVersion() function or if there was an error in the download.. This function copies the RAM loader to memory and passes it the needed information to download the new code bin file. It then reboots into the RAM program. When the download is finished the board will reset into the new program. If there is an error, the RAM loader will exit back to the original program. There is no way to return the error status at this time. If the download fails after the flash is modified, the RAM loader will retry the download rather than reboot. 3.5 RAM Loaders, Download to Main Flash Link: WebDL.lib requires the RAM loader to be specified in the RAM_FILE parameter on the web page. The other libs include the RAM loader in flash. Softools Loaders: The loaders are as follows: WL-All.bin - Supports any board with an Ethernet interface WL-Realtek.bin - Supports boards using the Realtek chip (most) WL-ASIX.bin - Supports boards with the ASIX chip (RCM3200) WL-SMSC.bin - Supports boards with SMSC chip (RCM3400). WL-All-D.bin - Supports all boards and outputs debug info on port A at 57600 baud. 6

te: the loaders have power-loss recovery. This stores a copy of the RAM loader in flash and will reload it of power is lost. This uses an additional 47-56k of flash. This will be located just after the end of the.bin file. The RAM loader copy in flash is located as follows: Address = length of new.bin file. Address is rounded up by sector size. If this address would conflicts with the User block, it is moved to the end of the second flash. te: When RAM_FILE is defined in the web page, the lib will require enough xmem to store the RAM program (usually 47-56k.) The RAM_FILE parameter will override any RAM loader stored in flash. There are debug versions (WL-xxx-D.bin) of these files. They are bigger in size and a bit slower. They will output debugging information on Port A of the Rabbit CPU at 57600 baud, 8-bit, no parity. This allows the 'DIAG' connector of the Z-World programming cable to be used with a program like Hyperterm. The debug versions are important for resolving problems with the download. Once the user code has been tested, the normal versions can be used to save flash space. The RAM loader uses a stripped-down TCP/IP stack. UDP, DNS or DHCP. CheckWebUpdate() uses DNS to resolve the domain names (if needed.) It then builds a table with the current IP address, netmask, gateway and the port and URL information. This is passed to the RAM loader. This way the RAM loader needs no compile-time configuration and can eliminate much of the TCP/IP stack. te: When RAM_FILE is defined in the web page, the lib will require enough xmem to store the RAM program (usually 48-73k.) The RAM_FILE parameter will override any RAM loader stored in flash. 3.6 Download to reserved Flash option Link: Use WebDL-FC.lib or WebDL-FC-D.lib The library has the option to copy the program.bin file to either the upper half of flash or to the second flash if two, 256k Flash chips are used. The following function controls this operation: void set_flash_start(long addr); Where addr is a long address to store the.bin file. For 512k Flash this would normally be 0x40000l (start of 2nd half of flash.) There are two flash copy RAM programs (located in the flashcopy directory): 7

flashcopy.bin - Smallest copy program flashcopy-d.bin - Copies and outputs debug information on port A at 115,200 Baud. These.bin files are included in the library. As they are small (5-9k) there is no real advantage on including it in the web page. 3.7 Download to Serial Flash on RCM3300 or RCM3700 Link: Use the WebDL-SFlash-33k.lib or WebDL-SFlash-37k.lib. (WebDL-SFlash-33k-D.lib or WebDL-SFlash-37k-D.lib for debug output). Use the function: set_flash_start(long addr), to set the start address to write to flash. This is a linear address. The demo application includes code to use the serial flash on the DCM3300 or RCM3700. Link the appropriate library either SF-RCM3300.lib or SF-RCM3700.lib. This mode is actually the same as External Storage mode, except it uses the serial flash lib and the serial flash handlers in the webtest.c file. The RAM loader is either flashcopy-33k.bin for the RCM3300, or sflashcopy-37k.bin for the RCM3700. There are also -D versions that include debug output on serial port A at 115,200 baud. The library already includes the RAM loader, so there is little need to specify it in the web page. 3.8 Download to External Storage or User-supplied Flash. Link: Use the WebDL-SFlash.lib or WebDL-SFlash-D.lib. This version of the library requires a user-supplied routine to save the.bin file. To support external storage, you will have to provide your own flash_init() and write_sector() routines. See the webtest.c demo for an example. flash_init() should return 0 if the init is successful. It should set _sector_size to the block size that data is stored in. This must be less than 4096. write_sector should return 0 if there is no error. In you main app, call set_flash_start(long addr) with the linear address to start writing to. This will be passed to your RAM loader. The length to copy is determined from the web page headers (Content Length.) The above will copy the program to your external storage. Next, a program to copy the external data to main flash will be needed. 8

This program should do the following: 1. Compiled to run in RAM. 2. Read a long int at memory location 0x20; this will be the param passed to set_flash_start(); 3. Read a long int at memory location 0x24; this will be the number of bytes to copy. 4. Read the device starting at location specified at 0x20 and copy to main flash starting at 0. 5. Reboot via a watchdog reset. te: locations 0x20-27 are not used by Softools, so we use the space to store parameters. tes: 1. use the flash_init() and write_sector() routines from web_dl.lib in your program to write to flash. 2. reboot can be done as follows: void reboot(void) { #asm ld a, 0x5a ioi ld (WDTCR),a xor a ioi ld (WDTTR),a xxx: jr xxx#endasm } 3. Define the following to access the params: const long int * start_address=(long int *)0x20; const long int * copy_len=(long int)0x24; Then in your code you can use *start_address and *copy_len to access passed parameters. The directory FC_Source contains the source to the flashcopy and sflashcopy programs. te: this was built using the Softools 1.65 compiler. There is a cstart158.asm file for earlier versions of WinIde; use that instead of CStart_ram.c te: if you change the project to use a CStart158.asm, make sure it is still linked first in the locate modules! tes: flashcopy.prj - project for all loaders except the RCM3300 sflashcopy sflashcopy33k.prj - project for the RCM3300 sflash copy. The flash_init() and read_sector() routines use the sflash libs for the RCM3300/3700. You would replace these with code to access your external device. 9

The Project needs the following defines in each Model: ASM defines : RAM_COMPILE - set to 1 for RAM loaders, 0-for code that will run under the debugger. INC_DEBUG - 1 for version that runs under the debugger, 0 for all others (saves 2k.) C Defines: SFLASH_COPY - define if using serial flash (undefined is Flash->flash copy.) DEBUG - Define to enable debug output on serial port A. Leave undefined for reduced code size. To debug, run the sflash copy RAM program (use the debug model) under the debugger. Manually set *copy_len and *copy_addr to valid values. You should be able to debug it reading your external storage routines and writing to main flash. 3.9 Debugging The Library First enable debug messages by linking with the -D versions of the libraries. Downloading itself can not be debugged under the IDE. It will reset and halt the CPU when the RAM loader is run. Set a breakpoint on the call to GetWebUpdate(). Run the app under the debugger. It should show all kinds of debug text. Look at the format of the GET commands and the responses from the web server. Once it gets the version and is ready to run the update, then begin to debug the RAM loader. Use the debug versions of the RAM loaders. Compile your program to flash. Use the Diag connector on the programming cable and use a terminal program like HyperTerm set up at 57600 baud, 8-bit, no parity, no handshaking. te: TerraTerm works better. The sample program includes code to redirect stdio to serial port A. This helps with debugging. Compile to flash and reset the board and you should see the debug output in the terminal program. Once the code operates properly, link with the non -D versions of the libraries to save code and data space. The non-debug RAM loaders can then also be used. 4.0 Samples A sample application, webtest.c is included. Read the file comments on how to use it. This sample can operate in all 3 modes. To use the download over LAN to Flash: 10

Undefine COPY2FLASH and EXTERNAL_STORAGE. To download to unused flash area: Define COPY2FLASH and FLASH_BIN_START To download to RCM3700 serial flash: Define COPY2FLASH, FLASH_BIN_START and EXTERNAL_STORAGE and link with the appropriate library. The Web directory contains the test.html file used by the webtest.c. Edit this file and store it along with the.bin files on your web server. A simple web server, Webster.exe has been included. This will turn any PC into a simple web server. It is great for debugging as it includes lots of debug info and is small. It is also secure as it does not open security holes like IIS does. It is secure as it only supports sending web files, no scripting, embedded applications, activex etc. 11