Developing a Winlink/Saildocs Document Retrieval Schedule
The principal use I make of my SSB radio is the acquisition of weather data and forecasts via the Winlink and Saildocs servers.  This weather data is acquired via emails sent and received through the Winlink and SailMail networks.

There are several categories of decisions each Winlink/Saildocs user must deal with to gain effective use of this data:

  1. Which weather documents to retrieve.  This decision is a function of both the bandwidth available when connected (how many files and of what size can be downloaded) and the ability of the sailor to use the reports.  The limited bandwidth of both the networks and the sailor's brain make it necessary to restrict the number of documents requested to those that are most important to the voyage.
  2. When to acquire specific weather documents.  This decision is a function of both the existing propagation conditions (what time of day can you reliably connect) and the scheduled availability of the requested documents.  The later issue is very important as weather data is very perishable.  A forecast that was prepared 6 hours ago may no longer represent the current outlook of the forecaster. 

The purpose of this web page is to document how I deal with the decisions in the second category.

I will use my planned trans-Atlantic sail from Lagos, PT to the Chesapeake Bay in 2007 as the basis for this discussion of document scheduling.  I have made preliminary decisions on which documents I want to have available during the voyage and they are listed below:
  • GRIB file twice each day to cover the next 72 hours
  • Ship and Buoy reports at least once each day
  • Surface Analysis once each day
  • Met Area II Text Forecast for the first half of the voyage (east of 40 deg W. Lon)
  • NOAA Forecaster Map Discussion for the second half of the voyage (west of 40 deg W. Lon.), twice each day.  This is actually a text message from the duty NOAA forecaster explaining how he/she intends to adjust and reconcile the various numerical models used by NOAA and other meteorological organizations.  It is an amplification on the information contained in the GRIB files.  Unfortunately the discussion is limited to the NOAA area of responsibility in the Atlantic Ocean (west of 40 deg W. Lon.).
The first step in building a document retrieval schedule is to examine the needed documents to determine when they are available in the Saildocs and Winlink systems.

In the process of determining a document's availability we need to take into account the different ways Saildocs and Winlink acquire the documents from the Internet.  While I've not seen anything documented for either system, it appears that Saildocs has a scheduled retrieval time for most documents in its library.  At a scheduled time the Saildocs server will download the document from its URL.  When Saildocs receives a request for some documents that are downloaded frequently (every hour or less), the request may be held until the next schedule retrieval of the document.  Then when that retrieval takes place the weather document is forwarded back to the requestor.  For other documents with less frequent retrievals (e.g., every 6 hours) Saildocs may immediately send what ever version of the document is currently stored on the server.

Again, the Saildocs retrieval methodology I described above is just based on my experience using this service.  The actual methodology might be different, and of course could change at any time.

Winlink appears to retrieve the document from the Internet at the time of the request and sends it to the requestor within minutes.

So, if my understanding of these systems is correct, with Saildocs you are assured of getting a recent version of the requested document, but you may have to wait up to an hour after your request for some documents to be available.  With Winlink you can expect to receive the document within a few minutes of your request, but that document may be updated shortly after you receive it and your copy may be obsolete before you even have a chance to download it.

Usually you can find the document availability information in Winlink or Saildocs message to which the document is attached.

In the screen capture on the right is a message from the Saildocs catalogue with the 00z GRIB file attached.  On the first line of the message (highlighted in yellow) are the file name and the date and time the file was acquired from the Internet.  This GRIB was extracted at 04:48z.  Notice that the time this GRIB was sent to my email address was 08:05z, by which time this forecast over 3 hours old.  The GRIB files are normally updated every 6 hours at approximately 4 hours 50 minutes after the valid time.  So the next update to this GRIB file (for 06z) should be available around 10:40z or in a little less than 3 hours from the time this message was received.

Therefore to get the latest version of a given GRIB file it is best to request it be delivered between 5 and 6 hours after one of the base times (00z, 06z, 12z, and 18z) for the document.

This is a screen capture of a Winlink message containing the NOAA Map Discussion.  In this message the time the document was retrieved from the Internet is shown on the second line (green highlight) and the time the document was created or last updated is shown on the fifth line (yellow highlight).

I believe this document is generated at least every 6 hours by the NOAA duty forecaster, but the schedule is not very  precise.  It appears to be posted anywhere from 1 to 2 hours after the GRIB under discussion is available from Saildocs.  So the discussion of the model output contained in the 00z GRIB forecast is posted between 06 - 07z (the GRIB itself being available around 05z).  If I wanted to acquire both this discussion and the associated GRIB in a single connection I would wait until 07z to make that connection.

The 18z document in the screen capture above discusses the data in the 12z GRIB, and the 06z document discusses the 00z GRIB.  So it would be of benefit to download these documents at the same time as the associated GRIBs.  Therefore I would schedule my retrieval of these document pairs at 19z and 07z, respectively.

This is the WL2K message containing the latest NOAA Surface Analysis for the eastern North Atlantic.

In this case WL2K retrieved the document from the Internet at the time it generated this message.  That is, WL2K does not store this document on its server, but retrieves it at the time the system receives the request.  This document was retrieved from the NOAA web server at 07:50z (green highlight), the same time this message was sent to my email address.  The document was published on the NOAA server at 04:04z (yellow highlight).  This is the 00z Surface Analysis.

So it would appear that the Surface Analysis is available a little over 4 hours after the valid period of the data and about 50 minutes prior to the availability of the GRIB file based on the same data.

So if I want to have a Surface Analysis and GRIB file based on the same data I should plan to retrieve this document at the same time as the GRIB file, or around 5 hours after base time.  For a 00z Surface Analysis and GRIB I should request those files be delivered around 05:00z.  If I wanted the NOAA Map Discussion document as well, I might wait until 07:00z and retrieve all 3 documents at one time.

This is the WL2K email containing the MET Area II Forecast.

Once again WL2K has retrieved this document from the French Meteo web site at the time of my request (green highlight).  The forecast valid time is 09z and was last updated on the Meteo web site at 08:30z.

So the latest MET Area II Forecast should be available every 6 hours starting at 00z.

In my web page on using Ship/Buoy Weather Reports from Saildocs I established that I would want to receive only the 00z, 06z, 12z or 18z reports and that they should be requested at approximately 30 minutes after the valid time of the reports.
So we know when each of the desired documents is available through either Saildocs or Winlink and when we should request the document get the most recent update.  Now we have to work that information into a document retrieval schedule.  That schedule will be largely dependent on the radio propagation status for the Winlink and SailMail frequencies during the voyage.  Although normally the best propagation occurs during the hours just before sunrise and after sunset, and usually at the lower frequencies (7 - 10 mHz) available on Winlink and SailMail.  However, in my experience here in Europe those lower frequencies are very noisy during those hours and I usually have my best success for a quick and solid connection during the afternoon at the mid or high frequencies (12 - 18 mHz).  I expect that once I am well offshore those lower frequencies will clear up somewhat and I should schedule my Winlink and SailMail connections for the early evening and early morning hours.  I also have to reserve the hour before and up to 2 hours after 20:00z for Herb Hilgenberg's Net.

Therefore early in the voyage I will be looking to connect to Winlink or SailMail between 02:00z and 04:00z in the morning and between 22:00z to 00:00z in the evening.  As we progress further west those connection times will shift to later hours.  As we near the East Coast of the USA the connection times will likely be between 07:00z and 09:00z in the morning and 03:00z and 05:00z in the evening.  Therefore the likely connection times during the entire voyage will be somewhere between 22:00z and 09:00z.  The following table summarizes the document availability during those connections times.

Hour GRIB Ship Surface Met II NOAA Map
2200 Valid: 12:00z

Avail: 17:00z

Valid: 18:00z

Avail: 18:30z

Valid: 18:00z

Avail: 22:00z

Valid: 18:00z

Avail: 18:00z

Valid: 12:00z

Avail: 19:00z

2300 Valid: 18:00z

Avail: 23:00z

Valid: 18:00z

Avail: 18:30z

Valid: 18:00z

Avail: 22:00z

Valid: 18:00z

Avail: 18:00z

Valid: 12:00z

Avail: 19:00z

0000 Valid: 18:00z

Avail: 23:00z

Valid: 00z

Avail: 00:30z

Valid: 18:00z

Avail: 22:00z

Valid: 00:00z

Avail: 00:00z

Valid: 12:00z

Avail: 19:00z

0100 Valid: 18:00z

Avail: 23:00z

Valid: 00z

Avail: 00:30z

Valid: 18:00z

Avail: 22:00z

Valid: 00:00z

Avail: 00:00z

Valid: 18:00z

Avail: 01:00z

0200 Valid: 18:00z

Avail: 23:00z

Valid: 00z

Avail: 00:30z

Valid: 18:00z

Avail: 22:00z

Valid: 00:00z

Avail: 00:00z

Valid: 18:00z

Avail: 01:00z

0300 Valid: 18:00z

Avail: 23:00z

Valid: 00z

Avail: 00:30z

Valid: 18:00z

Avail: 22:00z

Valid: 00:00z

Avail: 00:00z

Valid: 18:00z

Avail: 01:00z

0400 Valid: 18:00z

Avail: 23:00z

Valid: 00z

Avail: 00:30z

Valid: 00:00z

Avail: 04:00z

Valid: 00:00z

Avail: 00:00z

Valid: 18:00z

Avail: 01:00z

0500 Valid: 00:00z

Avail: 05:00z

Valid: 00z

Avail: 00:30z

Valid: 00:00z

Avail: 04:00z

Valid: 00:00z

Avail: 00:00z

Valid: 18:00z

Avail: 19:00z

0600 Valid: 00:00z

Avail: 05:00z

Valid: 06:00z

Avail: 06:30z

Valid: 00:00z

Avail: 04:00z

Valid: 06:00z

Avail: 06:00z

Valid: 18:00z

Avail: 19:00z

0700 Valid: 00:00z

Avail: 05:00z

Valid: 06:00z

Avail: 06:30z

Valid: 00:00z

Avail: 04:00z

Valid: 06:00z

Avail: 06:00z

Valid: 00:00z

Avail: 07:00z

0800 Valid: 00:00z

Avail: 05:00z

Valid: 06:00z

Avail: 06:30z

Valid: 00:00z

Avail: 04:00z

Valid: 06:00z

Avail: 06:00z

Valid: 00:00z

Avail: 07:00z

0900 Valid: 00:00z

Avail: 05:00z

Valid: 06:00z

Avail: 06:30z

Valid: 00:00z

Avail: 04:00z

Valid: 06:00z

Avail: 06:00z

Valid: 00:00z

Avail: 07:00z

The first thing that is apparent from this analysis is that there is no one best connect time to get the documents I want.  No matter when I connect at least one of the documents is approaching obsolescence and is probably not worth the connect time to download it.

In developing a connection and document retrieval schedule, one other thing needs to be kept in mind - the difference between a subscription for a Winlink document and a Saildocs document.  A document subscription is one of the best ways to implement a document retrieval schedule.  However, the differences between Winlink and Saildocs have implemented document scheduling have an effect on not only how you set up those subscriptions, but also on when and how often you must connect to the network.

These differences reflect the basic differences that exist in many client-server applications, whether to put the application on the server or the client.  There is seldom one best choice and the differences between Winlink and Saildocs demonstrate this.  Winlink uses a client-driven subscription process and Saildocs is server-driven.  Each approach has its advantages and disadvantages for the cruising sailor.

Document subscription is a useful tool for scheduling document retrieval, but these subscriptions behave very differently depending on from which library the document is retrieved and which network is used for the connection.  These differences and the associated Advantages and Disadvantages are summarized below.

Network Subscription Methodology Advantages Disadvantages  
Winlink Client-Based: When you establish a repeated request for a given document that schedule is stored in the Airmail client program.  If you request a document at at specific time, Airmail will generate a Winlink document request message at that time.  No document is actually requested until you connect to Winlink. 
  1. When a subscription is no longer needed you can turn it off within the Airmail client, no Winlink/SailMail connection is requires.
  1. There is no guarantee that the document requested will be retrieved and sent during the initial connection.  Often a second connection is required to retrieve the document.
  2. In order to insure that the document for a given valid period is retrieved you must successfully connect to Winlink and transmit the document request within a timeline window.  If you miss this window, the document retrieved may be for a different valid time period.
 
Saildocs Server-based.  When you submit a document subscription that information is store on the Saildocs server.  You only need to make subsequent connections to Winlink or SailMail to retrieve the documents that will have been queued for your email address at the times requested.
  1. The scheduled request for a document is automated and does not require a Winlink/SailMail connection for each iteration of the subscription.
  1. If you want to stop a Saildocs subscription before its expiration date you must connect to Winlink/SailMail and send a cancellation message.
  2. If you use Winlink to request a Saildocs subscription, Saildocs will send the subscribed documents to your Winlink email address at the subscription schedule.  If you cannot connect to Winlink for an extended period of time, the documents will continue to be queued to your email address.  When you finally connect to Winlink you will have to download the obsolete documents as well as the single current one. 
  3. This does not happen if the connection is via SailMail.  Because SailMail is integrated with the Saildocs server it will replace the older documents with a brief message telling you that a more current document is on your queue.
 

So Winlink has the advantage when connections are frequent or when the schedule changes often.  Saildocs has the advantage of requiring minimal connections to acquire the required documents.  Of course subscriptions are not necessary to manage a document retrieval schedule, they just automate the process.  Sometimes that automation is of significant benefit, but sometimes it gets in the way.  I use subscriptions sparingly, primarily for the GRIB and ship reports from Saildocs, where there is a specific time I want the document captured.

The documents I request for a given connection time will be based on the document availability table above.  For example, if the earliest propagation will allow a good connection is 22z, I will likely wait until 23z to get the latest GRIB file on the same connection.

   How Did It Really Work?
Bob Calt and I departed Lagos, PT on May 6, 2007 and arrived in the Chesapeake Bay on June 17, 2007 with stops at Madeira and Bermuda.  Use the  "Atlantic Circle" menu to view pictures and descriptions of the voyage.  Below I describe how my acquisition of weather documents actually worked on the cruise.
After testing my planned document schedules for a week in the Lagos Marina I decided to remove the surface analysis chart from my plans.  These documents are very large (15-20KB) and chew up too much connect time.  So upon departure my document schedule included the following:
  • GRIB files via a SailDocs subscription twice each day at 06Z and 12Z
  • East and Central Atlantic Buoy Reports once each day at 0030Z
  • Met Area II Forecast twice each day at 06Z and 12Z

As we neared 40W Longitude I dropped the East Atlantic Buoy reports and MetArea II forecast and added the West Atlantic Buoy Reports, the Met Area IV forecast and the NOAA Map Discussion.

I found the Met Area Forecasts to be of minimal value as they are so general in their forecasts.  Their main value was the positioning of major weather systems that were outside of the GRIB charts I was receiving.  The Buoy reports (really ship reports) also were not of great value as we sailed well south the normal shipping lanes between North America and Europe.  The closest ship reports were normally several hundred miles from our position.  Still they had some value in validating the GRIB wind forecasts.

So I depended primarily on the GRIB files for my weather data and they proved to be very accurate, particularly in predicting the formation of minor storm systems in the Western Atlantic.  We were able to navigate around these storm systems during the final week of the voyage from Madeira to Bermuda.  During this period I increased my acquisition of the GRIBs from twice each day to four times each day.  Of course the daily input from Herb Hilgenberg was of equal value to the GRIBs for this navigation.

For the passage from Lagos to Madeira and for the first few days out of Madeira I used Winlink exclusively for document retrieval and email.  However connections to Winlink were very poor.  Either I could not connect to any of the stations or if I did connect the transmission errors forced a very slow transmission rate.  Two days out of Madeira I decided to try SailMail and got much more reliable connections.  Eventually I stopped using Winlink altogether until in the vicinity of Bermuda when I was able to get decent connections to a few North American Winlink sites.  Even then the SailMail connections were much better.  The only problem I had with SailMail was my accumulated connection time quickly exceeded that allowed.  However it only took an email to the SailMail Sysop to have my accumulated connection time reset to zero.  I never had a connection rejected by SailMail because of my accumulated connection time.

Below are links to the two Excel Workbooks I used as logs of my connections to Winlink and SailMail.

Overall this strategy served my pretty well.  I deviated from the schedule quite a bit due to Winlink connection difficulties.  However, having done this research and developed a document retrieval strategy made it much easier to work around the Winlink problems and allowed me to take maximum advantage of the connections I had with SailMail.  I also got great support from the SailMail Sysop as my dependence on SailMail for all documents pushed my connect time well over the 90 minutes/week normally allowed.  I sent an email to the Sysop when I was getting close to the limit requesting additional time.  The Sysop said she had re-set my connect time to zero, but she actually must have done more than that.  After the reset, when we were approaching Bermuda I increased my GRIB file retrievals and my SailMail connect time quickly grew to more than 200 minutes (twice the limit).  In spite of that SailMail continued to allow me to connect and retrieve all of the documents I wanted.  If you let SailMail know you have a connection time problem, they will insure that does not prevent you from making your connections while at sea.