Propagation during the 2023 Solar Eclipse

On Thursday the 20th of April, 2023 at 04:17:56 UTC the world was subjected to a rare event, a hybrid solar eclipse. In Perth I experienced a partial eclipse and people lucky enough to be directly in line, places like Ningaloo Reef, Exmouth and Barrow Island, experienced a total eclipse. Timor-Leste had the experience of the peak total eclipse.

At the time I went into my shack and refreshed the WSPR or Weak Signal Propagation Reporter beacon map I have open and noticed that my beacon wasn't reported. I sagely nodded my head, that makes sense, no Sun, no propagation and I got on with my life.

Last week a fellow amateur, Will VK6UU, asked if anyone had any VK6 specific HF propagation reports to make. Being the data geek that I am, I thought to myself, "Aha! I can do some data analytics on the WSPR dataset that I have." So, the die was cast for a few enjoyable hours of importing 2.4 gigabytes of compressed data into a database and constructing a set of SQL queries to see what I could learn.

Before getting stuck in, I spent a few hours thinking about the problem. How could I go about doing this? Propagation information is notoriously fickle. You have to consider the obvious things like the Solar Index and the Geomagnetic Index which vary considerably. Then there's the nature of the various reports themselves. Not everyone has their beacon on all the time, not everyone has their receiver on all the time. Weekends are more popular than weekdays and popularity overall is growing exponentially. The solar cycle is on the way to its peak, so there's that variation to consider and if that's not enough, how should you compare the Signal To Noise ratio between weak and strong beacons?

With all that in hand I set about constructing a plan. I created a folder to hold my charts and SQL queries, intent on uploading that to GitHub when the work was done.

For my very first test I thought I'd count the number of reports per band in a 24 hour window around the eclipse. I imported all the WSPR records that had a VK6 callsign, either as the transmitter or the receiver, given that I was interested in learning if stations transmitting from VK6 could be heard elsewhere and inversely, could VK6 stations hear any other stations?

As my first effort, I created a scatter-plot to get a sense of what kind of numbers I was looking at. The initial result was interesting. Around the eclipse itself there was no propagation. This wasn't unexpected, since that's what I'd seen on the day at the time on my own map. I changed my data to use a cumulative count per band to see if any band was particularly different and then discovered that there was no propagation at all, on any band.

That seemed ... odd.

So, I had a look at the source data and discovered a gap, which accounted for what my chart was showing. I added a fake record for the eclipse time itself, just so I could see where on the chart this gap was. Turns out that for VK6 stations, the gap is just over five hours, but it's not centred around the eclipse. There's a four hour window before the eclipse and a one hour window after it.

Then I started looking at all the reports from across the world. To give you a sense of scale, across April 2023 the dataset has nearly 139 million rows. It's 12 gigabytes in size. By contrast, in March of 2008 when the first reports started, there were just over 93 thousand reports in a 7 megabyte file. Charting this shows exponential growth, hitting a million reports in July of 2009, 10 million reports in January 2016 and 100 million reports in October of 2021.

So, the eclipse and global propagation. The results came in and the reports are that there was no propagation, on any band at any point during the just under two hours and 12 minutes before the eclipse and the 38 minutes following it.

That ... or the WSPRnet.org database was down during the eclipse.

So, unfortunately I cannot tell you what propagation was like during the eclipse, since it appears that those records don't exist.

Looks like we'll have to wait until 2031 when we can try this again. We'll all be a little older and wiser by that time and perhaps we can come up with a way to ensure that the global central WSPR data server is running without downtime, scaled to match the growing requirements and paid for by a benevolent organisation with deep pockets.

I did start considering making lemonade from my lemons and charting the kinds of down time the WSPR server has, but just looking over the various discussion groups showed that this is going to be painful. On the plus side, I learnt about SUM OVER and LAG functions in SQL, so there's that.

I must confess that if we're going to seriously use WSPR as a propagation analysis tool we need to fix these kinds of issues. I have no doubt that running WSPRnet.org is a massive enterprise and that it costs real time and money to make that happen.

So, who's up for the challenge and will the real owner of WSPRnet.org please raise their hand?

Finally, if by chance you were running a WSPR receiver during the 2023 Solar eclipse you might want to consider looking at sharing your logs, since they're potentially the only record still remaining.

I'm Onno VK6FLAB