7 new charts & 119 updated rasterized ENC charts
see GeoGarage news
Friday, July 26, 2024
Liftoff for the Norwegian satellites that will provide broadband to circumpolar Arctic
The two new satellites developed by Space Norway will provide broadband communication to the whole Arctic. Illustration: Space Norway
From ArcticToday by Atle Staalesen, The Independant Barents Observer
The two satellites that are up for launch by Elon Musk’s SpaceX will open a new era for broadband communication across the Arctic.
It will benefit the growing number of operators in the region, including allied defense forces.
“With these satellites Norway gets control over strategically important communication services in an area that until now had been without broadband coverage,” Norway’s Defense Minister Bjørn Arild Gram says in a comment.
“It will strengthen Norwegian security and national control,” he underlines.
Until now, there has been little or no broadband connection beyond 75° North, and air traffic, shippers and expeditions have depended on one-way voice link or Iridium satellite phone.
The new broadband connection will benefit aircraft, research vessels, fishing vessels, cruise ships, the Coast Guard, expeditions and other operators in the Arctic, the Norwegian Ministry of Defense informs.
Among the prime beneficiaries will be Norway’s Armed Forces and NATO allies.
“The defense forces will now get access to crucial communications capacities, which will be important for cooperation with allied forces in the North,” Minister Gram explains.
The two satellites that have been under development for the past five years will be launched in mid-July. They will be launched with a Falcon-9 reusable launch carrier at the Vandenberg Space Force Base in California.
The new broadband communication is expected to be in operation from late fall this year.
Norway is making major investments in space tech and intends to play a key role in the future development of space.
“Norway’s unique geographical location, our competence and capacity within development and launch of satellites, as well as downloading of satellite data, gives Norway a comparative advantage in cooperation with the USA, EU and other allies,” Bjørn Arild Gram argues.
The Norwegian satellite project is run by state-owned company Space Norway and its HEOSAT subsidiary.
The project is named the Arctic Satellite Broadband Mission (ASBM) and is based on a contract with Northrop Grumman Space Systems.
The latter will build the two satellites that are based on the GEOstar 3 platform.
According to Space Norway, the satellite payloads will include X-band for the Norwegian Armed Force, Ka-band for Viasat and a payload for the US military.
One of the satellites will include also a radiation monitoring instrument that is to gather information about the radiation environment in the satellite orbits.
Links :
Thursday, July 25, 2024
How to catch a Russian submarine
Submarine crews of around 130 people share four sinks and three toilets, with little to no contact with the outside world - 'it's challenging', admits Captain Ryan Ramsey Credit: John Clews/Crown Copyright
From The Telegraph by Dominic Nicholls
A Navy frigate commander and a British submarine captain on playing cat and mouse in a ‘little bubble of GCHQ’
When a torpedo starts coming straight at you, there’s only one thing a submarine crew can do – snap into “full evasion” mode.
Captain Ryan Ramsey issues swift orders.
His crew responds immediately, twisting the Royal Navy’s Trafalgar-class submarine to manoeuvre the boat beneath the waves in an effort to shake off the weapon bearing down fast on their position.
It would, though, take a miracle for a submarine of nearly 6,000 tons to outrun a torpedo.
And that miracle never comes.
The torpedo smashes into the fin, the large central structure on the submarine sometimes called the conning tower.
As it hits the boat, a loud bang reverberates hard through the hull, a high-strength steel alloy designed to withstand the extreme pressure experienced at the submarine’s maximum – and classified – diving depth.
It is lucky, then, that it is only an exercise, and that the Dutch torpedo has not been fitted with an explosive warhead.
The depth setting on the torpedo has been accidentally but incorrectly adjusted, meaning that the weapon should, on this particular exercise, never have been able to go deep enough to see the British submarine lurking below.
Nevertheless, the incident “focused the mind,” says Ramsey, who has now retired.
Ramsey was in the submarine service for 23 years.
For three of those, he was captain of the fast-attack HMS Turbulent, and deployed in a “cat and mouse” game of anti-submarine warfare.
It quickly becomes apparent that before we even talk about Russian submarine commanders (five of which Ramsey met at a function in 2009: “I learned so much in that vodka session that I applied later on”) the most challenging aspect of their work was the sea itself.
Commander Ramsey was in the submarine service for 23 years, three of those as captain of the fast-attack HMS Turbulent
Credit: South West News Service
“I always found it amazing that when we took a submarine off on operations, we were going to take the most complex platform that the British military owns into something that’s less explored than space,” Ramsey says.
Tom Sharpe, a retired Royal Navy commander with 20 years service at sea – on the surface – agrees.
“When you let go all lines and sail, you’re on operations irrespective of whether the enemy is present,” says Sharpe, whose tenure included command of an anti-submarine frigate.
“You’re in harm’s way from forces that far exceed anything an enemy can throw at you.
If you end up in it, your chances of survival diminish really, really quickly.”
And that’s just on the outside.
Inside the vessel, whether above or below the surface, conditions are equally harsh.
Submariners work a six-hours on, six-hours off shift pattern.
“That’s relentless in itself,” Ramsey says.
“You’re not getting a full night’s sleep, you might get a maximum of four hours at a time.”
“You can sustain this almost indefinitely but it’s not much fun,” Sharpe adds.
“Before long, it feels more like existing than living.”
The submarine crew of around 130 people share four sinks and three toilets.
“It’s challenging,” Ramsey observes, in his typically understated fashion.
“You’ve got a load of people in a steel tube and no contact with the outside world.
You receive a telegram maybe once a week, of 150 words, from your family.
You can’t speak to them.
You can’t pass messages to them.
Occasionally there’ll be friction on board.
People have arguments.
The important bit is to let those arguments play out because otherwise tensions build.
If they overstep, then you intervene.”
It sounds like a combustible environment.
So why do countries invest so much time and money into sub-surface warfare?
“Nations can exercise political intent using under-sea assets effectively,” Ramsey says.
“They can threaten, they can spy, they can gather intelligence.
They can do all kinds of things.”
He says submarines should be thought of less as metal cigar tubes trying to find other metal cigar tubes and more as “little bubbles of GCHQ”.
HMS Turbulent returns to Devonport Plymouth in 2003 after serving in the second Gulf War
Credit: Barry Batchelor/Pa
“Submarines are the first line and last line of defence for the United Kingdom.
The last line of defence is obviously the strategic ballistic missile submarines with the Trident D5 [nuclear missiles].
But the first line of defence is intelligence gathering.
And that’s going to places in the world to gather intelligence and assess future capability to make sure that we’re ready to defend against it.”
Of course, there are many countries trying to do the same to Britain.
Finding hostile submarines is a team sport.
He says for the surface fleet, the most important concern is to stay outside an enemy submarine’s torpedo range of “high single digit miles”, adding: “If you’re outside of that you can do whatever you like as a ship.”
But he sounds a note of caution: “If you’re a single frigate against a single submarine, you’re going to be in trouble.
There are no absolutes in war fighting, there are so many variables in the business of anti-submarine warfare, but a singleton frigate against a submarine? It’s advantage to the submarine every time.”
Hunting Russian submarines starts with intelligence gathering.
Through satellite imagery and other sources, the British military would usually know when a Russian boat has deployed from its home base of Murmansk in the Russian arctic region.
Such knowledge would be highly classified and not shared widely around the Nato alliance.
Underwater sensors located on the seafloor in the “Greenland-Iceland-UK gap” would send a silent alert back to headquarters when a Russian vessel crossed into the high North Atlantic.
Long-range patrol planes, such as the RAF’s P-8 Poseidon multi-role maritime patrol aircraft, which is equipped with sensors and weapons systems for anti-submarine and surface warfare, would refine the search, perhaps aided by a friendly submarine.
After that, a close-in search would be conducted by frigates using towed-array sonars trailing behind the ship, dipping deep into the ocean to listen for the tell-tale sounds of man-made technology beneath the waves.
The ships would send out “pings”, basically a sound that would hopefully reach an enemy submarine and be reflected back.
“You’re giving away your position but in this case it doesn’t matter,” Sharpe says.
“As long as you’re outside their weapon range you are safe.”
However, “finding a well-run nuclear submarine on passive only [not emitting any noise] is a game probably only for another well-run submarine, not frigates,” he concedes.
Only when satisfied the Russian vessel had been found would the frigate’s commanding officer launch the on-board Merlin helicopter to locate the submarine and, if ordered, launch a weapon.
In the case of the Royal Navy helicopters this would be a Sting Ray torpedo with its sophisticated acoustic homing system and 45kg explosive charge, which is powerful enough to punch through the double hulls of modern submarines.
Sound can travel vast distances underwater and a frigate’s variable depth sonar will, in the right conditions, pick up noise from hundreds of miles away.
Exactly how far sound will travel underwater is governed by a number of factors.
Commanders above and below the surface are aided by highly sophisticated systems to construct mental models of the sea, where depth, temperature, salinity and distance from land all combine to form “layers” in the water.
Sonar pings can bounce off these layers in the right conditions, meaning submarines can hide below – potentially close to ships, but utterly unseen.
Submarine captains have to use their judgment to position their boat correctly in the layers to hunt or hide.
Grueling ‘Perisher’ course
Ramsey trained British and American would-be commanding officers on the Royal Navy’s gruelling “Perisher” course.
This unforgiving final hurdle, that tests future submarine commanders in every aspect of their work before they can be appointed, has just a 60 per cent pass rate.
Success or failure is a very human construct, Ramsey says.
“Ego has a huge part to play in this.
You see some captains that have massive egos that push the limits way beyond where they should do.
“We would get into contact with the enemy and start trailing them [but] we need to tell somebody so they can position other forces.
But breaking off to go back up to periscope depth to transmit, to tell somebody, means you lose the tactical advantage immediately.
“I always found that to be the real challenge.
How far do we push this before we pull out and say, ‘right, he’s going in this direction, you need to put other forces in to intercept’?
“Anti-submarine warfare is by far the biggest game of chess that you could ever participate in.
“I describe it as being in a really dark room where you’ve got a knife, the other guy’s got a knife, you can’t hear him and he can’t hear you, but you’re looking round, hoping, waiting for the person to make the first sound so you can get in there and deal with them.”
Has Ramsey been in contact with Russian submarines? “I’ve been in contact with many submarines in my time,” he replies cryptically.
But being detected by an enemy submarine is “a different game,” he says.
“You’re no longer in tactical control.
You need to work out how you’re going to evade, get out of there, and come back another time.
You don’t know whether they’re going to fire weapons or not.
You don’t know what their rules of engagement are; it’s not like we’ve spoken to them before we started this.
We have no idea what their rules of engagement are, we only know what ours are.
It’s exceptionally tense.”
Sharpe agrees: “Anti-submarine warfare is a complex and confusing business and the captain who can impose order on that the quickest is the one that inevitably ends up winning.”
The last line of defence is obviously the strategic ballistic missile submarines with the Trident D5 [nuclear missiles].
But the first line of defence is intelligence gathering.
And that’s going to places in the world to gather intelligence and assess future capability to make sure that we’re ready to defend against it.”
Of course, there are many countries trying to do the same to Britain.
Finding hostile submarines is a team sport.
He says for the surface fleet, the most important concern is to stay outside an enemy submarine’s torpedo range of “high single digit miles”, adding: “If you’re outside of that you can do whatever you like as a ship.”
But he sounds a note of caution: “If you’re a single frigate against a single submarine, you’re going to be in trouble.
There are no absolutes in war fighting, there are so many variables in the business of anti-submarine warfare, but a singleton frigate against a submarine? It’s advantage to the submarine every time.”
Hunting Russian submarines starts with intelligence gathering.
Through satellite imagery and other sources, the British military would usually know when a Russian boat has deployed from its home base of Murmansk in the Russian arctic region.
Such knowledge would be highly classified and not shared widely around the Nato alliance.
Underwater sensors located on the seafloor in the “Greenland-Iceland-UK gap” would send a silent alert back to headquarters when a Russian vessel crossed into the high North Atlantic.
Long-range patrol planes, such as the RAF’s P-8 Poseidon multi-role maritime patrol aircraft, which is equipped with sensors and weapons systems for anti-submarine and surface warfare, would refine the search, perhaps aided by a friendly submarine.
After that, a close-in search would be conducted by frigates using towed-array sonars trailing behind the ship, dipping deep into the ocean to listen for the tell-tale sounds of man-made technology beneath the waves.
The ships would send out “pings”, basically a sound that would hopefully reach an enemy submarine and be reflected back.
“You’re giving away your position but in this case it doesn’t matter,” Sharpe says.
“As long as you’re outside their weapon range you are safe.”
However, “finding a well-run nuclear submarine on passive only [not emitting any noise] is a game probably only for another well-run submarine, not frigates,” he concedes.
Only when satisfied the Russian vessel had been found would the frigate’s commanding officer launch the on-board Merlin helicopter to locate the submarine and, if ordered, launch a weapon.
In the case of the Royal Navy helicopters this would be a Sting Ray torpedo with its sophisticated acoustic homing system and 45kg explosive charge, which is powerful enough to punch through the double hulls of modern submarines.
Sound can travel vast distances underwater and a frigate’s variable depth sonar will, in the right conditions, pick up noise from hundreds of miles away.
Exactly how far sound will travel underwater is governed by a number of factors.
Commanders above and below the surface are aided by highly sophisticated systems to construct mental models of the sea, where depth, temperature, salinity and distance from land all combine to form “layers” in the water.
Sonar pings can bounce off these layers in the right conditions, meaning submarines can hide below – potentially close to ships, but utterly unseen.
Submarine captains have to use their judgment to position their boat correctly in the layers to hunt or hide.
Grueling ‘Perisher’ course
Ramsey trained British and American would-be commanding officers on the Royal Navy’s gruelling “Perisher” course.
This unforgiving final hurdle, that tests future submarine commanders in every aspect of their work before they can be appointed, has just a 60 per cent pass rate.
Success or failure is a very human construct, Ramsey says.
“Ego has a huge part to play in this.
You see some captains that have massive egos that push the limits way beyond where they should do.
“We would get into contact with the enemy and start trailing them [but] we need to tell somebody so they can position other forces.
But breaking off to go back up to periscope depth to transmit, to tell somebody, means you lose the tactical advantage immediately.
“I always found that to be the real challenge.
How far do we push this before we pull out and say, ‘right, he’s going in this direction, you need to put other forces in to intercept’?
“Anti-submarine warfare is by far the biggest game of chess that you could ever participate in.
“I describe it as being in a really dark room where you’ve got a knife, the other guy’s got a knife, you can’t hear him and he can’t hear you, but you’re looking round, hoping, waiting for the person to make the first sound so you can get in there and deal with them.”
Has Ramsey been in contact with Russian submarines? “I’ve been in contact with many submarines in my time,” he replies cryptically.
But being detected by an enemy submarine is “a different game,” he says.
“You’re no longer in tactical control.
You need to work out how you’re going to evade, get out of there, and come back another time.
You don’t know whether they’re going to fire weapons or not.
You don’t know what their rules of engagement are; it’s not like we’ve spoken to them before we started this.
We have no idea what their rules of engagement are, we only know what ours are.
It’s exceptionally tense.”
Sharpe agrees: “Anti-submarine warfare is a complex and confusing business and the captain who can impose order on that the quickest is the one that inevitably ends up winning.”
Links :
Wednesday, July 24, 2024
Lessons learned from the subsea project to raise Shackleton’s Endurance
The Endurance sank in 1915 after being crushed in the ice
(Credit: © Royal Geographical Society-IBG)
From APM
Nico Vincent was Subsea Project Manager of the Endurance22 expedition.
On 5 March 2022, his team found the wreck of Endurance, Sir Ernest Shackleton’s ship, which sank off the coast of Antarctica 107 years ago.
In July, Vincent sat down with APM Podcast host Emma De Vita (Editor, Project journal) to explain how the project team used autonomous underwater vehicles (AUVs) to find the shipwreck, one of the most complex subsea projects ever undertaken.
Vincent has spent 30 years recovering deep-sea wrecks and shared his project management insights on what it takes to prepare for a once-in-a-lifetime project, how it feels to see such a piece of history and the lessons he took away from the successful project.
Below is an edited extract from his interview.
APM Podcast: To start with, could you give us an overview of the project?
Nico Vincent: The purpose of Endurance22 was to discover the wreck of Endurance and then to deliver a full survey of the wreck to allow scientific analysis.
The wreck is in the Weddell Sea; there is a lot of sea ice in this area and it is quite unpredictable.
The reason Endurance was so difficult to find was not because she was really lost, but because access to the area is so complicated.
As the wreck is protected by the Antarctic Treaty, we were forbidden to touch anything, so we needed to produce a high-resolution survey of the wreck to allow scientists and archaeologists to produce a scientific report.
On 5 March 2022, his team found the wreck of Endurance, Sir Ernest Shackleton’s ship, which sank off the coast of Antarctica 107 years ago.
In July, Vincent sat down with APM Podcast host Emma De Vita (Editor, Project journal) to explain how the project team used autonomous underwater vehicles (AUVs) to find the shipwreck, one of the most complex subsea projects ever undertaken.
Vincent has spent 30 years recovering deep-sea wrecks and shared his project management insights on what it takes to prepare for a once-in-a-lifetime project, how it feels to see such a piece of history and the lessons he took away from the successful project.
Below is an edited extract from his interview.
APM Podcast: To start with, could you give us an overview of the project?
Nico Vincent: The purpose of Endurance22 was to discover the wreck of Endurance and then to deliver a full survey of the wreck to allow scientific analysis.
The wreck is in the Weddell Sea; there is a lot of sea ice in this area and it is quite unpredictable.
The reason Endurance was so difficult to find was not because she was really lost, but because access to the area is so complicated.
As the wreck is protected by the Antarctic Treaty, we were forbidden to touch anything, so we needed to produce a high-resolution survey of the wreck to allow scientists and archaeologists to produce a scientific report.
Falklands Maritime Heritage Trust and Esther Horvath
Tell us about your role.
NV: The [Falklands Maritime Heritage Trust] asked the expedition leader to lead above the surface and me to drive all operations under the sea.
However, to run the operation I was obliged to control surface support as well, including navigation, the helicopter flight plan and the ice camp.
The project schedule for me started in February 2019, when the previous expedition lost its AUV on site and I was immediately contacted to build a new solution.
The first step was to launch the manufacturing of the Sabertooth AUVs.
Tell us more about Sabertooth.
NV: Sabertooth is a vehicle which is able to reach 3,000m.
It’s an AUV manufactured by Saab in Sweden, and the main reason why we choose that vehicle is because it’s quite versatile.
Usually, this kind of vehicle is flying fully autonomous underwater, but for this project we asked to use a tethered one.
This is a lesson learned from 2019, which was to keep permanent control and anticipate the need for an emergency ascent in an incident.
That’s really a game changer on the ice.
For such a complex and unique project, was it important to come with an open mind, to find creative solutions and be innovative?
NV: Absolutely.
I spent a year building procedures, a scope of work, a task plan, server planning, deployment on ice, helicopter flight plans, time computing schedules.
The reason why I was on board is because we were all open that nothing would run as expected.
The environmental conditions on site are so complicated that you cannot anticipate all scenarios.
Usually in the subsea industry the project manager stays in the office.
The complexity of the variables that we faced created the obligation that I be on board.
Falklands Maritime Heritage Trust and Nick Birtwistleorvath
What kind of skills did you need on team for this kind of project?
NV: Usually in the subsea industry, all the skills or expertise are quite disconnected from each other.
People have a very important task and they do not look at what others are doing.
When you are doing a project like Endurance22, the opposite needs to be true.
People must have very strong connections with each other.
The leadership may switch from one person to another according to expertise.
That means that your boss in the morning, because you are driving an AUV, can become your support in the afternoon, because you are doing electronics.
That is something that allows you to build a very strong relationship between people.
What was it like to work at such close quarters and under such pressure?
NV: The advantage when you work with recurrent people is that you know them.
Being in a difficult constraint, all living together during months and months, is something normal for us, because we do that all year.
But for Endurance22, in addition, we were on an icebreaker and we were in a polar area, which is quite unusual for us.
But life on board is something we are ready for.
We run a watch of 12 hours on, 12 hours off, for example.
So the full team is organised to work 12 hours per day.
On Endurance22 most of the guys did huge overtime, not only because we faced issues, but because they love what they do.
If you are able to work on something that is so exciting and attractive, people respond on their own, especially if they like each other.
Ernest Shackleton's lost ship, Endurance, has been found after 107 years.
This 4k footage shows the preserved vessel 3008 metres below the ocean surface.
The ship was discovered just four miles south of the location recorded at the time by ship's captain, Frank Worsley.
What have been the highs and the lows of working on Endurance22?
NV: Highs? The discovery.
Lows are a very long list [laughs].
Each dive was a stressful time.
You have to keep in mind that the previous team lost another vehicle after two dives.
So my goal was not only to find Endurance, but to come back with my vehicle.
We had eight emergency ascents over 32 dives.
But seeing the vehicle back on deck after the last dive with the full scope accomplished was an extremely good time.
The team faced the worst conditions.
We had -22°C with 35 knots of wind, which is equivalent to -40°C.
The equipment was freezing on deck; the people were absolutely freezing on deck.
They never said anything.
They accomplished a huge task and I am very proud of them.
Listen to the full interview with Nico Vincent here or search ‘APM Podcast’ on your preferred podcast app.
Links :
- Maritime Executive : UK Seeks to Expand Protections for the Wreck of Shackleton's Endurance
- GeoGarage blog : Endurance: Shackleton's lost ship is found in Antarctic / Will anyone ever find Shackleton's lost ship? / Shackleton's Endurance: Modern star maps hint at famous ... / Explorer Shackleton's last ship found on ocean floor / Antarctic expedition to renew search for Shackleton's Extraordinary 1915 photos from Ernest Shackleton's disastrous ... / Antarctic Weddell expedition targets Shackleton's lost ship / Walking in Shackleton's footsteps / Ernest Shackleton voyage to be retraced by modern-day ... / Shackleton Death or Glory - Rough Seas / 11 of History's most famous sea voyages / Henry Worsley's journey wasn't foolhardy / Scott and Shackleton logbooks prove Antarctic sea ice is ... /
Tuesday, July 23, 2024
Meteorological data rescue operations improve Copernicus reanalysis
Weather record treasures await to be discovered : C3S, WMO launch joint data rescue efffort portal
From Copernicus
Data rescue is a relatively little-known field, but it is essential to complete our knowledge of the past climate, in particular in the most remote areas of the planet, where weather records are scarce.
Reanalysis datasets such as the Copernicus Climate Change Service’s ERA5 resolve these “gaps” with sophisticated computing operations, but the more information we have about the real conditions, the better the job it can do.
That’s why C3S is involved in Satellite Data Rescue and In situ Observations rescue operations.
On the occasion of the launch of a joint C3S-World Meteorological Organization data rescue portal, we take an in-depth look at these fascinating but sometimes overwhelming activities.
On 22 June 1911, the German cargo sailing vessel Peking left the harbour of Hamburg to reach Valparaiso, Chile, on 14 September.
This first trip, along with the following 19 trips made by the Peking for the F. Laeisz shipping company were carefully recorded in the meteorological “Tagebücher”, the ship logs.
The objective of these meticulous records was to gather information about the best shipping routes.
The Peking sailing on the Elbe estuary (Source: Stiftung Hamburg Maritim, © Hans Hartz)
The handwritten ship logs of the Peking, but also from meteorological stations across Germany and overseas were carefully conserved by the German Meteorological Service, Deutscher Wetterdienst (DWD) for decades.
Millions of weather records have already been digitised and serve to improve our knowledge of the past weather, ingested in reanalysis datasets such as the Copernicus Climate Change Service’s ERA5.
But millions of other meteorological records await.
That era of the pioneer weather data recording has left behind a legacy of tonnes, kilometres, entire rooms and buildings of climate data records in paper.
Now let’s multiply the case of DWD by the number of other meteorological services, agricultural associations, environmental agencies, research and exploration societies and other entities around Europe and the world.
Trillions of handwritten weather logs, that shed light on the past climate before the automated records and the satellite era, wait to be transcribed and ingested into the modern weather and climate datasets.
The task is colossal.
View of some shelves from the DWD Maritime Archive in Hamburg, Germany. Source: Deutscher Wetterdienst
The new portal, a joint effort to harmonise global records
Until now, WMO and C3S led the two main international initiatives to harmonise, share and gather old weather files.
Now the two organizations, with the cooperation of national weather agencies, have joined forces to present a unified portal under the leadership of the Royal Netherlands Meteorological Institute (KNMI).
The new portal provides the tools, guidelines and access to global resources to set the standards and help international data rescue efforts move forward.
Paul Poli, C3S’ In-situ Observations Manager welcomes the arrival of the new portal.
“The data rescue community is a dedicated one and does not fear tackling such a daunting task.
In fact, the name they historically chose for themselves says more than a long story: I-DARE, which is short for International Data Rescue.
We owe it to all the individuals involved, that we do everything we can to maximise the impact of their efforts.
By merging the activities with WMO and other international partners we are helping operationalise the operations and this will have a huge impact in the breadth, diversity, quality, and usability of the data.”
“There are many activities like ours, and they’re fragmented all over the world,” says Axel Andersson, from DWD’s Marine Climate Monitoring Division, “and the new portal gives an overview, a point where we can submit the data and make sure it will be used and provides a systematic way of submitting the data.”
“Integrating our activities will give us a global reach, and it was a natural fit,” said Peter Thorne, Professor of Physical Geography (Climate Science) that leads the C3S data rescue project.
KNMI was also a natural choice because it has supported the I-DARE community for years and it was involved in WMO data rescue.
“The objective is to provide a state-of-the-art set of guidelines and a centralised portal for our members, just to try to work more efficiently,” said Peer Hechler, leading the Data Rescue at WMO.
The new portal is an enabler, but “it requires people to actively participate in it, to register their projects, to keep abreast of their projects.
This joint portal is a part of the toolbox, but we need to do the hard yards of actually getting these data rescued, and that will happen through multiple methods and multiple groups,” stated Peter Thorne.
‘Scheepsjournaal van het Nederlandse schil Noordbeveland uit 1761’: logbooks of the Dutch vessel Noordbeveland from 1761.These logbooks contain precious meteorological information, on the condition of applying efficient methods to transcribe them.
Courtesy: Royal Netherlands Meteorological Institute (KNMI)
Artificial intelligence and human intelligence
Some years ago DWD made an estimate of how long it would take to fully digitise all the data available in the German marine archive in Hamburg, and they came up with some impressive numbers: 300 person-years would be needed.
So, 300 people dedicated to digitising for a full year or one person for 300 years.
And this is the case for a single member of the 193 WMO members, even if data rescue operations sometimes don’t refer to centuries-old files.
“The data situations are very different around the world.
In Cambodia, for example, where I was on a mission, data rescue applies for the decade of the 2000s because their records from that period are still only available on paper and not digitised,” points out KNMI’s Gerard van der Schrier.
Only a few WMO member have the resources to tackle this almost “endless task”.
The rapid advances in artificial intelligence could mean a significant push for data rescue activities, but it is still challenging to read handwriting and correctly tabulate the data.
And a human is still needed to input the physical data records to the machine, in particular for century-old papers that require handling with additional care.
“There are huge opportunities coming down the line from AI and OCR (optical character recognition) technologies, which for a long time we thought would not be capable [of the task].
But in the last few months we have seen a sea change in that.
If you could get to the point where AI could do 80% or 90% and leave humans to verify and do the remaining 10% or 20% you would get to a whole new scale,” says Peter Thorne, who adds that the real bottleneck is digitalisation.
Logbook of a German ship kept at the DWD Maritime Archive in Hamburg, Germany.Source: Deutscher Wetterdienst
“Until these techniques are perfected, WMO still recommends to manually key in the data,” says Peer Hechler.
“I believe it will take some years to fully develop.
And AI won’t solve all the problems, because very often the archives are sitting on a shelf, those lucky ones that have a shelf, but nobody knows what’s in there.
Very often there’s very difficult work of sorting the archives that can only be done manually,” adds Hechler.
Data rescue is a field where citizen science and student initiatives have played an important role, given the amount of work needed and the relatively reduced resources available.
Anonymous contributors and weather passionate volunteers have helped digitise millions of observations and some initiatives have given rise to pages like Old Weather, which has helped digitise data from Arctic exploration, whaling or World War II.
A recent example was the Rainfall Rescue project, with some 16,000 volunteers digitising 5.2 million observations in just 16 days.
ACRE (Atmospheric Circulation Reconstructions over the Earth) and IEDRO (International Environmental Data Rescue Organization) are some of the key international players in data rescue besides WMO, Copernicus and the national meteorological agencies.
“Data rescue involves a huge number of people, and there’s not a single modality,” says Peter Thorne.
Many data rescue volunteers become somehow addicted to log the files while discovering unheard weather events and meteorological conditions in the process.
Anonymous contributors and weather passionate volunteers have helped digitise millions of observations and some initiatives have given rise to pages like Old Weather, which has helped digitise data from Arctic exploration, whaling or World War II.
A recent example was the Rainfall Rescue project, with some 16,000 volunteers digitising 5.2 million observations in just 16 days.
ACRE (Atmospheric Circulation Reconstructions over the Earth) and IEDRO (International Environmental Data Rescue Organization) are some of the key international players in data rescue besides WMO, Copernicus and the national meteorological agencies.
“Data rescue involves a huge number of people, and there’s not a single modality,” says Peter Thorne.
Many data rescue volunteers become somehow addicted to log the files while discovering unheard weather events and meteorological conditions in the process.
Even if the personal and collective citizen science initiatives are highly appreciated and necessary, data rescue operations need to be coordinated and to follow precise format rules and guidelines to be efficiently used and integrated into global datasets, which is the main ‘raison d’être’ of the new data rescue portal.
Much of the old meteorological data was saved on punch cards.This is a picture of the archive of these punch cards.
It illustrates that progress of technology is a thread to safe and sustainable archiving of old data.
Courtesy: Royal Netherlands Meteorological Institute (KNMI)
Is it worth the effort?
But why so much effort? Why undertake a mission that – even more before the arrival of AI— is deemed almost impossible to achieve? The reasons are many and varied.
Faced with a chaotic weather system, humanity has always kept track of the changes in weather, to try to better understand the best routes for trade, travel, the best geographical areas to settle in and more.
Axel Andersson reminds us that the early shipping logs were at the foundations of the International Meteorological Organization, the predecessor of the current WMO “already in the 19th century they started to exchange data to produce climatological charts.
This international data exchange system is still relevant today and is of crucial importance for WMO.” he says.
The modern data rescue serves to support reanalysis, either by filling gaps or as a tool to verify the performance of a reanalysis dataset.
“The old data form data-sparse regions are particularly important.
It could have a big impact on regions where we don’t have any data.
We have input data collections for storm surges and storm floods in the 19th century.
It’s about learning from the past to understand the future,” says Axel Andersson.
Having previously been involved in the generation of century-long reanalyses, Paul Poli jumps on the question of whether focusing efforts on in-situ observations is not just a thing of the past, when today so many new satellites abound.
His response is an animation showing the evolution of the global observing system since 1940.
As the data coverage illustrates, the foundational role of in-situ observations is clearer.
“Improve this coverage, he adds, and the quality of reanalysis in the 1970s or before will improve for everyone using these data today – keep in mind we estimate that a quarter million people are using ECMWF reanalysis data, and if you count all the other reanalysis products, then globally this is probably far more.”
Observations assimilated in ERA5
Professor at Reading University Ed Hawkins, a great advocate of data rescue, and a dedicated data rescue volunteer himself, demonstrated in 2022 how historical weather observations improved risk quantification for severe storms.
“Data rescue is one of the key priorities of WMO since its foundation, because our members have a lot of observations from the past, and for predicting our climate we need huge samples of observational data.
We need these observations to feed our computers in order to get a good analysis of the climate and develop climate models,” explains Peer Hechler.
Peter Thorne points out that, while climate models evolve thanks to the advances in computing and climate science, “the raw observations that inform all these wonderful datasets, including reanalyses, they are forever.
Without rescued data none of these state-of-the-art datasets would exist,” he says, “If we’re blind to the past we’re going to be surprised in the future.”
Data rescue also serves research, and not only in climate and weather but also in social sciences.
Beyond the clear benefits of ingesting the data into modern databases, there is also a sort of moral obligation face to the people that gathered those data, often at a great risk or in harsh conditions.
There was a time when gathering weather records was somehow heroic.
With heavy and rudimentary instruments and often in remote areas bashed by extreme weather, the early meteorology data observers would spend weeks or months gathering daily data records and maintaining their instruments in good shape.
KNMI’s Gerard van der Schrier said that one of the aspects that keeps him passionate about data rescue is to discover major weather events.
“You dive into those old records, and you discover events that you know about because they had an impact, and were told by the newspapers, for example hurricanes with impacts in crops or houses being lost.
Then looking at this data actually makes meteorology become alive, this gives you a vision on how a particular event developed and impacted a region,” said van der Schrier, citing the example of Indonesia, where there’s a lot of data because large farms kept weather records.
There are for example very detailed records of the eruption of the Krakatoa in 1883 that tell, at high resolution of how the weather and climate changed in the aftermath of the eruption.
“It gives a view that we didn’t have before, and I think that is the value of these old records.”
New portal, new perspectives for data rescue
C3S and WMO expect that the new portal will help national meteorological services, volunteers and researchers give a new impetus to data rescue activities.
“I'm quite happy about the last two or three years when we worked closely together with Copernicus as well as with our other global partners like IEDRO and ACRE.
We all came together and try to bundle our resources and try to provide to the world single solutions, a single portal and a single set of guidelines.
I believe this was much needed,” concludes Peer Hechler.
“It is great to see the data rescue community coming together to maximise the efforts.
The new perspectives opened by artificial intelligence can be a game changer for data rescue activities around the world.
We are very excited with merge of this activity together with WMO and we hope we will receive millions of additional observations that we didn’t know about before,” said Paul Poli.
Peter Thorne agrees: “Creating data that’s usable as a global archive is a huge undertaking but it’s hugely necessary.”
Links :
- Vimeo : Data rescue and reprocessing
- Copernicus : C3S reconstructs the D Day weather using ERA5 data
- GeoGarage blog : Old weather “time machine” opens a treasure trove for researchers / Decades of detailed weather reports pulled from old ... / Citizen science: trawl World War I navy records for weather data / Old sailor logs show how frighteningly fast the Arctic is ... / NOAA, National Archives team up with citizen-scientists to ...
Subscribe to:
Posts (Atom)