Minimize Martime AI-NAV

Martime AI-NAV (Artificial Intelligence / Machine Learning Sensor Fusion for Autonomous Vessel Navigation)

References

November 16, 2020: A day of ferry trips between Finland and Estonia became some of the best documented voyages in maritime history. Cameras, sensors, radio and satellite navigation receivers and even microphones recorded every instant of the crossings over the Baltic, gathering raw data for a new ESA-led project applying AI to the situational awareness of shipping – as an important step to full autonomy. 1)

The Tallink shipping company’s new 212.2 m-long Megastar passenger and car ferry was fitted with data-gathering devices for its sailings on the busy stretch of sea between Helsinki and Tallinn.

MaritimeAI-NAV_Auto4

Figure 1: Photo of the Tallink Megastar ferry (photo: AS Tallink Grupp)

The testing was overseen by a team from the Finnish Geospatial Research Institute (FGI) for an ESA project called Artificial Intelligence / Machine Learning Sensor Fusion for Autonomous Vessel Navigation, or Maritime AI-NAV for short.

“Our aim is to show how AI and machine learning can be applied to achieve autonomous situational awareness, so that a ship can reliably sense its own environment,” notes Sarang Thombre of FGI.

MaritimeAI-NAV_Auto3

Figure 2: A day of ferry trips between Finland and Estonia became some of the best documented voyages in maritime history. Cameras, sensors, radio and satellite navigation receivers and even microphones recorded every instant of the crossings over the Baltic, gathering raw data for a new ESA-led project applying AI to the situational awareness of shipping – as an important step to full autonomy (image credit: FGI)

“Such autonomous systems would initially be deployed in support of human crews, for enhanced safety and efficiency – with crewless ships a much longer-term goal.

“The most experienced human ship captains will have the least trust in any single navigational device but will rather continuously cross reference between them. Similarly, our autonomous functionality will not be overly reliant on a single data source but combine and verify data from multiple sensors.

“Having gathered many gigabytes of data during this 20 August field campaign, we are now using it to train and test our data-fusing algorithms. A follow-up seagoing test will then verify their performance in practice.”

MaritimeAI-NAV_Auto2

Figure 3: Photo of an installed Satnav antenna (photo credit: FGI)

The Maritime AI-NAV team plans in employing a variety of sensor types, including satellite navigation receivers – also making use of Europe’s Galileo system – inertial measurement units, monocular and stereo cameras, standard radar, ‘laser radar’ lidar and an array of microphones, along with Automatic Identification System radio signals. These AIS signals transmit position, size and routing information of all vessels above a certain class, as well as fixed infrastructure such as oil rigs or wind turbines.

Dr Thombre adds: “Satellite navigation lets the ship know where it is in the sea, while the other sensors let it know what is around it, which is essential for identifying and avoiding any obstacles.

“The different data sources operate across a variety of ranges – so radar and AIS provide longer range detection out to the horizon, while cameras and lidars come into their own at shorter distances. Plus we had a trio of microphones aboard the Megastar, determining the angle of arrival of sound from other ships. The challenge now is to fully integrate all these sources using AI and machine learning, to build up a holistic picture.”

Maritime AI-NAV is supported through ESA’s Navigation Innovation and Support Program (NAVISP), working with European industry and academia to develop innovative navigation technology.

FGI is joined in the Maritime AI-NAV consortium by Aalto University’s Sensor Informatics and Medical Technology group and maritime IT startup Fleetrange Ltd.

MaritimeAI-NAV_Auto1

Figure 4: Gathering data from ferry trips (image credit: FGI)

MaritimeAI-NAV_Auto0

Figure 5: Storing sensor data (photo credit: FGI)

Maritime transport is currently facing new challenges such as significant increases in transport volumes, more stringent environmental requirements and a shortage of seafarers in the future.

One of the many new concepts having the potential to overcome these challenges is autonomous ship navigation. In particular, the concept is expected to allow for more efficient and competitive ship operations while reducing the vessels’ environmental impact.

The objectives of this activity are to:

• Study artificial intelligence and machine learning techniques for the combined use of multiple sensors in maritime PNT receivers for autonomous vessel navigation;

• Implement a proof-of-concept prototype autonomous navigation PNT receiver using the techniques identified;

• Study the feasibility of implementing an autonomous vessel navigation PNT service leveraging current and future European GNSS using the techniques identified.

The tasks to be performed will include:

• State-of-the-art review of artificial intelligence algorithms (neural networks, machine learning, decision trees, etc.) applied to sensor fusion for general navigation;

• Analysis of operational requirements in terms of at least the accuracy and integrity of unmanned vessels for at least two representative maritime operations;

• Study, define and trade-off artificial intelligence methods and techniques for the combination of data from multiple sensors and propose more defined architectures for autonomous vessel navigation;

• Create a proof-of-concept autonomous navigation PNT receiver prototype using different sensors and systems (based on COTS components) to test the artificial intelligence methods and techniques identified;

• Perform laboratory tests and simulations on the proof-of-concept prototype in a variety of situations with a particular focus on assessing performance with regard not only to accuracy but more importantly to resilience and integrity (for example under interference situations) linked to the selected maritime operations;

• Perform a training phase and testing campaign on a vessel using the proof-of-concept autonomous navigation PNT receiver prototype in order to assess the performance of the artificial intelligence methods and techniques implemented;

• Identify areas and technologies that will require further evolutions to achieve operational requirements whose feasibility is not possible with current and planned European GNSS systems.

The activity outcome will be instrumental for the development of autonomous vessel navigation and future standards on the topic. The proof-of-concept prototype will constitute an initial tool available to ESA, industry and third parties to test future options for maritime autonomous vessel navigation.

The Executive will ensure coordination with EC and GSA, in particular for the task related to analysis of operational requirements and for dissemination of joint information papers at IMO (International Maritime Organization) and IALA (International Association of Marine Aids to Navigation and Lighthouse Authorities).

Table 1: Artificial Intelligence / Machine Learning Sensor Fusion for autonomous Vessel Navigation 2)


1) ”Baltic ferry gathers data for self-aware sailing,” ESA Applications, 16 November 2020, URL: https://www.esa.int/Applications/Navigation/Baltic_ferry_gathers_data_for_self-aware_sailing

2) ”Artificial Intelligence / Machine Learning Sensor Fusion for autonomous Vessel Navigation,” ESA, 13 March 2019, URL: https://navisp.esa.int/project/details/51/show

The information compiled and edited in this article was provided by Herbert J. Kramer from his documentation of: ”Observation of the Earth and Its Environment: Survey of Missions and Sensors” (Springer Verlag) as well as many other sources after the publication of the 4th edition in 2002. - Comments and corrections to this article are always welcome for further updates (herb.kramer@gmx.net).

References    Back to top

Satellite Missions