Remote sensing or navigation systems for maritime security

The global fleet of vessels has grown significantly over the years, with more than?2.2 billion deadweight tons?in 2023, compared to less than?1.5 billion tons?in 2011.?Despite a slight decreasing trend, the European Maritime Security Agency reports an increase in marine accidents involving EU ships within domestic waters—from about?1,300 incidents in 2011 to approximately 2,500 in 2022. To enhance safety, security, and emergency management, effective marine traffic surveillance is essential.

Various systems have been developed to help ships locate their positions at sea, including the widely used?Global Positioning System (GPS). However, for comprehensive maritime safety, we need systems that go beyond individual ship positioning. These systems should provide a clear picture of the current situation in a specific area and predict its likely evolution. The?International Maritime Organization (IMO)?sets standards and rules for deploying and using such systems. One notable example is the?Automatic Identification System (AIS), which relies on collaborative vessel traffic services.

However, there are scenarios where collaborative vessel services are not operational or reliable due to system failures, spoofing, or other malicious actions. In such cases,?remote sensing?becomes crucial for ensuring safety and security. Remote sensing technologies allow us to monitor vessel activities, detect anomalies, and take appropriate actions.?For instance, researchers have combined data from?X-band marine radar, optical cameras, and AIS?to track vessel activities in marine protected areas.

Remote sensing plays a vital role in maritime monitoring and vessel identification, especially when collaborative systems are unavailable or compromised. By leveraging advanced technologies, we can enhance safety, protect marine environments, and respond effectively to targeted events.

Wherever and whenever the collaborative vessel traffic services are not operational or not reliable for system failures, spoofing or other malicious actions, or some vessel is suspected of sending falsified messages, remote sensing is the only possibility to properly ensure safety and security and take the appropriate reactions/countermeasures for any targeted event. Several problems are still open to cope with the wide range of applications related to maritime surveillance without the help of collaborative systems, even though many technologies and platforms are operational for detecting and locating even the faintest objects on the sea surface, ranging from optics in various bands to radio/acoustic waves and from satellite to underwater platforms.

Marine traffic understanding is not only ship detection and tracking: a remote-sensed target recognized as a vessel should at least be assigned to its specific ship type and possibly identified as an individual ship, even without relying on its active collaboration. Moreover, having information about the current course and velocity would be necessary to estimate the most likely target positions in a specified time frame. Finally, a complete situational awareness would be achieved if the transported passengers and cargo could be identified and the complete ship behavior could be evaluated. To this end, it would be indispensable to integrate data from different specialized sensors and other available information, such as meteorological, geographical and historical data. Data science, including deep learning and artificial intelligence as well as pattern recognition, image analysis and statistical signal processing, is thus essential to provide a complete assessment of the maritime traffic status, for safety, security and general maritime traffic management purposes. The need for advances in information technology is also recognized by the IMO up to possible “negative consequences”. It is apparent that despite the open problems, the mandatory use of specific technologies has produced a marked decrease in the effects of marine accidents: the number of injuries produced by contact or collision events in EU waters was 136 in 2014 against 43 in 2022. Also, the number of ships lost or damaged and the related pollution events is steadily decreasing. The multi-faceted interests backing maritime activities all need efficient awareness and communication systems, so all the goals mentioned above are pursued by several international and governmental agencies, as well as many commercial companies for security, scheduling and fleet management purposes, and are the subject of active research from academic, government and private institutions. As remote sensing is the ultimate resource for situational awareness in the absence of collaborative actions, in a systematic way, which I like to describe the basics of this system in this newsletter.

Concepts of Remote Sensing

Remote sensing, also called earth observation, refers in a general sense to the instrumentation, techniques and methods used to observe, or sense, the surface of the earth, usually by the formation of an image in a position, stationary or mobile, at a certain distance remote from that surface. In remote sensing electromagnetic radiation coming from an object, in case of earth observation this object is the earth’s surface, is being measured and translated into information about the object or into processes related to the object. In the former measurement phase the following components are relevant:

? the source of the electromagnetic radiance

? the path through the atmosphere

? the interaction with the object

? the recording of the radiation by a sensor. These comprise the remote sensing system.

The second phase can be considered to cover the following components:

? transmission, reception and (pre)processing of the recorded radiance

? interpretation and analysis of the remote sensing data

? creation of the final product.

Sources of electromagnetic radiation

In remote sensing we restrict ourselves to the use of electromagnetic radiation as a characteristic of numerous physical processes. All materials with a temperature above 0 K have the power to emit electromagnetic energy. Objects on or near the earth’s surface are able to reflect or scatter incident electromagnetic radiation emitted by a source, which may be artificial, e.g., flashlight, laser or microwave radiation, or natural, such as the sun. In the visible, near-infrared (NIR) and middle-infrared (MIR) part of the electromagnetic spectrum, we are measuring solar radiation reflected by objects at the earth’s surface. In the thermal-infrared (TIR) part, we are measuring emitted radiation by objects at the earth’s surface, be it that this radiation is originating from the sun. In the microwave part of the spectrum, both reflection of solar light and emission occur at very low energy rates. As a result, radiation mostly is transmitted to the earth’s surface by an antenna on board the remote sensing system and, subsequently, we measure the amount of radiation that is reflected (backscattered) towards the same antenna. The latter type of system is generally referred to as an active remote sensing system.

Object – radiation interaction

When electromagnetic radiation interacts with an object at the Earth's surface, several processes can occur: transmission, absorption, and reflection. The relative magnitude of these processes depends on the properties of the object itself. In remote sensing, we can quantify the amount of reflected solar radiation across different wavelengths, known as spectral reflectance.

Water predominantly absorbs incoming radiation and reflects only a small portion, particularly in the visible part of the spectrum. At longer wavelengths, water does not reflect significant amounts of radiation. On the other hand, soils typically exhibit a smooth spectral reflectance curve. However, distinct spectral features can be observed in narrow bands due to absorption by minerals and iron oxide, which can be detected using imaging spectrometers. These spectral features provide valuable information for various applications such as environmental monitoring, agriculture, and geological mapping.

In maritime environments, the interaction between electromagnetic radiation and the Earth's surface presents unique characteristics and challenges. Broader features in spectral reflectance in such environments are influenced by the presence of water bodies, salt content, and marine vegetation, among other factors.

In coastal regions, spectral reflectance is significantly affected by the optical properties of water. The absorption and scattering of light by water molecules and suspended particles contribute to distinctive features in the reflectance spectrum. For instance, in the visible spectrum, water absorbs shorter wavelengths of light more strongly than longer wavelengths, resulting in reduced reflectance. This absorption pattern is crucial for understanding water quality parameters, such as turbidity and suspended sediment concentration, which have significant implications for marine ecosystems and coastal management.

Additionally, the presence of dissolved organic matter and phytoplankton in coastal waters can introduce specific absorption features in the reflectance spectrum, particularly in the blue and red regions. These features provide valuable information about primary productivity and ecosystem dynamics in coastal and marine environments.

In regions influenced by tidal dynamics and estuarine processes, spectral reflectance may exhibit temporal variations linked to changes in water depth, sediment resuspension, and the mixing of freshwater and marine water masses. These dynamics influence the optical properties of water and subsequently affect the reflectance characteristics observed by remote sensing instruments.

Furthermore, the presence of coastal vegetation, such as mangroves, salt marshes, and seagrasses, introduces additional complexity to spectral reflectance patterns. These vegetation types exhibit distinct spectral signatures influenced by their biochemical composition, canopy structure, and water content. Understanding these spectral characteristics is essential for monitoring coastal vegetation dynamics, habitat mapping, and assessing ecosystem health and resilience.

Overall, in maritime environments, broader features in spectral reflectance are intricately linked to the complex interplay between water properties, coastal morphology, and biological processes. Remote sensing techniques play a crucial role in capturing and analyzing these spectral signatures, providing valuable insights for coastal management, environmental monitoring, and ecosystem conservation efforts.

In the thermal infrared part of the spectrum the amount of emitted radiation is measured. This amount can be related to the temperature of the feature observed. This provides information on, e.g., the (evapo)transpiration of the surface and thus gives relevant information for energy balance studies. An important property of the long wavelengths used in the microwave region is that they are not susceptible to atmospheric scattering. As a result they can penetrate through cloud cover, haze and all but the heaviest rainfall. A passive microwave sensor detects the naturally emitted microwave energy within its field of view. This emitted energy is related to the temperature and moisture properties of the emitting object. Since the amounts of emitted energy generally are very small, a passive microwave sensor is therefore characterized by a low spatial resolution.

Active microwave sensors provide their own source of illumination. They are called radars and measure the amount of energy scattered back towards the radar antenna. The radar echo is depending on the properties of the radar system like frequency, polarisation and the viewing geometry, and on the properties of the object like the roughness and electrical properties. So, with radar we get information on object properties like the geometry (terrain topography), roughness (height variations in relation to the applied wavelength) and moisture (determining the electrical properties of a soil or vegetation).

Sensors Instruments capable of measuring electromagnetic radiation are called sensors. They can be classified as follows:

Passive sensors do not have their own source of radiation. They are sensitive only to radiation from a natural origin, usually reflected sunlight or the energy emitted by an earthly object. The classical example of a passive imaging sensor is the camera, which records the distribution of radiation from an object on a photosensitive emulsion spread out on a film. Other examples are the multi-spectral scanner, the thermal scanner and the microwave radiometer. Both sensor and object are passive.

Active sensors have a built-in source of radiation. The object is passive. Examples are radar (radio detection and ranging) and lidar (light detection and ranging).

It is clear that the design and use of remote sensing systems should be preceded by many considerations depending on specific applications.

Transmission, reception and (pre-)processing

Transmission of sensor data from the maritime environment involves transferring electronic signals from the sensor platform to a receiving and processing station. This transmission must account for the often remote and dynamic nature of maritime operations, including factors such as vessel movement, weather conditions, and potential communication constraints. Additionally, data transmission may be affected by the presence of water bodies, which can introduce signal attenuation and interference.

At the receiving and processing station, the transmitted data are processed into usable images, both in digital and/or hardcopy formats. Given the unique characteristics of maritime environments, such as varying sea states, changing weather conditions, and complex coastal topography, specialized processing techniques may be required to account for these factors and generate accurate and reliable images.

Preprocessing operations are essential to correct for sensor- and platform-specific radiometric and geometric distortions inherent in the acquired data. Radiometric corrections address variations in scene illumination, viewing geometry, atmospheric conditions, and sensor noise and response. These corrections are particularly crucial in maritime environments where factors such as sunlight reflection off water surfaces, atmospheric haze, and sensor noise can significantly impact data quality.

Moreover, in the maritime context, it may be desirable to convert and calibrate the data to known (absolute) radiation or reflectance units to facilitate comparison between different datasets and enable quantitative analysis. Calibration ensures that the data accurately represent the physical properties of the observed features, allowing for meaningful interpretation and analysis.

Overall, the transmission, reception, and preprocessing of remote sensing data in maritime environments require specialized techniques and considerations to account for the unique challenges posed by the marine environment. Effective data processing and calibration are essential for extracting valuable information from remote sensing imagery and supporting various applications, including maritime surveillance, environmental monitoring, and coastal management.

Digital recordings in maritime remote sensing allow for the application of various manipulations using digital image processing and pattern recognition techniques. These methods enable the extraction of valuable information from remote sensing data collected over water bodies, coastal areas, and maritime environments.

Three main categories of information can be derived from remote sensing data in the maritime context:

1.???? Classification: Assigning class labels to individual pixels or objects in an image enables the creation of thematic maps specific to maritime features. This includes mapping land cover types such as coastal vegetation, mangroves, seagrass beds, and built structures like ports and harbors. Classification techniques tailored to maritime environments must account for unique spectral signatures, spatial patterns, and temporal dynamics associated with coastal and marine features.

2.???? Estimation of Object Properties: Remote sensing allows for the estimation of various object properties relevant to maritime applications. For example, it enables the assessment of biomass in coastal wetlands, the classification of different types of coral reefs or the monitoring of changes in beach morphology. These estimations provide valuable insights into the health and dynamics of marine ecosystems, supporting conservation efforts and coastal management practices.

3.???? Observing Vegetation Properties: In the maritime environment, vegetation characteristics play a crucial role in ecosystem dynamics and habitat mapping. However, several variables must be considered when analyzing vegetation properties using remote sensing. These variables include irradiance, atmospheric conditions, sensor viewing geometry, and intrinsic vegetation parameters such as growth stage, moisture content, leaf area index, and soil background. Understanding and accounting for these variables are essential for accurately interpreting remote sensing data and deriving meaningful insights into maritime vegetation dynamics.

Overall, digital image analysis and interpretation in the maritime environment offer valuable tools for monitoring coastal and marine ecosystems, supporting environmental management, and addressing challenges related to coastal resilience, habitat conservation, and sustainable maritime development.

In the context of remote sensing, extracting information about the Earth's surface and its features relies on various detection methods based on spectral, spatial, temporal, and polarization characteristics of images.

Spectral characteristics refer to the electromagnetic radiation's wavelength or frequency and its reflective or emissive properties. Different surface materials and features exhibit unique spectral signatures, allowing for their identification and characterization through spectral analysis.

Spatial characteristics encompass the viewing angle of the sensor, as well as the shape, size, position, site, distribution, and texture of objects on the Earth's surface. Spatial information enables the delineation and classification of features based on their spatial arrangement and structural properties.

Temporal characteristics involve monitoring changes in time and position over successive image acquisitions. Temporal analysis facilitates the detection of dynamic processes such as land cover changes, vegetation growth, urban expansion, and natural disasters.

Polarization characteristics pertain to the effects of objects in relation to the polarization conditions of the transmitter and receiver. Polarimetric remote sensing techniques exploit the polarization properties of electromagnetic waves to enhance information extraction and discrimination of surface features, particularly in areas with complex surface properties or atmospheric conditions.

Integration of these detection methods enables comprehensive analysis and interpretation of remote sensing data, leading to a better understanding of Earth's surface dynamics, environmental changes, and spatial patterns. These technical approaches play a crucial role in various applications, including environmental monitoring, land use planning, natural resource management, and disaster assessment and mitigation.

In the maritime environment, the output from remote sensing takes various forms and serves as valuable input for further analysis, often integrated into a Geographic Information System (GIS). Remote sensing data can be analyzed to derive information relevant to maritime applications, which can then be utilized in conjunction with other spatial datasets within a GIS framework.

On one hand, information within a GIS aids in the analysis and interpretation of remote sensing data captured over maritime regions. GIS facilitates the integration of diverse spatial datasets, including bathymetric maps, coastal morphology data, marine habitat classifications, and oceanographic parameters, enhancing the contextual understanding of remote sensing imagery.

Conversely, the results of remote sensing analyses conducted in the maritime domain, such as sea surface temperature distributions, ocean color variations, or coastal land cover classifications, can be seamlessly integrated into a GIS environment. This integration enables the combination of remote sensing-derived information with other datasets, such as ship traffic patterns, marine protected areas, or pollution sources, for comprehensive spatial analysis and decision-making.

For example, a remote sensing-derived sea surface temperature map can be combined with oceanographic data to study marine thermal dynamics and identify areas prone to coral bleaching. Likewise, a maritime land cover classification map can be integrated with coastal zoning regulations to assess habitat vulnerability to coastal development activities.

Overall, the integration of remote sensing data into GIS platforms enhances our ability to study and manage maritime environments effectively. By combining remote sensing-derived information with other spatial datasets, researchers and decision-makers can gain valuable insights into marine processes, support sustainable maritime development, and address challenges related to marine conservation, resource management, and coastal resilience.

The deployment of remote sensing technology has shown promising results in mitigating the effects of marine accidents. By leveraging advanced technologies, we can not only enhance safety and security but also protect marine environments and respond efficiently to targeted events.

It's crucial for us to stay updated on the latest advancements in remote sensing technology and its applications in safeguarding our seas. Together, we can work towards a safer and more secure maritime domain.