PhiSat-1 Nanosatellite Mission
PhiSat-1 (Φ-Sat-1) is the first European satellite to demonstrate how onboard artificial intelligence can improve the efficiency of sending Earth observation data back to Earth. This revolutionary artificial intelligence technology will fly on one of the two CubeSats that make up the FSSCat (Federated Satellite System) mission – a Copernicus Masters winning idea. 1)
As the overall 2017 Copernicus Masters winner, FSSCat, was proposed by Spain’s UPC (Universitat Politècnica de Catalunya) and developed by a consortium of European companies and institutes.
FSSCat is an innovative mission concept consisting of two federated 6U CubeSats in support of the Copernicus Land and Marine Environment services. They carry a dual microwave payload (a GNSS-Reflectometer and a L-band radiometer with interference detection/mitigation), and a multispectral optical payload to measure soil moisture, ice extent, and ice thickness, and to detect melting ponds over ice. It also includes a radio/optical inter-satellite link and an Iridium intersatellite link to test some of the techniques and technologies for upcoming satellite federations. FSSCat will be the precursor of a constellation of federated small satellites for Earth observation achieving high temporal resolution and moderate spatial resolution in a cost-effective manner. 2)
«The Federated Satellite System 6U tandem mission for sea ice and soil moisture monitoring captured the interest of the challenge experts immediately. Not only because the mission concept shows a high degree of well thought through technical novelties, but also because it will provide data that is complementary to the Sentinel fleet. This is especially true for the soil moisture monitoring component, which is not part of the current Sentinel portfolio. The FSSCat mission development is good to go and due to its disruptive approach, we are confident that it will be seen as a breakthrough in procuring future small missions at ESA.»
The two CubeSats will collect data, which will be made available through the Copernicus Land and Marine Environment services, using state-of-the-art dual microwave and hyperspectral optical instruments. They also carry a set of intersatellite communication technology experiments.
During Φ-week (9-13 September 2019 at ESA/ESRIN), ESA’s Director of Earth Observation Programs, Josef Aschbacher, said, “We see that there is huge interest in Φ-Sat and thanks to our partners, it is ready to be launched. We live in exciting times, the pace at which digital technology is developing coupled with the wealth of satellite information being delivered and, indeed, the growing demand for such data, means there are many opportunities to make a step change for the future of Earth observation. And, with Φ-Sat – Europe’s first artificial intelligence in space – we are going to do just this.”
The hyperspectral camera on one of the CubeSats will collect an enormous number of images of Earth, some of which will not be suitable for use because of cloud cover. To avoid downlinking these less than perfect images back to Earth, the Φ-Sat artificial intelligence chip will filter them out so that only usable data are returned.
Marco Esposito, from cosine Remote Sensing, the company that led the development of the artificial intelligence algorithm, explained, “While compact, the instrument – which covers the visible and near infrared with hyperspectral capability, enhanced with bands in the thermal infrared – is very powerful and will acquire terabytes of data that can be used to monitor vegetation changes and to assess water quality, for example.
“However, generating this amount of data actually poses a problem, as the data have to be handled efficiently so that they can reach the users in a timely manner. With Φ-Sat we have effectively given the instrument its own brain, which processes the data onboard to detect clouds in the images. This not only ensures better quality data, but makes the delivery much more efficient.”
ESA’s Massimiliano Pastena, noted, “Indeed, this will be the first satellite to demonstrate the use of artificial intelligence in orbit and we are very much looking forward to it being launched in the coming months.”
Φ-Sat-1 is the first scientific and technology initiative of the Earth Observation directorate of the European Space Agency (ESA) in the NewSpace field. 3) Φ-Sat-1 is the result of an enhancement of the FSSCAT mission using AI (Artificial Intelligence) and improved imaging capabilities of the hyperspectral camera HyperScout-1 VNIR imager. cosine Remote Sensing is leading the development and execution of the demonstration, including the highly integrated spectral imaging in the VNIR and TIR, as well as the integration of state-of-art AI accelerators including the first AI algorithms for cloud screening. The HyperScout-2 spectral camera is the successor of HyperScout-1, a miniaturized reflective optical instrument equipped with enhanced processing capabilities, in orbit since February 2018 as part ESA GOMX-4B technology demonstration mission.
Φ-Sat-1 will demonstrate the capabilities of small instruments for scientific applications, and will validate in-orbit innovative technologies. 4) The technology demonstration includes the miniaturization and highly integration of visible, near infrared and thermal spectral channels, as well as state-of-art processors and machine learning algorithms. The hardware co-registration of the VNIR and TIR channels will enable a variety of applications for terrain classification and change detection. Φ-Sat-1 will demonstrate an AI-based inference engine for cloud detection and will be launched as enhancement of the FSSCat mission, 5) during the VEGA SSMS (Small Satellite Mission Service) Proof Of Concept flight.
Sensor complement (HyperScout-2)
The HyperScout product series consists of very compact spectral imagers, in the order of 1-2 kg. The instruments are rather inexpensive, which are not meant to replace or compete with larger institutional satellites like Sentinel or Landsat, but more to complement large observation spacecrafts, being exploited for early detection of anomalies, as well as other applications. HyperScout is the one of the first hyperspectral imagers with such a compact envelope, therefore enabling applications which rely on the potential of very high temporal resolution (i.e. order of hours) in order to detect anomalies as early as possible and to refer to other space or ground assets for more detailed investigations.
A spectral imager with such a small envelope becomes an attractive solution for deployment in constellations, and therefore offers to the user the possibility of achieving high revisit times over regions of interest. The HyperScout concept is a combination of state of the art optics packed in an extremely compact format, in combination with very powerful processing abilities. It was first designed to be installed on CubeSat and small satellites, however it can be easily incorporated into larger platforms given the small engineering budgets and the compact volume.
HyperScout-2 is based on the building blocks of HyperScout-1 that was successfully demonstrated in flight by cosine Remote Sensing and ESA onboard the ESA GOMX-4B satellite and it is operational since February 2018. 6) The HyperScout-2 specifications are reported in Table 1. The optical front-end is based on a monolithically and reflective TMA configuration, based on freeform optical design and manufactured using diamond turning machining technique. The dual channel implementation envisages a beam splitter at the end of the front end optics, diverting the shorter wavelengths towards the VNIR sensor, and allowing the longer wavelengths to impinge on the TIR sensor.
Table 1: HyperScout-2 specifications
Figure 1: Left: HyperScout-1 Protoflight Model, Right: HyperScout-2 Protoflight Model (image credit: cosine Remote Sensing, ESA/ESTEC)
This configuration was envisaged during other ESA activities related to navigation and planetary science, with goal of designing a multispectral camera for relative navigation. 7) A very compact relay has been conceived in order to enhance the numerical aperture of the TIR channel. The VNIR Focal Plane Array (FPA) is a CMOS sensor with a linearly variable hyperspectral filtering element that separates the different wavelengths from 400 to 1000 nm, while the TIR FPA is based on a microbolometer and a set of four spectral filters from 8 to 14 µm. Both FPAs are 2D sensors operated in push broom mode. The TIR sensor is actively thermally controlled via a thermoelectric cooler (TEC) to limit the dependency of the sensor readout on the sensor temperature fluctuations.
HyperScout-2 has the following subsystems:
• Telescope assembly;
• VNIR Focal Plane Array (VNIR FPA);
• TIR Focal Plane Array (TIR FPA);
• Back-End Electronics unit (BEE);
• On-Board Data Handling (OBDH);
• Mass Memory Units (MMUs): operated in hot and cold spare configuration;
• Instrument Control Unit (ICU): main point of contact from the S/C bus for data interface and main instrument control;
• Eyes of Things board (EOT): data processing board for on-board artificial intelligence algorithms.
The architectural diagram of HyperScout-2 is depicted in Figure 2, while the subsystems locations in the payload is indicated in Figure 3. The back-end electronics (BEE) is a compact and versatile FPGA based single board image acquisition system. The On-Board Data Handling (OBDH) is physically installed on the BEE. The Instrument Control Unit (ICU) controls and monitors the operational status of the payload subsystems. The Vision Processing Unit (VPU) consists of a Myriad 2 processor located on the Eyes of Things (EoT) board. The ICU can independently power the BEE, OBDH, the EoT board and each MMU. The FPAs are independently powered via the BEE. All subsystems have independent latch-up and overcurrent protection.
Figure 3: Illustration of the HyperScout-2 instrument (image credit: cosine Remote Sensing, ESA/ESTEC)
The majority of the HyperScout subsystems can be operated independently from each other, offering an additional degree of freedom to optimize the power budget by powering up only the components that play a role within the specific operating mode. The ICU is always powered to serve as the contact point for communication with the S/C bus and to monitor the status of the payload throughout the orbit. During acquisition the FPAs, BEE and MMU are mainly used. In this phase for example, the OBDH will consume minimal power transferring the image data to disk. During processing both FPAs and BEE are powered off completely. The OBDH will normally consume the most power in the processing mode. The dependency of the subsystems per operating mode is reported in Figure 4.
The frequency of the observation mode can vary according to the mission planning. The acquired data is cached in the instrument MMUs for subsequent processing or transmission to ground, as requested by the S/C bus. The processing operating mode can be run using different processing subsystems. For this scope HyperScout-2 offers a powerful computational set of processors, including a FPGA, CPU, GPU and a dedicated board for AI inference represented by an Eyes Of Things (EOT) board consisting of the Intel® Movidius™ Myriad™ vision processing unit (VPU) designed for accelerating machine vision tasks. The Intel's Movidius™ Myriad™ 2 VPU (Ref. 5) (second generation VPU from Movidius™, an Intel® company) is specialized for AI, vision and imaging applications where both performance and low power consumption are important.
HyperScout-2 is therefore capable of performing a vast range of processing operations, that can be continuously updated during flight, even with completely new applications.
Figure 5: Right: Intel's Movidius Myriad 2 board, Left: as integrated on the top of the HyperScout-2 electronics stack (image credit: cosine Remote Sensing, ESA/ESTEC).
HyperScout-2 calibration and applications
In-flight Calibration approach: HyperScout-2 has been characterized in the laboratory before integration into the spacecraft. For what concerns in-flight calibration, typically on-board stimuli are used to calibrate the full optical chain for the long waves infrared channel, such as blackbodies. For a miniaturized instrument like HyperScout this is currently not an option. A lean system approach is adopted that relies on vicarious calibration in combination with a thermally stable instrument and detector.
For the absolute calibration of the HyperScout VNIR spectral channel, instrumented test sites of RadCalNet are considered. Test sites consist of large, homogeneous, cloud-free areas used as radiance or reflectance reference targets. The method relies on the comparison of satellite date with a simultaneous ground truth measurements. Two methods are distinguished:
• Reflectance based: the reflectance of the earth surface is measured on-ground, together with extinction depth and other meteorological parameters. Subsequently a radiative transfer model is used to convert these values to a TOA (Top of Atmosphere) radiance, which can be compared to the sensor data. These methods allow absolute radiometric calibration with a accuracy <5%.
• Radiance based: the radiance of the earth scene is measured at an altitude much above aerosol scattering. After correction for residual scattering and absorption, the radiance can be compared directly to the sensor for absolute radiometric calibration within 2.8% accuracy.
Pseudo invariant characterization sites (PICS), are sites with a high temporal or spatial stability. Desert sites are the most promising candidates as they are characterized by large, homogeneous areas and constant atmospheric conditions (no clouds). An extensive catalogue with site information is available on the websites of the USGS and CEOS. The most common application is to monitor the relative radiometric response of the sensor, without the need of an on-ground reference measurement.
Data will be crossed calibrated on the long term using Sentinel-2A/-2B, acquired over the same region of interest (ROI). For the purposes of cross-calibration of HyperScout, the Sentinel-2 satellites are considered thanks to the good match between the spectral bands of the Sentinel Multispectral Instrument (MSI) and HyperScout and the in-orbit HyperScout data collection opportunity provided by the FSSCat spacecraft with limited costs for cosine.
Figure 6: Diagram of a cross acquisition between HyperScout and Sentinel satellites. Given the 180º separation between the Sentinel-2A and Sentinel-2B orbit, the crossed acquisition with HyperScout can be performed with only one Sentinel at the time (image credit: cosine Remote Sensing, ESA/ESTEC)
For the TIR channel in-flight calibration, a spacecraft maneuver with the instrument pointed at deep space is used to measure the instrument thermal background signal. As there is no black body calibrator on-board the payload for in-flight calibration, two options are considered: 1) intercalibration or cross-calibration, 2) vicarious calibration. Intercalibration calibrates a satellite instrument by relating its measurements to those of a well calibrated reference satellite instrument. This technique relates measurements in similar spectral bands. Well-calibrated reference satellite instruments that match each TIR band of HyperScout-2 are for example Sentinel-3, MODIS or Landsat-8. Vicarious calibration uses natural or artificial sites on the surface of the earth to calibrate satellite instruments. Within this technique, well-calibrated ground-based or airborne radiometers take measurements of a spectrally and spatially homogeneous test sites at the time of the satellite instrument overpass.
Two methods can be used to perform vicarious calibration in the thermal infrared:
• Radiance-based method;
• Temperature-based method.
The radiance-based method requires that the spectral response of the well-calibrated reference radiometer matches the spectral response of the satellite instrument. In practice, this condition is most likely not going to be fulfilled. Consequently, this method is also not considered to be a suitable option for inflight monitoring and calibration. In contrast to the radiance-based method, the temperature-based method in principle does not impose restrictions on the spectral response of the instrument. Hence, this method is considered a suitable option for in-flight monitoring and calibration. The following calibration sites could be envisaged for the in-flight calibration of HyperScout:
• Lake Tahoe on the California/Nevada border
- ideal thermal calibration target due to thin atmosphere above the lake thanks to the high altitude
- does not freeze in winter as it is very deep
- annual temperature ranges from about 4 to 20°C
- excluding instrument noise, uncertainty in the predicted at-sensor spectral radiance for TIR bands and skin surface temperature of 300 K is about 0.43 K - expressed in equivalent apparent temperature
- freely available data
• Salton Sea in Southern California
- less ideal target than Lake Tahoe due to thicker atmosphere (lower altitude)
- annual temperature variation ranges from about 4 to 35°C (extends temperature range compared to lake Tahoe)
- excluding instrument noise, uncertainty in the predicted at-sensor spectral radiance for TIR bands and skin surface temperature of 300 K is about 0.43 K - expressed in equivalent apparent temperature
- freely available data
• NOAA Ocean and Great Lakes
- larger surface area than Lake Tahoe and Salton Sea
- temperature variation ranges from about 3 to 30°C depending on the season and location
- excluding instrument noise, uncertainty in the predicted at-sensor spectral radiance for TIR bands and skin surface temperature of 300 K is about 0.48 K - expressed in equivalent apparent temperature
- freely available data.
cosine is also leading a Dutch consortium for the development and commercialization of a miniaturized spectroradiometric calibration stimuli, expected to be integrated and tested in orbit in late 2020 / early 2021. This miniaturized calibration source will be considered for future flights of HyperScout related products as well as third parties compact spectrometers.
HyperScout-2 data products
A number of data products have been selected for HyperScout-1 and this is being extended for the HyperScout-2 mission adding the potential applications enabled by the coexistence of reflectance and thermal information of the target. The target applications are: agriculture, thermal inertia and soil moisture, urban heat island, fire hazard and monitoring, and water quality.
In-Orbit Test Bed: A Platform for Artificial Intelligence Experiments
Applying Artificial Intelligence processing algorithms on the data acquired by HyperScout-2 directly onboard and in real time may represent a big leap forward in inferring and delivering Earth Observation derived information at a very high temporal resolution and with a very short timeliness. Alternatively, an interesting use case is the screening of the data before being downloaded. This is the case for the first demonstration that will be performed in orbit as part of the Φ-Sat-1 mission, where AI is leveraged to select the data to be downloaded based on the cloud coverage of the acquired scene. The application has been selected by ESA as the presence of clouds is one of the fundamental problems of optical remote sensing and it becomes particularly relevant considering the large data volume generated by Earth Observation missions based on hyperspectral imaging.
However, it should be noted that the HyperScout-2 processing chain has been designed in order to accept third party AI based software that can be run on the either on the VPU, or CPU/GPU, for testing purposes before being employed in operational scenario. The processing chain is depicted in Figure 7.
The first step prepares the data and corrects it for known aberrations. After the raw data is corrected, the spectral cube can be computed in order to produce spectral bands. The second pre-processing step is mandatory if the VPU is used. In this case the spectral cube is prepared and loaded into the VPU, ready to be analyzed by the AI algorithm. The inference step is use-case based, and as discussed can be performed on the VPU if high time efficiency is required.
Figure 8: Φ-Sat-1 is the first experiment to demonstrate how onboard artificial intelligence can improve the efficiency of sending Earth observation data back to Earth. This revolutionary artificial intelligence technology will fly on one of the two CubeSats that make up the Federated Satellite Systems (FSSCat) mission (image credit: ESA)
Launch: The Φ-Sat-1 CubeSat mission (6U), as enhancement of the FSSCat mission, was launched as a passenger payload at 01:51 UTC, 03:51 CEST on 3 September, 2020 (22:51 local time on 2 September in Kourou).The launch vehicle was the VEGA SSMS (Small Spacecraft Mission Service) Proof Of Concept flight. 8)
Orbit: Sun-synchronous orbit; target orbit for the 7 microsatellites: altitude of 515 km, inclination of 97.45º; target orbit for the 46 nanosatellites: altitude of 530 km, inclination = 97.51º. The nominal mission duration (from liftoff to separation of the 53 satellites) is: 1 hour, 44 minutes and 56 seconds.
Passenger payloads (53) of the Vega rideshare mission
Arianespace has realized the first European “rideshare” mission for small satellites, with 53 satellites onboard the Vega launcher for 21 customers from 13 different countries. With this new SSMS (Small Spacecraft Mission Service) shared launch concept, Arianespace demonstrates its ability to respond – in an innovative and competitive manner – to institutional and commercial requirements of the growing market for small satellites. The total satellite launch mass was 1,327 kg. 9)
With the demonstration of its new SSMS service, Arianespace is strengthening its position in the growing market for small satellites. This service will soon be supplemented by the MLS (Multi Launch Service) – a similar offer available on Ariane 6, allowing Arianespace to increase the number of affordable launch opportunities for small satellites and constellations.
• ESAIL is a maritime microsatellite with a mass of 112 kg for AIS (Automatic Identification System) ship tracking operated by exactEarth. Is was built by a European manufacturing team led by the satellite prime contractor Luxspace. ESAIL features an enhanced multiple antenna-receiver configuration for global detection of AIS messages and high-resolution spectrum capture, which will enable the demonstration of advanced future services such as VDES (VHF Data Exchange System) message reception. 10)
• Lemur-2, eight 3U CubeSats built by Spire Global Inc., San Francisco, CA . These satellites carry two payloads for meteorology and ship traffic tracking. The payloads are: STRATOS GPS radio occultation payload and the SENSE AIS payload.
• TriSat is a 3U CubeSat (5 kg) imaging mission led by the University of Maribor, Slovenia. The mission is focused on remote sensing by incorporating a miniaturized multispectral optical payload as the primary instrument, providing affordable multispectral Earth observation in up to 20 non-overlapping bands in NIR-SWIR (Near to Short Wave Infrared) spectrum.
• The launch integrator company Spaceflight Inc. of Seattle WA is providing its services for four different customers with a total of 28 satellites. These are:
a) NewSat-6 (also written as ÑuSat-6), is a low Earth orbit commercial remote sensing microsatellite (43.5 kg) designed and manufactured by Satellogic S.A. with HQs in Argentina, a vertically integrated geospatial analytics company that is building the first Earth observation platform with the ability to remap the entire planet at both high-frequency and high-resolution. This is Satellogic’s 11th spacecraft in orbit, equipped with multispectral and hyperspectral imaging capabilities and it will be added to the company’s growing satellite constellation.
The spacecraft is named "Hypatia" after the philosopher, astronomer, and mathematician (350-415 A.D.) who lived in Alexandria, Egypt, and was a symbol of learning and science. She was renowned in her own lifetime as a great teacher and a wise counselor and became seen as an icon for women's rights and a precursor to the feminist movement. In line with Satellogic's NewSats already in orbit, Hypatia is equipped with sub-meter multispectral and 30 m hyperspectral cameras. This NewSat Mark IV is also equipped with new technologies in service of Satellogic's research and development of Earth-observation capabilities. Upon successful commissioning, these new capabilities will be available to existing Satellogic customers.
b) 14 Flock-4v, 3U CubeSats, next-generation SuperDove satellites of Planet Inc., San Francisco, they will join its constellation of 150 Earth-imaging spacecraft.
c) SpaceBEE, 12 (.25U) picosatellites of Swarm Technology which provide affordable global connectivity.
d) Tyvak-0171, an undisclosed minisatellite of Tyvak, developed by Maxar with a mass of 138 kg.
• Planet Inc. of San Francisco launches a total of 26 Flock 4v SuperDoves on this mission. They will be split into two batches on the same launch: 14 of them will be housed inside and deployed from ISL’s QuadPack deployers and the remaining 12 will be deployed from D-Orbit’s InOrbit Now (ION) freeflying deployment platform. 11)
• Athena, a communications minisatellite mission (138 kg) of PointView Tech LLC, a subsidiary of Facebook. The objective is to provide broadband access (internet connectivity) to unserved and underserved areas throughout the world.
• AMICalSat, a 2U CubeSat, an educational mission, developed by CSUG (University of Grenoble Alpes, France) and MSU-SINP (Lomonosov Moscow State University-Skobeltsyn Institute of Nuclear Physics, Russia). The objective is to take pictures of the Northern light in order to reconstruct the particle precipitation into the polar atmosphere. The payload is a very compact, ultra-sensitive wide filed imager (f=23mm, aperture f/1.4). Firstly, AMICal Sat will observe auroras using nadir pointing, i.e. by determining the center of the Earth to map and link the geographical position of the auroral oval and its internal structures with solar activity. Secondly, the CubeSat will perform image capture ‘in limbo’ through tangential orientation with the Earth to capture the vertical profile of the auroras and match an altitude to their various emissions.
• PICASSO, a 3U CubeSat mission (mass of 3.8 kg) developed for ESA ( European Space Agency) led by BISA (Belgian Institute for Space Aeronomy), in collaboration with VTT Technical Research Center of Finland Ltd, Clyde Space Ltd. (UK) and the CSL (Centre Spatial de Liège), Belgium. The goal is to develop and operate a scientific 3U CubeSat.
• GHGSat-C1 of GHGSat Inc., Montreal, Canada, is the first of two nanosatellites (~16 kg) as the commercial follow-on to the GHGSat-D (CLAIRE) demonstration satellite developed and launched by UTIAS/SFL of Toronto in 2016. GHGSat monitors industries greenhouse gas (GHG) and air quality gas (AQG) emissions, including: oil & gas, power generation, mining, pulp & paper, pipelines (natural gas), landfill, chemicals, metals & aluminum, cement, agriculture, and transportation.
• NEMO-HD of SPACE-SI (Slovenian Center of Excellence for Space Sciences and Technologies) is a microsatellite (65 kg) developed at UTIAS/SFL of Toronto, Canada in cooperation with SPACE-SI. The NEMO-HD (Next-generation Earth Monitoring and Observation-High Definition) satellite is a high precision interactive remote sensing mission for acquiring multispectral images and real time HD video.
• FSSCat (Federated Satellite Systems on Cat) is the winner of the 2017 Copernicus Master “ESA Sentinel Small Satellite Challenge (S3)”. Proposed by the Universitat Politèctica de Catalunya (UPC) and developed by a consortium composed of UPC (ES), Deimos Engenharia (PT), Golbriak Space (EE), COSINE (NL) and Tyvak International (IT).
• Phi-Sat-1 (Φ-Sat-1) is the first on-board ESA initiative (6U CubeSat) on Artificial Intelligence (AI) promoted by the Φ Department of the Earth Observation Directorate and implemented as an enhancement of the FSSCat mission. Among mission objectives, scientific goals are Polar Ice and Snow monitoring, soil moisture monitoring, terrain classification and terrain change detection (i.e. hazard detection and monitoring, water quality), while technological goals are optical Inter-Satellite Link (OISL) demonstration.
• The RTAFSAT-1 (Royal Thai Air Force Satellite-1) mission, also referred to as NAPA-1, is a 6U CubeSat, the first remote sensing CubeSat mission for Thailand. The satellite will carry out an Earth Observation Demonstration mission with SCS Gecko Camera and Simera TriScape-100 payloads; the designed lifetime is 3 years.
• DIDO-3, a commercial 3U CubeSat mission of SpacePharma. The objective is to gather data by researching the effects of a microgravity environment on biological materials. SpacePharma from Israel will be is on board of SSMS POC with DIDO-3 Nanosatellite to perform biological experiment under Microgravity for several customers involved in pharmaceutical business, supported by Italian Space Agency (ASI) and Israeli Space Agency (ISA). Dido-3 will be monitored from the Ground Station developed by SpacePharma in Switzerland.
• SIMBA (Sun-Earth Imbalance), a 3U CubeSat mission led by the Royal Meteorological Institute Belgium, The objective is to measure the TSI (Total Solar Irradiance) and Earth Radiation Budget climate variables with a miniaturized radiometer instrument. This mission will help in the study of the global warming. This science mission will have a design lifetime of 3 years and the satellite performances will be monitored from ground station located in The Netherlands.
• TARS-1, a 6U CubeSat of Kepler Communications, developed at ÅAC Clyde Space for IoT (Internet of Things) applications. TARS-1 features deployable solar arrays, software defined radios (SDR), a narrowband communications payload and high gain antennas.
• OSM-1 Cicero, the first nanosatellite developed in Monaco by OSM (Orbital Solutions Monaco engineers, a 6U CubeSat with a mass of ~10 kg) based on the Tyvak Nano-Satellite Systems design. OSM plans to build nanosatellites to gather environment and climate data.
• TTU100, a 1U CubeSat developed at the Tallin University of Technology, Estonia. The objective is to test earth observation cameras and high-speed X-band communications. It will perform remote sensing in the visible and IR electromagnetic spectrum.
• UPMSat-2 (Universidad Politecnica de Madrid Satellite-2), a demonstration microsatellite (45 kg) of IRD-UPM.
The FSSCat mission will demonstrate for the first time worldwide a reliable optical intersatellite link (O-ISL) between two 6U CubeSats flying in LEO (Low Earth Orbit). We have developed a technology demonstration with a full duplex O-ISL terminal of 1.5U in size (15cm x 10cm x 10cm), 85mm optical aperture, suitable for LEO to LEO operations on coarse pointing CubeSats (pointing accuracy < 0.5º).
The terminal adopts a novel adaptive variable divergence laser mechanism with a hybrid payload/platform seek-and-track pointing algorithm, and an amplitude modulated signal to ensure reliable optical communications at 1 Gbit/s at a nominal intersatellite distance of up to 2,000 km in LEO. In addition of establishing an intersatellite link, the terminal provides the ability for simultaneous imaging through the optical aperture and the improvement of the attitude determination capability of the hosting satellite through the mixing of optical information in the overall platform attitude feedback control loop. 12)
Figure 9: FSSCat is a constellation of two 6U CubeSats that provide data on Earth’s ice and soil moisture content to complement the Sentinel fleet. FSSCat took the top prize at the 2017 Copernicus Masters Competition (image credit: UPC)
Status and events of Φ-Sat-1 (PhiSat-1)
• October 20, 2020: As ubiquitous as artificial intelligence has become in modern life — from boosting our understanding of the cosmos to surfacing entertaining videos on your phone — AI hasn’t yet found its way into orbit. 13)
- That is until Sept. 2, when an experimental satellite about the size of a cereal box was ejected from a rocket’s dispenser along with 45 other similarly small satellites. The satellite, named PhiSat-1, is now soaring at over 17,000 mph (27,500 km/h) in sun-synchronous orbit about 329 miles (530 km) overhead.
- PhiSat-1 contains a new hyperspectral-thermal camera and onboard AI processing thanks to an Intel® Movidius™ Myriad™ 2 Vision Processing Unit (VPU) — the same chip inside many smart cameras and even a $99 selfie drone here on Earth. PhiSat-1 is actually one of a pair of satellites on a mission to monitor polar ice and soil moisture, while also testing intersatellite communication systems in order to create a future network of federated satellites.
- The first problem the Myriad 2 is helping to solve? How to handle the large amount of data generated by high-fidelity cameras like the one on PhiSat-1. “The capability that sensors have to produce data increases by a factor of 100 every generation, while our capabilities to download data are increasing, but only by a factor of three, four, five per generation,” says Gianluca Furano, data systems and onboard computing lead at the European Space Agency, which led the collaborative effort behind PhiSat-1.
- At the same time, about two-thirds of our planet’s surface is covered in clouds at any given time. That means a whole lot of useless images of clouds are typically captured, saved, sent over precious down-link bandwidth to Earth, saved again, reviewed by a scientist (or an algorithm) on a computer hours or days later — only to be deleted.
- “And artificial intelligence at the edge came to rescue us, the cavalry in the Western movie,” says Furano. The idea the team rallied around was to use onboard processing to identify and discard cloudy images — thus saving about 30% of bandwidth.
- “Space is the ultimate edge,” says Aubrey Dunne, chief technology officer of Ubotica. The Irish startup built and tested PhiSat-1’s AI technology, working in close partnership with cosine, maker of the camera, in addition to the University of Pisa and Sinergise to develop the complete solution. “The Myriad was absolutely designed from the ground up to have an impressive compute capability but in a very low power envelope, and that really suits space applications.”
- The Myriad 2, however, was not intended for orbit. Spacecraft computers typically use very specialized “radiation-hardened” chips that can be “up to two decades behind state-of-the-art commercial technology,” explains Dunne. And AI has not been on the menu.
- Dunne and the Ubotica team performed “radiation characterization,” putting the Myriad chip through a series of tests to figure out how to handle any resulting errors or wear-and-tear.
Figure 10: Intel Powers First Satellite with AI on Board. An experimental ESA satellite about the size of a cereal box called PhiSat-1 is the first-of-its-kind with onboard artificial intelligence. Its Intel Movidius Myriad 2 Vision Processing Unit (VPU) identifies and discards cloudy images that aren't useful — thus saving about 30% of bandwidth. The satellite's mission is to test the viability of satellites with AI, and help monitor the health of our planet (image credit: Intel Corporation)
- ESA “had never tested a chip of this complexity for radiation,” says Furano. “We were doubtful we could test it properly ... we had to write the handbook on how to perform a comprehensive test and characterization for this chip from scratch.”
- The first test, 36 straight hours of radiation-beam blasting at CERN in late 2018, “was a very high pressure situation,” Dunne says. But that test and two follow-ups “luckily turned out well for us.” The Myriad 2 passed in off-the-shelf form, no modifications needed.
- This low-power, high-performance computer vision chip was ready to venture beyond Earth’s atmosphere. But then came another challenge.
- Typically, AI algorithms are built, or “trained,” using large quantities of data to “learn” — in this case, what’s a cloud and not a cloud. But given the camera was so new, “we didn’t have any data,” says Furano. “We had to train our application on synthetic data extracted from existing missions.”
- All this system and software integration and testing, with involvement of a half-dozen different organizations across Europe, took four months to complete. “We were very proud to be able to be so quick and so efficiently flexible, to put everything on board in such a short time,” says Max Pastena, PhiSat-1 officer at ESA. As far as spacecraft development goes, the timeline “is a miracle,” adds Furano.
- “Intel has given us background support on the Myriad device when we’ve needed it, to enable PhiSat-1’s AI using our CVAI Technology,”
says Dunne. “That’s very much appreciated.”
- Unfortunately, a series of unrelated events — delays with the rocket, the coronavirus pandemic and unfriendly summer winds — meant the teams had to wait more than a year to find out if PhiSat-1 would function in orbit as planned.
- The Sept. 2 launch from French Guiana — a first-of-its-kind satellite ride-share run by Arianespace — went fast and flawlessly. For the initial verification, the satellite saved all images and recorded its AI cloud detection decision for each, so the team on the ground could verify that its implanted brain was behaving as expected.
- After a three-week deep breath, Pastena was able to proclaim: “We have just entered the history of space.”
- ESA announced the joint team was “happy to reveal the first-ever hardware-accelerated AI inference of Earth observation images on an in-orbit satellite.”
- By only sending useful pixels, the satellite will now “improve bandwidth utilization and significantly reduce aggregated downlink costs” — not to mention saving scientists’ time on the ground.
- Looking forward, the usages for low-cost, AI-enhanced teensy satellites are innumerable — particularly when you add the ability to run multiple applications.
- “Rather than having dedicated hardware in a satellite that does one thing, it’s possible to switch networks in and out,” says Jonathan Byrne, head of the Intel Movidius technology office. Dunne calls this “satellite-as-a-service.”
- Consider: When flying over areas prone to wildfire, a satellite can spot fires and notify local responders in minutes rather than hours. Over oceans, which are typically ignored, a satellite can spot rogue ships or environmental accidents. Over forests and farms, a satellite can track soil moisture and the growth of crops. Over ice, it can track thickness and melting ponds to help monitor climate change.
- Many of these possibilities will soon be tested: ESA and Ubotica are working together on PhiSat-2, which will carry another Myriad 2 into orbit. PhiSat-2 will be “capable of running AI apps that can be developed, easily installed, validated and operated on the spacecraft during their flight using a simple user interface.”
- For Intel, the potential impact is unquestionable. As Pastena puts it, we can eventually understand “the pulse of our planet.”
• September 28, 2020: ESA’s 2020 -week event kicked off this morning with a series of stimulating speeches on Digital Twin Earth, updates on Φ-Sat-1, which was successfully launched into orbit earlier this month, and an exciting new initiative involving quantum computing. 14)
Figure 11: Digital Twin Earth will help visualize, monitor and forecast natural and human activity on the planet. The model will be able to monitor the health of the planet, perform simulations of Earth’s interconnected system with human behavior, and support the field of sustainable development, therefore, reinforcing Europe’s efforts for a better environment in order to respond to the urgent challenges and targets addressed by the Green Deal (image credit: ESA)
Digital Twin Earth
- The third edition of the Φ-week event, which is entirely virtual, focuses on how Earth observation can contribute to the concept of Digital Twin Earth – a dynamic, digital replica of our planet which accurately mimics Earth’s behavior. Constantly fed with Earth observation data, combined with in situ measurements and artificial intelligence, the Digital Twin Earth provides an accurate representation of the past, present and future changes of our world.
- Today’s session opened with inspiring statements from ESA’s Director General, Jan Wörner, ESA’s Director of Earth Observation Programs, Josef Aschbacher, ECMWF’s Director General, Florence Rabier, European Commission’s Deputy Director General for Defence Industry and Space, Pierre Delsaux, as well as Director General of DG CONNECT at the European Commission, Roberto Viola.
- Pierre Delsaux commented, “As our EU Commission President repeated recently during her State of the Union speech, it’s clear we need to address climate change. The Copernicus program offers us some of the best instruments, satellites, to give us a complete picture of our planet's health. But space is not only a monitoring tool, it is also about applied solutions for our economy to make it more green and more digital.”
- Roberto Viola said, “Φ-week is the week for disruptive technology and it is communities like this that our European programs were designed to support.”
- Florence Rabier added, “Machine learning and artificial intelligence could improve the realism and efficiency of the Digital Twin Earth – especially for extreme weather events and numerical forecast models.”
- Jan Wörner concluded, “Φ-week is the perfect example of the New Space approach focusing on disruptive innovation, artificial intelligence, agility and flexibility.”
- During the week, experts will come together to discuss the role of artificial intelligence for the Digital Twin Earth concept, its practical implementation, the infrastructure requirements needed to build the Digital Twin Earth, and present ideas on how industries and the science community can contribute.
- Earlier this month, on 3 September, the first artificial intelligence (AI) technology carried onboard a European Earth observation mission, Φ-Sat-1, was launched from Europe’s spaceport in French Guiana. An enhancement of the Federated Satellite Systems mission (FSSCat), the pioneering artificial intelligence technology is the first experiment to improve the efficiency of sending vast quantities of data back to Earth.
- Today, ESA, along with cosine remote sensing, are happy to reveal the first ever hardware-accelerated AI inference of Earth observation images on an in-orbit satellite – performed by a Deep Convolutional Neural Network, developed by the University of Pisa.
- Φ-Sat-1 has successfully enabled the pre-filtering of Earth observation data so that only relevant part of the image with usable information are downlinked to the ground, thereby improving bandwidth utilization and significantly reducing aggregated downlink costs.
- Initial data downlinked from the satellite has shown that the AI-powered automatic cloud detection algorithm has correctly sorted hyperspectral Earth observation imagery from the satellite’s sensor into cloudy and non-cloudy pixels.
Figure 12: Cloud mask from Φ-Sat-1. Φ-Sat-1 has successfully enabled the pre-filtering of Earth observation data so that only relevant part of the image with usable information are downlinked to the ground, thereby improving bandwidth utilization and significantly reducing aggregated downlink costs. AI-computed cloud mask tile where the colors represent the confidence/probability of cloud (dark blue is no cloud, yellow/green means high cloud probability), image credit: cosine remote sensing B.V.
Figure 13: Lake Tharthar, Iraq. Φ-Sat-1 has successfully enabled the pre-filtering of Earth observation data so that only relevant part of the image with usable information are downlinked to the ground, thereby improving bandwidth utilization and significantly reducing aggregated downlink costs. This is the first hyperspectral VNIR-TIR full co-registered image (image credit: cosine remote sensing B.V.)
- Massimiliano Pastena, Φ-sat-1 Technical Officer at ESA, commented, “We have just entered the history of space.”
- Today’s successful application of the Ubotica Artificial Intelligence technology, which is powered by the Intel Movidius Myriad 2 Vision Processing Unit, has demonstrated real on-board data processing autonomy.
- Aubrey Dunne, Co-Founder and Vice President of Engineering at Ubotica Technologies, said, “We are very excited to be a key part of what is to our knowledge the first ever demonstration of AI applied to Earth Observation data on a flying satellite. This is a watershed moment both for onboard processing of satellite data, and for the future of AI inference in orbital applications.”
- As the overall 2017 Copernicus Masters winner, FSSCat, was proposed by Spain’s Universitat Politècnica de Catalunya and developed by a consortium of European companies and institutes including Tyvak International.
- Also mentioned in his opening speech this morning, Josef Aschbacher made a special announcement regarding an exciting new ESA initiative, the “EOP AI-enhanced Quantum Initiative for EO - QC4EO” in collaboration with the European Organization for Nuclear Research (CERN).
- Quantum computing has the potential to improve performance, decrease computational costs and solve previously intractable problems in Earth observation by exploiting quantum phenomena such as superposition, entanglement and tunnelling.
- The initiative involves creating a quantum capability which will have the ability to solve demanding Earth observation problems by using artificial intelligence to support programs such as Digital Twin Earth and Copernicus. The initiative will be developed at the Φ-lab – an ESA laboratory at ESA/ESRIN (ESA’s Center for Earth Observation) in Frascati, Italy, which embraces transformational innovation in Earth observation.
Figure 14: Quantum computing has the potential to improve performance, decrease computational costs and solve previously intractable problems in Earth observation by exploiting quantum phenomena such as superposition, entanglement and tunnelling (image credit: IBM)
- ESA and CERN enjoy a long-standing collaboration, centered on technological matters and fundamental physics. This collaboration will be extended to link to the CERN Quantum Technology Initiative, which was announced in June 2020 by the CERN Director General, Fabiola Gianotti.
- Through this partnership, ESA and CERN will create new synergies, building on their common experience in big data, data mining and pattern recognition.
- Giuseppe Borghi, Head of the Φ-lab, said, “Quantum computing together with AI are perhaps the most promising breakthrough to come along in computer technology. In the coming years, we will see more Earth or space science disciplines employing current or future quantum computing techniques to solve geoscience problems.”
- Josef Aschbacher added, “ESA will exploit the broad range of specialized expertise available at ESA and we will place ourselves in a unique position and take a leading role in the development of quantum technologies in the Earth observation domain.”
- Alberto Di Meglio, Coordinator of the CERN Quantum Technology Initiative, said, “Quantum technologies are a rapidly growing field of research and their applications have the potential to revolutionize the way we do science. Preparing for that paradigm change, by building knowledge and tools, is essential. This new collaboration on quantum technologies bears great promise.”
1) ”First Earth observation satellite with AI ready for launch,” ESA, 12 September 2019, URL: http://www.esa.int/Our_Activities/Observing_the_Earth
2) ”Smallsats win big prize at Copernicus Masters,” ESA, 8 November 2017, URL: http://www.esa.int/Our_Activities/Observing_the_Earth
3) Marco Esposito, Bernardo Carnicero Domınguez, Massimiliano Pastena, Nathan Vercruyssen, Simon Silvio Conticello, Chris van Dijk, Pierluigi Foglia Manzillo, Rick Koeleman,”Highly integration of hyperspectral, thermal and artificial intelligence for the ESA PhiSat-1 mission,” Proceedings of the 70th IAC (International Astronautical Congress), Washington DC, USA, 21-25 October 2019, URL: https://iafastro.directory/iac/proceedings/IAC-19/IAC-19/B4/4/manuscripts/IAC-19,B4,4,4,x54611.pdf
4) Massimiliano Pastena, Bernardo Carnicero Domínguez, Pierre Philippe Mathieu, Aman d a Re ga n,Marco Esposito, Simon Conticello, Chris Van Dijk, Nathan Vercruyssen, Pierluigi Foglia Manzillo, Rick Koelemann, John Hefele, ” Earth Observation Directorate NewSpace Initiatives,” Proceedings of the 70th IAC (International Astronautical Congress), Washington DC, USA, 21-25 October 2019, paper: SSC19-V-05, URL: https://digitalcommons.usu.edu/cgi/viewcontent.cgi?article=4391&context=smallsat
5) A Camps, A. Golkar, A. Gitierrez, J. Ruit-de-Azua, J. Muñoz‐Martin, L. Fernandez-Capon, C. Diez, A. Aguilella, S. Briatore, N. Garzaniti, F. Nichele, R. Mozzillo, M. Vanotti, M. Esposito, B. Carnicero, G. Filippazzo, and A. Regan, “FSSCat: A Cubesat-based Tandem Mission for Earth Observation of the Polar Regions” Living Planet Symposium ‘19, Milan, May 2019
6) Marco Esposito, Alessandro Zuccaro Marchi, “In-Orbit Demonstration of the first hyperspectral imager for nanosatellites” ICSO ( International Conference on Space Optics) 2018, Chania, Greece
Pierluigi Foglia Manzillo, Ljubisa Babic, Marco Esposito, Cristian
Corneliu Chitu, Marco Canetri, Paolo Corradi, Jesus Gil Fernandez,
Michael Kueppers. ”TIRI: A Multi-Purpose Thermal Infrared Payload
for Planetary Exploration” Proceedings of the 69th IAC
(International Astronautical Congress) Bremen, Germany, 1-5 October
2018, paper: IAC-18,A3,4B,4, URL: https://iafastro.directory/
8) ”Vega return to flight proves new rideshare service,” ESA Enabling & Support, 3 September 2020, URL: https://www.esa.int/Enabling_Support/Space_Transportation
”With Vega, Arianespace successfully performs the first European
mission to launch multiple small satellites,” Arianespace Press
Release, 3 September 2020, URL: https://www.arianespace.com
11) Mike Safyan, ”Planet’s First Launch of 2020: 26 SuperDoves on a Vega,” Planet, 13 February 2020, URL: https://www.planet.com/pulse/planets-first-launch-of-2020-26-superdoves-on-a-vega/
12) ”Overview of the FSSCAT Optical Intersatellite Link (O-ISL) - Technology Demonstration,” EPFL, 2019, URL: https://memento.epfl.ch/event/overview-of-the-fsscat-optical-intersatellite-link/
13) ”Intel Powers First Satellite with AI on Board,” Intel, 20 October 2020, URL: https://newsroom.intel.de/news/intel-powers-first-satellite-with-ai-on-board/#gs.jvbakw
”Digital Twin Earth, quantum computing and AI take centre stage
at ESA’s Φ-week,” ESA Applications, 28 September 2020,
The information compiled and edited in this article was provided by Herbert J. Kramer from his documentation of: ”Observation of the Earth and Its Environment: Survey of Missions and Sensors” (Springer Verlag) as well as many other sources after the publication of the 4th edition in 2002. - Comments and corrections to this article are always welcome for further updates (firstname.lastname@example.org).