The Earth is constantly hit by energetic particles originating from galactic sources. The flux of these particles is altered by the magnetized solar wind in the heliosphere and the Earth's magnetic field. For this reason, the ability of a particle to approach a spacecraft in LEO depends on its energy and the position of the spacecraft within the Earth' magnetosphere. Moreover, there are some areas (radiation belts) where the particles are trapped for a long time, and therefore the flux of energetic particles is particularly high. Occasionally, SEP contribute to the energetic particle flux too. DOSTEL is one of the instruments aboard the \ac{ISS} that monitors the radiation field within the European module Columbus. Because being installed inside the \ac{ISS}, particles produced by the interaction between the "primary" radiation and the ISS materials are also measured. To describe the observations in such a complex radiation field, we follow the method by Caballero-Lopez and Moraal (2012) in order to compute the so-called yield function using precise measurements of the proton and Helium energy spectra obtained by AMS and the systematic variation of the DOSTEL measurements within the Earth's magnetosphere

For the 2007 International Forum on Landslide Disaster Management framework, our team performed several numerical simulations on both theoretical and natural cases of granular flows. The objective was to figure out the ability and the limits of our numerical model in terms of reproduction and prediction. Our benchmarking exercises show that for almost all the cases, the model we use is able to reproduce observations at the field scale. Calibrated friction angles are almost similar to that used in other models and the shape of the final deposits is in good agreement with observation. However, as it is tricky to compare the dynamics of natural cases, these exercises do not allow us to highlight the good ability to reproduce the behavior of natural landslides. Nevertheless, by comparing with analytical solution, we show that our model presents very low numerical dissipation due to the discretization and to the numerical scheme used. Finally, in terms of mitigation and prediction, the different friction angles used for each cases figure out the limits of using such model as long as constitutive equations for granular media are not known.

Computer vision techniques have attracted a great interest in precision agriculture, recently. The common goal of all computer vision-based precision agriculture tasks is to detect the objects of interest (e.g., crop, weed) and discriminating them from the background. The Weeds are unwanted plants growing among crops competing for nutrients, water, and sunlight, causing losses to crop yields. Weed detection and mapping is critical for site-specific weed management to reduce the cost of labor and impact of herbicides. This paper investigates the use of color and texture features for discrimination of Soybean crops and weeds. Feature extraction methods including two color spaces (RGB, HSV), gray level Co-occurrence matrix (GLCM), and Local Binary Pattern (LBP) are used to train the Support Vector Machine (SVM) classifier. The experiment was carried out on image dataset of soybean crop, obtained from an unmanned aerial vehicle (UAV), which is publicly available. The results from the experiment showed that the highest accuracy (above 96%) was obtained from the combination of color and LBP features.

We investigate the problem of privacy preserving distributed matrix multiplication in edge networks using multi-party computation (MPC). Coded multi-party computation (CMPC) is an emerging approach to reduce the required number of workers in MPC by employing coded computation. Existing CMPC approaches usually combine coded computation algorithms designed for efficient matrix multiplication with MPC. We show that this approach is not efficient. We design a novel CMPC algorithm; PolyDot coded MPC (PolyDot-CMPC) by using a recently proposed coded computation algorithm; PolyDot codes. We exploit "garbage terms" that naturally arise when polynomials are constructed in the design of PolyDot-CMPC to reduce the number of workers needed for privacy-preserving computation. We show that entangled polynomial codes, which are consistently better than PolyDot codes in coded computation setup, are not necessarily better than PolyDot-CMPC in MPC setting.

Object detection with Transformers (DETR) has achieved a competitive performance over traditional detectors, such as Faster R-CNN. However, the potential of DETR remains largely unexplored for the more challenging task of arbitrary-oriented object detection problem. We provide the first attempt and implement Oriented Object DEtection with TRansformer ($\bf O^2DETR$) based on an end-to-end network. The contributions of $\rm O^2DETR$ include: 1) we provide a new insight into oriented object detection, by applying Transformer to directly and efficiently localize objects without a tedious process of rotated anchors as in conventional detectors; 2) we design a simple but highly efficient encoder for Transformer by replacing the attention mechanism with depthwise separable convolution, which can significantly reduce the memory and computational cost of using multi-scale features in the original Transformer; 3) our $\rm O^2DETR$ can be another new benchmark in the field of oriented object detection, which achieves up to 3.85 mAP improvement over Faster R-CNN and RetinaNet. We simply fine-tune the head mounted on $\rm O^2DETR$ in a cascaded architecture and achieve a competitive performance over SOTA in the DOTA dataset.

Contrastive self-supervised learning (SSL) learns an embedding space that maps similar data pairs closer and dissimilar data pairs farther apart. Despite its success, one issue has been overlooked: the fairness aspect of representations learned using contrastive SSL. Without mitigation, contrastive SSL techniques can incorporate sensitive information such as gender or race and cause potentially unfair predictions on downstream tasks. In this paper, we propose a Conditional Contrastive Learning (CCL) approach to improve the fairness of contrastive SSL methods. Our approach samples positive and negative pairs from distributions conditioning on the sensitive attribute, or empirically speaking, sampling positive and negative pairs from the same gender or the same race. We show that our approach provably maximizes the conditional mutual information between the learned representations of the positive pairs, and reduces the effect of the sensitive attribute by taking it as the conditional variable. On seven fairness and vision datasets, we empirically demonstrate that the proposed approach achieves state-of-the-art downstream performances compared to unsupervised baselines and significantly improves the fairness of contrastive SSL models on multiple fairness metrics.

The self-supervised learning (SSL) paradigm is an essential exploration area, which tries to eliminate the need for expensive data labeling. Despite the great success of SSL methods in computer vision and natural language processing, most of them employ contrastive learning objectives that require negative samples, which are hard to define. This becomes even more challenging in the case of graphs and is a bottleneck for achieving robust representations. To overcome such limitations, we propose a framework for self-supervised graph representation learning - Graph Barlow Twins, which utilizes a cross-correlation-based loss function instead of negative samples. Moreover, it does not rely on non-symmetric neural network architectures - in contrast to state-of-the-art self-supervised graph representation learning method BGRL. We show that our method achieves as competitive results as the best self-supervised methods and fully supervised ones while requiring fewer hyperparameters and substantially shorter computation time (ca. 30 times faster than BGRL).

Predicting the permeability of porous media in saturated and partially saturated conditions is of crucial importance in many geo-engineering areas, from water resources to vadose zone hydrology or contaminant transport predictions. Many models have been proposed in the literature to estimate the permeability from properties of the porous media such as porosity, grain size or pore size. In this study, we develop a model of the permeability for porous media saturated by one or two fluid phases with all physically-based parameters using a fractal upscaling technique. The model is related to microstructural properties of porous media such as fractal dimension for pore space, fractal dimension for tortuosity, porosity, maximum radius, ratio of minimum pore radius and maximum pore radius, water saturation and irreducible water saturation. The model is favorably compared to existing and widely used models from the literature. Then, comparison with published experimental data for both unconsolidated and consolidated samples, we show that the proposed model estimate the permeability from the medium properties very well.

Unsupervised domain adaptation (UDA) and semi-supervised learning (SSL) are two typical strategies to reduce expensive manual annotations in machine learning. In order to learn effective models for a target task, UDA utilizes the available labeled source data, which may have different distributions from unlabeled samples in the target domain, while SSL employs few manually annotated target samples. Although UDA and SSL are seemingly very different strategies, we find that they are closely related in terms of task objectives and solutions, and SSL is a special case of UDA problems. Based on this finding, we further investigate whether SSL methods work on UDA tasks. By adapting eight representative SSL algorithms on UDA benchmarks, we show that SSL methods are strong UDA learners. Especially, state-of-the-art SSL methods significantly outperform existing UDA methods on the challenging UDA benchmark of DomainNet, and state-of-the-art UDA methods could be further enhanced with SSL techniques. We thus promote that SSL methods should be employed as baselines in future UDA studies and expect that the revealed relationship between UDA and SSL could shed light on future UDA development. Codes are available at \url{https://github.com/YBZh}.

Much environmental enforcement in the United States has historically relied on either self-reported data or physical, resource-intensive, infrequent inspections. Advances in remote sensing and computer vision, however, have the potential to augment compliance monitoring by detecting early warning signs of noncompliance. We demonstrate a process for rapid identification of significant structural expansion using Planet's 3m/pixel satellite imagery products and focusing on Concentrated Animal Feeding Operations (CAFOs) in the US as a test case. Unpermitted building expansion has been a particular challenge with CAFOs, which pose significant health and environmental risks. Using new hand-labeled dataset of 145,053 images of 1,513 CAFOs, we combine state-of-the-art building segmentation with a likelihood-based change-point detection model to provide a robust signal of building expansion (AUC = 0.86). A major advantage of this approach is that it can work with higher cadence (daily to weekly), but lower resolution (3m/pixel), satellite imagery than previously used in similar environmental settings. It is also highly generalizable and thus provides a near real-time monitoring tool to prioritize enforcement resources in other settings where unpermitted construction poses environmental risk, e.g. zoning, habitat modification, or wetland protection.

Crowdsourced vehicle-based observations have the potential to improve forecast skill in convection-permitting numerical weather prediction (NWP). The aim of this paper is to explore the characteristics of vehicle-based observations of air temperature. We describe a novel low-precision vehicle-based observation dataset obtained from a Met Office proof-of-concept trial. In this trial, observations of air temperature were obtained from built-in vehicle air-temperature sensors, broadcast to an application on the participant's smartphone and uploaded, with relevant metadata, to the Met Office servers. We discuss the instrument and representation uncertainties associated with vehicle-based observations and present a new quality-control procedure. It is shown that, for some observations, location metadata may be inaccurate due to unsuitable smartphone application settings. The characteristics of the data that passed quality-control are examined through comparison with United Kingdom variable-resolution model data, roadside weather information station observations, and Met Office integrated data archive system observations. Our results show that the uncertainty associated with vehicle-based observation-minus-model comparisons is likely to be weather-dependent and possibly vehicle-dependent. Despite the low precision of the data, vehicle-based observations of air temperature could be a useful source of spatially-dense and temporally-frequent observations for NWP.

Change detection is the process of identifying pixelwise differences in bitemporal co-registered images. It is of great significance to Earth observations. Recently, with the emergence of deep learning (DL), the power and feasibility of deep convolutional neural network (CNN)-based methods have been shown in the field of change detection. However, there is still a lack of effective supervision for change feature learning. In this work, a feature constraint change detection network (FCCDN) is proposed. We constrain features both in bitemporal feature extraction and feature fusion. More specifically, we propose a dual encoder-decoder network backbone for the change detection task. At the center of the backbone, we design a nonlocal feature pyramid network to extract and fuse multiscale features. To fuse bitemporal features in a robust way, we build a dense connection-based feature fusion module. Moreover, a self-supervised learning-based strategy is proposed to constrain feature learning. Based on FCCDN, we achieve state-of-the-art performance on two building change detection datasets (LEVIR-CD and WHU). On the LEVIR-CD dataset, we achieve an IoU of 0.8569 and an F1 score of 0.9229. On the WHU dataset, we achieve an IoU of 0.8820 and an F1 score of 0.9373. Moreover, for the first time, the acquisition of accurate bitemporal semantic segmentation results is achieved without using semantic segmentation labels. This is vital for the application of change detection because it saves the cost of labeling.

Following N.Kozyrev's idea about the influence of the gravitational fields of the Sun and the Moon on the Earth's crust, we consider a low-frequency resonance of the Earth's crust blocks is happening before the occurrence of the earthquake. The resonance affects several geophysical parameters and is associated with many short-term precursory phenomena; the most notable are the release of underground gases (radon), the air temperature rises, change of thermal radiation in the troposphere, and an increase in the electron density in the ionosphere. During the final stage of the earthquake genesis, long-period gravity-seismic waves called the Kozyrev-Yagodin ( KaY-) wave are formed and moves from the periphery to the epicenter of the future earthquake. We discuss that by integrating information from different stages of earthquake genesis: the network observations of KaY waves and satellite thermal radiation, we significantly could advance the accuracy and reliability of short-term earthquake forecasting, previously unachievable.

It is a well-established principle that cross-correlating seismic observations at different receiver locations can yield estimates of band-limited inter-receiver Green's functions. This principle, known as seismic interferometry, is a powerful technique that can transform noise into signals which allow us to remotely image and interrogate subsurface Earth structures. In practice it is often necessary and even desirable to rely on noise already present in the environment. Theory that underpins many applications of ambient noise interferometry makes an assumption that the noise sources are uncorrelated in space and time. However, many real-world noise sources such as trains, highway traffic and ocean waves are inherently correlated both in space and time, in direct contradiction to the current theoretical foundations. Applying standard interferometric techniques to recordings from correlated energy sources makes the Green's function liable to estimation errors that so far have not been fully accounted for theoretically nor in practice. We show that these errors are significant for common noise sources, always perturbing and sometimes obscuring the phase one wishes to retrieve. Our analysis explains why stacking may reduce the phase errors, but also shows that in commonly-encountered circumstances stacking will not remediate the problem. This analytical insight allowed us to develop a novel workflow that significantly mitigates effects arising from the use of correlated noise sources. Our methodology can be used in conjunction with already existing approaches, and improves results from both correlated and uncorrelated ambient noise. Hence, we expect it to be widely applicable in real life ambient noise studies.

Methane emissions from oil and gas (O&G) production and transmission represent a significant contribution to climate change. These emissions comprise sporadic releases of large amounts of methane during maintenance operations or equipment failures not accounted for in current inventory estimates. We collected and analyzed hundreds of very large releases from atmospheric methane images sampled by the TROPOspheric Monitoring Instrument (TROPOMI) over 2019 and 2020 to quantify emissions from O&G ultra-emitters. Ultra-emitters are primarily detected over the largest O&G basins of the world, following a power-law relationship with noticeable variations across countries but similar regression slopes. With a total contribution equivalent to 8-12% of the global O&G production methane emissions, mitigation of ultra-emitters is largely achievable at low costs and would lead to robust net benefits in billions of US dollars for the six major producing countries when incorporating recent estimates of societal costs of methane.

To detect and segment objects in images based on their content is one of the most active topics in the field of computer vision. Nowadays, this problem can be addressed using Deep Learning architectures such as Faster R-CNN or YOLO, among others. In this paper, we study the behaviour of different configurations of RetinaNet, Faster R-CNN and Mask R-CNN presented in Detectron2. First, we evaluate qualitatively and quantitatively (AP) the performance of the pre-trained models on KITTI-MOTS and MOTSChallenge datasets. We observe a significant improvement in performance after fine-tuning these models on the datasets of interest and optimizing hyperparameters. Finally, we run inference in unusual situations using out of context datasets, and present interesting results that help us understanding better the networks.

This paper provides theoretical and practical arguments regarding the possibility of predicting strong and major earthquakes worldwide. Many strong and major earthquakes can be predicted at least two to five months in advance, based on identifying stressed areas that begin to behave abnormally before strong events, with the size of these areas corresponding to Dobrovolsky formula. We make predictions by combining knowledge from many different disciplines: physics, geophysics, seismology, geology, and earth science, among others. An integrated approach is used to identify anomalies and make predictions, including satellite remote sensing techniques and data from ground-based instruments. Terabytes of information are currently processed every day with many different multi-parametric prediction systems applied thereto. Alerts are issued if anomalies are confirmed by a few different systems. It has been found that geophysical patterns of earthquake preparation and stress accumulation are similar for all key seismic regions. The same earthquake prediction methodologies and systems have been successfully applied in global practice since 2013, with the technology successfully used to retrospectively test against more than 700 strong and major earthquakes since 1970.

Ecosystem services are the direct and indirect contributions of an ecosystem to human well-being and survival. Ecosystem valuation is a method of assigning a monetary value to an ecosystem with its goods and services,often referred to as ecosystem service value (ESV). With the rapid expansion of cities, a mismatch occurs between urban development and ecological development, and it is increasingly urgent to establish a valid ecological assessment method. In this study, we propose an ecological evaluation standard framework by designing an ESV monitoring workflow based on the establishment of multi-level grids. The proposed method is able to capture multi-scale features, facilitates multi-level spatial expression, and can effectively reveal the spatial heterogeneity of ESV. Taking Haian city in the Jiangsu province as the study case, we implemented the proposed dynamic multi-level grids-based (DMLG) to calculate its urban ESV in 2016 and 2019. We found that the ESV of Haian city showed considerable growth (increased by 24.54 million RMB). Negative ESVs are concentrated in the central city, which presented a rapid trend of outward expansion. The results illustrated that the ongoing urban expanse does not reduce the ecological value in the study area. The proposed unified grid framework can be applied to other geographical regions and is expected to benefit future studies in ecosystem service evaluation in terms of capture multi-level spatial heterogeneity.

Energetic electrons inside Earth's outer Van Allen belt pose a major radiation threat to space-borne electronics that often play vital roles in our modern society. Ultra-relativistic electrons with energies greater than or equal to two Megaelectron-volt (MeV) are of particular interest due to their high penetrating ability, and thus forecasting these >=2 MeV electron levels has significant meaning to all space sectors. Here we update the latest development of the predictive model for MeV electrons inside the Earth's outer radiation belt. The new version, called PreMevE-2E, focuses on forecasting ultra-relativistic electron flux distributions across the outer radiation belt, with no need of in-situ measurements except for at the geosynchronous (GEO) orbit. Model inputs include precipitating electrons observed in low-Earth-orbits by NOAA satellites, upstream solar wind conditions (speeds and densities) from solar wind monitors, as well as ultra-relativistic electrons measured by one Los Alamos GEO satellite. We evaluated a total of 32 supervised machine learning models that fall into four different classes of linear and neural network architectures, and also successfully tested ensemble forecasting by using groups of top-performing models. All models are individually trained, validated, and tested by in-situ electron data from NASA's Van Allen Probes mission. It is shown that the final ensemble model generally outperforms individual models overs L-shells, and this PreMevE-2E model provides reliable and high-fidelity 25-hr (~1-day) and 50-hr (~2-day) forecasts with high mean performance efficiency values. Our results also suggest this new model is dominated by non-linear components at low L-shells (< ~4) for ultra-relativistic electrons, which is different from the dominance of linear components at all L-shells for 1 MeV electrons as previously discovered.

This paper investigates regex CQs with string equalities (SERCQs), a subclass of core spanners. As shown by Freydenberger, Kimelfeld, and Peterfreund (PODS 2018), these queries are intractable, even if restricted to acyclic queries. This previous result defines acyclicity by treating regex formulas as atoms. In contrast to this, we propose an alternative definition by converting SERCQs into FC-CQs -- conjunctive queries in FC, a logic that is based on word equations. We introduce a way to decompose word equations of unbounded arity into a conjunction of binary word equations. If the result of the decomposition is acyclic, then evaluation and enumeration of results become tractable. The main result of this work is an algorithm that decides in polynomial time whether an FC-CQ can be decomposed into an acyclic FC-CQ. We also give an efficient conversion from synchronized SERCQs to FC-CQs with regular constraints. As a consequence, tractability results for acyclic relational CQs directly translate to a large class of SERCQs.

This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision. Challenges in adapting Transformer from language to vision arise from differences between the two domains, such as large variations in the scale of visual entities and the high resolution of pixels in images compared to words in text. To address these differences, we propose a hierarchical Transformer whose representation is computed with \textbf{S}hifted \textbf{win}dows. The shifted windowing scheme brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection. This hierarchical architecture has the flexibility to model at various scales and has linear computational complexity with respect to image size. These qualities of Swin Transformer make it compatible with a broad range of vision tasks, including image classification (87.3 top-1 accuracy on ImageNet-1K) and dense prediction tasks such as object detection (58.7 box AP and 51.1 mask AP on COCO test-dev) and semantic segmentation (53.5 mIoU on ADE20K val). Its performance surpasses the previous state-of-the-art by a large margin of +2.7 box AP and +2.6 mask AP on COCO, and +3.2 mIoU on ADE20K, demonstrating the potential of Transformer-based models as vision backbones. The hierarchical design and the shifted window approach also prove beneficial for all-MLP architectures. The code and models are publicly available at~\url{https://github.com/microsoft/Swin-Transformer}.

The size distribution of marine microplastics provides a fundamental data source for understanding the dispersal, break down, and biotic impacts of the microplastics in the ocean. The observed size distribution at the sea surface generally shows, from large to small sizes, a gradual increase followed by a rapid decrease. This decrease has led to the hypothesis that the smallest fragments are selectively removed by sinking or biological uptake. Here we propose a new model of size distribution without any removal of material from the system. The model uses an analogy with black-body radiation and the resultant size distribution is analogous to Planck's law. In this model, the original large plastic piece is broken into smaller pieces once by the application of "energy" or work by waves or other processes, under two assumptions, one that fragmentation into smaller pieces requires larger energy and the other that the probability distribution of the "energy" follows the Boltzmann distribution. Our formula well reproduces observed size distributions over wide size ranges from micro- to mesoplastics. According to this model, the smallest fragments are fewer because large "energy" required to produce such small fragments occurs more rarely.

Understanding how environmental characteristics affect bio-diversity patterns, from individual species to communities of species, is critical for mitigating effects of global change. A central goal for conservation planning and monitoring is the ability to accurately predict the occurrence of species communities and how these communities change over space and time. This in turn leads to a challenging and long-standing problem in the field of computer science - how to perform ac-curate multi-label classification with hundreds of labels? The key challenge of this problem is its exponential-sized output space with regards to the number of labels to be predicted.Therefore, it is essential to facilitate the learning process by exploiting correlations (or dependency) among labels. Previous methods mostly focus on modelling the correlation on label pairs; however, complex relations between real-world objects often go beyond second order. In this paper, we pro-pose a novel framework for multi-label classification, High-order Tie-in Variational Autoencoder (HOT-VAE), which per-forms adaptive high-order label correlation learning. We experimentally verify that our model outperforms the existing state-of-the-art approaches on a bird distribution dataset on both conventional F1 scores and a variety of ecological metrics. To show our method is general, we also perform empirical analysis on seven other public real-world datasets in several application domains, and Hot-VAE exhibits superior performance to previous methods.

Recently, neural architecture search (NAS) has been exploited to design feature pyramid networks (FPNs) and achieved promising results for visual object detection. Encouraged by the success, we propose a novel One-Shot Path Aggregation Network Architecture Search (OPANAS) algorithm, which significantly improves both searching efficiency and detection accuracy. Specifically, we first introduce six heterogeneous information paths to build our search space, namely top-down, bottom-up, fusing-splitting, scale-equalizing, skip-connect and none. Second, we propose a novel search space of FPNs, in which each FPN candidate is represented by a densely-connected directed acyclic graph (each node is a feature pyramid and each edge is one of the six heterogeneous information paths). Third, we propose an efficient one-shot search method to find the optimal path aggregation architecture, that is, we first train a super-net and then find the optimal candidate with an evolutionary algorithm. Experimental results demonstrate the efficacy of the proposed OPANAS for object detection: (1) OPANAS is more efficient than state-of-the-art methods (e.g., NAS-FPN and Auto-FPN), at significantly smaller searching cost (e.g., only 4 GPU days on MS-COCO); (2) the optimal architecture found by OPANAS significantly improves main-stream detectors including RetinaNet, Faster R-CNN and Cascade R-CNN, by 2.3-3.2 % mAP comparing to their FPN counterparts; and (3) a new state-of-the-art accuracy-speed trade-off (52.2 % mAP at 7.6 FPS) at smaller training costs than comparable state-of-the-arts. Code will be released at https://github.com/VDIGPKU/OPANAS.

Niche and neutral theory are two prevailing, yet much debated, ideas in ecology proposed to explain the patterns of biodiversity. Whereas niche theory emphasizes selective differences between species and interspecific interactions in shaping the community, neutral theory supposes functional equivalence between species and points to stochasticity as the primary driver of ecological dynamics. In this work, we draw a bridge between these two opposing theories. Starting from a Lotka-Volterra (LV) model with demographic noise and random symmetric interactions, we analytically derive the stationary population statistics and species abundance distribution (SAD). Using these results, we demonstrate that the model can exhibit three classes of SADs commonly found in niche and neutral theories and found conditions that allow an ecosystem to transition between these various regimes. Thus, we reconcile how neutral-like statistics may arise from a diverse community with niche differentiation.

In the paper, we propose an adaptive data-driven model-based approach for filling the gaps in time series. The approach is based on the automated evolutionary identification of the optimal structure for a composite data-driven model. It allows adapting the model for the effective gap-filling in a specific dataset without the involvement of the data scientist. As a case study, both synthetic and real datasets from different fields (environmental, economic, etc) are used. The experiments confirm that the proposed approach allows achieving the higher quality of the gap restoration and improve the effectiveness of forecasting models.

Agent-Based Models are a powerful class of computational models widely used to simulate complex phenomena in many different application areas. However, one of the most critical aspects, poorly investigated in the literature, regards an important step of the model credibility assessment: solution verification. This study overcomes this limitation by proposing a general verification framework for Agent-Based Models that aims at evaluating the numerical errors associated with the model. A step-by-step procedure, which consists of two main verification studies (deterministic and stochastic model verification), is described in detail and applied to a specific mission critical scenario: the quantification of the numerical approximation error for UISS-TB, an ABM of the human immune system developed to predict the progression of pulmonary tuberculosis. Results provide indications on the possibility to use the proposed model verification workflow to systematically identify and quantify numerical approximation errors associated with UISS-TB and, in general, with any other ABMs.

Deep learning on graphs has attracted significant interests recently. However, most of the works have focused on (semi-) supervised learning, resulting in shortcomings including heavy label reliance, poor generalization, and weak robustness. To address these issues, self-supervised learning (SSL), which extracts informative knowledge through well-designed pretext tasks without relying on manual labels, has become a promising and trending learning paradigm for graph data. Different from SSL on other domains like computer vision and natural language processing, SSL on graphs has an exclusive background, design ideas, and taxonomies. Under the umbrella of graph self-supervised learning, we present a timely and comprehensive review of the existing approaches which employ SSL techniques for graph data. We construct a unified framework that mathematically formalizes the paradigm of graph SSL. According to the objectives of pretext tasks, we divide these approaches into four categories: generation-based, auxiliary property-based, contrast-based, and hybrid approaches. We further describe the applications of graph SSL across various research fields and summarize the commonly used datasets, evaluation benchmark, performance comparison and open-source codes of graph SSL. Finally, we discuss the remaining challenges and potential future directions in this research field.

Deep learning has made revolutionary advances to diverse applications in the presence of large-scale labeled datasets. However, it is prohibitively time-costly and labor-expensive to collect sufficient labeled data in most realistic scenarios. To mitigate the requirement for labeled data, semi-supervised learning (SSL) focuses on simultaneously exploring both labeled and unlabeled data, while transfer learning (TL) popularizes a favorable practice of fine-tuning a pre-trained model to the target data. A dilemma is thus encountered: Without a decent pre-trained model to provide an implicit regularization, SSL through self-training from scratch will be easily misled by inaccurate pseudo-labels, especially in large-sized label space; Without exploring the intrinsic structure of unlabeled data, TL through fine-tuning from limited labeled data is at risk of under-transfer caused by model shift. To escape from this dilemma, we present Self-Tuning to enable data-efficient deep learning by unifying the exploration of labeled and unlabeled data and the transfer of a pre-trained model, as well as a Pseudo Group Contrast (PGC) mechanism to mitigate the reliance on pseudo-labels and boost the tolerance to false labels. Self-Tuning outperforms its SSL and TL counterparts on five tasks by sharp margins, e.g. it doubles the accuracy of fine-tuning on Cars with 15% labels.

Deep models trained in supervised mode have achieved remarkable success on a variety of tasks. When labeled samples are limited, self-supervised learning (SSL) is emerging as a new paradigm for making use of large amounts of unlabeled samples. SSL has achieved promising performance on natural language and image learning tasks. Recently, there is a trend to extend such success to graph data using graph neural networks (GNNs). In this survey, we provide a unified review of different ways of training GNNs using SSL. Specifically, we categorize SSL methods into contrastive and predictive models. In either category, we provide a unified framework for methods as well as how these methods differ in each component under the framework. Our unified treatment of SSL methods for GNNs sheds light on the similarities and differences of various methods, setting the stage for developing new methods and algorithms. We also summarize different SSL settings and the corresponding datasets used in each setting. To facilitate methodological development and empirical comparison, we develop a standardized testbed for SSL in GNNs, including implementations of common baseline methods, datasets, and evaluation metrics.

Recent years have seen a proliferation of attention mechanisms and the rise of Transformers in Natural Language Generation (NLG). Previously, state-of-the-art NLG architectures such as RNN and LSTM ran into vanishing gradient problems; as sentences grew larger, distance between positions remained linear, and sequential computation hindered parallelization since sentences were processed word by word. Transformers usher in a new era. In this paper, we explore three major Transformer-based models, namely GPT, BERT, and XLNet, that carry significant implications for the field. NLG is a burgeoning area that is now bolstered with rapid developments in attention mechanisms. From poetry generation to summarization, text generation derives benefit as Transformer-based language models achieve groundbreaking results.

Ground observatory and satellite-based determinations of temporal variations in the geomagnetic field probe a decadal to annual time scale range where Earth's core slow, inertialess convective motions and rapidly propagating, inertia-bearing hydromagnetic waves are in interplay. Here we numerically model and jointly investigate these two important features with the help of a geodynamo simulation that (to date) is the closest to the dynamical regime of Earth's core. This model also considerably enlarges the scope of a previous asymptotic scaling analysis. Three classes of hydrodynamic and hydromagnetic waves are identified in the model output, all with propagation velocity largely exceeding that of convective advection: axisymmetric, geostrophic Alfv\'en torsional waves, and non-axisymmetric, quasi-geostrophic Alfv\'en and Rossby waves. The contribution of these waves to the geomagnetic acceleration amounts to an enrichment and flattening of its energy density spectral profile at decadal time scales, thereby providing a constraint on the extent of the $f^{-4}$ range observed in the geomagnetic frequency power spectrum. The flow and magnetic acceleration energies carried by waves both linearly increase with the ratio of the magnetic diffusion time scale to the Alfv\'en time scale, highlighting the dominance of Alfv\'en waves in the signal and the stabilising control of magnetic dissipation at non-axisymmetric scales. Extrapolation of the results to Earth's core conditions supports the detectability of Alfv\'en waves in geomagnetic observations, either as axisymmetric torsional oscillations or through the geomagnetic jerks caused by non-axisymmetric waves. In contrast, Rossby waves appear to be too fast and carry too little magnetic energy to be detectable in geomagnetic acceleration signals of limited spatio-temporal resolution.

Sleep has a profound influence on the physiology of body systems and biological processes. Molecular studies have shown circadian-regulated shifts in protein expression patterns across human tissues, further emphasizing the unique functional, behavioral and pharmacokinetic landscape of sleep. Thus, many pathological processes are also expected to exhibit sleep-specific manifestations. Nevertheless, sleep is seldom utilized for the study, detection and treatment of non-sleep-specific pathologies. Modern advances in biosensor technologies have enabled remote, non-invasive recording of a growing number of physiologic parameters and biomarkers. Sleep is an ideal time frame for the collection of long and clean physiological time series data which can then be analyzed using data-driven algorithms such as deep learning. In this perspective paper, we aim to highlight the potential of sleep as an auspicious time for diagnosis, management and therapy of nonsleep-specific pathologies. We introduce key clinical studies in selected medical fields, which leveraged novel technologies and the advantageous period of sleep to diagnose, monitor and treat pathologies. We then discuss possible opportunities to further harness this new paradigm and modern technologies to explore human health and disease during sleep and to advance the development of novel clinical applications: From sleep medicine to medicine during sleep.

For highly automated driving above SAE level~3, behavior generation algorithms must reliably consider the inherent uncertainties of the traffic environment, e.g. arising from the variety of human driving styles. Such uncertainties can generate ambiguous decisions, requiring the algorithm to appropriately balance low-probability hazardous events, e.g. collisions, and high-probability beneficial events, e.g. quickly crossing the intersection. State-of-the-art behavior generation algorithms lack a distributional treatment of decision outcome. This impedes a proper risk evaluation in ambiguous situations, often encouraging either unsafe or conservative behavior. Thus, we propose a two-step approach for risk-sensitive behavior generation combining offline distribution learning with online risk assessment. Specifically, we first learn an optimal policy in an uncertain environment with Deep Distributional Reinforcement Learning. During execution, the optimal risk-sensitive action is selected by applying established risk criteria, such as the Conditional Value at Risk, to the learned state-action return distributions. In intersection crossing scenarios, we evaluate different risk criteria and demonstrate that our approach increases safety, while maintaining an active driving style. Our approach shall encourage further studies about the benefits of risk-sensitive approaches for self-driving vehicles.

Coalbed methane embedded in coal seams, is an unconventional energy resource as well as a hazardous gas existing in mining industries, which attracts lots of global attention. As the largest coal producer, the mining industry in China had to deal with many hazards induced by methane for decades. To solve this issue, underground methane extraction is commonly used in underground coal mines. However, underground methane extraction is hampered by low production rate and low efficiency because of slow gas emission from coal primarily controlled by gas desorption and permeability. It is well known that temperature has a great impact on gas sorption. The higher the temperature the larger the desorption rate. As the depth of coal mines increases beyond 1000m coal mines suffer elevated air temperatures caused by the natural geothermal gradient. The elevated temperature in such mines provides a potential economical way for geothermal energy extraction and utilization in deep coal mines which can largely cut the expenses of installation and operation maintenance. Therefore, a novel method is proposed to enhance underground methane extraction by deep heat stimulation. This paper mainly presents an assessment of previous and ongoing research in the related field and provides a first feasibility analysis of this method applied in the underground environment. The technique proposed in this early appraisal is deemed significant for coalbed methane drainage enhancing the productivity of deep coal mines by geothermal technology and can also be extended for many applications in relevant areas such as shale gas, and tight oil.

The goal of this paper is to review and critically assess different methods to monitor key process variables for ethanol production from lignocellulosic biomass. Because cellulose-based biofuels cannot yet compete with non-cellulosic biofuels, process control and optimization are of importance to lower the production costs. This study reviews different monitoring schemes, to indicate what the added value of real-time monitoring is for process control. Furthermore, a comparison is made on different monitoring techniques to measure the off-gas, the concentrations of dissolved components in the inlet to the process, the concentrations of dissolved components in the reactor, and the biomass concentration. Finally, soft sensor techniques and available models are discussed, to give an overview of modeling techniques that analyze data, with the aim of coupling the soft sensor predictions to the control and optimization of cellulose to ethanol fermentation. The paper ends with a discussion of future needs and developments.

Remote sensing for archaeological investigations using surface response is reasonably well established, however, remote subsurface exploration is limited by depth and penetration and ground resolution. Furthermore, the conservation of archaeological sites requires constant monitoring capability, which is often not feasible between annual field seasons, but may be provided by modern satellite coverage. Here we develop an approach using Sentinel-1 C-band radar backscatter, and Sentinel-2 multispectral data, to map and characterise the site of Qubbet el-Hawa, Egypt. The multispectral bands analysed show similar sensitivity to satellite imagery. However, the radar backscatter is sensitive to exposed known structures, as well as disturbances to soil textural/composition profile due to excavation/erosion. Sub-resolution features such as causeways manifest as a 'radar-break' in the backscatter - a discontinuity in otherwise continuous radar units. Furthermore, the finite subsurface response in the backscatter under the arid conditions of the site means we are able to delineate some shallow subsurface structures and map their orientation beneath the surface in areas not yet excavated. The sensitivity of Sentinel-1 backscatter to soil disturbance and human activity at Qubbet el-Hawa, and the short (~12 day) recurrence time of the satellites, makes it an important tool in heritage conservation.

Increasing aridification of continental areas due to global climate change has impacted freshwater availability, particularly in extremely dry landmasses, such as Australia. Multiple demands on water resources require integrated basin management approaches, necessitating knowledge of total water storage, and changes in water mass. Such monitoring is not practical at continental scales using traditional methods. Satellite gravity has proven successful at documenting changes in total water mass at regional scales, and here we use data from the Grace and Grace-FO missions, spanning 2002 - 2020, to track regional water budget trends in Australia most heavily utilised basin systems, including the Murray-Darling Basin. The period of analysis covers the Millennium drought (2002-2009) and 2010-11 heavy flooding events, which contribute significant signal variability. However our extended datasets demonstrate a negative trend in the geoid anomaly over the Murray-Darling Basin of -1.5mm, equivalent to a water loss rate of -0.91 Gt yr-1. With the exception of northern Australia, similar scale geoid declines are observed in most Australian basin systems analysed - implying declining total water storage. Long-term declines in water availability require concerted management plans, balancing the requirements of agriculture and industry, with domestic use, traditional owners, and healthy freshwater ecosystems.

Knowledge on the temporal and size distribution of particulate matter (PM) in air as well as on its elemental composition is a key information for source appointment, for the investigation of their influence on environmental processes and for providing valid data for climate models. A prerequisite is that size fractionated sampling times of few hours must be achieved such that anthropogenic and natural emissions can be correctly identified. While cascade impactors allow for time- and size-resolved collection of airborne PM, total reflection X-ray fluorescence (TXRF) allows for element-sensitive investigation of low sample amounts thanks to its detection sensitivity. However, during quantification by means of TXRF it is crucial to be aware of the limits of TXRF in order to identify situations where collection times or pollution levels were exceedingly long or high. It will be shown by means of grazing incidence X-ray fluorescence (GIXRF), where different reflection conditions are probed, that a self consistent quantification of elemental mass depositions can be performed in order to validate or identify issues in quantification by means of TXRF. Furthermore, monitors of validity for a reliable quantification of the elemental composition of PM by means of TXRF will be introduced. The methodological approach presented can be transferred to tabletop instrumentation in order to guarantee a reliable quantification on an element sensitive basis of the PM collected. This aspect is highly relevant for defining appropriate legislation and measures for health and climate protection and for supporting their enforcement and monitoring.

In the last few years, the release of BERT, a multilingual transformer based model, has taken the NLP community by storm. BERT-based models have achieved state-of-the-art results on various NLP tasks, including dialog tasks. One of the limitation of BERT is the lack of ability to handle long text sequence. By default, BERT has a maximum wordpiece token sequence length of 512. Recently, there has been renewed interest to tackle the BERT limitation to handle long text sequences with the addition of new self-attention based architectures. However, there has been little to no research on the impact of this limitation with respect to dialog tasks. Dialog tasks are inherently different from other NLP tasks due to: a) the presence of multiple utterances from multiple speakers, which may be interlinked to each other across different turns and b) longer length of dialogs. In this work, we empirically evaluate the impact of dialog length on the performance of BERT model for the Next Response Selection dialog task on four publicly available and one internal multi-turn dialog datasets. We observe that there is little impact on performance with long dialogs and even the simplest approach of truncating input works really well.

The two main drivers of climate change on sub-Milankovic time scales are re-assessed by means of a multiple regression analysis. Evaluating linear combinations of the logarithm of carbon dioxide concentration and the geomagnetic aa-index as a proxy for solar activity, we reproduce the sea surface temperature (HadSST) since the middle of the 19th century with an adjusted $R^2$ value of around 87 per cent for a climate sensitivity (of TCR type) in the range of 0.6 K until 1.6 K per doubling of CO$_2$. The solution of the regression is quite sensitive: when including data from the last decade, the simultaneous occurrence of a strong El Ni\~no on one side and low aa-values on the other side lead to a preponderance of solutions with relatively high climate sensitivities around 1.6 K. If those later data are excluded, the regression leads to a significantly higher weight of the aa-index and a correspondingly lower climate sensitivity going down to 0.6 K. The plausibility of such low values is discussed in view of recent experimental and satellite-borne measurements. We argue that a further decade of data collection will be needed to allow for a reliable distinction between low and high sensitivity values. Based on recent ideas about a quasi-deterministic planetary synchronization of the solar dynamo, we make a first attempt to predict the aa-index and the resulting temperature anomaly for various typical CO$_2$ scenarios. Even for the highest climate sensitivities, and an unabated linear CO$_2$ increase, we predict only a mild additional temperature rise of around 1 K until the end of the century, while for the lower values an imminent temperature drop in the near future, followed by a rather flat temperature curve, is prognosticated.

A biomedical relation statement is commonly expressed in multiple sentences and consists of many concepts, including gene, disease, chemical, and mutation. To automatically extract information from biomedical literature, existing biomedical text-mining approaches typically formulate the problem as a cross-sentence n-ary relation-extraction task that detects relations among n entities across multiple sentences, and use either a graph neural network (GNN) with long short-term memory (LSTM) or an attention mechanism. Recently, Transformer has been shown to outperform LSTM on many natural language processing (NLP) tasks. In this work, we propose a novel architecture that combines Bidirectional Encoder Representations from Transformers with Graph Transformer (BERT-GT), through integrating a neighbor-attention mechanism into the BERT architecture. Unlike the original Transformer architecture, which utilizes the whole sentence(s) to calculate the attention of the current token, the neighbor-attention mechanism in our method calculates its attention utilizing only its neighbor tokens. Thus, each token can pay attention to its neighbor information with little noise. We show that this is critically important when the text is very long, as in cross-sentence or abstract-level relation-extraction tasks. Our benchmarking results show improvements of 5.44% and 3.89% in accuracy and F1-measure over the state-of-the-art on n-ary and chemical-protein relation datasets, suggesting BERT-GT is a robust approach that is applicable to other biomedical relation extraction tasks or datasets.

Global lockdowns in response to the COVID-19 pandemic have led to changes in the anthropogenic activities resulting in perceivable air quality improvements. Although several recent studies have analyzed these changes over different regions of the globe, these analyses have been constrained due to the usage of station-based data which is mostly limited upto the metropolitan cities. Also, the quantifiable changes have been reported only for the developed and developing regions leaving the poor economies (e.g. Africa) due to the shortage of in-situ data. Using a comprehensive set of high spatiotemporal resolution satellites and merged products of air pollutants, we analyze the air quality across the globe and quantify the improvement resulting from the suppressed anthropogenic activity during the lockdowns. In particular, we focus on megacities, capitals and cities with high standards of living to make the quantitative assessment. Our results offer valuable insights into the spatial distribution of changes in the air pollutants due to COVID-19 enforced lockdowns. Statistically significant reductions are observed over megacities with mean reduction by 19.74%, 7.38% and 49.9% in nitrogen dioxide (NO2), aerosol optical depth (AOD) and PM 2.5 concentrations. Google Earth Engine empowered cloud computing based remote sensing is used and the results provide a testbed for climate sensitivity experiments and validation of chemistry-climate models. Additionally, Google Earth Engine based apps have been developed to visualize the changes in a real-time fashion.

How large ecosystems can create and maintain the remarkable biodiversity we see in nature is probably one of the biggest open questions in science, attracting attention from different fields, from Theoretical Ecology to Mathematics and Physics. In this context, modeling the stable coexistence of species competing for limited resources is a particularly challenging task. From a mathematical point of view, coexistence in competitive dynamics can be achieved when dominance among species forms intransitive loops. However, these relationships usually lead to species' relative abundances neutrally cycling without converging to a stable equilibrium. Although in recent years several mechanisms have been proposed, models able to explain species coexistence in competitive communities are still limited. Here we identify locality in the interactions as one of the simplest mechanisms leading to stable species coexistence. We consider a simplified ecosystem where individuals of each species lay on a spatial network and interactions are possible only between nodes within a certain distance. Varying such distance allows to interpolate between local and global competition. Our results demonstrate, within the scope of our model, that species coexist reaching a stable equilibrium when two conditions are met: individuals are embedded in space and can only interact with other individuals within a short distance. On the contrary, when one of these ingredients is missing, large oscillations and neutral cycles emerge.

Satellite remote sensing has been widely used in the last decades for agricultural applications, {both for assessing vegetation condition and for subsequent yield prediction.} Existing remote sensing-based methods to estimate gross primary productivity (GPP), which is an important variable to indicate crop photosynthetic function and stress, typically rely on empirical or semi-empirical approaches, which tend to over-simplify photosynthetic mechanisms. In this work, we take advantage of all parallel developments in mechanistic photosynthesis modeling and satellite data availability for advanced monitoring of crop productivity. In particular, we combine process-based modeling with the soil-canopy energy balance radiative transfer model (SCOPE) with Sentinel-2 {and Landsat 8} optical remote sensing data and machine learning methods in order to estimate crop GPP. Our model successfully estimates GPP across a variety of C3 crop types and environmental conditions even though it does not use any local information from the corresponding sites. This highlights its potential to map crop productivity from new satellite sensors at a global scale with the help of current Earth observation cloud computing platforms.

While the main causes of the temporal gravity variations observed by the GRACE space mission result from water mass redistributions occurring at the surface of the Earth in response to climatic and anthropogenic forcings (e.g., changes in land hydrology, in ocean mass, in mass of glaciers and ice sheets), solid Earth's mass redistributions are also recorded by these observations. This is the case, in particular, for the Glacial Isostatic Adjustment (GIA) or the viscous response of the mantle to the last deglaciation. However, it is only recently showed that the gravity data also contain the signature of flows inside the outer core and their effects on the core-mantle boundary (CMB). Detecting deep Earth's processes in GRACE observations offers an exciting opportunity to provide additional insight on the dynamics of the core-mantle interface. Here, we present one aspect of the GRACEFUL (GRavimetry, mAgnetism and CorE Flow) project, i.e. the possibility to use the gravity field data for understanding the dynamic processes inside the fluid core and core-mantle boundary of the Earth, beside that offered by the geomagnetic field variations.

The terrestrial magnetopause is the boundary that shields the Earth's magnetosphere on one side from the shocked solar wind and its embedded interplanetary magnetic field on the other side. In this paper, we show observations from two of the Time History of Events and Macroscales Interactions during Substorms (THEMIS) satellites, comparing dayside magnetopause crossings with flank crossings near the terminator. Macroscopic properties such as current sheet thickness, motion, and current density are examined for a large number of magnetopause crossings. The results show that the flank magnetopause is typically thicker than the dayside magnetopause and has a lower current density. Consistent with earlier results from Cluster observations, we also find a persistent dawn-dusk asymmetry with a thicker and more dynamic magnetopause at dawn than at dusk.

This paper focuses on the problem of online golf ball detection and tracking from image sequences. An efficient real-time approach is proposed by exploiting convolutional neural networks (CNN) based object detection and a Kalman filter based prediction. Five classical deep learning-based object detection networks are implemented and evaluated for ball detection, including YOLO v3 and its tiny version, YOLO v4, Faster R-CNN, SSD, and RefineDet. The detection is performed on small image patches instead of the entire image to increase the performance of small ball detection. At the tracking stage, a discrete Kalman filter is employed to predict the location of the ball and a small image patch is cropped based on the prediction. Then, the object detector is utilized to refine the location of the ball and update the parameters of Kalman filter. In order to train the detection models and test the tracking algorithm, a collection of golf ball dataset is created and annotated. Extensive comparative experiments are performed to demonstrate the effectiveness and superior tracking performance of the proposed scheme.

Declustering aims to divide earthquake catalogs into independent events (mainshocks), and dependent (clustered) events, and is an integral component of many seismicity studies, including seismic hazard assessment. We assess the effect of declustering on the frequency-magnitude distribution of mainshocks. In particular, we examine the dependence of the b-value of declustered catalogs on the choice of declustering approach and algorithm-specific parameters. Using the catalog of earthquakes in California since 1980, we show that the b-value decreases by up to 30% due to declustering with respect to the undeclustered catalog. The extent of the reduction is highly dependent on the declustering method and parameters applied. We then reproduce a similar effect by declustering synthetic earthquake catalogs with known b-value, which have been generated using an Epidemic-Type Aftershock Sequence (ETAS) model. Our analysis suggests that the observed decrease in b-value must, at least partially, arise from the application of the declustering algorithm on the catalog, rather than from differences in the nature of mainshocks versus fore- or aftershocks. We conclude that declustering should be considered as a potential source of bias in seismicity and hazard studies.

Recent advances in Artificial Intelligence (AI) have revived the quest for agents able to acquire an open-ended repertoire of skills. However, although this ability is fundamentally related to the characteristics of human intelligence, research in this field rarely considers the processes that may have guided the emergence of complex cognitive capacities during the evolution of the species. Research in Human Behavioral Ecology (HBE) seeks to understand how the behaviors characterizing human nature can be conceived as adaptive responses to major changes in the structure of our ecological niche. In this paper, we propose a framework highlighting the role of environmental complexity in open-ended skill acquisition, grounded in major hypotheses from HBE and recent contributions in Reinforcement learning (RL). We use this framework to highlight fundamental links between the two disciplines, as well as to identify feedback loops that bootstrap ecological complexity and create promising research directions for AI researchers.