Self-supervision based deep learning classification approaches have received considerable attention in academic literature. However, the performance of such methods on remote sensing imagery domains remains under-explored. In this work, we explore contrastive representation learning methods on the task of imagery-based city classification, an important problem in urban computing. We use satellite and map imagery across 2 domains, 3 million locations and more than 1500 cities. We show that self-supervised methods can build a generalizable representation from as few as 200 cities, with representations achieving over 95\% accuracy in unseen cities with minimal additional training. We also find that the performance discrepancy of such methods, when compared to supervised methods, induced by the domain discrepancy between natural imagery and abstract imagery is significant for remote sensing imagery. We compare all analysis against existing supervised models from academic literature and open-source our models for broader usage and further criticism.

Air pollution has been on continuous rise with increase in industrialization in metropolitan cities of the world. Several measures including strict climate laws and reduction in the number of vehicles were implemented by several nations. The COVID-19 pandemic provided a great opportunity to understand the daily human activities effect on air pollution. Majority nations restricted industrial activities and vehicular traffic to a large extent as a measure to restrict COVID-19 spread. In this paper, we analyzed the impact of such COVID19-induced lockdown on the air quality of the city of New Delhi, India. We analyzed the average concentration of common gaseous pollutants viz. sulfur dioxide (SO$_2$), ozone (O$_3$), nitrogen dioxide (NO$_2$), and carbon monoxide (CO). These concentrations were obtained from the tropospheric column of Sentinel-5P (an earth observation satellite of European Space Agency) data. We observed that the city observed a significant drop in the level of atmospheric pollutant's concentration for all the major pollutants as a result of strict lockdown measures. Such findings are also validated with pollutant data obtained from ground-based monitoring stations. We observed that near-surface pollutant concentration dropped significantly by 50% for PM$_{2.5}$, 71.9% for NO$_2$, and 88% for CO, after the lockdown period. Such studies would pave the path for implementing future air pollution control measures by environmentalists.

Detecting changes on the Earth, such as urban development, deforestation, or natural disaster, is one of the research fields that is attracting a great deal of attention. One promising tool to solve these problems is satellite imagery. However, satellite images require huge amount of storage, therefore users are required to set Area of Interests first, which was not suitable for detecting potential areas for disaster or development. To tackle with this problem, we develop the novel tool, namely GRASP EARTH, which is the simple change detection application based on Google Earth Engine. GRASP EARTH allows us to handle satellite imagery easily and it has used for disaster monitoring and urban development monitoring.

Most traffic flow control algorithms address switching cycle adaptation of traffic signals and lights. This work addresses traffic flow optimisation by self-organising micro-level control combining Reinforcement Learning and rule-based agents for action selection performing long-range navigation in urban environments. I.e., vehicles represented by agents adapt their decision making for re-routing based on local environmental sensors. Agent-based modelling and simulation is used to study emergence effects on urban city traffic flows. An unified agent programming model enables simulation and distributed data processing with possible incorporation of crowd sensing tasks used as an additional sensor data base. Results from an agent-based simulation of an artificial urban area show that the deployment of micro-level vehicle navigation control just by learned individual decision making and re-routing based on local environmental sensors can increase the efficiency of mobility in terms of path length and travelling time.

Natural ecosystems, in particular on the microbial scale, are inhabited by a large number of species. The population size of each species is affected by interactions of individuals with each other and by spatial and temporal changes in environmental conditions, such as resource abundance. Here, we use a generic population dynamics model to study how, and under what conditions, a periodic temporal environmental variation can alter an ecosystem's composition and biodiversity. We demonstrate that using time scale separation allows one to qualitatively predict the long-term population dynamics of interacting species in varying environments. We show that the notion of Tilman's R* rule, a well-known principle that applies for constant environments, can be extended to periodically varying environments if the time scale of environmental changes (e.g., seasonal variations) is much faster than the time scale of population growth (doubling time in bacteria). When these time scales are similar, our analysis shows that a varying environment deters the system from reaching a steady state, and stable coexistence between multiple species becomes possible. Our results posit that biodiversity can in part be attributed to natural environmental variations.

Modeling water flow in unsaturated soils is vital for describing various hydrological and ecological phenomena. Soil water dynamics is described by well-established physical laws (Richardson-Richards equation (RRE)). Solving the RRE is difficult due to the inherent non-linearity of the processes, and various numerical methods have been proposed to solve the issue. However, applying the methods to practical situations is challenging because they require well-defined initial and boundary conditions. Here, we present a physics-informed neural networks (PINNs) method that approximates the solution to the RRE using neural networks while concurrently matching available soil moisture data. Although the ability of PINNs to solve partial differential equations, including the RRE, has been demonstrated previously, its potential applications and limitations are not fully known. This study conducted a comprehensive analysis of PINNs and carefully tested the accuracy of the solutions by comparing them with analytical solutions and accepted traditional numerical solutions. We demonstrated that the solutions by PINNs with adaptive activation functions are comparable with those by traditional methods. We showed that soil moisture dynamics in layered soils with discontinuous hydraulic conductivities are correctly simulated by PINNs with domain decomposition. We demonstrated that the unspecified upper boundary condition can be estimated from sparse soil moisture measurements. Nevertheless, there remain challenges that require further development. Chiefly, PINNs are sensitive to the initialization of NNs and are significantly slower than traditional numerical methods.

Self-supervised learning (SSL) of graph neural networks is emerging as a promising way of leveraging unlabeled data. Currently, most methods are based on contrastive learning adapted from the image domain, which requires view generation and a sufficient number of negative samples. In contrast, existing predictive models do not require negative sampling, but lack theoretical guidance on the design of pretext training tasks. In this work, we propose the LaGraph, a theoretically grounded predictive SSL framework based on latent graph prediction. Learning objectives of LaGraph are derived as self-supervised upper bounds to objectives for predicting unobserved latent graphs. In addition to its improved performance, LaGraph provides explanations for recent successes of predictive models that include invariance-based objectives. We provide theoretical analysis comparing LaGraph to related methods in different domains. Our experimental results demonstrate the superiority of LaGraph in performance and the robustness to decreasing of training sample size on both graph-level and node-level tasks.

Pretrained language models based on the transformer architecture have shown great success in NLP. Textual training data often comes from the web and is thus tagged with time-specific information, but most language models ignore this information. They are trained on the textual data alone, limiting their ability to generalize temporally. In this work, we extend the key component of the transformer architecture, i.e., the self-attention mechanism, and propose temporal attention - a time-aware self-attention mechanism. Temporal attention can be applied to any transformer model and requires the input texts to be accompanied with their relevant time points. It allows the transformer to capture this temporal information and create time-specific contextualized word representations. We leverage these representations for the task of semantic change detection; we apply our proposed mechanism to BERT and experiment on three datasets in different languages (English, German, and Latin) that also vary in time, size, and genre. Our proposed model achieves state-of-the-art results on all the datasets.

The CAMS air quality multi-model forecasts have been assessed and calibrated for PM10, PM2.5, O3, NO2, and CO against observations collected by the Regional Monitoring Network of the Liguria region (northwestern Italy) in the years 2019 and 2020. The calibration strategy used in the present work has its roots in the well-established Ensemble Model Output Statistics (EMOS) through which a raw ensemble forecast can be accurately transformed into a predictive probability density function, with a simultaneous correction of biases and dispersion errors. The strategy also provides a calibrated forecast of model uncertainties. As a result of our analysis, the key role of pollutant real-time observations to be ingested in the calibration strategy clearly emerge especially in the shorter look-ahead forecast hours. Our dynamic calibration strategy turns out to be superior with respect to its analogous where real-time data are not taken into account. The best calibration strategy we have identified makes the CAMS multi-model forecast system more reliable than other raw air quality models running at higher spatial resolution which exploit more detailed information from inventory emission. We expect positive impacts of our research for identifying and set up reliable and economic air pollution early warning systems.

Gait is an essential manifestation of depression. Laboratory gait characteristics have been found to be closely associated with depression. However, the gait characteristics of daily walking in real-world scenarios and their relationships with depression are yet to be fully explored. This study aimed to explore associations between depression symptom severity and daily-life gait characteristics derived from acceleration signals in real-world settings. In this study, we used two ambulatory datasets: a public dataset with 71 elder adults' 3-day acceleration signals collected by a wearable device, and a subset of an EU longitudinal depression study with 215 participants and their phone-collected acceleration signals (average 463 hours per participant). We detected participants' gait cycles and force from acceleration signals and extracted 20 statistics-based daily-life gait features to describe the distribution and variance of gait cadence and force over a long-term period corresponding to the self-reported depression score. The gait cadence of faster steps (75th percentile) over a long-term period has a significant negative association with the depression symptom severity of this period in both datasets. Daily-life gait features could significantly improve the goodness of fit of evaluating depression severity relative to laboratory gait patterns and demographics, which was assessed by likelihood-ratio tests in both datasets. This study indicated that the significant links between daily-life walking characteristics and depression symptom severity could be captured by both wearable devices and mobile phones. The gait cadence of faster steps in daily-life walking has the potential to be a biomarker for evaluating depression severity, which may contribute to clinical tools to remotely monitor mental health in real-world settings.

Effective biological utilization of wood biomass is necessary worldwide. Since several insect larvae can use wood biomass as a nutrient source, studies on their digestive mechanism are expected to speculate a novel rule in wood biomass processing. Here, the relationships of inhabitant bacteria involved in carbon and nitrogen metabolism in the intestine of beetle larvae, an insect model, are investigated. Bacterial analysis of larval feces showed enrichment of members of which could include candidates for plant growth promotion, nitrogen cycle modulation, and/or environmental protection. The abundances of these bacteria were not necessarily positively correlated with the abundance in the habitat, suggesting that they might be selectively enriched in the intestines of larvae. Further association analysis predicted that carbon and nitrogen metabolism in the intestine was affected by the presence of the other common bacteria, the populations of which were not remarkably altered in the habitat and feces. Based on hypotheses targeting these selected bacterial groups, structural estimation modeling analyses statistically suggested that their metabolism of carbon and nitrogen and their stable isotopes, {\delta}13C and {\delta}15N, may be associated with fecal enriched bacteria and other common bacteria. In addition, other causal inference analyses, such as causal mediation analysis, linear non-Gaussian acyclic model (LiNGAM), and BayesLiNGAM, did not necessarily affirm the existence of prominent bacteria involved in metabolism, implying its importance as the bacterial groups for metabolism rather than a remarkable bacterium. Thus, these observations highlight a multifaceted view of symbiotic bacterial groups utilizing carbon and nitrogen from wood biomass in insect larvae as a cultivator of potentially environmentally beneficial bacteria.

According to general relativity theory (GRT), by comparing the frequencies between two precise clocks at two different stations, the gravity potential (geopotential) difference between the two stations can be determined due to the gravity frequency shift effect. Here, we provide experimental results of geopotential difference determination based on frequency comparisons between two remote hydrogen atomic clocks, with the help of common-view satellite time transfer (CVSTT) technique. For the first time we apply the ensemble empirical mode decomposition (EEMD) technique to the CVSTT observations for effectively determining the geopotential-related signals. Based on the net frequency shift between the two clocks in two different periods, the geopotential difference between stations of the Beijing 203 Institute Laboratory (BIL) and Luojiashan Time--Frequency Station (LTS) is determined. Comparisons show that the orthometric height (OH) of LTS determined by the clock comparison is deviated from that determined by the Earth gravity model EGM2008 by (38.5$\pm$45.7)~m. The results are consistent with the frequency stabilities of the hydrogen clocks (at the level of $10^{-15}$~day$^{-1}$) used in the experiment. Using more precise atomic or optical clocks, the CVSTT method for geopotential determination could be applied effectively and extensively in geodesy in the future.

With this work we are explaining the "You Only Look Once" (YOLO) single-stage object detection approach as a parallel classification of 10647 fixed region proposals. We support this view by showing that each of YOLOs output pixel is attentive to a specific sub-region of previous layers, comparable to a local region proposal. This understanding reduces the conceptual gap between YOLO-like single-stage object detection models, RCNN-like two-stage region proposal based models, and ResNet-like image classification models. In addition, we created interactive exploration tools for a better visual understanding of the YOLO information processing streams: https://limchr.github.io/yolo_visualization

Evidence of fast variations in the Earth's core field are seen both in magnetic observatory and satellite records. We present here how they have been identified at the Earth's surface from ground-based observatory records and how their spatio-temporal structure is now characterised by satellite data. It is shown how their properties at the core mantle boundary are extracted through localised and global modelling processes, paying particular attention to their time scales. Finally are listed possible types of waves in the liquid outer core, together with their main properties, that may give rise to these observed fast variations.

Full-waveform inversion (FWI), a popular technique that promises high-resolution models, has helped in improving the salt definition in inverted velocity models. The success of the inversion relies heavily on having prior knowledge of the salt, and using advanced acquisition technology with long offsets and low frequencies. Salt bodies are often constructed by recursively picking the top and bottom of the salt from seismic images corresponding to tomography models, combined with flooding techniques. The process is time-consuming and highly prone to error, especially in picking the bottom of the salt (BoS). Many studies suggest performing FWI with long offsets and low frequencies after constructing the salt bodies to correct the miss-interpreted boundaries. Here, we focus on detecting the BoS automatically by utilizing deep learning tools. We specifically generate many random 1D models, containing or free of salt bodies, and calculate the corresponding shot gathers. We then apply FWI starting with salt flooded versions of those models, and the results of the FWI become inputs to the neural network, whereas the corresponding true 1D models are the output. The network is trained in a regression manner to detect the BoS and estimate the subsalt velocity. We analyze three scenarios in creating the training datasets and test their performance on the 2D BP 2004 salt model. We show that when the network succeeds in estimating the subsalt velocity, the requirement of low frequencies and long offsets are somewhat mitigated. In general, this work allows us to merge the top-to-bottom approach with FWI, save the BoS picking time, and empower FWI to converge in the absence of low frequencies and long offsets in the data.

Humankind has spread worldwide supported by cultural and technological knowledge, but the environmental sustainability on the human niche evolution depends on a new human beings relationship with the biosphere. Human lifestyles nowadays are very Antropocentric and in many ways deleterious to the other life forms. Here we try to identify future scenarios, where the less deleterious is the Natural-Technological Model that points the urgent need to change the evolutionary direction of the human niche seeking the resumption of original ecological relations. New cultural habits and novel technologies, thereby, would reverse the current anthropogenic impacts. The middle way is the Bio-Anthropogenic Model that predicts the success of the emerging ecosystems and the symbiotic relationship of humans and anthropogenic-favored species, hybrids, aliens and genetically modified organisms. For such, we must also change our way of life and adopt new conscious ways of consumption aiming at the socio-environmental good. Lastly, the Wear Out Model, which depends only on maintaining current patterns of human expansion. The lack of investments on new technologies and new cultural habits, added to the current patterns of human niche evolution that are based on the massive exploitation of world resources, will lead to a fearsome scenario with a precarious global health, biodiversity losses and food scarcity. This theoretical models indicates some pathways and can help us to choose a better future.

Wakes behind spacecraft caused by supersonic drifting positive ions are common in plasmas and disturb in situ measurements. We review the impact of wakes on observations by the Electric Field and Wave double-probe instruments on the Cluster satellites. In the solar wind, the equivalent spacecraft charging is small compared to the ion drift energy and the wake effects are caused by the spacecraft body and can be compensated for. We present statistics of the direction, width, and electrostatic potential of wakes, and we compare with an analytical model. In the low-density magnetospheric lobes, the equivalent positive spacecraft charging is large compared to the ion drift energy and an enhanced wake forms. In this case observations of the geophysical electric field with the double-probe technique becomes extremely challenging. Rather, the wake can be used to estimate the flux of cold (eV) positive ions. For an intermediate range of parameters, when the equivalent charging of the spacecraft is similar to the drift energy of the ions, also the charged wire booms of a double-probe instrument must be taken into account. We discuss an example of these effects from the MMS spacecraft near the magnetopause. We find that many observed wake characteristics provide information that can be used for scientific studies. An important example is the enhanced wakes used to estimate the outflow of ionospheric origin in the magnetospheric lobes to about ${10}^{26}$ cold (eV) ions/s, constituting a large fraction of the mass outflow from planet Earth.

With the advent of Transformer, which was used in translation models in 2017, attention-based architectures began to attract attention. Furthermore, after the emergence of BERT, which strengthened the NLU-specific encoder part, which is a part of the Transformer, and the GPT architecture, which strengthened the NLG-specific decoder part, various methodologies, data, and models for learning the Pretrained Language Model began to appear. Furthermore, in the past three years, various Pretrained Language Models specialized for Korean have appeared. In this paper, we intend to numerically and qualitatively compare and analyze various Korean PLMs released to the public.

We present our approach to modeling over 20 years of the solar wind-magnetosphere-ionosphere system using version 5 of the Grand Unified Magnetosphere-Ionosphere Coupling Simulation (GUMICS-5). As input we use 16-s resolution magnetic field and 1-min plasma measurements by the Advanced Composition Explorer (ACE) satellite from 1998 to 2020. The modeled interval is divided into 28 h simulations, which include 4 h overlap. We use a maximum magnetospheric resolution of 0.5 Earth radii (Re) up to about 15 Re from Earth and decreasing resolution further away. In the ionosphere we use a maximum resolution of approximately 100 km poleward of +-58 degrees magnetic latitude and decreasing resolution towards the equator. With respect to the previous version GUMICS-4, we have parallelized the magnetosphere of GUMICS-5 using the Message Passing Interface and have made several improvements which have e.g. decreased its numerical diffusion. We compare the simulation results to several empirical models and geomagnetic indices derived from ground magnetic field measurements. GUMICS-5 reproduces observed solar cycle trends in magnetopause stand-off distance and magnetospheric lobe field strength but consistency in plasma sheet pressure and ionospheric cross-polar cap potential is lower. Comparisons with geomagnetic indices show better results for Kp index than for AE index. The simulation results are available at https://doi.org/10.23729/ca1da110-2d4e-45c4-8876-57210fbb0b0d, consisting of full ionospheric files and size-optimized magnetospheric files. The data used for Figures is available at https://doi.org/10.5281/zenodo.6641258. Our extensive results can serve e.g. as a foundation for a combined physics-based and black-box approach to real-time prediction of near-Earth space, or as input to other physics-based models of the inner magnetosphere, upper and middle atmosphere, etc.

Recent years of research in Natural Language Processing (NLP) have witnessed dramatic growth in training large models for generating context-aware language representations. In this regard, numerous NLP systems have leveraged the power of neural network-based architectures to incorporate sense information in embeddings, resulting in Contextualized Word Embeddings (CWEs). Despite this progress, the NLP community has not witnessed any significant work performing a comparative study on the contextualization power of such architectures. This paper presents a comparative study and an extensive analysis of nine widely adopted Transformer models. These models are BERT, CTRL, DistilBERT, OpenAI-GPT, OpenAI-GPT2, Transformer-XL, XLNet, ELECTRA, and ALBERT. We evaluate their contextualization power using two lexical sample Word Sense Disambiguation (WSD) tasks, SensEval-2 and SensEval-3. We adopt a simple yet effective approach to WSD that uses a k-Nearest Neighbor (kNN) classification on CWEs. Experimental results show that the proposed techniques also achieve superior results over the current state-of-the-art on both the WSD tasks

With the sweeping digitalization of societal, medical, industrial, and scientific processes, sensing technologies are being deployed that produce increasing volumes of time series data, thus fueling a plethora of new or improved applications. In this setting, outlier detection is frequently important, and while solutions based on neural networks exist, they leave room for improvement in terms of both accuracy and efficiency. With the objective of achieving such improvements, we propose a diversity-driven, convolutional ensemble. To improve accuracy, the ensemble employs multiple basic outlier detection models built on convolutional sequence-to-sequence autoencoders that can capture temporal dependencies in time series. Further, a novel diversity-driven training method maintains diversity among the basic models, with the aim of improving the ensemble's accuracy. To improve efficiency, the approach enables a high degree of parallelism during training. In addition, it is able to transfer some model parameters from one basic model to another, which reduces training time. We report on extensive experiments using real-world multivariate time series that offer insight into the design choices underlying the new approach and offer evidence that it is capable of improved accuracy and efficiency. This is an extended version of "Unsupervised Time Series Outlier Detection with Diversity-Driven Convolutional Ensembles", to appear in PVLDB 2022.

Today, the AI community is obsessed with 'state-of-the-art' scores (80% papers in NeurIPS) as the major performance metrics, due to which an important parameter, i.e., the environmental metric, remains unreported. Computational capabilities were a limiting factor a decade ago; however, in foreseeable future circumstances, the challenge will be to develop environment-friendly and power-efficient algorithms. The human brain, which has been optimizing itself for almost a million years, consumes the same amount of power as a typical laptop. Therefore, developing nature-inspired algorithms is one solution to it. In this study, we show that currently used ANNs are not what we find in nature, and why, although having lower performance, spiking neural networks, which mirror the mammalian visual cortex, have attracted much interest. We further highlight the hardware gaps restricting the researchers from using spike-based computation for developing neuromorphic energy-efficient microchips on a large scale. Using neuromorphic processors instead of traditional GPUs might be more environment friendly and efficient. These processors will turn SNNs into an ideal solution for the problem. This paper presents in-depth attention highlighting the current gaps, the lack of comparative research, while proposing new research directions at the intersection of two fields -- neuroscience and deep learning. Further, we define a new evaluation metric 'NATURE' for reporting the carbon footprint of AI models.

Difficulties identifying appropriate biodiversity impact metrics remain a major barrier to inclusion of biodiversity considerations in environmentally responsible investment. We propose and analyse a simple science-based local metric: the sum of proportional changes in local species abundances relative to their global species abundances, with a correction for species close to extinction. As we show, this metric quantifies changes in the mean long-term global survival probability of species. It links mathematically to a widely cited global biodiversity indicator, the Living Planet Index, for which we propose an improved formula that directly addresses the known problem of singularities caused by extinctions. We show that, in an ideal market, trade in our metric would lead to near-optimal allocation of resources to species conservation. We further show that the metric is closely related to several other metrics and indices already in use. Barriers to adoption are therefore low. Used in conjunction with metrics addressing ecosystem functioning and services, potential areas of application include biodiversity related financial disclosures and voluntary or legislated no net biodiversity loss policies.

This paper presents a deep neural architecture, for Natural Language Sentence Matching (NLSM) by adding a deep recursive encoder to BERT so called BERT with Deep Recursive Encoder (BERT-DRE). Our analysis of model behavior shows that BERT still does not capture the full complexity of text, so a deep recursive encoder is applied on top of BERT. Three Bi-LSTM layers with residual connection are used to design a recursive encoder and an attention module is used on top of this encoder. To obtain the final vector, a pooling layer consisting of average and maximum pooling is used. We experiment our model on four benchmarks, SNLI, FarsTail, MultiNLI, SciTail, and a novel Persian religious questions dataset. This paper focuses on improving the BERT results in the NLSM task. In this regard, comparisons between BERT-DRE and BERT are conducted, and it is shown that in all cases, BERT-DRE outperforms BERT. The BERT algorithm on the religious dataset achieved an accuracy of 89.70%, and BERT-DRE architectures improved to 90.29% using the same dataset.

Learning to act from observational data without active environmental interaction is a well-known challenge in Reinforcement Learning (RL). Recent approaches involve constraints on the learned policy or conservative updates, preventing strong deviations from the state-action distribution of the dataset. Although these methods are evaluated using non-linear function approximation, theoretical justifications are mostly limited to the tabular or linear cases. Given the impressive results of deep reinforcement learning, we argue for a need to more clearly understand the challenges in this setting. In the vein of Held & Hein's classic 1963 experiment, we propose the "tandem learning" experimental paradigm which facilitates our empirical analysis of the difficulties in offline reinforcement learning. We identify function approximation in conjunction with fixed data distributions as the strongest factors, thereby extending but also challenging hypotheses stated in past work. Our results provide relevant insights for offline deep reinforcement learning, while also shedding new light on phenomena observed in the online case of learning control.

For many countries like Russia, Canada, or the USA, a robust and detailed tree species inventory is essential to manage their forests sustainably. Since one can not apply unmanned aerial vehicle (UAV) imagery-based approaches to large-scale forest inventory applications, the utilization of machine learning algorithms on satellite imagery is a rising topic of research. Although satellite imagery quality is relatively low, additional spectral channels provide a sufficient amount of information for tree crown classification tasks. Assuming that tree crowns are detected already, we use embeddings of tree crowns generated by Autoencoders as a data set to train classical Machine Learning algorithms. We compare our Autoencoder (AE) based approach to traditional convolutional neural networks (CNN) end-to-end classifiers.

Oriented object detection in remote sensing images has made great progress in recent years. However, most of the current methods only focus on detecting targets, and cannot distinguish fine-grained objects well in complex scenes. In this technical report, we analyzed the key issues of fine-grained object recognition, and use an oriented feature alignment network (OFA-Net) to achieve high-performance fine-grained oriented object recognition in optical remote sensing images. OFA-Net achieves accurate object localization through a rotated bounding boxes refinement module. On this basis, the boundary-constrained rotation feature alignment module is applied to achieve local feature extraction, which is beneficial to fine-grained object classification. The single model of our method achieved mAP of 46.51\% in the GaoFen competition and won 3rd place in the ISPRS benchmark with the mAP of 43.73\%.

Detecting changes on the ground in multitemporal Earth observation data is one of the key problems in remote sensing. In this paper, we introduce Sibling Regression for Optical Change detection (SiROC), an unsupervised method for change detection in optical satellite images with medium and high resolution. SiROC is a spatial context-based method that models a pixel as a linear combination of its distant neighbors. It uses this model to analyze differences in the pixel and its spatial context-based predictions in subsequent time periods for change detection. We combine this spatial context-based change detection with ensembling over mutually exclusive neighborhoods and transitioning from pixel to object-level changes with morphological operations. SiROC achieves competitive performance for change detection with medium-resolution Sentinel-2 and high-resolution Planetscope imagery on four datasets. Besides accurate predictions without the need for training, SiROC also provides a well-calibrated uncertainty of its predictions. This makes the method especially useful in conjunction with deep-learning based methods for applications such as pseudo-labeling.

In many cyber-physical systems, imaging can be an important but expensive or 'difficult to deploy' sensing modality. One such example is detecting combustion instability using flame images, where deep learning frameworks have demonstrated state-of-the-art performance. The proposed frameworks are also shown to be quite trustworthy such that domain experts can have sufficient confidence to use these models in real systems to prevent unwanted incidents. However, flame imaging is not a common sensing modality in engine combustors today. Therefore, the current roadblock exists on the hardware side regarding the acquisition and processing of high-volume flame images. On the other hand, the acoustic pressure time series is a more feasible modality for data collection in real combustors. To utilize acoustic time series as a sensing modality, we propose a novel cross-modal encoder-decoder architecture that can reconstruct cross-modal visual features from acoustic pressure time series in combustion systems. With the "distillation" of cross-modal features, the results demonstrate that the detection accuracy can be enhanced using the virtual visual sensing modality. By providing the benefit of cross-modal reconstruction, our framework can prove to be useful in different domains well beyond the power generation and transportation industries.

While state-of-the-art contrastive Self-Supervised Learning (SSL) models produce results competitive with their supervised counterparts, they lack the ability to infer latent variables. In contrast, prescribed latent variable (LV) models enable attributing uncertainty, inducing task specific compression, and in general allow for more interpretable representations. In this work, we introduce LV approximations to large scale contrastive SSL models. We demonstrate that this addition improves downstream performance (resulting in 96.42% and 77.49% test top-1 fine-tuned performance on CIFAR10 and ImageNet respectively with a ResNet50) as well as producing highly compressed representations (588x reduction) that are useful for interpretability, classification and regression downstream tasks.

In this work we examine how fine-tuning impacts the fairness of contrastive Self-Supervised Learning (SSL) models. Our findings indicate that Batch Normalization (BN) statistics play a crucial role, and that updating only the BN statistics of a pre-trained SSL backbone improves its downstream fairness (36% worst subgroup, 25% mean subgroup gap). This procedure is competitive with supervised learning, while taking 4.4x less time to train and requiring only 0.35% as many parameters to be updated. Finally, inspired by recent work in supervised learning, we find that updating BN statistics and training residual skip connections (12.3% of the parameters) achieves parity with a fully fine-tuned model, while taking 1.33x less time to train.

Current methods of estimating the change in stress caused by injecting fluid into subsurface formations require choosing the type of constitutive model and the model parameters based on core, log, and geophysical data during the characterization phase, with little feedback from operational observations to validate or refine these choices. It is shown that errors in the assumed constitutive response, even when informed by laboratory tests on core samples, are likely to be common, large, and underestimate the magnitude of stress change caused by injection. Recent advances in borehole-based strain instruments and borehole and surface-based tilt and displacement instruments have now enabled monitoring of the deformation of the storage system throughout its operational lifespan. This data can enable validation and refinement of the knowledge of the geomechanical properties and state of the system, but brings with it a challenge to transform the raw data into actionable knowledge. We demonstrate a method to perform a gradient-based deterministic inversion of geomechanical monitoring data. This approach allows autonomous integration of the instrument data without the need for time consuming manual interpretation and selection of updated model parameters. The approach presented is very flexible as to what type of geomechanical constitutive response can be used. The approach is easily adaptable to nonlinear physics-based constitutive models to account for common rock behaviors such as creep and plasticity. The approach also enables training of machine learning-based constitutive models by allowing back propagation of errors through the finite element calculations. This enables strongly enforcing known physics, such as conservation of momentum and continuity, while allowing data-driven models to learn the truly unknown physics such as the constitutive or petrophysical responses.

Over the past years, topics ranging from climate change to human rights have seen increasing importance for investment decisions. Hence, investors (asset managers and asset owners) who wanted to incorporate these issues started to assess companies based on how they handle such topics. For this assessment, investors rely on specialized rating agencies that issue ratings along the environmental, social and governance (ESG) dimensions. Such ratings allow them to make investment decisions in favor of sustainability. However, rating agencies base their analysis on subjective assessment of sustainability reports, not provided by every company. Furthermore, due to human labor involved, rating agencies are currently facing the challenge to scale up the coverage in a timely manner. In order to alleviate these challenges and contribute to the overall goal of supporting sustainability, we propose a heterogeneous ensemble model to predict ESG ratings using fundamental data. This model is based on feedforward neural network, CatBoost and XGBoost ensemble members. Given the public availability of fundamental data, the proposed method would allow cost-efficient and scalable creation of initial ESG ratings (also for companies without sustainability reporting). Using our approach we are able to explain 54% of the variation in ratings R2 using fundamental data and outperform prior work in this area.

Despite its rise as a prominent solution to the data inefficiency of today's machine learning models, self-supervised learning has yet to be studied from a purely multi-agent perspective. In this work, we propose that aligning internal subjective representations, which naturally arise in a multi-agent setup where agents receive partial observations of the same underlying environmental state, can lead to more data-efficient representations. We propose that multi-agent environments, where agents do not have access to the observations of others but can communicate within a limited range, guarantees a common context that can be leveraged in individual representation learning. The reason is that subjective observations necessarily refer to the same subset of the underlying environmental states and that communication about these states can freely offer a supervised signal. To highlight the importance of communication, we refer to our setting as \textit{socially supervised representation learning}. We present a minimal architecture comprised of a population of autoencoders, where we define loss functions, capturing different aspects of effective communication, and examine their effect on the learned representations. We show that our proposed architecture allows the emergence of aligned representations. The subjectivity introduced by presenting agents with distinct perspectives of the environment state contributes to learning abstract representations that outperform those learned by a single autoencoder and a population of autoencoders, presented with identical perspectives of the environment state. Altogether, our results demonstrate how communication from subjective perspectives can lead to the acquisition of more abstract representations in multi-agent systems, opening promising perspectives for future research at the intersection of representation learning and emergent communication.

EEG signals are usually simple to obtain but expensive to label. Although supervised learning has been widely used in the field of EEG signal analysis, its generalization performance is limited by the amount of annotated data. Self-supervised learning (SSL), as a popular learning paradigm in computer vision (CV) and natural language processing (NLP), can employ unlabeled data to make up for the data shortage of supervised learning. In this paper, we propose a self-supervised contrastive learning method of EEG signals for sleep stage classification. During the training process, we set up a pretext task for the network in order to match the right transformation pairs generated from EEG signals. In this way, the network improves the representation ability by learning the general features of EEG signals. The robustness of the network also gets improved in dealing with diverse data, that is, extracting constant features from changing data. In detail, the network's performance depends on the choice of transformations and the amount of unlabeled data used in the training process of self-supervised learning. Empirical evaluations on the Sleep-edf dataset demonstrate the competitive performance of our method on sleep staging (88.16% accuracy and 81.96% F1 score) and verify the effectiveness of SSL strategy for EEG signal analysis in limited labeled data regimes. All codes are provided publicly online.

Can pre-trained BERT for one language and GPT for another be glued together to translate texts? Self-supervised training using only monolingual data has led to the success of pre-trained (masked) language models in many NLP tasks. However, directly connecting BERT as an encoder and GPT as a decoder can be challenging in machine translation, for GPT-like models lack a cross-attention component that is needed in seq2seq decoders. In this paper, we propose Graformer to graft separately pre-trained (masked) language models for machine translation. With monolingual data for pre-training and parallel data for grafting training, we maximally take advantage of the usage of both types of data. Experiments on 60 directions show that our method achieves average improvements of 5.8 BLEU in x2en and 2.9 BLEU in en2x directions comparing with the multilingual Transformer of the same size.

Quantifying the impact of climate change on future air quality is a challenging subject in air quality studies. An ANN model is employed to simulate hourly O3 concentrations. The model is developed based on hourly monitored values of temperature, solar radiation, nitrogen monoxide, and nitrogen dioxide which are monitored during summers (June, July, and August) of 2009-2012 at urban air quality stations in Tehran, Iran. Climate projections by HadCM3 GCM over the study area, driven by IPCC SRES A1B, A2, and B1 emission scenarios, are downscaled by LARS-WG5 model over the periods of 2015-2039 and 2040-2064. The projections are calculated by assuming that current emissions conditions of O3 precursors remain constant in the future. The employed O3 metrics include the number of days exceeding one-hour (1-hr) (120 ppb) and eight-hour (8-hr) (75 ppb) O3 standards and the number of days exceeding 8-hr Air Quality Index (AQI). The projected increases in solar radiation and decreases in precipitation in future summers along with summertime daily maximum temperature rise of about 1.2 and 3 celsius in the first and second climate periods respectively are some indications of more favorable conditions for O3 formation over the study area in the future. Based on pollution conditions of the violation-free summer of 2012, the summertime exceedance days of 8-hr O3 standard are projected to increase in the future by about 4.2 days in the short term and about 12.3 days in the mid-term. Similarly, based on pollution conditions of the polluted summer of 2010 with 58 O3 exceedance days, this metric is projected to increase about 4.5 days in the short term and about 14.1 days in the mid-term. Moreover, the number of Unhealthy and Very Unhealthy days in 8-hr AQI is also projected to increase based on pollution conditions of both summers.

Visual sensation and perception refers to the process of sensing, organizing, identifying, and interpreting visual information in environmental awareness and understanding. Computational models inspired by visual perception have the characteristics of complexity and diversity, as they come from many subjects such as cognition science, information science, and artificial intelligence. In this paper, visual perception computational models oriented deep learning are investigated from the biological visual mechanism and computational vision theory systematically. Then, some points of view about the prospects of the visual perception computational models are presented. Finally, this paper also summarizes the current challenges of visual perception and predicts its future development trends. Through this survey, it will provide a comprehensive reference for research in this direction.

Crop and weed monitoring is an important challenge for agriculture and food production nowadays. Thanks to recent advances in data acquisition and computation technologies, agriculture is evolving to a more smart and precision farming to meet with the high yield and high quality crop production. Classification and recognition in Unmanned Aerial Vehicles (UAV) images are important phases for crop monitoring. Advances in deep learning models relying on Convolutional Neural Network (CNN) have achieved high performances in image classification in the agricultural domain. Despite the success of this architecture, CNN still faces many challenges such as high computation cost, the need of large labelled datasets, ... Natural language processing's transformer architecture can be an alternative approach to deal with CNN's limitations. Making use of the self-attention paradigm, Vision Transformer (ViT) models can achieve competitive or better results without applying any convolution operations. In this paper, we adopt the self-attention mechanism via the ViT models for plant classification of weeds and crops: red beet, off-type beet (green leaves), parsley and spinach. Our experiments show that with small set of labelled training data, ViT models perform better compared to state-of-the-art CNN-based models EfficientNet and ResNet, with a top accuracy of 99.8\% achieved by the ViT model.

Scientists have long aimed to discover meaningful formulae which accurately describe experimental data. A common approach is to manually create mathematical models of natural phenomena using domain knowledge, and then fit these models to data. In contrast, machine-learning algorithms automate the construction of accurate data-driven models while consuming large amounts of data. The problem of incorporating prior knowledge in the form of constraints on the functional form of a learned model (e.g., nonnegativity) has been explored in the literature. However, finding models that are consistent with prior knowledge expressed in the form of general logical axioms (e.g., conservation of energy) is an open problem. We develop a method to enable principled derivations of models of natural phenomena from axiomatic knowledge and experimental data by combining logical reasoning with symbolic regression. We demonstrate these concepts for Kepler's third law of planetary motion, Einstein's relativistic time-dilation law, and Langmuir's theory of adsorption, automatically connecting experimental data with background theory in each case. We show that laws can be discovered from few data points when using formal logical reasoning to distinguish the correct formula from a set of plausible formulas that have similar error on the data. The combination of reasoning with machine learning provides generalizeable insights into key aspects of natural phenomena. We envision that this combination will enable derivable discovery of fundamental laws of science and believe that our work is an important step towards automating the scientific method.

Pretrained transformer-based models such as BERT have demonstrated state-of-the-art predictive performance when adapted into a range of natural language processing tasks. An open problem is how to improve the faithfulness of explanations (rationales) for the predictions of these models. In this paper, we hypothesize that salient information extracted a priori from the training data can complement the task-specific information learned by the model during fine-tuning on a downstream task. In this way, we aim to help BERT not to forget assigning importance to informative input tokens when making predictions by proposing SaLoss; an auxiliary loss function for guiding the multi-head attention mechanism during training to be close to salient information extracted a priori using TextRank. Experiments for explanation faithfulness across five datasets, show that models trained with SaLoss consistently provide more faithful explanations across four different feature attribution methods compared to vanilla BERT. Using the rationales extracted from vanilla BERT and SaLoss models to train inherently faithful classifiers, we further show that the latter result in higher predictive performance in downstream tasks.

Anomaly detection on time series is a fundamental task in monitoring the Key Performance Indicators (KPIs) of IT systems. Many of the existing approaches in the literature show good performance while requiring a lot of training resources. In this paper, the online matrix profile, which requires no training, is proposed to address this issue. The anomalies are detected by referring to the past subsequence that is the closest to the current one. The distance significance is introduced based on the online matrix profile, which demonstrates a prominent pattern when an anomaly occurs. Another training-free approach spectral residual is integrated into our approach to further enhance the detection accuracy. Moreover, the proposed approach is sped up by at least four times for long time series by the introduced cache strategy. In comparison to the existing approaches, the online matrix profile makes a good trade-off between accuracy and efficiency. More importantly, it is generic to various types of time series in the sense that it works without the constraint from any trained model.

Sea ice motions play an important role in the polar climate system by transporting pollutants, heat, water and salt as well as changing the ice cover. Numerous physics-based models have been constructed to represent the sea ice dynamical interaction with the atmosphere and ocean. In this study, we propose a new data-driven deep-learning approach that utilizes a convolutional neural network (CNN) to model how Arctic sea ice moves in response to surface winds given its initial ice velocity and concentration a day earlier. Results show that CNN computes the sea ice response with a correlation of 0.82 on average with respect to reality, which surpasses a set of local point-wise predictions and a leading thermodynamic-dynamical model, CICE5. The superior predictive skill of CNN suggests the important role played by the connective patterns of the predictors of the sea ice motion.

The pervasiveness of microplastics in global oceans is raising concern about its impacts on organisms. While quantifying its toxicity is still an open issue, sampling evidence has shown that rarely is marine microplastics found clean; rather, it is often contaminated by other types of chemical pollutants, some known to be harmful to biota and humans. To provide a first tool for assessing the role of microplastics as vectors of plastic-related organic pollutants (PROPs), we developed a data-informed model that accounts for the intertwined dynamics of Lagrangian microplastic particles transported by surface currents and the Eulerian advection-diffusion of chemicals that partition on them through seawater-particle interaction. Focusing on the Mediterranean Sea and using simple, yet realistic forcings for the input of PROPs, our simulations highlight that microplastics can mediate PROP export across different sub-seas. Particle origin, in terms of both source type (either coastal, riverine, or fishing-derived) and geographical location, seems to play a major role in determining the amount of PROPs conveyed by microplastics during their journey at sea. We argue that quantitative numerical modelling approaches can be focal to shed some light on the vast spatial and temporal scales of microplastics-PROPs interaction, complementary to much-needed on-field investigation.

The operation of satellites in very low Earth orbit (VLEO) has been linked to a variety of benefits to both the spacecraft platform and mission design. Critically, for Earth observation (EO) missions a reduction in altitude can enable smaller and less powerful payloads to achieve the same performance as larger instruments or sensors at higher altitude, with significant benefits to the spacecraft design. As a result, renewed interest in the exploitation of these orbits has spurred the development of new technologies that have the potential to enable sustainable operations in this lower altitude range. In this paper, system models are developed for (i) novel materials that improve aerodynamic performance enabling reduced drag or increased lift production and resistance to atomic oxygen erosion and (ii) atmosphere-breathing electric propulsion (ABEP) for sustained drag compensation or mitigation in VLEO. Attitude and orbit control methods that can take advantage of the aerodynamic forces and torques in VLEO are also discussed. These system models are integrated into a framework for concept-level satellite design and this approach is used to explore the system-level trade-offs for future EO spacecraft enabled by these new technologies. A case-study presented for an optical very-high resolution spacecraft demonstrates the significant potential of reducing orbital altitude using these technologies and indicates possible savings of up to 75% in system mass and over 50% in development and manufacturing costs in comparison to current state-of-the-art missions. For a synthetic aperture radar (SAR) satellite, the reduction in mass and cost with altitude were shown to be smaller, though it was noted that currently available cost models do not capture recent commercial advancements in this segment...

Mapping and monitoring crops is a key step towards sustainable intensification of agriculture and addressing global food security. A dataset like ImageNet that revolutionized computer vision applications can accelerate development of novel crop mapping techniques. Currently, the United States Department of Agriculture (USDA) annually releases the Cropland Data Layer (CDL) which contains crop labels at 30m resolution for the entire United States of America. While CDL is state of the art and is widely used for a number of agricultural applications, it has a number of limitations (e.g., pixelated errors, labels carried over from previous errors and absence of input imagery along with class labels). In this work, we create a new semantic segmentation benchmark dataset, which we call CalCROP21, for the diverse crops in the Central Valley region of California at 10m spatial resolution using a Google Earth Engine based robust image processing pipeline and a novel attention based spatio-temporal semantic segmentation algorithm STATT. STATT uses re-sampled (interpolated) CDL labels for training, but is able to generate a better prediction than CDL by leveraging spatial and temporal patterns in Sentinel2 multi-spectral image series to effectively capture phenologic differences amongst crops and uses attention to reduce the impact of clouds and other atmospheric disturbances. We also present a comprehensive evaluation to show that STATT has significantly better results when compared to the resampled CDL labels. We have released the dataset and the processing pipeline code for generating the benchmark dataset.

Coal mines are globally an important source of methane and also one of the largest point sources of methane. We present a high-resolution 0.1deg x 0.1deg bottom-up gridded emission inventory for methane emissions from coal mines in India and Australia, which are among the top five coal-producing countries in 2018. The aim is to reduce the uncertainty in local coal mine methane emissions and to improve the spatial localization to support monitoring and mitigation of these emissions. For India, we improve the spatial allocation of the emissions by identifying the exact location of surface and underground coal mines and we use a tier-2 Intergovernmental Panel on Climate Change (IPCC) methodology to estimate the emissions from each coal mine using country-specific emission factors. For Australia, we estimate the emission for each coal mine by distributing the state-level reported total emissions using proxies of coal production and the coal basin-specific gas content profile of underground mines. Comparison of our total coal mine methane emission from India with existing global inventories showed our estimates are about a factor 3 lower, but well within the range of the national Indian estimate reported to the United Nations framework convention on climate change (UNFCCC). For both countries, the new spatial distribution of the emissions shows a large difference from the global inventories. Our improved emissions dataset will be useful for air quality or climate modeling and while assessing the satellite methane observations.

This survey paper specially analyzed computer vision-based object detection challenges and solutions by different techniques. We mainly highlighted object detection by three different trending strategies, i.e., 1) domain adaptive deep learning-based approaches (discrepancy-based, Adversarial-based, Reconstruction-based, Hybrid). We examined general as well as tiny object detection-related challenges and offered solutions by historical and comparative analysis. In part 2) we mainly focused on tiny object detection techniques (multi-scale feature learning, Data augmentation, Training strategy (TS), Context-based detection, GAN-based detection). In part 3), To obtain knowledge-able findings, we discussed different object detection methods, i.e., convolutions and convolutional neural networks (CNN), pooling operations with trending types. Furthermore, we explained results with the help of some object detection algorithms, i.e., R-CNN, Fast R-CNN, Faster R-CNN, YOLO, and SSD, which are generally considered the base bone of CV, CNN, and OD. We performed comparative analysis on different datasets such as MS-COCO, PASCAL VOC07,12, and ImageNet to analyze results and present findings. At the end, we showed future directions with existing challenges of the field. In the future, OD methods and models can be analyzed for real-time object detection, tracking strategies.

The accurate estimation of terrestrial water and vegetation is a grand challenge in hydrometeorology. Many previous studies developed land data assimilation systems (LDASs) and provided global-scale land surface datasets by integrating numerical simulation and satellite data. However, vegetation dynamics has not been explicitly solved in these land reanalysis datasets. Here we present the newly developed land reanalysis dataset, ECoHydrological Land reAnalysis (ECHLA). ECHLA is generated by sequentially assimilating C- and X- band microwave brightness temperature satellite observations into a land surface model which can explicitly simulate the dynamic evolution of vegetation biomass. The ECHLA dataset provides semi-global soil moisture from surface to 1.95m depth, Leaf Area Index (LAI), and vegetation water content and is available from 2003 to 2010 and from 2013 to 2019. We assess the performance of ECHLA to estimate soil moisture and vegetation dynamics by comparing the ECHLA dataset with independent satellite and in-situ observation data. We found that our sequential update by data assimilation substantially improves the skill to reproduce the seasonal cycle of vegetation. Data assimilation also contributes to improving the skill to simulate soil moisture mainly in the shallow soil layers (0-0.15m depth). The ECHLA dataset will be publicly available and expected to contribute to understanding terrestrial ecohydrological cycles and water-related natural disasters such as drought.

The proliferation of remote sensing satellites has resulted in a massive amount of remote sensing images. However, due to human and material resource constraints, the vast majority of remote sensing images remain unlabeled. As a result, it cannot be applied to currently available deep learning methods. To fully utilize the remaining unlabeled images, we propose a Geographical Knowledge-driven Representation learning method for remote sensing images (GeoKR), improving network performance and reduce the demand for annotated data. The global land cover products and geographical location associated with each remote sensing image are regarded as geographical knowledge to provide supervision for representation learning and network pre-training. An efficient pre-training framework is proposed to eliminate the supervision noises caused by imaging times and resolutions difference between remote sensing images and geographical knowledge. A large scale pre-training dataset Levir-KR is proposed to support network pre-training. It contains 1,431,950 remote sensing images from Gaofen series satellites with various resolutions. Experimental results demonstrate that our proposed method outperforms ImageNet pre-training and self-supervised representation learning methods and significantly reduces the burden of data annotation on downstream tasks such as scene classification, semantic segmentation, object detection, and cloud / snow detection. It demonstrates that our proposed method can be used as a novel paradigm for pre-training neural networks. Codes will be available on https://github.com/flyakon/Geographical-Knowledge-driven-Representaion-Learning.