Bo-Ruei (Ray) Huang 黃柏睿

Learning from observation (LfO) aims to imitate experts by learning from state-only demonstrations without requiring action labels. Existing adversarial imitation learning approaches learn a generator agent policy to produce state transitions that are indistinguishable to a discriminator that learns to classify agent and expert state transitions. Despite its simplicity in formulation, these methods are often sensitive to hyperparameters and brittle to train. Motivated by the recent success of diffusion models in generative modeling, we propose to integrate a diffusion model into the adversarial imitation learning from observation framework. Specifically, we employ a diffusion model to capture expert and agent transitions by generating the next state, given the current state. Then, we reformulate the learning objective to train the diffusion model as a binary classifier and use it to provide “realness” rewards for policy learning. Our proposed framework, Diffusion Imitation from Observation (DIFO), demonstrates superior performance in various continuous control domains, including navigation, locomotion, manipulation, and games.

Volcanic eruptions significantly influence climate change by releasing substantial amounts of CO2, SO2, and H2S into the atmosphere, triggering acid rain and enhancing the greenhouse effect. The Orbiting Carbon Observatory (OCO) satellites, OCO-2 and OCO-3, monitor the column-averaged concentration of CO2 (XCO2). However, these eruptions also produce ash plumes, primarily composed of sulfate aerosols, which can compromise the accuracy of OCO measurements. The complex scattering and reflective properties of these volcanic aerosols, which have spatiotemporally varying density, color, and composition, pose challenges for precise modeling. Previous research has confirmed the efficacy of machine learning in extracting aerosol optical depth (AOD) from OCO-2 spectra, specifically in the Saudi Arabian desert (Chen et al., 2021). In this study, we enhance this approach to manage the complexity introduced by volcanic eruptions. We introduce a machine learning algorithm to extract essential aerosol information from OCO spectra, with CALIPSO data serving as the benchmark for training and validation. We aim to extract critical aerosol parameters, such as AOD and layer height (ALH), exploiting the inherent non-linear relationships between measured spectra and aerosol properties. The improved aerosol data will, in turn, boost the precision of XCO2 retrieved from OCO measurements (Hong et al., 2023). We will demonstrate the promising role of machine learning algorithms in gleaning crucial insights about volcanic aerosols from OCO measurements. Through enhanced aerosol constraints, we aim to improve the retrievals of gases like CO2 and SO2S, facilitating more precise evaluations of their impact on climate change. Our results add to the emerging knowledge on remote sensing techniques and open up new avenues for using machine learning to refine atmospheric models.