Vol. 43 · No. 2

In recent years, some deep learning dehazing methods based on atmospheric scattering model mostly solve the dehazing results by using depth convolution neural networks (CNNs) to estimate the medium transmission map in the model. However, these methods usually ignored the potential correlation between the transmission map and the atmospheric light in the atmospheric scattering model, which can lead to colour distortion and incomplete dehazing in the dehazing results. To address this problem, this paper first presents a novel Haze-Veil model to increase the correlation between the model parameters by constructing an atmospheric veil term. Then, based on the proposed model, a haze-relevant end-to-end network (HRN) is designed to estimate the parameters of this model and directly output the final clear image. In addition, a cost function is designed by defining multi-object constraint cost functions to further establish the connections between the statistical attributes of the hazy image and the out of HRN. Experiments on benchmark images, which include synthesized and real images, show that HRN effectively removes haze and outperforms most of the existing and state-of-the-art dehazing methods.

Existing deep-learning-based image steganalysis networks have problems such as large model sizes, significant runtime memory usage, and extensive computational operations, which hinder their deployment in many practical applications. To address these challenges, we applied model compression techniques to image steganalysis and designed a model called SASRNet, a slimming-assisted steganalysis residual network. We observed that the trainable scale factor of BN (batch normalization) layer in steganalysis network can be used as channel scaling factor for pruning. The channel-level sparsity of convolutional layers is enhanced by imposing L1 regularization on channel scaling factors and pruning less informative feature channels. With the goal of balancing performance and efficiency, the iterative algorithm is used to further compress the network to obtain a slimming- steganalysis detector. In contrast to many existing methods, our proposed method can be directly applied to steganalysis network architectures by introducing a minimal overhead to the training process. We have conducted extensive experiments on BOSSBase+BOWS2 dataset. Experiments show that, compared to the original steganalysis model, this method can achieve comparable performance with less than 5

As a well-known nonlinear tool, mathematical morphology (MM) is still active in image processing. Benefiting from the fixed structuring element (SE), traditional MM (TMM) gets solid theoretical foundation. However, due to the inherent diversity of pixels in an image, the rigid SE paradigm is not always practical. As a result, the development of morphology with adaptive SE, known as adaptive MM (AMM), has been a significant challenge. In this work, we present a novel approach for designing adaptive SE using the α-cut of a fuzzy set. By implementing dilation and erosion operations serially, we obtain an AMM (named SAMM) that is both adaptive to image content and robust to noise. Additionally, the operators in SAMM inherit important properties from TMM as much as possible. We provide theoretical proofs and simulated results to support our conclusion. Preliminary experiments on edge detection and noise reduction confirm the effectiveness of our SAMM both quantitatively and perceptually. In the denoising experiments, SAMM achieves the best performance in the nine algorithms involved, with its PSNR value surpassing the second-ranked approach by more than 0.6 dB overall. Additionally, its SSIM quantification metric also ranks prominently among the top performers.

Low-light image enhancement (LLIE) enables to serve high-level vision tasks and improve their efficiency. Retinex-based methods have well been recognized as a representative technique for LLIE, but they still suffer from inflexible regularization terms in decomposing illumination and reflectance. In this paper, we propose a new weighted fractional-order variational model based on the Retinex model. First, the constructed weighted fractional-order variational model estimates piecewise smoothed and weakly pixel-shifted illumination by aware structures and textures. Then, to solve this problem accurately, we chose a semi-decoupled approach and an alternating minimization method. Finally, the designed multi-illumination fusion method accurately enhances the structure-rich dark regions of the image through well-exposedness and local entropy weights, while realizing adaptive enhancement based on a naturalness-preserving parameter estimation algorithm. The results of subjective and objective experiments on several challenging low-light datasets demonstrate that our proposed method shows better competitiveness in enhancing low-light images compared with the state-of-the-art methods.

Blind Face Super-Resolution (BFSR) has recently gained widespread attention, which aims to super-resolve Low-Resolution (LR) face images with complex unknown degradation to High-Resolution (HR) face images. However, existing BFSR methods suffer from two major limitations. First, most of them are trained on synthetic degradation data pairs with pre-defined degradation models, which leads to poor performance due to the degradation mismatch between other unknown complex degradations in real-world scenarios. Second, some methods rely on hand-crafted face priors as constraints, such as facial landmarks and parsing maps, which require additional callouts and laborious hyperparameter tuning for real cases. To tackle these issues, we propose a simple and effective self-supervised cooperative learning framework via a conditional diffusion contraction method for BFSR, dubbed DifBFSR, which establishes the posterior distribution of HR images from degraded LR images with unknown degradation via a powerful diffusion model without expensive supervised training or additional constraint design. Specifically, we first transform the degraded LR face image to an intermediate HR face prediction with degradation-invariant by a simple Super-Resolution module (SRM), which only relies on self-supervised optimization. To enhance the face prediction, we propose a Contraction Filter Module (CFM) to gradually contract the restoration error by adaptive dynamic filtering, which efficiently leverages rich nature face prior encapsulated in the pre-trained diffusion model through conditional posterior sampling. Finally, by combining the SRM, CFM, and diffusion model in a self-supervised cooperative learning framework, DifBFSR can robustly handle unknown complex degradations, which favorably avoids the cumbersome training and parameter tuning. Extensive qualitative and quantitative experiments on complex degraded synthetic and real-world datasets show that our method outperforms state-of-the-art BFSR methods.

In this paper, we propose a lightweight dual-path convolutional neural network for image super-resolution (SR). We introduce shift convolution and propose a shift-channel attention (shift-ca) mechanism to build an effective network. Shift-ca produces an attentional map with a larger field of view, and its formulation is similar to channel attention and spatial attention. In addition, we propose the Local Shift-Channel Attention Feature Extraction (LCFE) module as the main part of the Dual Path Shift Attention Block (DPSAB). Using the dual-path structure allows us to reduce the network depth and retain more original features for the subsequent up-sampling compensation operation. In the final HR reconstruction module, we combine the nearest neighbor upsampling layer, convolutional layer, and activation layer to form the compensated nearest neighbor upsampling module (C-NUM) to improve the reconstruction quality with a small parameter cost. Our final model is the Dual Path Shift Attention Network (DPSAN), and it achieves similar performance to the lightweight network WMRN (36.38

In the realm of single-image super-resolution (SISR), generating high-resolution (HR) images from a low-resolution (LR) input remains a challenging task. While deep neural networks have shown promising results, they often require significant computational resources. To address this issue, we introduce a lightweight convolutional neural network, named BTAN, that leverages the connection between LR and HR images to enhance performance without increasing the number of parameters. Our approach includes a target transform module that adjusts output features to match the target distribution and improve reconstruction quality, as well as a spatial and channel-wise attention module that modulates feature maps based on visual attention at multiple layers. We demonstrate the effectiveness of our approach on four benchmark datasets, showcasing superior accuracy, efficiency, and visual quality when compared to state-of-the-art methods.

Existing approaches to image risk assessment start with the uncertainty of the model, yet ignore the uncertainty that exists in the data itself. In addition, the decisions made by the models still lack interpretability, even with the ability to assess the credibility of the decisions. This paper proposes a risk assessment model that unites a model, a sample and an external knowledge base, which includes: 1. The uncertainty of the data is constructed by masking the different decision-related parts of the image data with a random mask of probabilities. 2. A dynamically distributed dilated convolution method based on random directional field perturbations is proposed to construct the uncertainty of the model. The method evaluates the impact of different components on the decisions within the local region by locally perturbing the attention region of the dilated convolution. 3. A triadic external knowledge base with relative interpretability is presented to reason and validate the model's decisions. The experiments are implemented on the dataset of CT images of the stomach, which shows that our proposed method outperforms current state-of-the-art methods.

Facial expressions (FEs) spotting aims to split long videos into intervals of neutral expression, macro-expression, or micro-expression. Recent works mainly focus on feature descriptor or optical flow methods, suffering from difficulty capturing subtle facial motion and efficient temporal aggregation. This paper proposes a novel end-to-end network, named FESNet (Facial Expression Spotting Network), to solve the above challenges. The main idea is to model the subtle facial motion as local spatial discrepancy and incorporate temporal correlation by multi-scale temporal convolution. The FESNet comprises a local spatial discrepancy module (LSDM) and a multi-scale temporal aggregation module (MTAM). The LSDM first extracts the static spatial features from each frame by residual convolution and learns the inner spatial correlation by multi-head attention. Moreover, the subtle facial motion of facial expression is modeled as the discrepancy between the first frame and the current frame of the input interval, making frame-wise spatial proposals. Using the local spatial discrepancy features and proposals as input, the MTAM incorporates the temporal correlation by multi-scale temporal convolution and performs cascade refinement to make the final prediction. Furthermore, this paper proposes a smooth loss to ensure the temporal consistency of the cascade refined proposals from MTAM. Comprehensive experiments show that FESNet achieves competitive performance compared to state-of-the-art methods.

Generative adversarial networks (GANs) have been used as a solution to handwritten Chinese character automatic generation (HCCAG) in recent years. However, most existing GAN-based methods adopt a pixel-based strategy, which ignores the radical structure of Chinese characters. To achieve better HCCAG, a radical constraint-based GAN (RC-GAN) is proposed in this work. In the proposed method, a gated recurrent unit (GRU)-based radical learning network is designed to calculate the radical components among Chinese characters, and radical consistent loss is applied to train this module. Finally, the radical learning module is fused with a cyclic structure GAN to improve the performance of Chinese character generation. The experimental results show that compared with the existing GAN, the proposed method has better performance. Specifically, the proposed method can reduce the stroke error in the generated Chinese character images.

Stereo matching methods based on Convolutional Neural Network (CNN) have achieved a significant progress in recent years. However, they still cannot work well on generalization performance across a variety of datasets due to their poor robustness. In view of this, we aim to enhance the robustness in three main steps of stereo matching, namely cost computation, cost aggregation, and disparity refinement. For cost computation, we propose an atrous pyramid grouping convolution (APGC) module, which combines local context information with multi-scale features generated from CNN backbone, aiming to obtain a more discriminative feature representation. For cost aggregation, we provide a multi-scale cost aggregation (MSCA) module, which sufficiently and effectively fuses multiple cost volumes at three different scales into the 3D hourglass networks to improve initial disparity estimation. In addition, we present a disparity refinement (DR) module that employs the color guidance of left input image and several convolutional residual blocks to obtain a more accurate disparity estimation. With such three modules, we propose an end-to-end context-guided cost aggregation network (CGCANet) for robust stereo matching. To evaluate the performance of the proposed modules and CGCANet, we conduct comprehensive experiments on the challenging SceneFlow, KITTI 2015 and KITTI 2012 datasets, with a consistent and competitive improvement over the existing stereo matching methods.