arXane › Electrical Engineering and Systems Science › Electrical Engineering and Systems Science
Wednesday, 4 November 2026: 12 new papers, 1 cross-list and 5 replacements in Image and Video Processing (eess.IV), in arXiv's order, with full abstracts. The same papers as arXiv's own listing, set as a newspaper; every title links to arXiv.
Continuous video scales in cost with camera count, resolution and recording duration, yet many monitoring tasks seek events rather than complete images and can operate on a compact single-pixel measurement stream. Recovering a chosen interval as video is then desirable, but at 0.24\% of the pixel count the scene correlation image looks like noise and per-clip optimization takes minutes. Correlating the residual of an estimate with the coding patterns nevertheless gives a residual image, the gradient of a centered measurement loss. Lifted through a pretrained video decoder, its agreement with the true error exceeds that of control decoders. Video-latent inversion (VLI) then outperforms frame-wise image-latent reconstruction at 1{,}000 bucket values. Measurement-Gradient-Conditioned Video Flow (MGV-Flow), trained once to read the lifted residual image with preceding frames, reconstructs five frames in 1.55\,s at 22.9\,dB median PSNR, against 398\,s and 18.1\,dB for VLI, with the largest gains in high-motion scenes. Directional fidelity ranks 30 coding configurations and selects one needing a third fewer bucket values. On a physical system, MGV-Flow outperforms per-frame reconstruction without retraining. A sliding window over the bucket-value stream yields 3.2 frames per second on one RTX~4090 and, in theory, 20 frames per second on a data-centre accelerator. Demonstrations are available at https://cavedweller509.github.io/MGV-Flow.
Diabetic macular edema (DME) is a leading cause of severe vision loss among working-age adults. Although intravitreal anti-vascular endothelial growth factor (anti-VEGF) injections currently serve as the first-line therapy for DME, a notable subset of patients fails to respond despite the substantial cost and clinical burden, suggesting they may benefit more from alternative interventions. Automated, noninvasive pre-treatment response assessment could therefore spare non-responders from the current trial-and-error treatment paradigm. In this study, we propose a novel conditional adversarial diffusion framework that synthesizes post-treatment optical coherence tomography (OCT) images from baseline scans, enabling the derivation of each patient's initial response to anti-VEGF therapy. The model was developed with subject-level 5-fold cross-validation on a US cohort (n = 127) and externally validated on an independent Iranian cohort (n = 44). Benchmarked against fourteen baseline and state-of-the-art translation models, the proposed model achieved the best treatment-response prediction performance among the methods evaluated for that task on both cohorts, with balanced accuracies of 85.7% and 93.0% on the US cohort and 85.2% and 95.3% on the external Iranian cohort at the 10% and 20% central subfield thickness reduction thresholds, respectively. Ultimately, this framework offers a decision-support tool to help ophthalmologists anticipate initial anti-VEGF outcomes in DME prior to treatment initiation, toward personalized therapy selection. The source code will be made publicly available upon acceptance.
A brain MRI scan can carry an unknown mix of artifacts, so restoration has to be blind, yet most deep-learning methods are built to correct one known artifact. We present Dx-Rx, an agent in which a multimodal large language model (MLLM) measures suspected artifacts before it repairs them. To our knowledge, it is the first agent to restore whole 3-D MRI volumes, whereas earlier MRI agents restore 2-D slices. In a loop, it looks at slices of the 3-D volume, calls five diagnostic tools (noise, spike, ghosting, bias field, motion) and four restoration tools in any order it chooses, checks each repair, can return to any earlier version to change the order, and ends with a radiology-style report of its diagnostic reasons, the repairs it applied and their results. We simulated up to five artifacts on 148 whole 3-D T1-weighted volumes, at levels calibrated to radiologists' grades of real scans. An MLLM that only looked at the image, as earlier MRI agents do, removed at most 19.7 % of the artifact error. With the best of five MLLMs, Dx-Rx removed a mean 64.5 %, nearly matching the 64.6 % of the same tools given the ground-truth artifacts, which are never known for a clinical scan. With all five models, it made no volume worse and was on par with or better than an expert rule-based pipeline built from the same tools (62.3 %). Code and demo are public at https://itu-biai.github.io/measure-before-you-repair.
Longitudinal rodent positron emission tomography (PET) studies of stroke and neuromodulation rely on accurate registration to a common atlas space to detect group-level effects. Prior work applying ultra-high frequency low-intensity magnetic stimulation (UHF-LiMS) to a rat model of chronic capsule infarct used affine normalization to a Sprague Dawley MRI template without evaluating that choice. We compare a two-stage (affine and nonlinear) pipeline against an affine- only one for rat PET-to-atlas alignment on the same dataset. Across 32 longitudinal [18F]-FDG PET scans (8 rats, 4 timepoints), the nonlinear stage gave a higher Dice similarity coefficient on every scan, and the gain was significant at the rat level. Jacobian determinants showed no folding on the sampling grid. Group-level linear mixed-effects maps of change from baseline (AFNI 3dLME), computed identically under both pipelines, differ in the size, number and location of their suprathreshold clusters. In this dataset, the registration pipeline is therefore a variable that can change the interpretation of downstream group-level analyses.
State-of-the-art bronchoscopy guidance systems and computer-aided diagnosis methods have emerged as important tools for early lung cancer detection. Effective use of these tools depends on being able to accurately link a patient's bronchoscopic airway exam video to their pre-operative 3D chest CT scan. Unfortunately, the derivation of such links, which include bronchoscope depth and camera motion, from the exam video remains an open problem. Building on recently proposed 3D-vision foundation models, we propose DUNEDepth, a hybrid architecture that combines a shared DUNE ViT encoder, a twin MASt3R cross-attention decoder structure, and a prediction head that includes elements of the MASt3R and Reloc3r models. To fine-tune the architecture to our application, we utilize an integrated hybrid synthetic-supervised/self-supervised training strategy that draws on both bronchoscopic video and CT-derived virtual bronchoscopic views. Experiments drawing on bronchoscopic video sequences demonstrate that DUNEDepth more accurately derives depth information and camera motion trajectories than previously proposed self-supervised methods.
Non-line-of-sight (NLOS) imaging aims to sense objects outside the direct line of sight. While most previous work focuses on the canonical around-the-corner configuration, this work considers a more challenging occlusion-based NLOS setting in which targets are hidden by scene geometry. To avoid the challenge of collecting large real-world datasets, we have developed a photon-based Monte Carlo simulator to generate diverse synthetic training data and validate against experimental data. We trained an AI model to identify multi-bounce photon patterns associated with hidden targets. The model achieves an Average Precision (AP) of 0.91 on synthetic test data and, when transferred directly to 180 real measurements acquired with a TMF8829 SPAD flash LiDAR, achieves 86.67% accuracy and an AP of 0.95. These results show that simulated transient data can support hidden-target detection with a low-cost commercial sensor and provide an initial step toward more general occlusion-based NLOS sensing and, ultimately, hidden-scene reconstruction.
Improving learned LiDAR compression increasingly relies on heavier inference, making it harder to meet the constraints of real-time embedded systems. We present ELiCv2, which removes two sources of redundant computation in progressive coding. Its discrete entropy interface selects a precomputed integer CDF instead of materializing one from a per-voxel distribution, while its occupancy-driven kernel-map builder derives finer-level sparse-convolution kernel-maps directly from the decoded hierarchy, eliminating coordinate-based neighbor search. ELiCv2 reinvests the recovered computation budget in lightweight ego-motion-aware temporal conditioning that gathers local occupancy context from the aligned previous reconstruction. On four LiDAR datasets and an NVIDIA Jetson Thor, ELiCv2 achieves a mean BD-rate reduction of 30.97% over G-PCC with channel width C=32 while sustaining the 20Hz nuScenes capture rate. Increasing the channel width to C=64 improves the mean reduction to 35.22%.
Reconstructing three-dimensional (3D) coronary anatomy from sparse angiographic views could support percutaneous coronary intervention, but limited angular coverage and imperfect vessel segmentations introduce substantial geometric ambiguity. We propose a vessel-specific reconstruction framework combining a learned 3D diffusion prior, visual-hull initialization, and projection-based data consistency. The unconditional diffusion model is trained on coronary vessel masks without paired 2D--3D data and reused across different view counts without retraining. During inference, a scaled visual hull initializes reconstruction, while projection-based updates constrain diffusion refinement using patient-specific vessel observations. We evaluated the framework on simulated angiograms derived from ImageCAS computed tomography angiography volumes using two, four, and eight views at two contrast levels. Across 50 evaluation coronary-tree cases, the proposed method achieved higher mean Dice and clDice than visual hull, deep image prior (DIP), and hash-grid implicit neural representation (INR) at all view counts under reduced contrast. With eight views, improvements over DIP, the best-performing baseline at lower contrast, were approximately 0.13 in Dice and 0.22 in clDice. At higher contrast, performance remained competitive with hash-grid INR, the best-performing eight-view baseline. The preferred diffusion starting timestep depended on observation quality and view count, with excessive refinement reducing accuracy at higher contrast. Projection guidance improved both metrics across all conditions. Averaging stochastic reconstructions further improved accuracy, while sample disagreement localized errors with average-precision gains of 0.14--0.21 over error prevalence. These findings support diffusion-prior reconstruction from sparse vessel observations, although clinical validation remains necessary.
Robust Principal Component Analysis (RPCA) is essential in signal processing, with applications in hyperspectral imaging, face recognition, and video background modeling. Traditional RPCA methods often suffer from insufficient utilization of data priors and struggle to capture complex image structures. Conversely, advanced deep denoisers like DnCNN, FFDNet, and DRUNet excel at leveraging image data priors but lack physical model constraints and demonstrate poor robustness against structured noise in RPCA tasks. This paper proposes a novel Plug-and-Play (PnP) method for multi-channel RPCA, integrating both the physical generative model and cutting-edge deep priors. Extensive experiments on RGB and hyperspectral datasets show that PnP-PDA achieves better quantitative performance than conventional RPCA methods and state-of-the-art methods, including deep denoisers and tensor RPCA approaches, while preserving visual details.
Accurate segmentation of the mitral valve from three-dimensional transesophageal echocardiography (3D TEE) is severely restricted by the scarcity of publicly available expert- annotated volumetric datasets. To address this critical limitation, we investigated Fourier Domain Adaptation (FDA), utilizing general-purpose generative-AI images (from platforms like Chat- GPT, Google Gemini, and Microsoft MAI) as annotation-free style donors for 3D mitral valve segmentation. We employed a DeepMitral residual U-Net model on the public MVSeg2023 dataset, comparing six conditions over three runs: real-only training, traditional augmentation, FDA with each generative source, and a non-AI FDA control using real-image style donors, plus an optimization-matched real-only control and a \b{eta} sensitivity analysis. Case-level paired testing showed that FDA with these generative sources led to significant reductions in boundary- and surface-related errors across all three donor conditions relative to the primary real-only baseline. However, the impact on the core segmentation metric, Generalized Dice Score (GDS), proved generator-dependent. ChatGPT-FDA showed a significant GDS advantage over the optimization-matched control but not over the primary baseline, and Gemini-FDA showed no significant GDS difference. Notably, only MAI-FDA provided a statistically robust and significant increase in GDS compared to the primary baseline. The \b{eta} sensitivity was not robust across training runs. These findings confirm that FDA remains an important augmentation mechanism. While generative-AI images offer a practical means of acquiring style diversity without new annotations, the non-AI control performed comparably to ChatGPT-FDA, so we do not conclude that this approach is superior to training with real-image donors.
Quantitative MRI methods can offer tissue parameter maps (e.g. T1) that are comparable across scanners, sessions and subjects, together with multiple contrast-weighted images from a single scan. However, most data are acquired at 1 mm, which limits their use in neurodegenerative diseases that affect fine brain structures. We introduce, to our knowledge, the first diffusion-model-based reconstruction for quantitative MRI at 0.5 mm isotropic resolution from scans of less than 5 minutes, which has eightfold smaller voxel volume than clinical protocols. The key idea is to apply the prior to the synthesized contrast images rather than to the subspace coefficients. The prior is then independent of the temporal basis and the sequence, which improves generalization and allows a 2-D multiscale energy (MuSE) model trained on grayscale CelebA face images to be used without any MRI training data. The explicit energy makes reconstruction MAP estimation by a step-size-free majorize-minimize algorithm that regularizes a rotating subset of contrasts along different axes, which makes 3-D multi-contrast reconstruction tractable. For 3-D radial MPnRAGE at 7 T in 4.3 min, MuSE reduced the phantom NRMSE from 0.133 (CG-SENSE) and 0.091 (L1-wavelet) to 0.074 and the WM T1 interquartile range from 117 and 47 ms to 38 ms, and it was demonstrated in vivo at 7 T.
Zero echo time (ZTE) MRI is near-silent, but dead time gaps, head motion, and undersampling jointly degrade reconstruction. To develop an accelerated ZTE MRI reconstruction framework that jointly estimates the 3D image and rigid-motion parameters, this work introduces Energy-based Motion Correction (EnMoCo), a regularized maximum a posteriori (MAP) formulation that also compensates for missing k-space center data and angular undersampling artifacts. Solved by alternating between k-space rigid-motion updates and conjugate-gradient data-consistency updates, the framework incorporates regularization via a pre-trained Multi-Scale Energy-based model (MuSE) implemented in a data-efficient 2.5D form. Evaluations on simulated and in vivo 3D ZTE data, spanning dead time gaps of 1.5 to 5 samples, rotations up to 17 degrees and translations up to 10 mm, demonstrate that EnMoCo significantly improves PSNR and SSIM relative to CG-SENSE and Autofocusing baselines (achieving roughly 10 to 15 dB higher PSNR for gaps >= 3.5) while reducing motion blur and tissue contrast inconsistencies.
Saliency maps are offered as a safeguard for medical image classifiers, but whether they reveal individual errors at a new hospital remains largely untested. We trained a convolutional network and a vision transformer on NIH ChestX-ray14 and evaluated 303,768 decisions per model at the training hospital and at two external sites, in the USA and Vietnam, with radiologist-consensus labels. We scored each explanation by its anatomical enrichment, the area-normalized concentration of attribution in a segmented thoracic region, and tested whether it detects errors beyond the model's confidence. Grad-CAM localized findings less often than radiologists (hit rate at most 0.575 versus 0.888). Adding enrichment to a confidence-based error detector calibrated at the training hospital never raised its area under the curve by more than 0.011, and for the convolutional network the relation between enrichment and correctness reversed between hospitals, degrading the transferred detector. Model confidence, not saliency, should anchor deployment monitoring.