Shashwat Sarkar

Glioblastoma survival prediction is a vital challenge in brain tumor research, yet most existing approaches still use isolated or staged pipelines that separate tumor segmentation, feature design, and survival prediction. These staged setups often carry errors from one step to the next resulting in error accumulation, it depends on time-consuming handcrafted features, and fails to link early outputs directly to patient survival. To overcome these issues, we introduce a student–teacher framework that combines volumetric tumor segmentation and survival prediction through knowledge distillation. We introduce a segmentation framework which builds on Shifted Windows UNet Transformer (Swin-UNETR) backbone with attention modules and multi-scale pooling to segment detailed tumor structure from Magnetic Resonance Imaging (MRI) scans. We then use a multimodal Transformer to merge these imaging features with segmentation maps and patient data such as age and surgical resection. By employing an Exponential Moving Average (EMA) as the teacher of the Multimodal Transformer student, our approach transfers a stable risk signal while learning image patterns that connect with clinical outcomes. This student–teacher training eliminates the need for handcrafted radiomics and avoids the issues observed in staged pipelines. Overall, our student–teacher design frames survival prediction as a knowledge distillation problem, offering a more direct and meaningful path from raw scans to patient survival prediction.

Recognizing text extracted from multiple domains is complex and challenging because complexities vary from one domain to another. Most existing methods focus either on natural scene text or specific text type but not text of multiple domains, namely, scene, underwater, and drone texts. In addition, the state-of-the-art models ignore the vital cues that exist in multiple instances of the text. This paper presents a new method called the Student-Teacher-Assistant (STA) network, which involves dual CLIP models to exploit cues in multiple text instances. The model that uses ResNet50 in its image encoder is called helper CLIP, while the model that uses ViT in its image encoder is called primary CLIP. The proposed work processes both models simultaneously to extract visual and textual features through image and text encoders. Our work uses cosine similarity for the randomly chosen input image to detect instances similar to the input image. The input and similar instances are supplied to primary and helper CLIPs for visual and textual feature extraction. The outputs of dual CLIPs are fused in a different way through the alignment step for recognizing text accurately, irrespective of domains. To demonstrate the proposed model’s significance, experiments are conducted on a set of standard natural scene text datasets (regular and irregular), underwater images, and drone images. The results on three different domains show that the proposed model outperforms the state-of-the-art recognition models.

Text spotting in diverse domains, such as drone-captured images, underwater scenes, and natural scene images, presents unique challenges due to variations in image quality, contrast, text appearance, background complexity, and external factors like water surface reflections and weather conditions. While most existing approaches focus on text spotting in natural scene images, we propose a Domain-Independent Text Spotter (DITS) that effectively handles multiple domains. We innovatively combine the Real-ESRGAN, developed for regular image enhancement, with the DeepSolo, developed for scene text spotting, in an end-to-end fashion for text detection and spotting on images of different domains. The key idea behind our approach is that improving image quality and text-spotting accuracy are complementary goals. Real-ESRGAN enhances image quality, making the text more discernible, while DeepSolo, a state-of-the-art text spotting model, accurately localizes and recognizes text in the enhanced images. We validate the superiority of our proposed model by evaluating it on datasets from drone, underwater, and scene domains (ICDAR 2015, CTW1500, and Total-Text).