Publications

2025

1. AAAI

   Leveraging Textual Memory and Key Frame Reasoning for Full Video Understanding Using Off-the-Shelf LLMs and VLMs (Student Abstract)

   Harsh Dubey, and Chulwoo Pack

   In Proceedings of the AAAI Conference on Artificial Intelligence, 2025

   Abs HTML

   To address the limitations of current Large-scale Video-Language Models (LVLMs) in fine-grained understanding and long-term temporal memory, we propose a novel video understanding approach that integrates a Vision Language Model (VLM) and a Large Language Model (LLM) with a textual memory mechanism to ensure continuity and contextual coherence. In addition, we introduce a novel evaluation metric, VAD-Score (Video Automated Description Score), to assess precision, recall, and F1 scores for events, subjects, and objects. Our approach delivers competitive results on a diverse set of videos from the DREAM-1K dataset, spanning categories such as live-action, animation, shorts, stock, and YouTube, with a focus on fine-grained comprehension.

2. ACR

   PEARL: Perceptual and Analytical Representation Learning for Video Anomaly Detection

   Omeshamisu Anigala, Kwanghee Won, and Chulwoo Pack

   SIGAPP Appl. Comput. Rev., Apr 2025

   Abs HTML

   Video anomaly detection is crucial for applications like surveillance and autonomous systems. Traditional methods often rely solely on visual cues, missing valuable contextual data. This paper presents Perceptual and Analytical Representation Learning (PEARL), a novel method that combines perceptual (raw sensory input) and analytical (higher-level context) modalities. Specifically, we integrate visual information with object tracking data, along with the tracking data-specialized normalization method, DOT-Norm, leveraging ID switching to capture high-level contexts of abnormal movements. We evaluate early- and late-fusion strategies to enhance anomaly detection, particularly for irregular movements marked by frequent track ID switches. Our approach, tested on the UCSD-Ped1 dataset, outperforms the state-of-the-art by improving precision (+0.082), recall (+0.104), F1 score (+0.149), and AUC (+0.053). These findings highlight the potential of integrating analytical tracking data with perceptual video frames in a multimodal learning approach for anomaly detection, paving the way for future applications and research where knowledge-driven analytical modalities are crucial.

2024

1. IJDAR

   Perceptual cue-guided adaptive image downscaling for enhanced semantic segmentation on large document images

   Chulwoo Pack, Leen-Kiat Soh, and Elizabeth Lorang

   International Journal on Document Analysis and Recognition (IJDAR), Apr 2024

   Abs HTML Code

   Image downscaling is an essential operation to reduce spatial complexity for various applications and is becoming increasingly important due to the growing number of solutions that rely on memory-intensive approaches, such as applying deep convolutional neural networks to semantic segmentation tasks on large images. Although conventional content-independent image downscaling can efficiently reduce complexity, it is vulnerable to losing perceptual details, which are important to preserve. Alternatively, existing content-aware downscaling severely distorts spatial structure and is not effectively applicable for segmentation tasks involving document images. In this paper, we propose a novel image downscaling approach that combines the strengths of both content-independent and content-aware strategies. The approach limits the sampling space per the content-independent strategy, adaptively relocating such sampled pixel points, and amplifying their intensities based on the local gradient and texture via the content-aware strategy. To demonstrate its effectiveness, we plug our adaptive downscaling method into a deep learning-based document image segmentation pipeline and evaluate the performance improvement. We perform the evaluation on three publicly available historical newspaper digital collections with differences in quality and quantity, comparing our method with one widely used downscaling method, Lanczos. We further demonstrate the robustness of the proposed method by using three different training scenarios: stand-alone, image-pyramid, and augmentation. The results show that training a deep convolutional neural network using images generated by the proposed method outperforms Lanczos, which relies on only content-independent strategies.

2023

1. Ph.D.

   Enhancing Document Layout Analysis on Historical Newspapers: Visual Representation, Pseudo-Ground-Truth, and Downscaling

   Chulwoo Pack

   The University of Nebraska-Lincoln, Apr 2023
2. AHR

   Applying Image Analysis and Machine Learning to Historical Newspaper Collections

   Leen-Kiat Soh, Liz Lorang, Chulwoo Pack, and 1 more author

   The American Historical Review, Apr 2023

   HTML

2022

1. JOCCH

   Augmentation-based Pseudo-Ground truth Generation for Deep Learning in Historical Document Segmentation for Greater Levels of Archival Description and Access

   Chulwoo Pack, Yi Liu, Leen-Kiat Soh, and 1 more author

   Journal on Computing and Cultural Heritage (JOCCH), Apr 2022

   Abs HTML

   The successful use of deep learning solutions for document image segmentation typically relies on a large number of manually labeled ground truth examples, which is expensive to obtain for historical document images that have significant noise effects and variation. At the same time, successful applications of deep learning solutions for document image segmentation have rich potential to facilitate greater levels of description in archival collections (e.g., at and below the item-level). These greater levels of description are critical to increasing access and use of archival collections across an array of research domains. In response, this article investigates whether an augmentation-based approach to generating pseudo-ground truth can be effective with a limited number of labeled images in a document segmentation application. The rationale is that if we can decrease the cost of generating ground truth through augmentation-based approaches, we can use these approaches as part of the description and access pipelines for historical library and archival collections. In this initial exploration, we first generate synthetic images and corresponding pseudo-ground truth using a set of existing degradation-based augmentation models from a small number of labeled actual images. When generating synthetic images, we control the visual quality distortion based on OCR word-level confidence to avoid generating images unlikely to be present in the dataset. Then, we perform several investigations to examine the impact of incorporating pseudo-ground truth data in the training of the deep learning network dhSegment and further evaluate the use of multiple combinations of degradation models. We also assess the generalizability of the approach by applying the trained network on a larger dataset. Our investigations primarily use real-world datasets known to have significant noise effects. Results show that augmentation-based pseudo-ground truth generation is capable of improving segmentation performance with the use of the full original dataset and requires only 30% of the original dataset. Results also show that using more than three degradation models is likely to cause overfitting during training. Furthermore, we show that a segmentation network trained on pseudo-ground truth data has generalization capability.

2. COMPSAC

   An Intelligent Tutoring System for API Misuse Correction by Instant Quality Feedback

   Rui Zhao, Harvey Siy, Chulwoo Pack, and 2 more authors

   In 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC), Apr 2022

2021

1. JEI

   Visual domain knowledge-based multimodal zoning for textual region localization in noisy historical document images

   Chulwoo Pack, Leen-Kiat Soh, and Elizabeth Lorang

   Journal of Electronic Imaging, Apr 2021

   Abs HTML Code

   Document layout analysis, or zoning, is important for textual content analysis such as optical character recognition. Zoning document images such as digitized historical newspaper pages are challenging due to noise and quality of the document images. Recently, effective data-driven approaches, such as leveraging deep learning, have been proposed, albeit with the concern of requiring larger training data and thus incurring additional cost of ground truthing. We propose a zoning solution by incorporating a knowledge-driven document representation, gravity map, into a multimodal deep learning framework to reduce the amount of time and data required for training. We first generate a gravity map for each image, considering the centroid distance and area between a cell in a Voronoi tessellation and its content to encode visual domain knowledge of a zoning task. Second, we inject the gravity maps into a deep convolution neural network (DCNN) during training, as an additional modality to boost performance. We report on two investigations using two state-of-the-art DCNN architectures and three datasets: two sets of historical newspapers and a set of born-digital contemporary documents. Evaluations show that our solution achieved comparable segmentation accuracy using fewer training epochs and less training data compared to a naïve training scheme.

2020

1. Tech. Report

   Digital libraries, intelligent data analytics, and augmented description: A demonstration project

   Elizabeth Lorang, Leen-Kiat Soh, Yi Liu, and 1 more author

   Apr 2020

   HTML Blog
2. Tech. Report

   Final Presentation to the Library of Congress on Digital Libraries, Intelligent Data Analytics, and Augmented Description

   Elizabeth Lorang, Leen-Kiat Soh, Yi Liu, and 1 more author

   Apr 2020

   PDF Code

2019

1. Tech. Report

   Application of the Image Analysis for Archival Discovery Team’s First-Generation Methods and Software to the Burney Collection of British Newspapers

   Elizabeth Lorang, Leen-Kiat Soh, Chulwoo Pack, and 3 more authors

   Apr 2019

   HTML Supp
2. Tech. Report

   Virtual Wrap-Up Presentation: Digital Libraries, Intelligent Data Analytics, and Augmented Description

   Elizabeth Lorang, Leen-Kiat Soh, Yi Liu, and 1 more author

   Apr 2019
3. Tech. Report

   Document Images and Machine Learning: A Collaboratory between the Library of Congress and the Image Analysis for Archival Discovery (Aida) Lab at the University of Nebraska, Lincoln, NE

   Yi Liu, Chulwoo Pack, Leen-Kiat Soh, and 1 more author

   Apr 2019
4. Tech. Report

   Work-in-Progress Reports Submitted to the Library of Congress as Part of Digital Libraries, Intelligent Data Analytics, and Augmented Description

   Chulwoo Pack, Yi Liu, Leen-Kiat Soh, and 1 more author

   Apr 2019

2017

1. Master

   Optimized Multilayer Perceptron with Dynamic Learning Rate to Classify Breast Microwave Tomography Image

   Chulwoo Pack

   Apr 2017
2. ACR

   Computer aided diagnosis with boosted learning for anomaly detection in microwave tomography

   Chulwoo Pack, Seong-Ho Son, and Sung Shin

   ACM SIGAPP Applied Computing Review, Apr 2017

2016

1. SAC

   Optimized multilayer perceptron using dynamic learning rate based microwave tomography breast cancer screening

   Chulwoo Pack, Sung Shin, Hyung-Do Choi, and 2 more authors

   In Proceedings of the 31st Annual ACM Symposium on Applied Computing, Apr 2016

   HTML Code

2015

1. RACS

   Computer aided breast cancer diagnosis system with fuzzy multiple-parameter support vector machine

   Chulwoo Pack, Sung Shin, Seong Ho Son, and 1 more author

   In Proceedings of the 2015 Conference on research in adaptive and convergent systems, Apr 2015

   HTML
2. RACS

   Breast cancer classification of mammographic masses using improved shape features

   Sailesh Gc, Chulwoo Pack, Sung Shin, and 1 more author

   In Proceedings of the 2015 Conference on research in adaptive and convergent systems, Apr 2015
3. SAC

   Study of wireless mammography image transmission impacts on robust cyber-aided diagnosis systems

   Samaneh Aminikhanghahi, Sung Shin, Wei Wang, and 3 more authors

   In Proceedings of the 30th Annual ACM Symposium on Applied Computing, Apr 2015
4. III

   An Optimized Fuzzy Support Vector Machine Classifier using Breast Mammogram Tomography: Trade-off between Specificity and Sensitivity

   Chulwoo Pack, Samaneh Aminikhanghahi, Sung Shin, and 2 more authors

   International Information Institute (Tokyo). Information, Apr 2015