Unlocking Geological Archives with AI: Multimodal Retrieval and UI Design applied to Critical Mineral Intelligence

The Hartree National Centre for Digital Innovation (HNCDI) is a funded collaboration between IBM and UKRI STFC to support companies in learning how AI and machine learning can address their real-world data and challenges. British Geological Survey (UKRI BGS) benefited from a HNCDI project to develop an AI-driven pipeline for extracting critical mineral intelligence from a legacy archive of approximately 40,000 pages of UK mineral exploration reports dating from the 1970s-80s. These documents combine typewritten and handwritten text, maps, sketches, and tables. The size, age and multi-modality make manual exploration or traditional information extraction infeasible. Unlocking this data is valuable to identify UK occurrences of minerals needed for green technologies and the energy transition such as lithium, graphite, and rare earth elements.
Two complementary approaches were implemented:
1. OCR + Ontology Pipeline leveraging IBM’s Knowledge Hub for document ingestion, entity recognition, and structured retrieval, grounding answers using a domain ontology, and using state of the art open source OCR (OlmoOCR) for text detection.
2. Multimodal Vision-Language Model (VLM) approach that bypasses OCR entirely, enabling unified representation across text, tables, and images. This approach explores emerging techniques for scalable retrieval, including vector databases (Milvus), OCR-free vision RAG (ColPali), and multimodal embeddings (Nomic).
The prototype integrates a user-centric interface for query formulation, filtering of source pages, relevance visualization via attention heatmaps, and geospatial mapping of extracted entities. Responses include source citations to mitigate hallucination and support provenance—a critical requirement for verifiable geological decision-making. Evaluation remains challenging due to the absence of comprehensive ground truth in an otherwise unexplored archive.
This work demonstrates how advanced AI architectures can transform pre-digital archives into actionable knowledge, addressing scalability, hallucination mitigation, and multimodal fusion. Beyond critical minerals, the methodology generalizes to other archives, offering a blueprint for applying AI to knowledge extraction from complex geoscience archives at scale.