Faced with siloed data and time-consuming information retrieval, the Port of Antwerp-Bruges partnered with Superlinear to develop APICA Chat, an AI-driven assistant streamlining data access and operational efficiency.
Executive summary
Context
A few years ago, the data governance team at the Port of Antwerp-Bruges started on a quest to develop a comprehensive company-wide data catalog in the form of a conceptual graph. This data catalog would provide unambiguous definitions for all concepts related to Port operations, and would show how different concepts are related. By adding metadata - data about their data - it would act as the perfect complement to the plethora of data the Port has access to.
The problem: only tech-savvy employees could effectively combine the data catalog with structured data, limiting the overall improvement in company-wide data literacy.
Goal of the project
The goal of this project is to democratize data across all departments of the Port of Antwerp-Bruges. In a period in which all companies aim to become more data-driven, most are slowed down by the technical expertise required to gain actionable insights from their data. Whether it is programming knowledge, or the ability to navigate certain data catalog tools, data democratization across the whole company often remains a dream. We are here to help the Port realize that dream.
Solution
An agent that employees can interact with in natural language to obtain insights into both the Port’s data, and the company-wide data catalog. This agent can construct SQL queries to retrieve data from the databases, and can explore the data catalog to provide employees with an easy access point to the business glossary.
"This innovative solution revolutionizes data accessibility at the Port of Antwerp-Bruges. By leveraging a natural language interface, we empower every employee to effortlessly extract insights from our extensive data and comprehensive data catalog, without the need for technical skills. This project is a great step towards true data democratization, leading us into a new era of data-driven excellence." – Pieter Van Bouwel, Team Lead Data Science, Port of Antwerp-Bruges
Case study
The challenge
Like many companies, the Port of Antwerp-Bruges has a lot of data. Getting access to this data remains difficult for most employees. In general, they would have to ask a business analyst to write a SQL query and create a dashboard on top of it. To do this, the analyst will make use of the data catalog created by the data governance team to find the relevant concepts, and identify where they occur in the data. Unfortunately, this process hampers data democratization, and significantly slows down the time required to get insights from the data.
The solution
With the advent of large language models, the Port of Antwerp-Bruges realized that the data catalog they developed could now be shared across the whole company in a user-friendly way. Even more impressive, since these models are very capable SQL developers, it would be possible to create an agent that enables interacting with the database in natural language.
By combining information from the data catalog and the Port’s SQL databases, our solution offers two distinct but complementary tools:
Chat with your data catalog - an intuitive single point of access to a company’s business glossary.
This allows everyone in the company to quickly find the definition of a certain concept, to find out which other concepts are related to it, and where in the data this concept occurs. The Port’s data catalog contains 1500 concepts, and our solution helps employees navigate that glossary.Chat with your database: A SQL query creator that translates a natural language question into a SQL query
This allows non-technical people to get access to critical data. Before, they had to ask a business analyst to create a dashboard that should answer a set of questions. A process that takes hours at best, and days or weeks in most cases. With our solution, key insights can be obtained in a matter of seconds.
For the business analysts, this tool accelerates query construction, and takes away the burden of having to replicate simple queries. Instead, they can focus on complex requests with more impact.
The implementation
The Chat With your Data use case is a prime example of Agentic AI. When a question is received, the Data Catalog Agent selects the relevant concepts from the data catalog. This agent leverages graph algorithms to improve retrieval. Subsequently, the occurrences of each concept in the SQL database are traced, so that a holistic view of the concept is obtained. The Orchestrator Agent then decides whether the question requires a SQL query to be constructed, or whether the question can be answered based on the information collected by the Data Catalog Agent.
In case the question does require a query to be constructed, all information about the SQL tables considered relevant by the Data Catalog Agent is collected. The Query Constructor Agent then uses this information to iteratively construct the right query. This agent will perform self-healing modifications in case the query was not successful at first, will ask for clarification if the question is unclear, and will provide an explanation of the generated query.
Example
To illustrate the agent in practice, let us take a look at the following example: imagine a port co-worker needing to answer the question, "How many sea ships arrived at the port yesterday?" Even though this question may seem simple, it is actually quite complex upon closer examination. What exactly is meant by this question? What is a sea ship? What does arrival mean? These terms can be found in the data catalog, but technical knowledge is required to query it.
Even if one succeeds in understanding the question, finding an answer is difficult without a deep understanding of the underlying data model. Where can I find the data related to sea ships and arrivals, and how should I link these two data sources? This is all described in the data catalog, but again, it takes a lot of time for a technical person to navigate through it.
These difficulties are solved by our agent, which leverages the structure of the data catalog to answer both questions. Essentially, it functions as a combination of a domain expert and an SQL developer—a skill that typically takes a person a considerable amount of time to learn.
Results
These two tools offer tremendous benefits for data teams and other employees alike.
For data teams, we see three important benefits:
Constructing simple queries by hand is a thing of the past. APICA can now do this quite easily. As such, data teams can focus on complicated requests that add a lot of value.
The tool offers a plethora of insights for the data governance team. By interacting with APICA, they can see which concepts are still confusing, and can see what SQL tables are unclear.
We have seen first-hand how this led to a discussion of whether a new SQL table should be added, and how it led to a redefinition of various concepts.It is the best onboarding assistant you can imagine. Without writing a single line of code, APICA allows a new hire to get acquainted with the whole data catalog and databases in a matter of days.
For the broader organization, we see two main perks. First, from an operational point of view, we observe significant reductions in time to data. In a matter of seconds employees gain insights that would otherwise require a dashboard to be created by a business analyst. As a result, the decision-making process can be sped up significantly.
The following testimonial sums this up perfectly:
“For an urgent COO request, APICA’s Chat with your Data feature delivered in 20 minutes what used to take a full workday. It’s a game-changer.” – Evelien Baelden, Team Lead Port Intelligence, Port of Antwerp-Bruges
Second, the application offers a way to open up the business glossary in an intuitive way. As such, every employee in each department knows what a concept means and in what data sources it occurs. I think it is fair to say that in this way, we are able to realize the dream we envisioned at the start: to enhance data democratization across the Port of Antwerp-Bruges.
