KT Cloud Proposes Three Architectural Boundaries for Heterogeneous Data Integration

The KT Cloud Platform Engineering team has released design criteria for three architectural boundaries: 'Collection, Identification, and Consumption' to ensure the scalability of heterogeneous data integration systems, as monitoring tools and observability targets rapidly increase in multi-cloud environments. The complexity of integration systems arises not at the initial integration but at the point of 'adding new sources'; the core principle is to set each segment of the data flow as an isolation boundary, allowing changes to be absorbed internally.

Design of the Three Architectural Boundaries

• Collection Boundary (Source Adapter): Applies the adapter pattern to handle changes in source types. It separates core logic from adapters so that adding a new source requires only adding 'one adapter'. It adopts a monotonically increasing cursor-based Pull method instead of real-time Push, standardizing reprocessing and order guarantees.
• Identification Boundary (Entity Resolver): Centralizes and absorbs differences in entity identification keys (such as IP, Name+Region, etc.) across various sources in a single Resolver. Since hardcoding key matching logic in query lookups requires a full overhaul when introducing new keys, it ensures flexibility by separating common fields from extended (JSONB) attributes.
• Consumption Boundary (Read Model): Directs dashboards, reports, and other interfaces to go through a Read Model (dedicated query model) rather than directly joining core tables. This prevents changes to core table structures from propagating to all interfaces and avoids silent bugs where 'data is ingested but not visible on the screen'.

Commonization Principles and Operational Visibility

It is crucial to strike a balance by drawing abstraction boundaries only at points that 'change frequently and in the same way', while leaving areas with distinct contexts to individual handling. Instead of a fully integrated view, individual UNIONs were adopted to manage performance and complexity. By clarifying interface contracts with sources (monotonically increasing cursors, occurrence/resolution pairing), correction code was reduced by half. Additionally, matching failure data is exposed through a separate channel to prevent silent data loss, and schema single-source management is used to maintain consistency between documentation and implementation.