✦ Enterprise System Integration · Higher Education
Connecting Five Enterprise Systems Without Rebuilding Them
How we built a reusable, event-driven integration architecture to connect existing applications, standardise business data and handle complex data dependencies.
- 5enterprise systems connectedTeam-reported
- 600k / hourrecords processed, timetable acceptance testMeasured · acceptance test
- 2integration approaches, one shared libraryDelivered
- Backbone
- Apache Kafka
- Pattern
- Canonical contract
- Domains
- Records, offers, benefits, timetables
→ or swipe to begin
✦ The challenge
Where it started
A large higher-education organisation operated several independent systems supporting student records, academic programmes, scheduling and financial processes. They needed to exchange information, but had been developed independently, with different data structures, business rules and responsibilities.
The same student could be represented differently across applications. Some systems were authorised to update specific attributes, while others could only consume them. Consistency was hardest when updates arrived out of order, dependencies were missing or an external system supplied invalid data.
The challenge was to connect five existing systems without replacing them or creating a growing network of point-to-point integrations.
- 5Existing systems connected
- 4Business domains
- 2Integration approaches
- 1Shared integration library
✦ Our approach
How we tackled it
A canonical integration architecture built around a shared business data model. Each system connects through a dedicated bridge that translates its data into a standard contract, with Apache Kafka as the shared event backbone. Not every system is integrated the same way.
- 01
Contract-based integration
Systems the organisation controlsBusiness rules are enforced at the source. Validated changes are translated into the canonical contract and distributed through events. Lightweight.
- 02
Centrally validated integration
External or third-party systemsChanges pass through an extra validation layer and a canonical database. Invalid changes trigger retries, reprocessing and compensating actions, subject to the source system's capabilities.
The second approach gives stronger control over shared data at the cost of additional operational complexity. It is a deliberate trade-off, used only where validation cannot be controlled at the source.
✦ Architecture
Conceptual architecture
A conceptual architecture, not an infrastructure deployment diagram. Compensating actions depend on source-system capabilities and are not an atomic distributed rollback.
Drag to pan · Ctrl/⌘ + scroll or pinch to zoom · click a block to trace its connections
✦ Technology
The stack, and why
- Apache KafkaShared event backbone that decouples systems.
- ADAWS DMSChange data capture from legacy databases.
- Oracle GoldenGateChange data capture where Oracle databases are involved.
- TemporalWorkflow orchestration where required.
- DatadogMonitoring and operational visibility.
Hover or tap a technology to see why it is there.
All key engineering decisions
| Technology / pattern | Why it matters |
|---|---|
| Canonical Data Contracts | Standardise business information and explicitly define which system owns each attribute |
| Apache Kafka | Decouple systems through asynchronous event communication |
| Integration Bridges | Isolate system-specific transformations and enable independent deployment and scaling |
| AWS DMS / Oracle GoldenGate | Support change data capture from legacy databases without major changes to existing applications |
| Outbox / Inbox + Idempotency | Support at-least-once delivery, safe reprocessing and recovery from failures |
| Temporal / Datadog | Support workflow orchestration where required, alongside monitoring and operational visibility |
✦ Technical highlight
Reusable integration infrastructure
The most important decision was not the broker or the database. It was separating reusable integration infrastructure from system-specific business logic.
A shared library carries the common capabilities, so engineers connecting another system concentrate on business rules, mappings and source-specific behaviour instead of rebuilding reliability.
- DeliverReliable message delivery and persistence.
- RecoverConfigurable retries and automatic recovery.
- DeduplicateSHA-256 payload hashes, entity-level ordering, idempotent processing.
- ObserveStructured logging, health checks and operational metrics.
Consumers also scale independently through buffered Inbox queues, which matters when systems have very different processing capacities.
✦ Evidence
The numbers
Arithmetic model: point-to-point needs n(n−1)/2 connections; a bridge per system needs n.
Engineering simulation, not a measured delivery-time improvement. A potential 63% reduction when reusing the shared library.
Calculated from the measured rate of 600,000 records/hour (~167 per second). The rate itself comes from an acceptance test on the complete current timetable dataset, not from sustained production throughput.
✦ Results & impact
What we can say, and what we can't yet
The engineering team reports that the canonical integration platform is implemented and operating across five connected systems, covering student records, academic offerings and direct-benefit information. The timetable integration was separately validated through acceptance testing.
| Metric | Result | Evidence |
|---|---|---|
| Enterprise systems connected | 5 | Team-reported |
| Timetable processing throughput | 600,000 records/hour | Measured · acceptance test |
| Integration approaches | 2 | Delivered |
| Estimated effort for a new bridge | 38 → 14 working days | Modelled scenario |
Actual production success rates, incident reductions and financial savings have not yet been quantified. The throughput figure was an acceptance-test result on the complete current dataset, not a measure of sustained production throughput across the platform.
✦ What this project demonstrates
Principles that travel
A complex integration problem can be addressed without replacing business-critical applications. The value lies in combining data governance, reusable engineering and operational resilience in one architecture.
The same approach is relevant to ERPs, CRMs, financial systems, SaaS platforms and legacy applications.
- Define data ownership before moving data
- Integrate different systems in different ways, on purpose
- Separate reusable reliability from business logic
- Make failure a normal, recoverable state
✦ Your turn
Five systems, five dialects, one meeting.
Bring the tangle of connections you have outgrown. We will bring the translator, and a plan that leaves your existing applications exactly where they are.
Next case studyOne integrated operations platform in five months, without changing how people work overnight→