Skip to content

Shadow systems via duplication

Summary

Shadow systems via duplication appear when people copy data out of an official system and maintain it somewhere else.

This often starts as a practical solution:

  • “The CRM doesn’t show what we need.”
  • “The report is too limited.”
  • “We just need a quick Excel to manage this.”

So data is exported. A spreadsheet is created. A second tracker appears.

Over time, the copy becomes operational.

Now there are two systems holding similar information.

Both are used. Both are updated. Neither is fully trusted.

Touches lenses:
Information, Integration, Processing

Primary system feeding duplicated side spreadsheets

A simple example

Imagine a company that manages customers in a CRM.

The CRM tracks:

  • Customer name
  • Contract value
  • Status
  • Renewal date

The sales manager feels the CRM does not provide enough visibility into renewal risk.

So she exports all customers to Excel and adds new columns:

  • Risk score
  • Last call notes
  • Probability of renewal

At first, the spreadsheet is just an analysis tool.

But then:

  • Risk scores are updated only in Excel.
  • Sales discussions happen based on the spreadsheet.
  • The CRM is updated later — or not at all.

Now the spreadsheet is no longer just a report.

It is a shadow system.

If management asks:

“What is our renewal forecast?”

The answer depends on whether you look in the CRM or in the spreadsheet.

This is shadow duplication.

Recognition

You may have shadow systems via duplication if:

  • Teams regularly export data “to work with it properly”
  • Important decisions are based on spreadsheets, not the core system
  • Two files contain similar but not identical information
  • People say “the system is not flexible enough”
  • Reconciliation between files happens monthly

There is usually no malicious intent.

Shadow systems are created to get work done.

But they introduce structural risk.

What is really breaking

The problem is not that people use Excel.

The problem is that structure is being rebuilt outside the system.

When someone duplicates data, they often also:

  • Add new fields
  • Add informal rules
  • Recalculate values
  • Redefine statuses

These changes live only in the copy.

The official system remains unchanged.

Over time:

  • The duplicated version evolves.
  • The original version evolves separately.
  • There is no synchronization logic.

The two structures drift apart.

Two diverging data blocks evolving separately

Why this happens

Shadow duplication usually appears when:

  1. The core system cannot represent an important concept.
  2. A report cannot answer a key question.
  3. The workflow does not match reality.
  4. Access permissions block necessary edits.

Instead of changing the structure, people duplicate the data.

This feels safer and faster.

In many SMBs, this is common:

  • CRM + “Sales master.xlsx”
  • ERP + “Operations tracker.xlsx”
  • Accounting + “Management adjustments.xlsx”

The official system handles transactions. The spreadsheet handles understanding.

But the spreadsheet slowly becomes operational.

Once that happens, the organization now has parallel truth sources.

Why it is hard to see

Shadow systems often look harmless.

  • They start as temporary.
  • They sit in someone’s private folder.
  • They are shared informally.
  • They are described as “just a helper file.”

There is no error message.

The core system still functions.

The shadow system does not replace it entirely.

Instead, it complements it — quietly.

The real problem appears when:

  • A key employee leaves.
  • A file is outdated.
  • A number must be audited.
  • Two reports disagree.

Then it becomes clear that critical logic exists outside the official system.

A second example: pricing adjustments

Imagine an SMB that uses an ERP system for invoicing.

The ERP calculates price based on:

  • Product price
  • Discount
  • VAT

But the sales team often negotiates custom bundle deals.

Instead of updating the ERP pricing model, they:

  • Export invoices
  • Adjust pricing in a spreadsheet
  • Track “real deal values” separately

Finance reports revenue from the ERP. Sales reports revenue from the spreadsheet.

Both numbers are defensible. Neither is fully aligned.

When someone searches online:

  • “Why do CRM and accounting numbers not match?”
  • “Why do dashboards show inconsistent totals?”
  • “Why do reports differ between systems?”

Shadow duplication is often the cause.

Flow from primary system to duplicate to reconciliation loop

What it costs

Shadow systems via duplication create several long-term risks:

1. Loss of single source of truth

There is no clear owner of meaning.

2. Manual reconciliation

Time is spent comparing files instead of improving structure.

3. Hidden business rules

Important logic exists in cells, formulas, and notes.

4. Fragile knowledge

When the file owner leaves, the logic disappears.

5. Reduced automation

If key values live outside the system, automation cannot rely on them.

For SMBs, the cost is often invisible at first.

But as the business grows:

  • More exports are created.
  • More corrections are layered.
  • More meetings focus on “which number is right.”

The operational complexity increases without formal architecture.

How to test for it

Ask these questions:

  1. Are key decisions ever based on exported files instead of the system?
  2. Do spreadsheets contain fields that do not exist in the core system?
  3. Would operations break if a private Excel file disappeared?
  4. Are two teams maintaining overlapping data independently?

If the answer to any of these is yes, you likely have shadow duplication.

Another test:

If we delete all spreadsheets tomorrow, can the business still operate correctly?

If not, then the shadow system is not optional. It is structural.

Structural explanation

Every business system tries to model reality.

When that model is incomplete, people compensate.

Duplication is a workaround for missing structure.

Instead of expanding the system model, the organization creates a parallel one.

The key issue is not duplication itself. It is uncontrolled duplication.

When two systems hold the same entity — for example:

  • Customer
  • Order
  • Revenue
  • Contract
  • Inventory

but allow independent updates, you now have multiple state machines.

Without synchronization logic, divergence is inevitable.

This is not a people problem. It is a structural design gap.

How to resolve it

Fixing shadow systems does not mean banning spreadsheets.

It means restoring structural alignment.

Possible steps include:

  • Identifying duplicated entities
  • Listing fields that only exist in spreadsheets
  • Moving critical logic into the official system
  • Creating proper reporting views instead of exports
  • Clarifying ownership of each data concept

In some cases, the core system must be extended.

In other cases, a lightweight secondary system can exist — but with clear integration and ownership.

The goal is not centralization at any cost.

The goal is controlled structure.

If a spreadsheet contains business-critical logic, that logic should be modeled formally.

When structure is explicit:

  • Reports align
  • Automation becomes reliable
  • Trust increases

In short

Shadow systems via duplication happen when:

  • The official system is incomplete
  • People copy data to compensate
  • The copy becomes operational
  • Both versions evolve separately

The name of the file may say “temporary.”

But once decisions depend on it, it is no longer temporary.

The solution is not discipline alone.

The solution is structural clarity.