Key Takeaways:
-
The schedule hangs on approvals, not on run time. Planning a regulated migration by development days plans the smaller half.
-
Regulators wrote the requirement down long ago. EU GMP Annex 11 requires that data migrated to another format or system stays verifiably unchanged in value and meaning. The FDA's 21 CFR Part 11 demands audit trails for electronic records. Neither is new, and neither is optional.
-
Every transformation has to be justifiable. It is not enough that a value arrives correctly. It must be traceable which rule put it there.
-
Test data is especially difficult here. Real patient data must not sit in test environments. Synthetic data represents reality poorly. Both need planning.
-
Evidence is produced during the migration, not afterwards. Reconstructed proof is expensive and weak.
The main data migration risks in a regulated environment are not volume or technology. They are the burden of proof: for every migrated record it must later be traceable which source it came from, which rule changed it, and who approved that. This shifts effort out of development and into documentation and coordination, typically at a ratio of two to one against a comparable project outside regulation.
We are currently working on several applications in a medical context, and the contrast with our other projects is the same every time.
Why approvals set the pace
In an ordinary migration project the question is: does the import run? In a regulated one it is: who confirms that the import did the right thing, and how did that person recognize it?
That is not a formality. Annex 11, section 4.8 puts it in one sentence: "If data are transferred to another data format or system, validation should include checks that data are not altered in value and/or meaning during this migration process." Someone has to sign that check. Which means every test run has to produce an artifact a business-responsible person can read and judge. A log file with twelve thousand lines is not that. A report that sets counts, sums and deviations side by side per entity is.
Build that only at the end and you build it under time pressure, in a form nobody can check. Then it gets signed because the deadline is pressing. That is precisely the state regulation exists to prevent.
Transformations have to be justifiable
The most common finding in such projects is a transformation rule someone once introduced for a good reason that nobody can explain any more. A field gets truncated, a date shifted, a status value remapped.
Our approach: every rule carries one sentence of justification, next to the rule, not in a separate document. Whoever reads the rule reads the reason. That sounds trivial. It is the difference between a migration you can still defend in two years and one where you have to guess.
The test data problem
Real data does not belong in a test environment. Under GDPR Article 9, health data carries the strictest protection level there is, and US law draws the same line: HIPAA only releases patient data from its rules once it is de-identified under the standard in 45 CFR § 164.514. Synthetic data, on the other hand, does not know the special cases the migration fails on. That is not a contradiction you resolve. It is one you manage.
What works:
- Format-preserving anonymization instead of random values, so structure and distribution survive. The Article 29 Working Party opinion on anonymisation is the reference for what counts as anonymous and what is merely pseudonymized.
- A narrowly scoped real-data run in an environment with production-level protection, logged and time-limited.
- Special cases as a catalogue. Whatever is noticed in the real dataset gets described as a pattern and reproduced in the synthetic data. The property travels, not the record.
The third is the most valuable, because the catalogue grows across projects.
What that means for the plan
Three things we now build in from the start:
Approval loops as activities, not as buffer. If a business approval takes two weeks, the plan says two weeks. Not as a risk noted in the margin.
One check report per run, generated automatically. Part of the run, not follow-up work. Otherwise it no longer exists after the third run.
Roles clarified early. Who is responsible in business terms, who in data protection terms, who technically? These are rarely the same person. If one of them is missing, the project stalls somewhere nobody planned for.
The side effect
Projects in regulated environments are slower and more expensive. They also have a side effect we have come to appreciate: what emerges there in terms of traceability is useful in every other project too. A check report a business department can read helps everywhere. Outside regulation, it is just rarely demanded.
So we build our migration processes so the chain of evidence always runs alongside. Not because every project needs it, but because it cannot be created after the fact. If your migration has a regulator at the end of it, talk to us early. Early is when the evidence is cheap.
Sources
- European Commission: EU GMP Annex 11 — Computerised Systems, 2011
- FDA: 21 CFR Part 11 — Electronic Records; Electronic Signatures
- GDPR Article 9 — Processing of special categories of personal data
- HIPAA: 45 CFR § 164.514 — De-identification of protected health information
- Article 29 Working Party: Opinion 05/2014 on Anonymisation Techniques