Egil Hansen's .com

Evolving your C# types without migrating stored JSON

Data models evolve, but the JSON payloads serialized from the old shapes do not go away. They sit at rest in databases, document stores, caches, queues, and on disk, and they keep arriving over the wire from APIs, Event Hubs, and other systems that have not upgraded yet. The moment I rename a property or restructure a type, deserialization of all that existing JSON breaks.

My previous approach was to keep the old C# types around and map manually after deserialization: deserialize as UserV1, then convert to UserV2 somewhere downstream. It works, but the migration plumbing leaks into business logic and repository code, and every read path needs to know which historical shapes might show up.

Egil.SystemTextJson.Migration is my attempt at solving this properly. The idea: migration happens during deserialization, no matter where the payload came from. The old types stay in the codebase as a historical record of past schemas, the migration logic is declared once next to them, and the rest of the codebase only ever sees the type it wants to work with. Data at rest never needs to be touched, and senders and receivers do not have to upgrade in lockstep.

Here is the smallest possible example. The old and current shape of a User, with the migration declared on the current type:

[JsonMigratable(TypeDiscriminator = "user-v1")]
public record UserV1(string Name, int Age);

[JsonMigratable(TypeDiscriminator = "user-v2")]
public record UserV2(string FirstName, string LastName, int Age)
    : IMigrateFrom<UserV1, UserV2>
{
    public static bool TryMigrateFrom(UserV1 source, out UserV2 result)
    {
        var names = source.Name.Split(' ');
        result = new UserV2(names[0], names.ElementAtOrDefault(1) ?? "", source.Age);
        return true;
    }
}

Enable migration support on the serializer options and deserialize as usual:

var options = new JsonSerializerOptions(JsonSerializerDefaults.Web);
options.AddJsonMigrationSupport();

// A UserV1 payload is automatically migrated to UserV2:
var json = """{"$type":"user-v1","name":"Jane Doe","age":30}""";
UserV2 user = JsonSerializer.Deserialize<UserV2>(json, options)!;

That is the whole programming model. The getting started documentation and the recipes cover the rest, nested types, collections, dependency injection, source generation, failure handling, and more, so I will not repeat that here.

The migration logic does not have to live on the type itself either, it can go in a separate external migrator class instead, which keeps the data types clean and lets you add migrations to types you do not own.

Let’s talk about when this library is useful.

Use case: data at rest

This is the obvious one. JSON documents in a database, events in an event store, entries in a cache, messages parked in a queue. When the C# type changes, none of that data has to be rewritten. Add a migration from the old shape to the new one, and every read transparently produces the current type. The old types are kept around, but they are inert, no business logic references them, they only exist as input to migrations.

This means C# code can move forward at its own pace. A schema change is a new type version plus a small migration function, not a data migration project.

My workflow for local types: when a shape changes, I manually rename the old type (e.g. Customer → CustomerV1) without using the C# rename refactoring. I then add the new Customer type in the same namespace with the customer-v2 discriminator. Existing code keeps pointing to the clean Customer name, and the compiler flags exactly what needs updating. The canonical version always keeps the clean name, so it is immediately recognizable, and I name the file Customer.v1.cs so the historical versions stay colocated with the current Customer.cs.

A word of caution: this only works for types that are local to my own codebase. Renaming a type in a library shared with external users is a breaking change for them, so there the new version gets the new name instead, more on that in the next section.

Use case: contract libraries

A system publishes data to an Event Hub or exposes it via an API, and consumers use a shared .NET contract library to deserialize the payloads. Normally a breaking change to the contract forces every consumer to update their code at the same time as they update the contract library package.

With migration support, the contract library can ship both the old and the new types, with migrations between them in both directions, the library does not care which type is “newer”. Consumers update to the latest package version but keep coding against the old types; nothing in their codebase changes. Once consumers are on the new contract library package, the producer can start emitting the new payload shape, and it is automatically migrated at the receiving end to whatever type each consumer deserializes to. Consumers then move to the new types whenever it suits them.

A useful side effect: if a payload arrives with a discriminator the consumer’s version of the contract library does not know and cannot migrate from, deserialization throws a JsonException by default. Instead of silently producing a half-populated object, an incompatible payload shape going over the wire fails loud and explicit at the consumer side. This is configurable, you can opt to swallow the error and attempt a best-effort deserialization to the target type instead, globally or per type.

Use case: blue/green deployments

In a blue/green or rolling deployment, two versions of a distributed application run side by side, and they exchange messages with different versions of the same DTO. Instead of carefully sequencing the rollout so that no old-shape message ever reaches a new node (or the reverse), each side simply migrates incoming payloads to the DTO version it understands. The deployment order stops being a serialization concern. The only requirement is that all running versions of the application know about all payload types. This is usually done in two phases: first, deploy a version that introduces the new types and migrations but doesn’t emit them yet; then, deploy the version that starts sending the new shapes.

Use case: version-tolerant APIs

Migrations attack the payload shape directly, so one endpoint (or one consumer) can accept several historical shapes. Clients posting data to an API do not need version-specific endpoints, e.g. /v1/... and /v2/..., as long as the payload carries its type discriminator, they can submit any supported version, and the server migrates it to the type the endpoint expects automatically.

The advantage on the server side: the endpoint logic only ever deals with the latest version of the incoming payload. No branching on payload versions, no manual migration logic in handlers or controllers, older shapes are migrated before the endpoint code sees them, so the code stays clean and clients can upgrade at their own pace.

How does this compare to the alternatives?

Big-bang database migrations. Rewriting all stored JSON in a migration script is risky and slow on large datasets, and usually means downtime or a carefully choreographed deployment. Database triggers and background migration jobs reduce the risk but add operational machinery, and while the migration is running, the code that reads the JSON still has to support both versions until all data is migrated.

Hand-written migrate-on-read. This was my go-to before: keep the old types around, deserialize, map manually to the current type, and write the migrated JSON back to storage so each payload only pays the cost once. It is essentially the strategy this library automates, but hand-rolled, the version-handling code spreads across every read path, and it is easy to miss one. Declaring the migration once, next to the types, keeps it in a single place and out of the business logic, and the library’s migration tracking tells you which instances were migrated so you can keep the opportunistic write-back.

Custom JsonConverters. Writing a converter that reads the old format directly into the new type is the highest-performance option, no intermediate old-type instance, no copying data over. But writing custom converters by hand is cumbersome and easy to get wrong, especially for complex types, so I only consider it worth it when the absolute best performance is required. This library makes the opposite trade-off: accept the small cost of deserializing to the old type and migrating, and then write the upgraded payload back to storage afterwards, migration tracking tells you when to do this, so the migration work is done once per payload, not on every read.

Versioned API endpoints. Running /v1/ and /v2/ endpoints side by side works for request/response APIs, but it does nothing for data at rest or for messages already sitting in a queue, and every parallel version multiplies the surface area you maintain. With migrations, a single endpoint accepts several historical shapes instead.

Adopting it in a brownfield project

Existing projects are the interesting case, because the JSON that is already stored was written without a $type discriminator. There are two on-ramps:

Either way, adoption is incremental, one type at a time, no flag day. And because migration logic can live in an external migrator class, you can even add migrations for types you do not own, for example, types from a third-party package whose payloads are sitting in your database.

Trade-offs

There is no free lunch:

There is negligible performance impact on happy-path serialization and deserialization: the discriminator check is O(1), only the first JSON property is inspected, and the happy path benchmarks close to plain System.Text.Json throughput with zero extra library allocations. See the benchmarks in the README for details.

Feedback welcome

I have been running this in production for a couple of months now on a large project doing around 20 million reads and writes to blob storage every day, so the library has had a decent shake-out. That said, I am sure there are use cases and edge cases I have not hit yet. If you try it out and something does not work as expected, or you have ideas for improvements, please open an issue or leave a comment below, feedback and suggestions are very welcome.

Hope this helps.

Comments