[POC] FieldSerializer for Records

[theigl]

This is a POC for moving record serialization from RecordSerializer into FieldSerializer.

This approach has two main advantages:

1. Existing configuration options for the FieldSerializer can be used for records
2. Sub-classes of FieldSerializer (Compatible, Versioned, Tagged, ..) can be adjusted for records

JDK 17+ is required for this PR so it is a likely candidate for the next major version.

[theigl]

See #884 for the discussion about RecordSerializer vs FieldSerializer for records.

[julianhowarth]

Sorry to jump on a PR - let me know if I should post somewhere else.

I'm currently looking at serializing records with Kryo with the requirement that I have forward/backward compatibility. I've tested using this branch and it does exactly what I need, so I'd like to know the best way to move forward:

• wait for the next major release which includes this change, but I haven't seen anything published as to when that might be
• fork the repo and build my own version separately but I'd rather avoid this if possible
• extract the record serialization and create a separate serializer I can use for records, with the intention to remove it when the next major version is released

I appreciate any advice on the best way forward

[theigl]

Hi @julianhowarth! Thanks for reaching out and sorry for the late reply.

I've tested using this branch and it does exactly what I need

Can you tell me what kind of tests you performed? One of the main reasons (apart from lack of time) for not moving forward with this PR is that I wasn't sure if it really works in most real world use cases. If you performed realistic tests, that would improve my confidence in these changes.

So I'd like to know the best way to move forward

I would really like to do a release candidate for Kryo 6. The problem is that I'm currently the sole maintainer and my time to work on Kryo is quite limited. Kryo 5 is extremely stable and there hasn't been much demand for a next major release so progress has stalled a bit. If you are really interested in this feature and want to test it in production, I can try to set aside some time and build Kryo 6.0.0.RC1.

[julianhowarth]

Hi, sorry for my late reply

Can you tell me what kind of tests you performed? One of the main reasons (apart from lack of time) for not moving forward with this PR is that I wasn't sure if it really works in most real world use cases. If you performed realistic tests, that would improve my confidence in these changes.

So my application is around video streaming - it's been in production in various guises for 8+ years. Domain classes are things like descriptive text localised for different territories, video source information, image urls, durations of videos, timings of live events, series and seasons of videos etc. In total there are just over 100 classes most of which are Java records.

So the primary testing I have performed is that our production dataset (which as a gzipped json file is ~500MB containing ~500,000 video assets) serializes and deserializes with the Kryo snapshot. It does so successfully in 5 - 10s generally.

Whilst I have checked simple field addition in the root records, where I think I need to do a bit more is to check that adding fields further down in the nested structure also works as expected, but I can start working on that testing now.

I would really like to do a release candidate for Kryo 6. The problem is that I'm currently the sole maintainer and my time to work on Kryo is quite limited. Kryo 5 is extremely stable and there hasn't been much demand for a next major release so progress has stalled a bit. If you are really interested in this feature and want to test it in production, I can try to set aside some time and build Kryo 6.0.0.RC1.

So yes, I'd be very interested in trying out a release candidate as I'm now at the point where I need to work to finish off my serialisation component.