FST

This article introduces FST serialization

1 Introduction

FST serialization stands for Fast Serialization, which is a replacement implementation for Java serialization. Since the previous discussion highlighted two serious shortcomings of Java serialization, these have been significantly improved in FST. The characteristics of FST are as follows:

  1. Improved performance by 10 times compared to JDK serialization, with size reduction of more than 3-4 times
  2. Supports off-heap Maps and the persistence of off-heap Maps
  3. Supports serialization to JSON

2 Usage

2.1 Adding Dependencies

  1. <dependencies>
  2.     <dependency>
  3.       <groupId>org.apache.dubbo.extensions</groupId>
  4.       <artifactId>dubbo-serialization-fst</artifactId>
  5.       <version>3.3.0</version>
  6.     </dependency>
  7.     <dependency>
  8.         <groupId>de.ruedigermoeller</groupId>
  9.         <artifactId>fst</artifactId>
  10.         <version>3.0.3</version>
  11.     </dependency>
  12. </dependencies>

2.2 Configuring Activation

  1. # application.yml (Spring Boot)
  2. dubbo:
  3.  protocol:
  4.    serialization: fst

or

  1. # dubbo.properties
  2. dubbo.protocol.serialization=fst
  4. # or
  5. dubbo.consumer.serialization=fst
  7. # or
  8. dubbo.reference.com.demo.DemoService.serialization=fst

or

  1. <dubbo:protocol serialization="fst" />
  3.         <!-- or -->
  4. <dubbo:consumer serialization="fst" />
  6.         <!-- or -->
  7. <dubbo:reference interface="xxx" serialization="fst" />

3 Registering Serializable Classes

To fully leverage the high performance of Kryo and FST, it is best to register the classes that need to be serialized into the Dubbo system, implemented as follows.

Callback Interface

  1. public class SerializationOptimizerImpl implements SerializationOptimizer {
  3.     public Collection<Class> getSerializableClasses() {
  4.         List<Class> classes = new LinkedList<Class>();
  5.         classes.add(BidRequest.class);
  6.         classes.add(BidResponse.class);
  7.         classes.add(Device.class);
  8.         classes.add(Geo.class);
  9.         classes.add(Impression.class);
  10.         classes.add(SeatBid.class);
  11.         return classes;
  12.     }
  13. }

Then add in the XML configuration:

  1. <dubbo:protocol name="dubbo" serialization="kryo" optimizer="org.apache.dubbo.demo.SerializationOptimizerImpl"/>

After registering these classes, serialization performance may be greatly enhanced, particularly for small numbers of nested objects.

Of course, when serializing a class, it might also reference many other classes, such as Java collection classes.

In this case, we have automatically registered commonly used classes from the JDK, so you don’t need to register them again (though registering them again has no impact).

Including

  1. GregorianCalendar
  2. InvocationHandler
  3. BigDecimal
  4. BigInteger
  5. Pattern
  6. BitSet
  7. URI
  8. UUID
  9. HashMap
  10. ArrayList
  11. LinkedList
  12. HashSet
  13. TreeSet
  14. Hashtable
  15. Date
  16. Calendar
  17. ConcurrentHashMap
  18. SimpleDateFormat
  19. Vector
  20. BitSet
  21. StringBuffer
  22. StringBuilder
  23. Object
  24. Object[]
  25. String[]
  26. byte[]
  27. char[]
  28. int[]
  29. float[]
  30. double[]

Since registering the serializable classes is solely for performance optimization, it doesn’t matter if you forget to register some classes.

In fact, even without registering any classes, the performance of Kryo and FST is generally superior to that of Hessian and Dubbo serialization.

Of course, some may ask why we don’t use a configuration file to register these classes. This is because the number of classes to be registered is often large, leading to lengthy configuration files; furthermore, without good IDE support, writing and refactoring configuration files is much more cumbersome than dealing with Java classes; finally, these registered classes generally do not need to be dynamically modified after the project is compiled and packaged.

Additionally, some might find manually registering serializable classes relatively cumbersome and wonder if annotations could be used to mark them for automatic discovery and registration. However, the limitation of annotations here is that they can only mark classes that you can modify, while many classes referenced in serialization may be unmodifiable (e.g., from third-party libraries, JDK system classes, or classes from other projects). Moreover, adding annotations does slightly “pollute” the code, increasing the dependency of application code on the framework.

Apart from annotations, we could also consider other ways to automatically register serializable classes, such as scanning the classpath to automatically discover and register classes that implement the Serializable interface (including Externalizable). Of course, we know that there could be a large number of Serializable classes found in the classpath, so using package prefixes to some extent to limit the scanning range could also be considered.

Of course, in an automatic registration mechanism, it is particularly important to ensure that the provider and consumer register classes in the same order (or ID) to avoid misalignment, as the number of classes discoverable and registrable on both ends may be different.

No-Argument Constructor and Serializable Interface

If a serializable class does not include a no-argument constructor, its performance in Kryo serialization will be significantly reduced, as we will transparently use Java serialization to replace Kryo serialization. Therefore, adding a no-argument constructor to each serializable class is considered a best practice (of course, a Java class will have a no-argument constructor by default if no custom constructors are defined).

Moreover, while Kryo and FST don’t require the serializable classes to implement the Serializable interface, we still recommend that each serializable class does so, as this ensures compatibility with Java serialization and Dubbo serialization, thus potentially easing the use of some of the aforementioned automatic registration mechanisms in the future.

Feedback

Was this page helpful?

Yes No

Last modified September 30, 2024: Update & Translate Overview Docs (#3040) (d37ebceaea7)