Object serialization

What is object serialization ?

　 serialize (Serialization) It is the process of transforming the state information of an object into a form that can be stored or transmitted . 
　 This is the description of Baidu Encyclopedia , Also very easy to understand , for instance , I have one java object , I want to transmit to the remote program , How to transfer data ? 
　java Provides a serialization interface , As long as it's done Serializable Interface , You can java The object is serialized into bytes , Then it is transmitted as a stream , Then it is de sequenced into objects at the remote end , This achieves the purpose of transmitting messages . 
　 But this serialization has some drawbacks ,1 No cross language ,2, Large serialization volume  
　 So to support cross language , And improve sequence efficiency , Reduce the code stream size during transmission , Many companies or great gods have developed various serialization libraries ,hession,protostuff,kryo,jackson,xml etc. , such as json It is also a kind of serialization , The way , It is also the most widely used one at present , however json Strictly speaking, it is not a serialization , It is just a general way to describe information , A group of json Information , May correspond to different object patterns . For example, you can put a group json convert to java bean Objects can also be converted to map, The form is uncertain . Therefore, the general remote call will not adopt json As a serialization implementation .

Sequence type	Cross language	Advantages and disadvantages
hession	Support	Cross language , Small size after serialization , Faster
protostuff	Support	Cross language , Small size after serialization , Fast , But need Schema, It can generate
jackson	Support	Cross language , Small size after serialization , Faster , And with uncertainty
fastjson	Support	Cross language support is difficult , Small size after serialization , Faster , Only support java,c#
kryo	Support	Cross language support is difficult , Small size after serialization , Faster
fst	I won't support it	Cross language support is difficult , Small size after serialization , Faster , compatible jdk

|jdk| I won't support it | Large volume after serialization , Fast |

What serialization can do ?

As mentioned in the first section , Object serialization can be used to transfer messages , And in the form of streaming , Improve transmission efficiency , Use scenarios such as remote service invocation rpc

The serialization libraries are compared

0. Be prepared to rely on

        <!-- protostuff -->
        <dependency>
            <groupId>com.dyuproject.protostuff</groupId>
            <artifactId>protostuff-core</artifactId>
            <version>1.0.8</version>
        </dependency>
        <dependency>
            <groupId>com.dyuproject.protostuff</groupId>
            <artifactId>protostuff-runtime</artifactId>
            <version>1.0.8</version>
        </dependency>
        <!-- objenesis(support protostuff, You can use the parameterless construction method ) -->
        <dependency>
            <groupId>org.objenesis</groupId>
            <artifactId>objenesis</artifactId>
            <version>2.1</version>
        </dependency>
        <!-- hessian -->
        <dependency>
            <groupId>com.caucho</groupId>
            <artifactId>hessian</artifactId>
            <version>4.0.38</version>
        </dependency>
        <!-- jackson -->        
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
      <version>2.9.0</version>
        </dependency>
<!-- fastjson-->        
<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>fastjson</artifactId>
    <version>1.2.47</version>
</dependency>


                <!-- kryo  -->
            <dependency>
        <groupId>com.esotericsoftware</groupId>
        <artifactId>kryo</artifactId>
        <version>4.0.0</version>
    </dependency>
         <!-- fst  -->
    <dependency>
  <groupId>de.ruedigermoeller</groupId>
  <artifactId>fst</artifactId>
<version>2.57</version>
</dependency>

1. Write a serialization interface

public interface Serializer {

    <T> byte[] serialize(T obj);
    <T> Object deserialize(byte[] bytes, Class<T> clazz);
}

2. Implementation interface

hession Realization

public class HessianSerializer implements Serializer {

    @Override
    public <T> byte[] serialize(T obj){
        ByteArrayOutputStream os = new ByteArrayOutputStream();
        HessianOutput ho = new HessianOutput(os);
        try {
            ho.writeObject(obj);
        } catch (IOException e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
        return os.toByteArray();
    }

    @Override
    public <T> Object deserialize(byte[] bytes, Class<T> clazz) {
        ByteArrayInputStream is = new ByteArrayInputStream(bytes);
        HessianInput hi = new HessianInput(is);
        try {
            return hi.readObject();
        } catch (IOException e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }

}

jackson Realization

public class JacksonSerializer implements Serializer {
    private final static ObjectMapper objectMapper = new ObjectMapper();


    @Override
    public <T> byte[] serialize(T obj) {
        try {
            return objectMapper.writeValueAsBytes(obj);
        } catch (JsonProcessingException e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }
    @Override
    public <T> Object deserialize(byte[] bytes, Class<T> clazz)  {
        try {
            return objectMapper.readValue(bytes, clazz);
        } catch (JsonParseException e) {
            throw new IllegalStateException(e.getMessage(), e);
        } catch (JsonMappingException e) {
            throw new IllegalStateException(e.getMessage(), e);
        } catch (IOException e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }

}

fastjson Realization

import com.alibaba.fastjson.JSON;

public class FastJsonSerializer implements Serializer {


    @Override
    public <T> byte[] serialize(T obj) {
        try {
            return JSON.toJSONBytes(obj);
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }
    @Override
    public <T> Object deserialize(byte[] bytes, Class<T> clazz)  {
        try {
            return JSON.parseObject(bytes, clazz);
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }

}

prostfuff Realization

public class ProtostuffSerializer implements Serializer {
    private static Objenesis objenesis = new ObjenesisStd(true);
    private static Map<Class<?>, Schema<?>> cachedSchema = new ConcurrentHashMap<Class<?>, Schema<?>>();
    private static <T> Schema<T> getSchema(Class<T> cls) {
        @SuppressWarnings("unchecked")
        Schema<T> schema = (Schema<T>) cachedSchema.get(cls);
        if (schema == null) {
            schema = RuntimeSchema.createFrom(cls);
            if (schema != null) {
                cachedSchema.put(cls, schema);
            }
        }
        return schema;
    }

    @Override
    public <T> byte[] serialize(T obj) {
        @SuppressWarnings("unchecked")
        Class<T> cls = (Class<T>) obj.getClass();
        LinkedBuffer buffer = LinkedBuffer.allocate(LinkedBuffer.DEFAULT_BUFFER_SIZE);
        try {
            Schema<T> schema = getSchema(cls);
            return ProtostuffIOUtil.toByteArray(obj, schema, buffer);
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        } finally {
            buffer.clear();
        }
    }

    @Override
    public <T> Object deserialize(byte[] bytes, Class<T> clazz) {
        try {
            T message = (T) objenesis.newInstance(clazz);
            Schema<T> schema = getSchema(clazz);
            ProtostuffIOUtil.mergeFrom(bytes, message, schema);
            return message;
        } catch (Exception e) {
            throw new IllegalStateException(e.getMessage(), e);
        }
    }

}

kryo Realization

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import com.esotericsoftware.kryo.pool.KryoFactory;
import com.esotericsoftware.kryo.pool.KryoPool;

/**
 * @author  Urban Taoyuan 
 *
 */
public class KryoSerializer implements Serializer {

    /**
     * kryo  Not thread safe , So use pool control 
     */
    private static final KryoPool kryoPool = new KryoPool.Builder(
            new KryoFactory() {
                public Kryo create() {
                    Kryo kryo = new Kryo();
                    return kryo;

                }

            }).build();

    @Override
    public <T> byte[] serialize(T obj) throws Exception {
        try (
                Output output = new Output(new ByteArrayOutputStream())) {
            Kryo kryo = kryoPool.borrow();
            kryo.writeObject(output, obj);
            kryoPool.release(kryo);
            output.flush();
            return ((ByteArrayOutputStream) output.getOutputStream()).toByteArray();

        }

    }

    @Override
    public <T> Object deserialize(byte[] bytes, Class<T> clazz)
            throws Exception {
        try (Input input = new Input(new ByteArrayInputStream(bytes))) {
            Kryo kryo = kryoPool.borrow();
            T res = kryo.readObject(input, clazz);
            kryoPool.release(kryo);
            return res;
        }
    }
}

jdk Realization

public class JdkSerializer implements Serializer {

    @Override
    public <T> byte[] serialize(T obj) {

        try {
            ByteArrayOutputStream byteArr = new ByteArrayOutputStream();
            ObjectOutputStream out=new ObjectOutputStream(byteArr);
            out.writeObject(obj);
            out.flush();
            return byteArr.toByteArray();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return null;
    }

    @Override
    public <T> Object deserialize(byte[] bytes, Class<T> clazz) {
        try {
            ObjectInputStream input=new ObjectInputStream(new ByteArrayInputStream(bytes));
            return input.readObject();
        } catch (Exception e) {
            e.printStackTrace();
        }

        return null;
    }

}

fst Realization

import org.nustaq.serialization.FSTConfiguration;

public class FstSerializer implements Serializer {
    private static final FSTConfiguration configuration = FSTConfiguration
            .createStructConfiguration();

    @Override
    public <T> byte[] serialize(T obj) {
        return configuration.asByteArray(obj);
    }

    @Override
    public <T> Object deserialize(byte[] bytes, Class<T> clazz) {
        return configuration.asObject(bytes);
    }
}

3, Prepare sequence objects

Serialized objects need to implement Serializable, Some need it, some don't ,hession need , You also need to have a default parameterless constructor jackson need

public class User  implements Serializable{
    private String name;
    private String address;
    private Integer age;

    public User() {
        super();
    }
    public String getName() {
        return name;
    }
    public void setName(String name) {
        this.name = name;
    }
    public String getAddress() {
        return address;
    }
    public void setAddress(String address) {
        this.address = address;
    }
    public Integer getAge() {
        return age;
    }
    public void setAge(Integer age) {
        this.age = age;
    }
    public User(String name, String address, Integer age) {
        super();
        this.name = name;
        this.address = address;
        this.age = age;
    }

}

4. Serialization speed comparison

public class SerTest {
    public static void main(String[] args) {

        int len=10;
        test(new HessianSerializer(), "hession",len);
        test(new JacksonSerializer(), "jackson",len);
        test(new ProtostuffSerializer(),  "protostu",len);
        test(new JdkSerializer(),"jdk",len);
        test(new KryoSerializer(),"kryo",len);
        test(new FstSerializer(),"fst",len);
        test(new FastJsonSerializer(),"fastjson",len);
    }

    public static void test(Serializer s,String tag,int len){

        long cap=0;
        long sumTime=0;
        for(int i=0;i<len;i++){
            User u=new User("taoyuan"+i, i+"addr", i);
            long start=System.currentTimeMillis();
            byte[] serialize = s.serialize(u);
             cap+=serialize.length;
             s.deserialize(serialize, User.class);
             long cal= System.currentTimeMillis()-start;
             sumTime+=cal;

        }
          System.out.println(tag+"-->["+len+"] Total volume of object sequence :"+cap);
         System.out.println(tag+"->["+len+"] The total elapsed time of the object is :"+sumTime);
    }
}

result : 
hession–>[100] Total volume of object sequence :7780 
hession->[100] The total elapsed time of the object is :197 
jackson–>[100] Total volume of object sequence :4770 
jackson->[100] The total elapsed time of the object is :122 
protostu–>[100] Total volume of object sequence :2080 
protostu->[100] The total elapsed time of the object is :102 
jdk–>[100] Total volume of object sequence :21380 
jdk->[100] The total elapsed time of the object is :122 
kryo–>[100] Total volume of object sequence :2016 
kryo->[100] The total elapsed time of the object is :47 
fst–>[100] Total volume of object sequence :4880 
fst->[100] The total elapsed time of the object is :35 
fastjson–>[100] Total volume of object sequence :4770 
fastjson->[100] The total elapsed time of the object is :206

5 Conclusion

From the above results , The best serialization performance is protostuff and kryo 
jdk The serialization speed of is relatively fast , That is, the volume of the sequence is large . 
So when we need to develop a set rpc The service , May adopt kryo To serialize objects , Improve transmission efficiency

rpc Brief introduction to the principle of simple service implementation

Personal understanding

To achieve a set of rpc service , First of all, we have to understand our essence rpc 
rpc Simply put, serialization + Transfer protocol + Call the implementation + Service management

Serialization is the foundation and determines rpc Whether it supports cross language  
The transport protocol can be selected tcp,http,udp Such agreement  
Call implementation i.e rpc Business implementation of services  
Service management is the optimized management level based on the above , Load balancing , Call tracing, etc

Once we understand in essence rpc after , Write a simple rpc Service is also a very simple thing , Analyze and optimize the specific business , Each layer is separated , To optimize the , The changes won't be too big ,, Have the opportunity to , Let's write an article , Hand to hand teaching you how to achieve rpc 's blog post