当前位置:网站首页>Briefly describe the pitfalls of mobile IM development: architecture design, communication protocol and client
Briefly describe the pitfalls of mobile IM development: architecture design, communication protocol and client
2022-06-26 05:06:00 【JackJiang-】
1、 Preface
Developers who have experience in mobile terminal development have a deep understanding of it : Mobile IM Development of , With the traditional PC End IM There's a big difference , Unreliable wireless networks in particular 、 The limitation of mobile hardware resources , Leading to a complete mobile terminal IM Architecture design and implementation are full of a lot of challenges . This article will briefly describe the mobile terminal IM The most important aspects of architecture design and communication protocol selection , Hope for IM Some inspiration from developers .( This article is published synchronously in :http://www.52im.net/thread-289-1-1.html)
2、 Exchange of learning
- Instant messaging development communication group : 215891622 [ recommend ]
- Mobile IM Develop recommended articles :《 Beginner level one is enough : Develop mobile from scratch IM》
3、 summary
The advent of the mobile Internet era urges all our developers to start from the perspective of users , Create an application based on a specific scenario , Meet the needs of users . Usually , In these applications , Communication is essential . This requires entrepreneurs not only to spend time and energy to figure out what pain points users need in a particular scenario , Figure out how to address this need , And it may take more energy and time to solve the problems in the product “ communicate ” This technology node .
And to solve communication problems , You need a set IM System ( And it must support mobile terminals ). As IM Developers may soon become IM The developer's technicians ,IM The value and importance of . But in terms of technical implementation , It's not easy . Of course , Suppose you have 100 Users , Everything is easy , But suppose you have 100 ten thousand 、1000 Wan even 1 Million users , The simplest technical node cannot be solved well , Will become a disaster , Moreover, IM System ( Especially on the mobile end IM System ) There are still many technical difficulties and pits .
4、 About mobile IM Communication protocol
secondly , Let's see IM How to choose the protocol . Usually IM The agreements adopted are xmpp、mqtt、protobuf And other data communication private protocols , Let's analyze their advantages and disadvantages one by one .
1. XMPP agreement :
- advantage : be based on xml agreement , Easy to understand , Widely used , extensible .
- shortcoming : There's a lot of traffic , The mobile terminal also consumes electricity . The interaction process is complex . Many by pc Product use in the era , Not suitable for the mobile era IM product , Even if we are based on xmpp Improvement , Simplify the handshake process , Improve the file transfer mechanism , But its genes determine how to improve , He is not suitable for the era of mobile Internet IM product . Just like Sister Feng, no matter how cosmetic , Nor can it become a high circle .
2. MQTT agreement :
- advantage : Adapt to multiple platforms .
- shortcoming : Simple protocol , But you need to expand your friends , Group and other functions .
3. Private agreements :
- advantage : Follow one's inclinations , Define your own , The flow is small .
- shortcoming : Huge workload , Expandability of , It needs to be considered comprehensively .
4. Protobuf agreement :
- advantage : A very small 、 Very fast 、 It's simple , For one message data Protobuf The serialized size is JSON Of 1/10、XML Format 1/20、 Is binary serialized 1/10.
- shortcoming : Cannot represent complex data structures , But for IM Speaking of , Enough already . This agreement is highly recommended .
Add 1: For strong columns, it is recommended to use Protobuf, For the following reasons
- flexible 、 Efficient : flexible ( Convenient interface update )、 Efficient ( Efficiency goes through google The optimization of the , The transmission efficiency is higher than that of ordinary XML Much higher );
- Easy to use : Developers define structured message formats by following certain syntax , Then send it to the command line tool , The tool will automatically generate relevant classes , Can support java、c++、python And other language environments . By including these classes in the project , You can easily call relevant methods to complete the serialization and deserialization of business messages .
- Language support : Native support c++、java、python Wait up to 10 More than one language .
Add 2:Protobuf Mainly applicable to
- Need to exchange messages with other systems , Very sensitive to message size . that protobuf Fit , It has nothing to do with language , The message space is relatively small xml and json Wait, save a lot .
- Small data occasions . If you are big data , It's not suitable for .
- The project language is c++、java、python etc. , Because they can use google Source class library , Serialization and deserialization are very efficient . Other languages need to be written by a third party or yourself , The efficiency of serialization and deserialization is not guaranteed .
Overall speaking ,Protobuf It's still very easy to use , Tools used by many open source systems for data communication , stay google It is also the core basic library .
( More articles :《 The strong proposal will Protobuf As your IM application data transfer format 》、《 How to choose the data transmission format of instant messaging application 》、《 Combining theory with practice : A typical set of IM Communication protocol design details 》)
5、 Mobile IM Client's pit
Last , Let's learn more about the difficulties of the mobile terminal that need to be solved .
1. Traffic :
What kind of agreement 、 Picture thumbnail 、 The compression of attachments determines the size of traffic .
2. Power consumption :
(1) The smaller the flow , The lower the power consumption .(2) Heartbeat strategy , Reduce the number of heartbeats , Power consumption will be reduced .
3. The duration of the heartbeat :
wifi,2G,3G,4G, Move 、 telecom 、 Unicom , Different networks , Different operators ,NAT The failure time is different , So the time of heartbeat is different .
4. network connections :
cmnet and cmwap Lower connection processing mechanism .
5. Network instability :
The biggest feature of mobile terminal is the instability of network , In an unstable network state , How to ensure that messages arrive at the fastest speed ? How to avoid reconnection storm ? These need to be considered from the overall architecture , It also needs to adopt ingenious strategies at the mobile end to avoid .
( More articles : Mobile IM The technical problems that development needs to face )
6、 Mobile IM The pit of architecture design
First , Let's look at the mobile terminal IM Architecture design needs to consider .
1. Connector design :
The connector is mainly used to manage the long connection of the client . The best single connector at present 8G8 The core server can do 70 ten thousand —100 Wan's connection , And some developers can only do 4000 Left and right connections , There are several orders of magnitude . Where is the mystery here ?
2. The design of middleware :
Whether communication middleware is adopted ? What are the benefits of communication middleware ? If middleware is not used , How to manage the connection relationship between the connector and the logical server ?
3. Logical server :
Logical servers are usually simpler , It is mainly to divide the minimum granularity according to the business logic . But even so , There are still many developers who put together seemingly relevant but actually irrelevant logic , Such as authentication and message Put the servers together .
4. State server :
The status server mainly manages users online 、 Offline related status , The scheme of central node needs to be adopted , Otherwise the state will be out of sync . Here, we mainly need to consider the data storage mechanism corresponding to the state server , How to write , How to read ? In order to maximize the processing capacity and response speed of the status server .
5. Database design :
Database design is the most difficult , It is also the bottleneck of expansion . Because no matter for sql( Relational type ) Database or nosql( Non relational ) database , All have the limits of read-write processing , Then you need to consider how to partition the database ( According to what principle 、 What operation 、 Which users access the database on which node ). At the same time, we need to consider each atomic operation ( Like landing ) Which libraries need to be read , Write which libraries . Only these indicators are clear , You can assume that there is 100 Million concurrent users ,100 In the case of 10000 concurrent messages , Accurately evaluate how many servers the server needs , How to deploy .
6. other :
And the processing of device push , What kind of mechanism can ensure that messages are not lost , How to handle offline messages , wait . These are necessary and very complex function points and technical requirements , We need to adopt the correct architecture and strategy to achieve .
( More articles :http://www.52im.net/forum.php?mod=collection&action=view&ctid=7)
7、 Conclusion
The above difficulties and pit points are only recorded in a thousand words , But we should really solve these problems and meet the production and application standards , But I don't know how many days and nights it takes 、 How many lines of code to knock down , I'm afraid I've only really done IM Developers have a deeper understanding .( This article is published synchronously in :http://www.52im.net/thread-289-1-1.html)
appendix : more IM Technical articles
[1] Basic information of network programming :
《TCP/IP Detailed explanation - The first 11 Chapter ·UDP: User datagram protocol 》
《TCP/IP Detailed explanation - The first 17 Chapter ·TCP: Transmission control protocol 》
《TCP/IP Detailed explanation - The first 18 Chapter ·TCP Connection establishment and termination 》
《TCP/IP Detailed explanation - The first 21 Chapter ·TCP Over time and retransmission 》
《 Theoretical Classics :TCP Agreed 3 Shake hands with 4 Detailed explanation of the wave process 》
《 Combining theory with practice :Wireshark Caught analysis TCP 3 The second handshake 、4 The wave process 》
《 Computer network communication protocol diagram ( Chinese Collection Edition )》
《NAT Detailed explanation : The basic principle 、 Crossing technology (P2P Hole digging )、 Port aging, etc 》
《UDP What is the maximum size of a package in ?》
《Java New generation network programming model AIO Principle and Linux System AIO Introduce 》
《NIO Introduction to framework ( 3、 ... and ):iOS And MINA2、Netty4 cross-platform UDP Two way communication practice 》
《NIO Introduction to framework ( Four ):Android And MINA2、Netty4 cross-platform UDP Two way communication practice 》
>> More articles of the same kind ……
[2] of IM/ Push communication format 、 The choice of agreement :
《 Why? QQ It's using UDP Agreement, not TCP agreement ?》
《 Mobile instant messaging protocol selection :UDP still TCP?》
《 How to choose the data transmission format of instant messaging application 》
《 The strong proposal will Protobuf As your IM application data transfer format 》
《 Mobile IM The technical problems that development needs to face ( Including communication protocol options )》
《 Brief introduction of mobile terminal IM The pits developed : Architecture design 、 Communication protocol and client 》
《 Combining theory with practice : A typical set of IM Communication protocol design details 》
《58 Home real-time message system protocol design and other technical practice sharing 》
>> More articles of the same kind ……
[3] of IM/ Push the heartbeat to keep alive :
《Android Detailed explanation of process assurance : One article solves all your questions 》
《Android End message push summary : Realization principle 、 The heartbeat keeps alive 、 Problems encountered, etc 》
《 Why based on TCP The mobile end of the protocol IM We still need a heartbeat mechanism ?》
《 Wechat team original sharing :Android Version of wechat backstage to keep the real battle sharing ( Keep the process alive )》
《 Wechat team original sharing :Android Version of wechat backstage to keep the real battle sharing ( Network security chapter )》
《 Mobile IM practice : Realization Android Version of wechat's intelligent heartbeat mechanism 》
《 Mobile IM practice :WhatsApp、Line、 Heartbeat strategy analysis of wechat 》
>> More articles of the same kind ……
[4] of WEB End instant messaging development :
《 Novice entry post : In the history of the most complete Web Detailed explanation of the principle of end instant messaging technology 》
《Web End instant messaging technology inventory : Short polling 、Comet、Websocket、SSE》
《SSE Technical details : A brand new HTML5 Server push event technology 》
《Comet Technical details : be based on HTTP Long connection Web End real-time communication technology 》
《WebSocket Detailed explanation ( One ): A preliminary understanding WebSocket technology 》
《socket.io A little practice and thinking of message push 》
>> More articles of the same kind ……
[5] of IM Architecture design :
《 Talking about IM Architecture design of the system 》
《 Brief introduction of mobile terminal IM The pits developed : Architecture design 、 Communication protocol and client 》
《 A set of original distributed instant messaging (IM) System theoretical framework scheme 》
《 From zero to excellence : The evolution process of the technical framework of JD customer service instant messaging system 》
《 Mushroom Street im /IM Architecture selection of server development 》
《 tencent QQ1.4 Technology challenges and architecture evolution of 100 million online users PPT》
《 Wechat technical director talks about architecture : The way of WeChat —— The greatest truths are the simplest ( Speech full text )》
《 How to interpret 《 Wechat technical director talks about architecture : The way of WeChat —— The greatest truths are the simplest 》》
《 Fast fission : Witness the powerful background architecture of wechat 0 To 1 The evolution of ( One )》
《17 Year's practice : Technical methodology of Tencent's massive products 》
>> More articles of the same kind ……
[6] of IM Safe articles :
《 Instant messaging security ( One ): Understand and use... Correctly Android End encryption algorithm 》
《 Instant messaging security ( Two ): The combination encryption algorithm is discussed in IM Application in 》
《 Instant messaging security ( 3、 ... and ): Commonly used encryption and decryption algorithm and communication security 》
《 Instant messaging security ( Four ): The example analysis Android The risk of hard key coding in 》
《 Transport layer security protocol SSL/TLS Of Java Platform implementation and Demo demonstration 》
《 Combining theory with practice : A typical set of IM Communication protocol design details ( Including security layer design )》
《 Wechat new generation communication security solution : be based on TLS1.3 Of MMTLS Detailed explanation 》
《 From Ali OpenIM: Create a safe and reliable instant messaging service technology practice sharing 》
>> More articles of the same kind ……
[7] About real-time audio and video development :
《 Audio and video development of instant messaging ( One ): The theory of video codec 》
《 Audio and video development of instant messaging ( Two ): Digital video introduction of video coding and decoding 》
《 Audio and video development of instant messaging ( 3、 ... and ): Coding basis of video coding and decoding 》
《 Audio and video development of instant messaging ( Four ): Prediction technology of video codec 》
《 Audio and video development of instant messaging ( 5、 ... and ): Understanding the mainstream video coding technology H.264》
《 Audio and video development of instant messaging ( 6、 ... and ): How to start learning audio codec technology 》
《 Audio and video development of instant messaging ( 7、 ... and ): Introduction to audio fundamentals and coding principles 》
《 Audio and video development of instant messaging ( 8、 ... and ): Common real-time voice communication coding standards 》
《 Audio and video development of instant messaging ( Nine ): Echo and echo cancellation of real-time voice communication summary 》
《 Audio and video development of instant messaging ( Ten ): Echo cancellation in real-time voice communication Technical details 》
《 Audio and video development of instant messaging ( 11、 ... and ): Detailed explanation of packet loss compensation technology in real-time voice communication 》
《 Audio and video development of instant messaging ( Twelve ): Discussion on the framework of real-time audio and video chat for multiple people 》
《 Audio and video development of instant messaging ( 13、 ... and ): Real time video coding H.264 Characteristics and advantages of 》
《 Audio and video development of instant messaging ( fourteen ): Real time audio and video data transmission protocol 》
《 Audio and video development of instant messaging ( 15、 ... and ): Chat P2P And real-time audio and video applications 》
《 Audio and video development of instant messaging ( sixteen ): Some suggestions on the development of real-time audio and video in mobile terminal 》
《 Audio and video development of instant messaging ( seventeen ): Video coding H.264、V8 The past and this life 》
《 Open source real-time audio and video technology WebRTC Advantages and disadvantages 》
《 Conscience sharing :WebRTC Zero basic developer tutorial ( chinese )》
>> More articles of the same kind ……
[8] IM Develop comprehensive articles :
《 Mobile IM The technical problems that development needs to face 》
《 Development IM Is it better to use byte stream or character stream to design your own protocol ?》
《 Do you know the mainstream way of voice message chat ?》
《IM How to ensure the reliable delivery of messages in the system ( namely QoS Mechanism )》
《 Talk about mobile IM Optimization of login request in development 》
《 It's completely self-developed IM How to design “ Failure to retry ” Mechanism ?》
《 Technical test and analysis of the influence of wechat on the network ( Paper full text )》
《 The principle of instant messaging system 、 Technology and Application ( Technical papers )》
《 Open source IM engineering “ Mushroom street TeamTalk” The status quo of : An open source show with no end 》
>> More articles of the same kind ……
[9] Open source mobile IM Technical framework information :
《 Open source mobile IM Technical framework MobileIMSDK: Quick start 》
《 Open source mobile IM Technical framework MobileIMSDK: frequently asked questions 》
《 Open source mobile IM Technical framework MobileIMSDK: Stress test report 》
>> More articles of the same kind ……
[10] Articles on Push Technology :
《iOS The push service of APNs Detailed explanation : Design thinking 、 Technical principles and defects, etc 》
《Android End message push summary : Realization principle 、 The heartbeat keeps alive 、 Problems encountered, etc 》
《 Literacy stickers : know MQTT Communication protocol 》
《 One is based on MQTT Integrity of communication protocol Android push Demo》
《 Ask for advice android Message push :GCM、XMPP、MQTT The advantages and disadvantages of the three schemes 》
《 Mobile real-time message push technology analysis 》
《 Literacy stickers : Talking about iOS and Android The principle and difference of background real-time message push 》
《 Absolutely dry : be based on Netty Push service technology points to achieve massive access 》
《 Mobile IM practice : Google News push service (GCM) Research ( From wechat )》
《 Why wechat 、QQ In this way IM Tools not used GCM Service push message ?》
>> More articles of the same kind ……
[11] More instant messaging technology good article classification :
http://www.52im.net/forum.php?mod=collection&op=all
author :Jack Jiang ( Click the author name to enter Github)
Source :http://www.52im.net/space-uid-1.html
communication : Welcome to the instant messaging development exchange group 215891622
Discuss :http://www.52im.net/
Jack Jiang At the same time 【 original Java Swing Appearance engineering BeautyEye】 and 【 Lightweight mobile instant messaging framework MobileIMSDK】 The author of , You can go to download and exchange .
边栏推荐
猜你喜欢
Codeforces Round #800 (Div. 2)
6.1 - 6.2 Introduction à la cryptographie à clé publique
Guanghetong and anti international bring 5g R16 powerful performance to the AI edge computing platform based on NVIDIA Jetson Xavier nx
UWB超高精度定位系统原理图
86. (cesium chapter) cesium overlay surface receiving shadow effect (gltf model)
YOLOV5超参数设置与数据增强解析
C# 40. byte[]与16进制string互转
为什么许多shopify独立站卖家都在用聊天机器人?一分钟读懂行业秘密!
DBeaver 安装及配置离线驱动
LeetCode 19. 删除链表的倒数第 N 个结点
随机推荐
Differences between TCP and UDP
Image translation /gan:unsupervised image-to-image translation with self attention networks
Lstms in tensorflow_ Cell actual combat
zencart新建的URL怎么重写伪静态
Using Matplotlib to add an external image at the canvas level
What is UWB in ultra-high precision positioning system
[quartz] read configuration from database to realize dynamic timing task
图像翻译/GAN:Unsupervised Image-to-Image Translation with Self-Attention Networks基于自我注意网络的无监督图像到图像的翻译
202.2.9
Illustration of ONEFLOW's learning rate adjustment strategy
Douban top250
微信小程序保存图片的方法
Happy New Year!
[greedy college] Figure neural network advanced training camp
2022.2.17
C# 40. byte[]与16进制string互转
微信小程序保存圖片的方法
Multipass Chinese document - share data with instances
ssh连win10报错:Permission denied (publickey,keyboard-interactive).
Condition query