当前位置：网站首页>Time-series database selection and implementation practice for Xingsheng optimal monitoring scenarios

Time-series database selection and implementation practice for Xingsheng optimal monitoring scenarios

2022-08-05 17:37:00 【Tao si TDengine data】

小 T 导读：Xingsheng Optimization needs to judge whether the equipment is working through the data generated in real time、Check whether the communication is delayed、观测 SNMP OID Is the flow normal?,This ensures that operation and maintenance and network personnel find and fix problems in a timely manner.For efficient processing of various time series data,保障服务的稳定运行,在对比了 Elasticsearch、InfluxDB 和 TDengine After three products,They chose and landed TDengine.

企业介绍

湖南兴盛优选电子商务有限公司（Prosperity for short）,总部位于湖南长沙,Is an Internet that cares about people's livelihood“新零售”平台,The main positioning is to solve the daily needs of household consumers,Provide fruits and vegetables、肉禽水产、米面粮油、Selected products of all categories such as daily necessities.Xingsheng Preferred relies on community convenience stores,通过“预售+自提”mode to provide services to users,It is the only e-commerce company in the community whose valuation exceeds 150 亿美金的“独角兽”.

业务背景

In order to ensure the efficiency and stability of Internet services,We need to monitor all the company's node servers（包括云服务器）、交换机及路由器.We need to judge whether the device is working through the data generated in real time、Check whether the communication is delayed、观测 SNMP OID Is the flow normal?,This ensures that operation and maintenance and network personnel find and fix problems in a timely manner.

This type of data is very typical time series data,How to handle it efficiently？There are several very popular time series databases on the market now（Time Series Database）产品.How should we evaluate and choose the technology platform that suits our business scenario?？

产品调研

For this business scenario,We have investigated the following products：Elasticsearch、InfluxDB 和 TDengine.具体对比如下.

Elasticsearch

优点：可以分布式部署,Barrier-free insertion,Supports arbitrary field types,查询速度快.
缺点：It is only suitable for logging and the amount of concurrent data is not large,There is a performance problem in the time series data writing of a large number of devices.

InfluxDB

优点：支持无模式（Schemaless 写入）,限制较少.
缺点：When faced with large batches of data being inserted or read at the same time,Memory is easily full,导致死机.Especially the polling mechanism,When checking for expired data,内存占用特别大.此外,在读取数据时,read the list,可读性差,解析比较麻烦.

TDengine

优点：columnar storage and“一个设备一张表”The model fits our business scenario very well.此外,Also compatible with our previous use InfluxDB accustomed to insertion,代码可读性强,Support of a strong binding parameters.When performing queries on massive data,响应速度比 InfluxDB 更快.
缺点：Schemeless The support is still being improved.

Since this project needs to monitor all servers of our company in the future,The average corresponding to each OID 会有几百个,如果每 1-5 Collect and store once per second,The amount of concurrent data will be very large.因此,we eliminated the candidates Elasticsearch.

Next, we continue to compare InfluxDB 和 TDengine.InfluxDB Insufficient performance of a single node,Cluster closed source and performance unknown.反观 TDengine,Its cluster functionality is open source,And retains some of the core functions of the enterprise version.This allows us to directly very understanding TDengine 的优劣.从这方面考虑,我们选择了 TDengine.而且在实际使用中,我们又发现了 TDengine 的一大优势,其“一个设备一张表”The model fits well our actual scene.

系统架构

在引入 TDengine 之后,我们的系统架构如下图所示.

在该架构下,The front-end formulates the rules and issues（例如：流量阈值,Delay threshold）,Backend to see if you need to store rules,or check if the rules have changed,Then send the rules to ETCD to do timed task scheduling.We have three program tasks in total,通过 ETCD 管理,Timing, and all kinds of communications equipment,Capture the data you need according to different rules：

SNMP 引擎通过 OID Monitor various indicators of network equipment
TCPing The engine is used to monitor the server TCP 端口状态
ICMP The engine focuses on the time it takes to receive or return data

After the three types of data collection,will be processed uniformly,再写入 TDengine.

我们使用 5 A set of nodes is built TDengine 集群：

We chose modeless writing,在Automatically create table when writing,For the specific writing method, please refer to the official website.这样一来,The type corresponding to each server is a super table,We chose the table as far as possible,提升查询速度.

对于 SNMP Type of super table,Each subtable is a OID.

对于 TCPing 类型,我们按照 TCP Port to distinguish sublist.

对于 ICMP type of hypertable,我们以 task_id 和 task_type as a basis for distinguishing sub-lists：

Currently we have stored 500 Ten thousand sub-tables,On average, the device reporting frequency is approximately 3 秒 1 行,According to the total amount has reached the level of billions,The storage space occupied is only 70GB 左右.这应该得益于 TDengine Targeted use of columnar compression,Storage resource usage is small,The compression ratio is approximately 8% 左右.TDengine Can withstand write and storage pressure well.And inquiries,We will batch query the value of a specific time in part of the table,Most of the data is used for real-time monitoring and alarming.

经验总结

在使用过程中,The relatively more time-consuming is probably the exploration of modeless insertion..

对于从 InfluxDB Switched user,There may be some discomfort in the beginning.因为 TDengine not initially according to Schemaless 来设计的,This feature was added later,The system will eventually convert the schemaless to SQL 再进行写入,Just simplifies the user's operation.不过 TDengine It has been improving the relevant ecological adaptation,For example, for the storage of upper and lower case special characters,Improve metadata management,Accommodates various data types,也在逐步向 InfluxDB Freestyle writes close to.

除此之外,We also had a problem,目前 Go Connector query API Temporarily does not support multiple SQL splicing together unified execution.So we take concurrent reads,But performance may suffer a little,Looking forward to the next version to solve.

最后,TDengine The support team is quite responsible,Actively cooperate,Let's get started quickly with this lightweight and easy-to-use、High-performance time series database.At present, we only have access to some servers and equipment,In the future, we plan to connect all the servers of the company nationwide.,Will also recommend more departments of the company to use.一切顺利的话,We will also consider including warehouse delivery robots,More application scenarios such as logistics line equipment.