Netease Shufan: Data productivity platform 2.0 promotes "everyone uses data and always uses data"

With the rise of a new round of scientific and technological revolution and industrial reform , big data 、 Cloud computing 、 Artificial intelligence and other technologies are developing day by day , More and more enterprises begin to explore the ways and methods of business model innovation and business ecosystem reconstruction , Therefore, more and more enterprises are moving towards digital transformation . However, the digital transformation of enterprises is not achieved overnight , For the enterprise itself , It not only involves the upper level decision-making of the enterprise , Enterprises also need to transform their business systems , Innovation in culture ; For technical service providers , Not only does it need to help enterprises integrate digital technologies to sort out business transformation needs , We also need to be familiar with the needs and industry laws of enterprise digital transformation , It can cover the breadth of digital transformation of thousands of lines and industries , It can also meet the in-depth needs of different scenarios in the segmented industries .

all the time , Providing digital transformation services for enterprises is a “ The slope is long and the snow is thick ” Track , When Tencent 、 Ali 、 Huawei and other giants are rolling on this track B The snowball of end business , There are many people who want to B Enterprises that have a place in end-to-end services have a continuous layout , It means that the digital transformation service is already a highly competitive battlefield .

among , Netease, already famous in the Internet industry , Now, it is also becoming more and more stable on the road of helping enterprises' digital transformation .“ Labelling ” Is a major feature of Internet companies , Baidu search 、 Ali's e-commerce 、 Tencent's social life , However, in China's Internet landscape , It seems that there is no label applicable to NetEase . And that's why , Netease has formed its own diversified and polymorphic product and business lines .

Netease Shufan —— Focus on enterprise service track

Netease 2016 In, it officially entered the enterprise service market , So far, Netease has formed several sails 、 Netease smart enterprise 、 Netease Lingxi and other sub businesses . among , Netease Shufan focuses on the field of basic software , From improving enterprise software productivity 、 Data productivity 、 Start with intelligent productivity , Adhere to open source and openness , Enable with software technology 、 Serve customers , Help customers improve the competitiveness of the industry , Help its transformation process . Now it has become the core strategy for Netease to serve the enterprise market .

recently , Netease digital + The conference was held in Hangzhou , Netease Shufan has shown its growth process and achievements in the field of enterprise services in recent years , At the same time, it released a new market strategy and a series of heavy products and technologies .

Netease Shufan released several full link data productivity platforms at the meeting 2.0, Introduced several DataOps、 Logical data Lake 、 Data Lake engine 、 Machine learning platform, etc , Further improve the digital governance capability , Promote the digital transformation and upgrading of enterprises . These new or upgraded products will further accelerate the implementation of big data technology in enterprises , Help enterprises mine data value , Empowering more enterprises for digital transformation .

Along the way , Netease Shufan keeps innovating , Iteratively upgrade the product technology , In the enterprise service market . obviously , For how to more efficiently help enterprises upgrade their digitalization , Netease Shufan also has its own methodology .

As an important highlight of this year , There are several full link data productivity platforms from 1.0 To 2.0 What is the difference ？ How does the innovative technology enable the digitalization of enterprises ？ With relevant questions , Dataape conducted an in-depth interview with Yulihua, general manager of Netease's Shufan products , Focus on new products 、 New technology 、 New concept , Fully understand the advanced road of Netease Shufan enterprise service .

1.0 To 2.0, The data productivity platform has taken a big step

The productivity of enterprises in the digital age , Most of it comes from data productivity . Data as a core asset element in the digital age , It is indispensable for enterprises in digital transformation and upgrading , How to use data efficiently , Improve the data productivity of enterprises , It is the key to the digital transformation of enterprises . therefore , Today's enterprises need a full link technology to improve data productivity 、 Products and services .

Yulihua said ：“ To improve the productivity of full link data, enterprises need to pay attention to technology 、 Products or services to improve , With data in the middle 、 Data products （ This refers to the data productization without code ） There are two nodes , Gather data from the business through the data midrange , Then build data products based on the data middle platform , Provide executable decision guidance for business systems , Improve the big data decision-making level of enterprises , In the data center 、 Data products 、 Under the linkage of business system , Form a closed loop of enterprise data application , Release data value from business systems .”

Used to be 1.0 In the practice of , There are still some obstacles in the closed loop of full link data productivity , Among them, the data center side , The cost of physical aggregation of data from various systems is getting higher and higher , The inefficiency of data development slows down the delivery of applications , At the same time, the requirements for real-time data processing are increasing , On the data application side , The understanding of unstructured data needs more means .

be based on 1.0 Various pain points in the stage , Netease digital sail has released several full link data productivity platforms 2.0. Yulihua said ,“ Data productivity platform 2.0 Stage , We have improved our technical service capability , From the big data base 、 To the data center 、 And then to data applications , Each link needs better polishing to serve customers .”

When talking about how to make the enterprise's data midrange play its value , Yu Lihua believes that enterprises should develop a large number of data products based on the data platform , Truly accomplish “ Everybody uses data 、 Always use data ”, In this way, data productivity can be comprehensively improved . Yu Lihua also shared the successful practice of Netease cloud music in data products ,“ The member operation Department of Netease cloud music has hundreds of statements , Although the data are available , But it is difficult to look at the data , The main reason is that the data is not organized according to the idea of analysis and decision-making , yes ‘ In bulk ’ Of . Design one for this scenario ‘ Members run portals ’ Data products , The data is well ‘ organization ’ get up , Colleagues without professional data analysis skills can make good use of data .”

The technology behind data productivity “ Pushing hands ”

In this full link data productivity platform 2.0 In the version , Netease Shufan has launched several DataOps、 Logical data Lake 、 Data Lake engine Arctic And so on , There are several of them DataOps The technology represents the heavy upgrade of the data center . Yulihua introduced , Have number DataOps Technology aims to help enterprises solve the efficiency and quality problems in the process of data development , It can be said that DevOps The concept is applied to the field of data development , To shorten the insight cycle , Promote continuous project integration （CI） And continuous deployment （CD）.

DataOps It is in the big data field DevOps, Hope to use automation and instrumented means , For example, automated testing , Automatic deployment , Improve data R & D efficiency , Continuous delivery of data value . Yulihua said ,“ Data development lags far behind business system development , A lot of times , Data development engineer and test environment , There are no automated testing tools , Write code directly online , Do the test manually . This is due to the low efficiency of data middle office project delivery , One of the main reasons for poor data quality . Netease has several supports DataOps This is to solve the problem of R & D delivery efficiency of the data center .”

except DataOps, In the past few years , The lake and the warehouse are integrated （Lakehouse） As a new data management architecture , Gradually and independently appear in the application scenarios of many customers . For such a data management architecture , Yulihua believes that ,“ Data Lake technology is strong in general low-cost , The strong data warehouse technology lies in the high performance of metadata management , Lake warehouse integration can have the benefits of both , Very valuable , We also provide such technology . But no data management system can solve all the problems , This is one of the reasons why we have proposed several logical data lakes .”

Besides , Enterprises often think that , Gather the data together through the data center , To take full advantage of data value . But Yulihua pointed out ,“ After communicating with customers, we found that , It is very difficult for enterprises to build a physically centralized data center . Without requiring data migration , It is the basic demand of many enterprises to incorporate data into data management .”

According to introducing , Data midrange based on logical data Lake , Is a physical dispersion , A logically unified data center , The data center can be established in Oracle/MySQL/Vertica etc. 7 Class system , Realize unified development and management of data .

Facing the lake warehouse , Netease Shufan also released several real-time data Lake engines Arctic, It not only realizes the integrated storage of streams and batches , It also supports the seamless connection of the data management system in the data center , It can be said that it has greatly improved the real-time capability of the data center . Especially for massive logs 、 Events change frequently 、 High real-time data processing and other business scenarios , It has a very broad space for technical practice . The realization of integrated stream batch storage also solves a core technical problem for the integration of lake and warehouse .

In addition to the above highlights , Netease Shufan also launched several machine learning platforms and several BI Data preparation , The former helps enterprises understand and process structured data , Greatly improve the iterative efficiency of machine learning . The latter can help small and micro enterprises that lack professionals to establish self-service ETL Realize lightweight Lake warehouse .

Give control to the customer

In this Netease Digital ＋ At the conference , Netease Shufan officially released two strategic propositions “ Technology as the body 、 Service is the soul ” and “ Give control to the customer ”.“ Technology as the body 、 Service is the soul ” It includes three aspects , Focus on mature products and solutions ; Introduce industry senior talents , Build an expert team that understands the industry ; From consulting to planning 、 Implement the whole process of digital transformation services delivered to knowledge transfer .“ Give control to the customer ” It means that the technical architecture is open , Compatible with all kinds of infrastructure ; At the same time, the kernel is open source , Give priority to the mainstream choice of open source software .

It is worth noting that ,“ Give control to the customer ” It is an important strategic proposition of Netease , This means that Netease can count the sails and make the technology open and transparent , Do not lock technology , user-centric , Grow with users .

Based on this idea , Netease Shufan also released knowledge sharing this time 、 Training and communication platform —— There are several schools , It aims to provide users with all-round learning 、 Training and communication platform , Focus on learning and training services 、 Product Q & a service 、 Professional certification services, etc . Learning and training , Several schools offer video courses 、 Help document 、 Big data basic technology and other product courses ; Product Q & A , Build for users “ Have a craigslist , Understand when you have an answer ” Our community ; Professional certification , Launch big data 、BI And other vocational skills services .

writing ：Winter /  Data ape