Why use NoSQL with MySQL?

reply PDF Collect information

This is Wukong's first  87  Original articles

This article comes from my real interview experience .

The real name of this company is “ Sanzang ”, And my name “ The wu is empty ” It fits well , Tang Sanzang interviewed Wukong , be perfectly logical and reasonable , And a little fun , So I went for an interview . Sanzang company is a small factory , I'm the person in charge of technology , For interview results , It was announced at the end of the article .

The main contents of this paper are as follows ：

One 、MongoDB and MySQL

1. interviewer ： Look, your resume says MongoDB, Under the said MongoDB and MySQL The difference between! .

In fact, for this problem , I was prepared in advance , The resume says MongoDB, The interviewer will definitely ask MongoDB and MySQL The difference between .

MongoDB It's a non relational database （NoSQL), It's a document database , Document database is to solve the problems brought by relational database . The biggest characteristic is no-schema, Can store and read arbitrary data .

The stored data format is JSON（ perhaps BSON）.JSON We are all familiar with the format , such as Rest API Requested returned Response Namely JSON Format .JSON Format data and XML The difference in format is JSON It's simpler , There are not so many tags to define field names . in other words JSON It's self describing . in addition JSON Save the format in MongoDB In the after , Even read one JSON Fields that do not exist in do not result in SQL That kind of grammatical error .

MongoDB advantage

Because the document database has no-schema characteristic , There are several obvious benefits .

（1） There will be no error in the new field .

For example, add a to the user table nickname Field of , There is no need to execute statements that update the table structure like relational data . We can directly query this document to see the new fields .

（2） There will be no error in querying historical data .

As mentioned above, a new nickname field is added , But there is no such field in the historical data , If you query historical data , Then the returned data will not have this field , Although the query will not report an error , But when taking values , Returns the null. If the nickname field is used in the business code , Compatibility processing is required .

（3） Easily store complex data .

Because it's using JSON Storage , and JSON It can also represent complex data structures , For example, fields can be stored in arrays , Fields can be nested , And you can save many fields . Change MySQL, You need to design several tables to store .MongoDB The structure of storing data , It is especially suitable for the business scenario of e-commerce , For example, two different goods , The properties are very different , But with JSON Save and you can easily deal with .

But what about the document database shortcoming Well ？

MongoDB shortcoming

（1） at present 4.0 Previously, multi document transactions were not supported .

combination MongoDB Document model embedded array 、 Document support , The current single document transaction can meet the needs of most developers . In order to make MongoDB It can adapt to more application scenarios , Make development easier ,MongoDB 4.0 Multiple document transactions across one or more collections within a copy set will be supported , Ensure the atomicity of updates to multiple documents . And in the future MongoDB 4.2 edition , It also supports distributed transactions of sharded clusters .

Let's see MongoDB Functions supported by different versions ：

MongoDB Different versions of

MongoDB The transaction interface of is very simple , Developers just need to put Need to ensure the atomicity of the update sequence In a session Of Start business And Commit transaction Between . Here is Java Use MongoDB Sample code of transaction ：

（2） Association query is not supported .

We all know MySQL It supports Association query , That is, you can execute Join operation . For example, there are two tables ： User tables and order tables , There is a user's... In the order table id, And the gender only exists in the user table . If you want to buy a male user of a mobile phone , Query with association , It can be done in one step . But if you use MongoDB, You need to check twice , First query the users who buy mobile phones in the order table , Then query which of these users are male .

Two 、 Disadvantages of relational data

2. interviewer ： Why not use relational database for this project ？ What are the disadvantages of relational databases ？

Follow the interviewer's thinking , You know what the interviewer wants to ask is what are the shortcomings of relational database .

The shortcomings of relational database

（1） It stores row records .

Can't store array 、 Data in nested fields and other formats .

（2） It is inconvenient to extend the table structure .

An error will be reported if the operation does not exist , Adding columns requires execution SQL Statement . And you need to pay special attention to it , Because it takes a long time to update the table Lock table , This can have a serious impact on the online environment .

（3） It takes up a lot of memory .

When a relational database performs statistics and other operations on a large number of data tables , It takes up a lot of memory , Because even if it only operates on one column , It also reads the entire line of data from the storage device into memory .

（4） The performance of full-text search is poor

Be similar to MySQL Relational database , Only use like Matching for whole table scanning , Efficiency is very low . Today, , There are many scenarios that need to support fuzzy matching , And it has to support efficient lookup . For example, query the log information containing keywords , Or query the product list according to a product keyword .

In view of the above shortcomings , Our project uses two non relational data storage schemes ：MongoDB and ElasticSearch.

3、 ... and 、NoSQL Classification and characteristics of

3. interviewer ： What do you know NoSQL database ？ What are the characteristics of each ？

NoSQL(NoSQL = Not Only SQL ), meaning " not only SQL".

What I know is Redis、MongoDB、HBase、 Full text search engine Elasticsearch. They are different non relational storage solutions .

K-V Storage type

such as Redis, It can be used K-V Key value pairs to store data , The stored value can have several formats , Such as string、hash、list、set、bitmap etc. .

Document storage type

such as MongoDB, Stored JSON The format of the document , It solves the problem of table constraint in relational database , For example, an error will be reported when querying a field that does not exist . In addition, it also solves the problem of some storage formats , because JSON Can represent an array , You can also nest fields to store .

Column storage type

such as HBase, Store data in columns , Solve the big data scenario of I/O problem .

Relational databases are based on That's ok To store data , So called Row database . according to That's ok To store has the following advantages ：

• Reading a row of data can read multiple columns , Only one disk operation is needed to read the data of multiple columns into memory .
• Writing a row of data can write to multiple columns , The consistency of atomic data is guaranteed . The multi column write operation of column storage , May cause some columns to succeed , Some failures , Generate data inconsistencies .

Full text search engine

The most used place is the log system , There are similar scenes such as searching product information . The e-commerce website shown in the figure below .

Search for mobile phones

The log search used in our project is to use ELK.

Elasticsearch Namely ELK Medium E.Elasticsearch Full text search engine , Be careful ： He is a kind of NoSQL programme , Not at all NoSQL database .

Logstash Namely ELK Medium L. It is Elastic Stack One of the core products of , Can be used to aggregate and process data , And send the data to Elasticsearch.Logstash Is an open source server-side data processing pipeline , Allows you to index data to Elasticsearch Previously, data was collected from multiple sources at the same time , And enrich and transform the data .

Kibana Namely ELK Medium  K. It's a Elasticsearch Data visualization and management tools , Can provide real-time histogram 、 Linear graphs, etc .

As shown in the figure below ：

Search logs

The traditional relational database mainly carries out fast query through index , But if you put it in the context of full-text search , It won't work .

Let's see why relational databases are difficult to achieve efficient full-text search ：

• Because in full-text search , The search conditions can be arranged and combined at will , For example, fields A、B、C, Can be arranged into 6 Kind of , If you want to use indexes to support fast queries , You need to create multiple indexes , This is very troublesome , meanwhile , Multiple indexes also have an impact on the efficiency of data insertion .
• Fuzzy matching can only be used like Inquire about , and like Queries are whole table scans , Efficiency is very low .

I wrote an article before Elasticsearch Principle ：《 Don't just search the logs , Please understand some principles 》, Efficient full-text retrieval through inverted index . Let's take an example of inverted index to show you ：

If the database has the following movie records ：

1- A Chinese Odyssey

2- It's a big story about traveling to the West

3- Analysis of the big story of the journey to the West

4- Journey to the west to subdue demons

5- Exclusive analysis of dream journey to the West

participle , Break the whole sentence into words ：

Serial number	Save to ES The word	The corresponding movie record number
A	Westward journey	1,2, 3,4, 5
B	Big talk	1,2, 3
C	External transmission	2,4, 5
D	analysis	3,5
E	Subdue demons	4
F	dream	5
G	exclusive	5

retrieval ： An exclusive tour to the West

take An exclusive tour to the West Split into exclusive 、 Big talk 、 Westward journey

ES in A、B、G Record There is one of these three words , therefore 1,2, 3,4, 5 No. records have related words hit .

1 Record number one hit 2 Time , A、B There are ( hit 2 Time ) , and 1 There is a record of 2 Word , Correlation score ：2 Time /2 Word =1

2 Record number one hit 2 Word A、B All of them have ( hit 2 Time ) , and 2 There is a record of 2 Word , Correlation score ：2 Time /3 Word = 0.67

3 Record number one hit 2 Word A、B All of them have ( hit 2 Time ) , and 3 There is a record of 2 Word , Correlation score ：2 Time /3 Word = 0.67

4 Record number one hit 2 Word A There is ( hit 1 Time ) , and 4 There is a record of 3 Word , Correlation score ：1 Time /3 Word = 0.33

5 Record number one hit 2 Word A There is ( hit 2 Time ) , and 4 There is a record of 4 Word , Correlation score ：2 Time /4 Word = 0.5

So the order of the retrieved records is as follows ：

1- A Chinese Odyssey ( phase Relevance score ：1 )

2- It's a big story about traveling to the West ( Correlation score ：0.67 )

3- Analysis of the big story of the journey to the West ( Correlation score ：0.67 )

5- Exclusive analysis of dream journey to the West ( Correlation score ：0.5 )

4- Swim West to subdue the devil ( phase Relevance score ：0.33 )

Elasticsearch And mysql Comparison of

Serial number	Mysql	Elasticsearch
1	Mysql service	ES The cluster service
2	database Database	Indexes Index
3	surface Table	type Type
4	Record Records ( Line by line )	file Document ( JSON Format )

additional NoSQL And graphics databases , There is no expansion here .

Relational and NoSQL How to choose ？

4. interviewer ： Relational and NoSQL How to choose? ？

Relational and NoSQL Database selection , Consider several indicators , Data volume 、 Concurrency 、 The real time 、 Consistency requirements 、 Read / write separation 、 Security 、 Operation and maintenance, etc . According to these indicators , Software systems can be divided into several categories .

• Management system , Such as operation system , Prefer relational .
• High flow system , And multiple fields 、 The amount of data is growing fast , The preferred NoSQL.
• Log system , The preferred Elasticsearch
• Search system , Refers to the station search , Non universal search , Such as product search , The preferred Elasticsearch.
• Transactional systems , Such as inventory 、 transaction 、 Bookkeeping , Select relational type + cache + Agreement of conformity .
• Offline computing , Such as a large number of data analysis , Column database is preferred .
• Real time computing , Such as real-time monitoring , You can select the timing database , Or column database .

Interview results ： The technical director thought it was ok , but HR Not today , etc. HR Let me know next time . There will be no further notice . End .

Reference material ： 

https://mongoing.com/archives/5560 

https://time.geekbang.org/column/article/8377 

https://dzone.com/articles/history-databases-%E2%80%9Cno-tation%E2%80%9D 

https://www.runoob.com/mongodb/nosql.html

- END -

I wrote two PDF, Welcome to download .

Articles written on Sunday 《 bosses , Your website crashed 》 Forget to embed the video , Let me show you .

At the end of the article Read the original You can also go to this website .

I'm Wukong , Try to be strong , Become a super Saiya ！

This article is from WeChat official account. - Wukong chat structure （PassJava666）.
If there is any infringement , Please contact the [email protected] Delete .
Participation of this paper “OSC Source creation plan ”, You are welcome to join us , share .