Open source SPL enhances mangodb computing

MongoDB yes NoSQL The typical representative of database , Support the storage method of document structure. Data storage and use are more convenient , Data access efficiency is also very high , But the computing power is weak , It involves MongoDB The calculation, especially the complex calculation, will be troublesome , This requires a data processing engine with strong computing power to cooperate with it .

Open source integrator SPL It is a professional structured data computing engine , It has a rich computing class library and complete 、 Do not rely on the computing power of the database .SPL Provides an independent process calculation syntax , Especially good at complex calculation , Can enhance MongoDB Computing power , Complete group summary 、 Associated calculation 、 Subqueries are all out of the question .

Regular query

MongoDB Difficult connections JOIN operation , use SPL It's easy to handle ：

	A	B
1	=mongo_open("mongodb://127.0.0.1:27017/raqdb")	/ Connect MongDB
2	=mongo_shell(A1,"c1.find()").fetch()	/ get data
3	=mongo_shell(A1,"c2.find()").fetch()
4	=A2.join(user1:user2,A3:user1:user2,output)	/ Associated calculation
5	>A1.close()	/ Close the connection

 

A single table participates in operations many times , Reuse calculation results ：

	A	B
1	=mongo_open("mongodb://127.0.0.1:27017/raqdb")
2	=mongo_shell(A1,“course.find(,{_id:0})”).fetch()	/ get data
3	=A2.group(Sno).((avg   = ~.avg(Grade), ~.select(Grade>avg))).conj()	/ The calculation result is greater than the average
4	>A1.close()

 

IN Calculation ：

	A	B
1	=mongo_open("mongodb://localhost:27017/test")
2	=mongo_shell(A1,"orders.find(,{_id:0})")	/ get data
3	=mongo_shell(A1,"employee.find({STATE:'California'},{_id:0})").fetch()	/ Filter employee data
4	=A3.(EID).sort()	/ Take out EID And sort
5	=A2.select([email protected](SELLERID)).fetch()	/ Binary search
6	>A1.close()

 

Foreign key objectification , Foreign key pointers are not only convenient , It's also efficient ：

	A	B
1	=mongo_open("mongodb://localhost:27017/local")
2	=mongo_shell(A1,"Progress.find({},   {_id:0})").fetch()	/ obtain Progress data
3	=A2.groups(courseid;   count(userId):popularityCount)	/ Count by course group
4	=mongo_shell(A1,"Course.find(,{title:1})").fetch()	/ obtain Course data
5	=A3.switch(courseid,A4:_id)	/ Foreign key connection
6	=A5.new(popularityCount,courseid.title)	/ Create result sets
7	=A1.close()

 

APPLY A simple implementation of the algorithm ：

	A	B
1	=mongo_open("mongodb://127.0.0.1:27017/raqdb")
2	=mongo_shell(A1,"users.find()").fetch()	/ obtain users data
3	=mongo_shell(A1,"workouts.find()").fetch()	/ obtain workouts data
4	=A2.conj(A3.select(A2.workouts.pos(_id)).derive(A2.name))	/ Inquire about _id value workouts Record of sequence
5	>A1.close()

 

Set operations , Consolidated cross payment ：

	A	B
1	=mongo_open("mongodb://127.0.0.1:27017/raqdb")
2	=mongo_shell(A1,"emp1.find()").fetch()
3	=mongo_shell(A1,"emp2.find()").fetch()
4	=[A2,A3].conj()	/ Multi sequence set
5	=[A2,A3][email protected]()	/ Comparison and union of the whole line
6	=[A2,A3][email protected](_id,   NAME)	/ Key value comparison and union
7	=[A2,A3][email protected]()	/ Compare and find the intersection of the whole bank
8	=[A2,A3][email protected](_id,   NAME)	/ Key value comparison and intersection
9	=[A2,A3][email protected]()	/ Compare and calculate the difference set of the whole line
10	=[A2,A3][email protected](_id,   NAME)	/ Key value comparison difference set
11	>A1.close()

 

Find the member sequence number in the sequence ：

	A	B
1	=mongo_open("mongodb://localhost:27017/local)
2	=mongo_shell(A1,"users.find({name:'jim'},{name:1,friends:1,_id:0})")   .fetch()
3	=A2.friends.pos("luke")	/ from friends Get the member sequence number in the sequence
4	=A1.close()

 

Intersection of multi member sets ：

	A	B
1	[Chemical,   Biology, Math]	/ Course
2	=mongo_open("mongodb://127.0.0.1:27017/raqdb")
3	=mongo_shell(A2,"student.find()").fetch()	/ obtain student data
4	=A3.select(Lesson^A1!=[])	/ Check the records of at least one elective course
5	=A4.new(_id,   Name, ~.Lesson^A1:Lession)	/ Calculate the result
6	>A2.close()

Complex calculations

TOPN operation ：

	A	B
1	=mongo_open("mongodb://127.0.0.1:27017/test")
2	=mongo_shell(A1,"last3.find(,{_id:0};{variable:1})")	/ obtain last3 data , And press variable Sort
3	for A2;variable	=A3.top(3;-timestamp)	/ elect timestamp The latest 3 individual
4		[email protected]|B3	/ Append the selected document to B4 in
5	=B4.minp(~.timestamp)	/ elect timstamp The earliest document
6	>mongo_close(A1)

 

Aggregation of nested structures ：

	A
1	=mongo_open("mongodb://127.0.0.1:27017/raqdb")
2	=mongo_shell(A1,"computer.find()").fetch()
3	=A2.new(_id:ID,income.array().sum():INCOME,output.array().sum():OUTPUT)
4	>A1.close()

 

Merge multi generic subdocuments ：

	A	B	C
1	=mongo_open("mongodb://localhost:27017/local")
2	=mongo_shell(A1,"c1.find(,{_id:0};{name:1})")
3	=create(_id,   readUsers)		/ Create the result sequence table
4	for   A2;name	=A4.conj(acls.read.users|acls.append.users|acls.edit.users|acls.fullControl.users).id()	/ Remove all users Field
5		>A3.insert(0,   A4.name, B4)	/ Insert this group of data
6	=A1.close()

 

nesting List Query of sub documents

	A	B
1	=mongo_open("mongodb://localhost:27017/local")
2	=mongo_shell(A1,"Cbettwen.find(,{_id:0})").fetch()
3	=A2.conj((t=~.objList.data.dataList,   t.select((s=float([email protected]()(1)), s>6154   && s<=6155))))	/ Find the qualified string
4	=A1.close()

 

Cross summary ：

	A
1	=mongo_open("mongodb://localhost:27017/local")
2	=mongo_shell(A1,"student.find()").fetch()
3	=A2.group(school)
4	=A3.new(school:school,[email protected](5,sub1).(~.len()):sub1,[email protected](5,sub2).(~.len()):sub2)
5	=A4.new(school,sub1(5):sub1-5,sub1(4):sub1-4,sub1(3):sub1-3,sub1(2):sub1-2,sub1(1):sub1-1,sub2(5):sub2-5,sub2(4):sub2-4,sub2(3):sub2-3,sub2(2):sub2-2,sub2(1):sub2-1)
6	=A1.close()

 

Segment and group

	A	B
1	[3000,5000,7500,10000,15000]	/Sales Segmented interval
2	=mongo_open("mongodb://127.0.0.1:27017/raqdb")
3	=mongo_shell(A2,"sales.find()").fetch()
4	=A3.groups(A1.pseg(~.SALES):Segment;count(1):   number)	/ according to SALES Count the number of employees by interval grouping
5	>A2.close()

 

Sort and group

	A	B
1	=mongo_open("mongodb://127.0.0.1:27017/raqdb")
2	=mongo_shell(A1,"books.find()")
3	=A2.groups(addr,book;count(book):   Count)	/ Group count
4	=A3.groups(addr;sum(Count):Total)	/ Grouping statistics
5	=A3.join(addr,A4:addr,Total)	/ Associated calculation
6	>A1.close()

 

Data writing

Export to CSV：

	A	B
1	=mongo_open("mongodb://localhost:27017/raqdb")
2	=mongo_shell(A1,"carInfo.find(,{_id:0})")
3	=A2.conj((t=~,cars.car.new(t.id:id,   t.cars.name, ~:car)))	/ Yes car Fields are split into rows
4	=file("D:\\data.csv")[email protected](A3)	/ Export build csv file
5	>A1.close()

 

Update the database （MongoDB To MySQL）：

	A	B
1	=mongo_open("mongodb://localhost:27017/raqdb")	/ Connect MongDB
2	=mongo_shell(A1,"course.find(,{_id:0})").fetch()
3	=connect("myDB1")	/ Connect mysql
4	[email protected]("select   * from course2").keys(Sno, Cno)
5	>A3.update(A2:A4,   course2, Sno, Cno, Grade; Sno,Cno)	/ towards mysql Update data
6	>A1.close()

 

Update the database （MySQL To MongoDB）：

	A	B
1	=connect("mysql")	/ Connect mysql
2	[email protected]("select   * from course2")	/ Get the table course2 data
3	=mongo_open("mongodb://localhost:27017/raqdb")	/ Connect MongDB
4	=mongo_insert(A3,   "course",A2)	/ take MySQL surface course2 Import MongoDB aggregate course
5	>A3.close()

 

Hybrid computing

With the help of SPL It's also easy to achieve MongoDB Mix calculations with other data sources ：

	A	B
1	=mongo_open("mongodb://localhost:27017/test")	/ Connect MongDB
2	=mongo_shell(A1,"emp.find({'$and':[{'Birthday':{'$gte':'"+string(begin)+"'}},{'Birthday':{'$lte':'"+string(end)+"'}}]},{_id:0})").fetch()	/ Query the records of a certain period
3	=A1.close()	/ close MongoDB
4	=myDB1.query("select   * from cities")	/ obtain mysql Middle watch cities data
5	=A2.switch(CityID,A4:   CityID)	/ Foreign key link
6	=A5.new(EID,Dept,CityID.CityName:CityName,Name,Gender)	/ Create result sets
7	return   A6	/ return

SQL Support

SPL In addition to native grammar , It also provides the equivalent of SQL92 The standard SQL Support , have access to SQL Inquire about MongoDB 了 , For example, the previous correlation calculation ：

	A
1	=mongo_open("mongodb://127.0.0.1:27017/test")
2	=mongo_shell(A1,"c1.find()").fetch()
3	[email protected](A1,"c2.find()").fetch()
4	$select s.* from {A2} as s left join {A3}   as r on s.user1=r.user1 and s.user2=r.user2 where r.income>0.3

Application Integration

More Than This ,SPL Provides standards JDBC/ODBC And other application program interfaces , Integration call is very convenient . Such as JDBC Use ：

…
Class.forName("com.esproc.jdbc.InternalDriver");
Connection conn = DriverManager.getConnection("jdbc:esproc:local://");
PrepareStatement st=con.prepareStatement("call splScript(?)"); // splScript by spl Script name 
st.setObject(1,"California");
st.execute();
ResultSet rs = st.getResultSet();
…

With these features , enhance MongoDB Computing power is not just talk , Do you want to try downloading ？

SPL Information

• SPL download
• SPL Source code