Database operation through shell script

Original address :https://www.cnblogs.com/jiangxiaobo/p/9897041.html

---

When doing some daily operation and maintenance work , Often need some shell The script monitors the device performance and other parameters . In the past year of work , Most of them are monitoring some information in the database .

So I thought of using it shell+mysql+crontab To implement .

Attached below is passed shell Command line non interactive method of operating database ：

mysql -hhostname -Pport -uusername -ppassword -e relevant mysql Of sql sentence , No use mysql Run at the prompt of mysql, That is to say, you can shell In the operation mysql Methods .

#!/bin/bash

HOSTNAME="192.168.111.84"  # database information 

PORT="3306"

USERNAME="root"

PASSWORD=""

 

DBNAME="test_db_test"  # Database name 

TABLENAME="test_table_test" # The name of the table in the database 

 

# Create database 

create_db_sql="create database IF NOT EXISTS ${DBNAME}"

mysql -h${
    HOSTNAME} -P${PORT} -u${USERNAME} -p${PASSWORD} -e "${create_db_sql}"

 

# Create table 

create_table_sql="create table IF NOT EXISTS ${TABLENAME} ( name varchar(20), id int(11) default 0 )"

mysql -h${
    HOSTNAME} -P${PORT} -u${USERNAME} -p${PASSWORD} ${DBNAME} -e "${create_table_sql}"

 

# insert data 

insert_sql="insert into ${TABLENAME} values('billchen',2)"

mysql -h${
    HOSTNAME} -P${PORT} -u${USERNAME} -p${PASSWORD} ${DBNAME} -e "${insert_sql}"

 

# Inquire about 

select_sql="select * from ${TABLENAME}"

mysql -h${
    HOSTNAME} -P${PORT} -u${USERNAME} -p${PASSWORD} ${DBNAME} -e "${select_sql}"

 

# Update data 

update_sql="update ${TABLENAME} set id=3"

mysql -h${
    HOSTNAME} -P${PORT} -u${USERNAME} -p${PASSWORD} ${DBNAME} -e "${update_sql}"

mysql -h${
    HOSTNAME} -P${PORT} -u${USERNAME} -p${PASSWORD} ${DBNAME} -e "${select_sql}"

 

# Delete data 

delete_sql="delete from ${TABLENAME}"

mysql -h${
    HOSTNAME} -P${PORT} -u${USERNAME} -p${PASSWORD} ${DBNAME} -e "${delete_sql}"

mysql -h${
    HOSTNAME} -P${PORT} -u${USERNAME} -p${PASSWORD} ${DBNAME} -e "${select_sql}"

shell Script connection 、 Reading and writing 、 operation mysql Database instance

This article introduces how to do it in shell Intermediate reading and writing mysql database . Mainly introduced how in shell Middle link mysql database , How to be in shell Create database , Create table , Insert csv file , Read mysql database , export mysql The database for xml or html file , And analyzes the core sentences . The method introduced in this paper is applicable to PostgreSQL , relative mysql for ,shell Intermediate reading and writing PostgreSQL It will be easier .

1. Connect mysql database

shell The method of connecting to the database in is very simple , Just specify the user name , password , The name of the connected database , Then redirect , Input mysql The sentence of , As shown below ：

mysql -u USERNAME -p PASSWORD DATABASENAME <<EOF 2>/dev/null show databases; EOF

There are indeed many embedded mysql Script , There can also be variables in the script , But this statement cannot be written in a function , It can't be written in a loop

But this is not a good way , Any user who uses the script can see the account and password of the database user , To solve this problem , It can be used mysql A special configuration file of the database .mysql Database usage $HOME/.my.cnf File to read special startup commands and settings . One of the settings is initiated by the user account mysql The default password for the session . To set the default password in this file , You can add the following content ：

[client]
password = 123456

then , Don't forget to modify permissions ：
chmod 400 .my.cnf
In this way, you can access mysql Database , As shown below ：

#!/bin/bash
MYSQL=`which mysql`
$MYSQL test -u root << EOF show databases; show tables; select * from employees where salary > 4000; EOF

2. Create database

Connect to the database through the above method , Then input through redirection mysql sentence ,shell Intermediate reading and writing mysql It's almost over . Just put sql The sentence is written correctly , Execute through redirection , Here's an example ：

#!/bin/bash
##############################
# @file create_db_mysql.sh
# @brief create database and tables in mysql
# @author Mingxing LAI
# @version 0.1
# @date 2013-01-20
##############################
USER="root"
DATABASE="students"
TABLE="students"
######################
#crate database
mysql -u $USER << EOF 2>/dev/null CREATE DATABASE $DATABASE EOF
[ $? -eq 0 ] && echo "created DB" || echo DB already exists
######################
#create table
mysql -u $USER $DATABASE << EOF 2>/dev/null CREATE TABLE $TABLE( id int, name varchar(100), mark int, dept varchar(4) ); EOF
[ $? -eq 0 ] && echo "Created table students" || echo "Table students already exist" 
######################
#delete data
mysql -u $USER $DATABASE << EOF 2>/dev/null DELETE FROM $TABLE; EOF

This script is relatively simple , Just a few SQL sentence , There's nothing to explain , So let's see , How to read csv file , Then insert into mysql In the database .

3. Insert csv file

A student table is created above , The student number is shown in the table , full name , achievement , Is don't , Let's say I have a csv file , The contents are as follows ：

$cat data
1,Navin M,98,CS
2,Kavya N,70,CS
3,Nawaz O,80,CS
4,Hari S,80,EC
5,Alex M,50,EC
6,Neenu J,70,EC
7,Bob A,30,EC
8,Anu M,90,AE
9,Sruthi,89,AE
10,Andrew,89,AE

In order to csv Insert file into database , We need to read in line by line , Then put double quotation marks on the string , The final generated statement is as follows ：

insert into students VALUES(1, "Navin M", 98, "CS");

To parse csv file , The best tool is awk 了 , Specify the delimiter of the field as comma -F,,awk The fields are automatically split , Then print out a double quotation mark where you need it , You can easily get the following data ：

1, “Navin M”, 98, “CS” awk The code is as follows ：

query=`echo $line | awk -F, '{ printf("%s,\"%s\",%s,\"%s\"", $1, $2, $3, $4)}'`
statement=`echo "INSERT INTO $TABLE VALUES($query);"`
echo $statement

Yes, of course , You can also use other methods , however , There is hardly anything better than awk It's easier , The first 2 The two methods are as follows ：

oldIFS=$IFS
IFS=,
values=($line)
values[1]="\"`echo ${
      values[1]} | tr ' ' '#' `\""
values[3]="\"`echo ${
      values[3]}`\""

query=`echo ${
     values[@]} | tr ' #' ', '`
IFS=$oldIFS

statement=`echo "INSERT INTO $TABLE VALUES($query);"`
echo "$statement"

First, specify the field separator , take csv The file is parsed into an array , Then replace the space with a special symbol "#"（ Because in the later replacement , The array will be output at one time , The array uses spaces to separate fields , We will replace the space separating the array with commas , So here we replace the spaces in the data with "#"） , Double quotes a string , Finally, replace the space with a comma , hold "#“ Replace with space . This method is really maddening , I didn't understand it for the first time , In particular, why should we replace the space with ”#".

The complete procedure for inserting data is as follows ：

#!/bin/bash
#
# @file write_to_db_mysql.sh
# @brief wirte data to database in mysql
# @author Mingxing LAI
# @version 0.1
# @date 2013-01-20
#
USER="root"
DATABASE="students"
TABLE="students"
if [ $# -ne 1 ]; then
    echo $0 DATAFILE
    echo
    exit 2
fi
data=$1
while  read line;
do
# query=`echo $line | awk -F, '{ printf("%s,\"%s\",%s,\"%s\"", $1, $2, $3, $4)}'`
    oldIFS=$IFS
    IFS=,
    values=($line)
    values[1]="\"`echo ${
      values[1]} | tr ' ' '#' `\""
    values[3]="\"`echo ${
      values[3]}`\""
    query=`echo ${
     values[@]} | tr ' #' ', '`
    IFS=$oldIFS
    statement=`echo "INSERT INTO $TABLE VALUES($query);"`
# echo $statement

mysql -u $USER $DATABASE << EOF INSERT INTO $TABLE VALUES($query); EOF
done < $data
if [[ $? -eq 0 ]]; then
    echo "Wrote data into DB"
fi

4. Reading data

Know how to shell Middle link mysql , I also know how to shell Medium volume execution sql sentence , Reading data , There is no difficulty .

#!/bin/bash
#
# @file read_db_mysql.sh
# @brief read data from mysql
# @author Mingxing LAI
# @version 0.1
# @date 2013-01-20
#
USER="root"
DATABASE="students"
TABLE="students"

# use tail  Remove the meter head 
depts=`mysql -u $USER $DATABASE <<EOF | tail -n +2 SELECT DISTINCT dept FROM $TABLE; EOF`

for d in $depts; do
    echo Department: $d
    result="`mysql -u $USER $DATABASE << EOF set @i:=0; SELECT @i:[email protected]+1 as rank, name, mark FROM students WHERE dept="$d" ORDER BY mark DESC; EOF`"

echo "$result"
echo 
done

We can still do that mysql In the sentence , Use options to control the output format of data

-H  Output is html
-X  Output is xml

As shown below ：

#!/bin/bash
USER="root"
DATABASE="students"
TABLE="students"
mysql -u $USER $DATABASE -H << EOF select * from $TABLE EOF

html The readability of the format is poor , The output effect is as follows ：

<TABLE BORDER=1><TR><TH>id</TH><TH>name</TH><TH>mark</TH><TH>dept</TH></TR><TR><TD>1</TD><TD>Navin M</TD><TD>98</TD><TD>CS</TD></TR><TR><TD>2</TD><TD> Kavya N</TD><TD>70</TD><TD>CS</TD></TR><TR><TD>3</TD><TD> Nawaz O</TD><TD>80</TD><TD>CS</TD></TR><TR><TD>4</TD><TD>Hari S</TD><TD>80</TD><TD>EC</TD></TR><TR><TD>5</TD><TD>Alex M</TD><TD>50</TD><TD>EC</TD></TR><TR><TD>6</TD><TD>Neenu J</TD><TD>70</TD><TD>EC</TD></TR><TR><TD>7</TD><TD>Bob A</TD><TD>30</TD><TD>EC</TD></TR><TR><TD>8</TD><TD>Anu M</TD><TD>90</TD><TD>AE</TD></TR><TR><TD>9</TD><TD>Sruthi</TD><TD>89</TD><TD>AE</TD></TR><TR><TD>10</TD><TD>Andrew</TD><TD>89</TD><TD>AE</TD></TR></TABLE>

Poor readability can also be understood , Because people think , Don't you need to revise , Directly to html Just show the data in form .

id    name    mark    dept
1    Navin M    98    CS
2    Kavya N    70    CS
3    Nawaz O    80    CS
4    Hari S    80    EC
5    Alex M    50    EC
6    Neenu J    70    EC
7    Bob A    30    EC
8    Anu M    90    AE
9    Sruthi    89    AE
10    Andrew    89    AE

xml Formal data display is normal , Just put the top -H Switch to -X, Output is as follows ：

<?xml version="1.0"?>
<resultset statement="select * from students" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <row>
    <field name="id">1</field>
    <field name="name">Navin M</field>
    <field name="mark">98</field>
    <field name="dept">CS</field>
  </row>
  <row>
    <field name="id">2</field>
    <field name="name"> Kavya N</field>
    <field name="mark">70</field>
    <field name="dept">CS</field>
  </row>
</resultset>

Please reprint the original article freely . I would like to share with you , And make progress together .-- jiang coder