help one another in defense work

One 、 Project brief introduction ：        

        I've finished recently Linux Knowledge of operating system , I want to do a project to integrate my knowledge , I found that I saved a lot of files on my computer during my usual study , But computers are usually not convenient to carry , Think about whether you can save files in the cloud , When you are no longer next to the computer, you can get files through the mobile browser . Do what you say . First, release my project design drawings .

        The overall design of the project is ： The server on the ECS is used to compress and store the files uploaded by the client , Provide download service through the browser when users use , The local client monitors the specified directory , Upload the non hot files in the directory .

Two 、 Solution

1、 Determine whether the file is a hot file ？

      stay Linux In the kernel stat Structure , The last modification time of the file is saved in this structure , If the current time minus the last modification time > Set threshold （10s）, Indicates a non hot file

      stay Windows Next , Through documents etag Information , Get the size of the file and the last modification time , Organized into string, And saved in the initialization phase etag Compare information , If the two are different , Think that this is a non hot file .

2、HTTP How is the server built ？

      Used... In the project GitHub Open source on httplib library , Directly use the ready-made interfaces provided in the Library ,http The main functions provided by the server are ： Respond to file list requests 、 Respond to file upload request 、 Respond to file download request .

      stay httplib Open source library 、 Provides Server and Client class ,Server Class for HTTP The request type is encapsulated , For example, the file upload request of the client PUT, The encapsulated overloaded functions are as follows ：

std::shared_ptr<Response> Put(const char *path, const std::string &body,
                                const char *content_type);

  std::shared_ptr<Response> Put(const char *path, const Headers &headers,
                                const std::string &body,
                                const char *content_type);

  std::shared_ptr<Response> Put(const char *path, size_t content_length,
                                ContentProvider content_provider,
                                const char *content_type);

It's easy to understand , The resource path of the incoming request , Body data , And body data types , It can send data to the designated server .

Next, let's look at the server ： It should be able to distinguish different client requests , Respond accordingly ：

 Server &Get(const char *pattern, Handler handler);
  Server &Post(const char *pattern, Handler handler);
  Server &Post(const char *pattern, HandlerWithContentReader handler);
  Server &Put(const char *pattern, Handler handler);
  Server &Put(const char *pattern, HandlerWithContentReader handler);
  Server &Patch(const char *pattern, Handler handler);
  Server &Patch(const char *pattern, HandlerWithContentReader handler);
  Server &Delete(const char *pattern, Handler handler);
  Server &Options(const char *pattern, Handler handler);

You can see the different requests of the corresponding clients ,Server Class provides a series of response functions , The first parameter is the requested resource path , The second parameter is the corresponding callback function , Come out of the client's request . This is how I use these functions ：

You can see that the browser accesses the host IP And the request resource path after the port , The server can call the callback function developed by the programmer to process the request .

class Server
    {
        public:
            bool Start()
            {
                _server.Put("/(.*)",Upload);
                _server.Get("/list",List);
                _server.Get("/download/(.*)",Download);
                //(.*) Regular Expression Matching /download/ Any character after , To avoid having a file named list And above list Request obfuscation 
                // Three request callback functions are added to the routing table , What methods are called back for what requests 
               _server.listen("0.0.0.0",9000);
                return true;

            }// Start the network communication module 
        private:
            // File upload request  
            static void Upload(const httplib::Request & req,httplib::Response &rsp)
            {
               /* rsp.status=200;
                rsp.set_content("upload",6,"text/html");
            */
                std::string filename=req.matches[1];
                std::string pathname=BACKUP_DIR+filename;
                FileUtil::Write(pathname,req.body);// Write data to the backup folder , Create if file does not exist 
                data_manage.Insert(filename,filename);
                rsp.status=200;
                return;
            }

            // File list request 
            static void List(const httplib::Request & req,httplib::Response &rsp)
            {
              /*  rsp.status=200;
               // set_content( Body data , Length of body data , annotation type -Content-Type)
               // rsp.body=upload;
               //rsp.set_header("content-Type","text/html");
               
               rsp.set_content("list",4,"text/html");
                */
                std::vector<std::string> list;
                data_manage.GetAllName(&list);
                std::stringstream tmp;
                tmp<<"<html><body><hr />";
                for(int i=0;i<list.size();++i)
                {
                    tmp<<"<a href='/download/"<<list[i]<<"'>"<<list[i]<<"</a>";
                    //tmp<<"<a href='/download/a.txt'>"<<"a.txt"<< "</a>"
					// Organize into hyperlinks , Send it to the server when clicking on the file name 
                    tmp<<"<hr />";
                }
                tmp<<"<hr /></body></html>";
                 rsp.set_content(tmp.str().c_str(),tmp.str().size(),"text/html");
                 rsp.status=200;
                return ;

            }
            // File download processing callback function 
            static void Download(const httplib::Request & req,httplib::Response &rsp)
            {
                std::string filename =req.matches[1];
                if(data_manage.Exists(filename)==false)
                {
                    rsp.status=404;
                    return ;
                }
                std::string pathname =BACKUP_DIR+filename;
                if(data_manage.IsCompress(filename)==true)
                {

                    std::string gzfile;
                    data_manage.GETGzname(filename,&gzfile);
                    std::string gzpathname=GZFILE_DIR+gzfile;
                    
                    CompressUtil::UnCompress(gzpathname,pathname);
                    data_manage.Insert(filename,filename);
                    unlink(gzpathname.c_str());

                }
                
                    FileUtil::Read(pathname,&rsp.body);
                    rsp.set_header("Content-Type","application/octet-stream");// Binary stream file download 
                    rsp.status=200;
                    return ;
            }
        private:
            std::string _file_dir;// File upload backup path 
            httplib::Server _server;


    };

explain ：download The request is triggered by a hyperlink on the browser , It can be downloaded directly in the browser , Callback functions have only two parameters , So it is defined as a static function , There is no implied this The pointer .

3、 Why does the server use read-write locks

      Because multiple execution flows are targeted at critical resources in the file management module hash object , Read and write , Read write lock is characterized by read sharing and write mutual exclusion , It can improve the response speed of the program , Document management module hash The same is true of object operation logic .

4、 What is the workflow of the document management module ？

      The file management module of the server uses a hash object , adopt pair Of first and second Mark the status of the file , On the server , When initializing the load , Yes pair Put the two elements in the same file name , Indicates that it is not compressed , When the file is identified as non hotspot and compressed , Change the correspondence pair Medium second Parameters , Become the name of the compressed package , At this time, it is considered that the file has been compressed .

    The same principle applies to the file management module of the client , It's just pair Of second Saved as a file etag Information .

3、 ... and 、 Project address ：https://github.com/lixuhui123/cloud_backup