Getting started with go web programming: validators

Preface

Network authentication can be a challenge . There is a saying in Web Principles that are widely circulated in development ：

We can't trust anything from the client user form .

So we have to validate all the incoming data before using it . Realization REST API yes Go Typical use cases for applications . API The accepted data with wrong format may cause serious errors in other parts of the system .

The best case scenario is that your database has mechanisms to prevent the storage of malformed data . If you don't do that , This data can cause errors and unexpected behavior in your customer facing applications （ such as SQL Inject ）.

In this article , We'll show you how to Go Send validation to REST API The data of .

Manually verify the input

Simple REST API

This is an easy one REST API Example , Use gorilla/mux Package building . It's a great HTTP Router , Especially for REST API. API Provide a path for an endpoint /user. For the sake of simplicity , It only accepts... From all users HTTP GET And create a user's HTTP Post. Besides , It has no persistent database , Instead, you use slices to store users in memory .

package mainimport (    "encoding/json"    "log"    "net/http"    "strings"    "github.com/gorilla/mux")type User struct {    ID                 int    FirstName          string    LastName           string    FavouriteVideoGame string    Email               string}func main() {    router := mux.NewRouter()    router.HandleFunc("/user", PostUser).Methods(http.MethodPost)    router.HandleFunc("/user", GetUsers).Methods(http.MethodGet)    log.Fatal(http.ListenAndServe(":8081", router))}var users = []User{}var id = 0func validateEmail(email string) bool {    // This is obviously not a good validation strategy for email addresses    // pretend a complex regex here    return !strings.Contains(email, "@")}func PostUser(w http.ResponseWriter, r *http.Request) {    user := User{}    json.NewDecoder(r.Body).Decode(&user)    // We don't want an API user to set the ID manually    // in a production use case this could be an automatically    // ID in the database    user.ID = id    id++    users = append(users, user)    w.WriteHeader(http.StatusCreated)}func GetUsers(w http.ResponseWriter, r *http.Request) {    w.Header().Add("Content-Type", "application/json")    if err := json.NewEncoder(w).Encode(users); err != nil {        log.Println(err)        w.WriteHeader(http.StatusInternalServerError)        return    }}

Now let's look at how to manually validate what is provided to this in the request body API Of POST Input to the handler .

Manually verify the input

Sometimes we ask the user to enter some fields , But they failed to complete the field . For example, in the previous section , When we need a user name . You can use len Function to get the length of the field , To ensure that the user has entered something .

    if len(r.Form["username"][0])==0{        // code for empty field    }

Suppose we want to use Post When the handler creates a user, it sets up FirstName、LastName and Email. Besides , We want the email field to be a valid email address . An easy way to do this is to manually validate fields , As shown below ：

if user.FirstName == "" {   errs = append(errs, fmt.Errorf("Firstname is required").Error())}if user.LastName == "" {   errs = append(errs, fmt.Errorf("LastName is required").Error())}if user.Email == "" || validateEmail(user.Email) {   errs = append(errs, fmt.Errorf("A valid Email is required").Error())}

Complete example ：

package mainimport (    "encoding/json"    "fmt"    "log"    "net/http"    "strings"    "github.com/gorilla/mux")type User struct {    ID                 int    FirstName          string    LastName           string    FavouriteVideoGame string    Email               string}func main() {    router := mux.NewRouter()    router.HandleFunc("/user", PostUser).Methods(http.MethodPost)    router.HandleFunc("/user", GetUsers).Methods(http.MethodGet)    log.Fatal(http.ListenAndServe(":8081", router))}var users = []User{}var id = 0func validateEmail(email string) bool {    // That's obviously not a good validation strategy for email addresses    // pretend a complex regex here    return !strings.Contains(email, "@")}func PostUser(w http.ResponseWriter, r *http.Request) {    user := User{}    json.NewDecoder(r.Body).Decode(&user)    errs := []string{}    if user.FirstName == "" {        errs = append(errs, fmt.Errorf("Firstname is required").Error())    }    if user.LastName == "" {        errs = append(errs, fmt.Errorf("LastName is required").Error())    }    if user.Email == "" || validateEmail(user.Email) {        errs = append(errs, fmt.Errorf("A valid Email is required").Error())    }    if len(errs) > 0 {        w.Header().Add("Content-Type", "application/json")        w.WriteHeader(http.StatusBadRequest)        if err := json.NewEncoder(w).Encode(errs); err != nil {        }        return    }    // We don't want an API user to set the ID manually    // in a production use case this could be an automatically    // ID in the database    user.ID = id    id++    users = append(users, user)    w.WriteHeader(http.StatusCreated)}func GetUsers(w http.ResponseWriter, r *http.Request) {    w.Header().Add("Content-Type", "application/json")    if err := json.NewEncoder(w).Encode(users); err != nil {        log.Println(err)        w.WriteHeader(http.StatusInternalServerError)        return    }}

You can see that this verification method is very verbose . We have to define a custom function to validate common things , Like email addresses . Let's see how to improve this .

It's just a simple pass validateEmail Function to verify whether the mailbox contains @ character ：

func validateEmail(email string) bool {    // That's obviously not a good validation strategy for email addresses    // pretend a complex regex here    return !strings.Contains(email, "@")}

In fact, a better way is to use regular expressions to verify E-mail The effectiveness of the ：

    if m, _ := regexp.MatchString(`^([\w\.\_]{2,10})@(\w{1,}).([a-z]{2,4})$`, r.Form.Get("email")); !m {        fmt.Println("no")    }else{        fmt.Println("yes")    }

Use the structure tag to validate the input

stay Go A more common way to validate structures in is to use structure tags . There are many packages that perform structure validation through structure tags . We will use... Here https://github.com/go-playground/validator： The Validator Implement structure and single field value validation based on tags .

Use go get github.com/go-playground/validator/v10 Installation .

This not only enables us to use structure tags for verification , It also provides a number of predefined validation methods , An E-mail address, for example .

We will perform this verification on the structure , But it doesn't discuss how to fill the structure . We can assume that the data will be parsed JSON Payload 、 Enter explicit padding or other methods from the form to populate .

If your data needs another validator , Please check the documentation of the validator package . The verifier you need is most likely provided in the package 80 Under multiple validators .

package mainimport (    "encoding/json"    "log"    "net/http"    "github.com/go-playground/validator/v10"    "github.com/gorilla/mux")type User struct {    ID                 int    `validate:"isdefault"`    FirstName          string `validate:"required"`    LastName           string `validate:"required"`    FavouriteVideoGame string    Email               string `validate:"required,email"`}func main() {    router := mux.NewRouter()    router.HandleFunc("/user", PostUser).Methods(http.MethodPost)    router.HandleFunc("/user", GetUsers).Methods(http.MethodGet)    log.Fatal(http.ListenAndServe(":8081", router))}var users = []User{}var id = 0func PostUser(w http.ResponseWriter, r *http.Request) {    user := User{}    json.NewDecoder(r.Body).Decode(&user)    validate := validator.New()    err := validate.Struct(user)    if err != nil {        validationErrors := err.(validator.ValidationErrors)        w.Header().Add("Content-Type", "application/json")        w.WriteHeader(http.StatusBadRequest)        responseBody := map[string]string{"error": validationErrors.Error()}        if err := json.NewEncoder(w).Encode(responseBody); err != nil {        }        return    }    // We don't want an API user to set the ID manually    // in a production use case this could be an automatically    // ID in the database    user.ID = id    id++    users = append(users, user)    w.WriteHeader(http.StatusCreated)}func GetUsers(w http.ResponseWriter, r *http.Request) {    w.Header().Add("Content-Type", "application/json")    if err := json.NewEncoder(w).Encode(users); err != nil {        log.Println(err)        w.WriteHeader(http.StatusInternalServerError)        return    }}

above validationError.Error() Returns a string , This string summarizes each failed validation in the structure . therefore BadRequest The response still has very detailed information about what went wrong .

We changed validation to use the validator package , Now verify according to the following rules ：

• ID Fields should not be set by the user , So we verify that it has int The default value of , namely 0

• FullName and LastName It's necessary

• Email fields are required , And use a predefined email validator to verify

Use a custom validator to validate input

So now we use the validator package and use the structure tag to validate the structure . But how do we verify a structure field that cannot be verified by the tag provided by the library ？

Suppose we want to blacklist some video games . Because we don't want anyone in our system to like PUBG or Fortnite Waiting for game users . under these circumstances , We can define a custom validate Mark values and let validate The package uses it like this ：

First, we define a validation function ：

func GameBlacklistValidator(f1 validator.FieldLevel) bool {    gameBlacklist := []string{"PUBG", "Fortnite"}    game := f1.Field().String()    for _, g := range gameBlacklist {        if game == g {            return false        }    }    return true}

Then we use the validator instance to register the function and the corresponding tag .

...    validate := validator.New()    validate.RegisterValidation("game-blacklist", GameBlacklistValidator)...

Now we are User Add a label to the definition of the structure .

type User struct {    ID                 int    `validate:"isdefault"`    FirstName          string `validate:"required"`    LastName           string `validate:"required"`    FavouriteVideoGame string `validate:"game-blacklist"`    Email              string `validate:"required,email"`}

Recommended validation Libraries

Awesome Go Under the project, there are libraries for verification . Here are the recommendations ：

• checkdigit - Provide check digit algorithms (Luhn, Verhoeff, Damm) and calculators (ISBN, EAN, JAN, UPC, etc.).

• gody - :balloon: A lightweight struct validator for Go.

• govalid - Fast, tag-based validation for structs.

• govalidator - Validators and sanitizers for strings, numerics, slices and structs.

• govalidator - Validate Golang request data with simple rules. Highly inspired by Laravel’s request validation.

• jio - jio is a json schema validator similar to joi.

• ozzo-validation - Supports validation of various data types (structs, strings, maps, slices, etc.) with configurable and extensible validation rules specified in usual code constructs instead of struct tags.

• terraform-validator - A norms and conventions validator for Terraform.

• validate - Go package for data validation and filtering. support validate Map, Struct, Request(Form, JSON, url.Values, Uploaded Files) data and more features.

• validate - This package provides a framework for writing validations for Go applications.

• validator - Go Struct and Field validation, including Cross Field, Cross Struct, Map, Slice and Array diving.

• Validator - A lightweight model validator written in Go.Contains VFs:Min, Max, MinLength, MaxLength, Length, Enum, Regex.

According to the introduction, pick one of your own projects ~

summary

verification REST API Input is critical to prevent malformed data in your application . You can write your own validation logic , But in most cases , It is best to use a well maintained validation package , For example, the above recommendation .

This allows you to use tags in the structure to configure authentication , And keep the logic of running validation simple . If you have a special use case that requires unusual validation capabilities , You can still define your own extensions for the validator package .

• Package validatorValidation

• Awesome Go：https://awesome-go.com/validation/

• Client-side form validation

• Validate REST Input in Go

• Verification of inputs