当前位置：网站首页>Do you want to have a robot that can make cartoon avatars in three steps?

Do you want to have a robot that can make cartoon avatars in three steps?

2022-07-24 05:07:00 【Huawei cloud developer Alliance】

Abstract ：3 A robot that can make cartoon avatars in one step , Want to have one ？

author ： Hu Qi

Just send keywords , You can start the cartoon processing program of chat robot avatar , Generate animation pictures . How to achieve it ？ The general idea is based on ModelArts AI Gallery Quick verification AnimateGanv2 Animation ability of realistic photos , be based on Flask Quickly deploy animation services , be based on Wechaty Quickly develop anime chat robots .

Why AnimeGAN?

Comic (Animation & Comic) As a common form of artistic expression in daily life , In children's education 、 Movies 、 It is widely used in advertising and other fields ; But animation creation is difficult 、 Cycle is long 、 Difficult to develop , The requirements for creators are also very strict , Generally speaking, a good animation work requires the creator to master the lines 、 texture 、 Painting skills such as color and shadow ; If ordinary people want to create their own animation works quickly, they have to use tools . In recent years , With the continuous development of the meta universe , People have more and more demand for animation , For example, generate a quadratic self portrait 、 Make NFT Art, etc , Therefore, animation style migration seems to become “ Alchemist ” One of our favorite research directions .

The transfer of image style has to start from pix2pix and CycleGAN Speaking of , These two are based on GAN The style transfer algorithm based on has laid a technical foundation for the generation of animated cartoon pictures , Follow up CartoonGAN Through the semantic content loss function and the edge enhanced adversarial loss function, the quality of the generated animation pictures is higher . and AnimeGAN be based on CartoonGAN improvement , A more lightweight generator architecture and grayscale style loss are proposed 、 There are three new loss functions of gray level confrontation loss and color reconstruction loss , Make its stylized visual effect surpass CartoonGAN.

After all ,AnimeGAN It makes xinhaicheng feel very interesting ; And for us ordinary people , There is a possibility of creation , It only takes a real picture to pass AI Output master animation works ！

What is? AnimeGAN ？

mention AnimeGAN , I have to talk about its development , One work Asher Chan from 2019 Submitted the first Git Commit so far , Three versions have been iterated ,AnimeGANv3 Released the executable program and the hot portrait sketch just released （ Click on my quick experience ：Run in ModelArts - AnimeGANv3 Portrait sketch generation ）.

AnimeGAN It realizes the transformation of photos of real-world scenes into animation style images . AnimeGANv1 Three loss functions are proposed ： Grayscale style loss 、 Color reconstruction loss and gray level confrontation loss .
It's solved ：

1. The generated image has no obvious animation style texture
2. The generated image lost the contents of the original image ;
3. The parameters of the network need large storage capacity .

AnimeGANv2 Fixed problems in the previous version , For example, there are high-frequency artifacts in the image generated by the model ; It is easier to train and can directly achieve the effect of the paper ; Further reduce the network parameters to make the generator smaller ; Use style data with higher image quality as much as possible . The author didn't republish his paper because he didn't think it was innovative .

AnimeGANv3 Based on and Google Business license for , The author does not provide the source code for the time being . However, graphical user interface programs are currently available (AnimeGANv3.exe) And the pre training model (onnx.zip) , At present, we can directly in Windows Experience the image or video to animation style . As of the author's posting , The author was updated by AnimeGANv3_PortraitSketch Used to generate portraits , That is, the effect of the above figure .

I have the honor to experience AnimeGAN Fun , On the left in the figure below is Huawei cloud ModelArts Results of running on , Animation avatars can be obtained by recognizing the key points of the face and then performing style transfer ; On the right is AnimeGANv3.exe The result of running locally , We don't need a relational environment and code execution , Simple operation can generate animation picture ; In the middle is the protagonist of this sharing – A chat robot that animates pictures .

stay AI Gallery Verify in advance

We usually think that “ data 、 Algorithm 、 Calculate the force ” Are the three elements of artificial intelligence , Today, , In the era of big data , The acquisition of reliable and high-quality data becomes simple ; High quality algorithms have become popular with the development of open source culture ; However, computing power has indeed become a constraint on the popularity of artificial intelligence “ Land Rover ”, Even AnimeGAN The author of also sighed “ The publication of the paper was delayed only because it was borrowed for only one year 2080ti”. For the author , Fortunately, there are universal benefits AI Huawei cloud ModelArts,AI Gallery Is in ModelArts On the basis of the development of ecological community , Provides Notebook Code samples 、 Data sets 、 Algorithm 、 Model 、Workflow etc. AI Sharing of digital assets , Let's understand it as AI Session of the Github.

Data aspect ,AI Gallery The data module of supports the sharing and downloading of data sets ; And datasets are supported License Declarative , This is similar to the open source community ;
An algorithm ,AI Gallery The algorithm module of supports algorithm sharing and subscription ; And the algorithm supports cashing , Similar to an algorithm mall ;
In terms of computational power , The user is in AI Gallery By clicking “Run in ModelArts” Can be Notebook The case is ModelArts The console opens quickly 、 Operation and secondary development , At present, limited free computing power is provided .

Transform through learning AI Gallery Existing AnimeGAN Case study , We can run fast NoteBook And get anime avatars .

be based on Wechaty Quickly build robots

Talking about the development of chat robot , As a front-end Engineer , The author believes that the quickest way is import {Wechaty} from "wechaty";, Yes ,Wechaty Is an open source dialogue robot SDK, Support Personal number WeChat . It is a use Typescript Built Node.js application . Support multiple wechat access schemes , Including web pages ,ipad,ios,windows, android etc. . Support at the same time Linux, Windows, Darwin(OSX/Mac) and Docker Multiple platforms . Therefore, it is very simple for us to realize the chat robot ！（PS： except token It's a little expensive , Of course, if you have the ability and creativity, you can join the official funding program ）.

Chat robot has , How to connect next AnimeGAN What about our ability ？ In order to provide services for robots , We need to deploy an application to provide an interface to the front end , Let's use Flask Quickly deploy avatar animation services .

from flask import *
# import request
import os
import uuid
import numpy as np
from animeGANv2 import *

app = Flask(__name__,template_folder='view')
app.config['MAX_CONTENT_LENGTH'] = 0.5 * 1024 * 1024  # 3MB

#  Convert picture file 
@app.route('/postdata', methods=['POST'])
def postdata():
    print(request)
    f = request.files['content']
    print(f)
    user_input = request.form.get("name")
    basepath = os.path.dirname(__file__)  #  The path of the current file 
    src_imgname = str(uuid.uuid1()) + ".jpg"
    upload_path = os.path.join(basepath, 'static/srcImg/')

    if os.path.exists(upload_path)==False:
        os.makedirs(upload_path)
    f.save(upload_path + src_imgname)
    # img = cv2.imread(upload_path + src_imgname, 1)

    save_path = os.path.join(basepath, 'static/resImg/')
    if os.path.exists(save_path) == False:
        os.makedirs(save_path)
    fileSize = os.path.getsize(upload_path+src_imgname)
    if(fileSize / 1024 / 1024 > 1):
        resSets = dict()
        resSets["value"] = 10
        resSets["resurl"] = "http://127.0.0.1:5000" +'/static/resImg/' + src_imgname
    else:
        inference_from_file(upload_path+src_imgname,os.path.join(save_path, src_imgname))
        resSets = dict()
        resSets["value"] = 10
        resSets["resurl"] = "http://127.0.0.1:5000" +'/static/resImg/' + src_imgname
    return json.dumps(resSets, ensure_ascii=False)

#  Convert picture links 
@app.route('/postdataUrl', methods=['POST'])
def postdataUrl():
    url = request.values['content']
    print(url)
    user_input = request.form.get("name")
    basepath = os.path.dirname(__file__)  #  The path of the current file 
    src_imgname = str(uuid.uuid1()) + ".jpg"

    save_path = os.path.join(basepath, 'static/resImg/')
    if os.path.exists(save_path) == False:
        os.makedirs(save_path)
    inference_from_url(url,os.path.join(save_path, src_imgname))
    resSets = dict()
    resSets["value"] = 10
    resSets["resurl"] = "http://127.0.0.1:5000" +'/static/resImg/' + src_imgname
    return json.dumps(resSets, ensure_ascii=False)

if __name__ == '__main__':
   app.run(threaded=True)

The complete code can be found in ：GitHub - hu-qi/MDG-AnimeGANv2

The general effect is as follows ：