当前位置：网站首页>6. Object storage

6. Object storage

2022-07-25 02:26:00 【MASA team】

What is object storage

At work , We often need to put the contents of documents （ File or binary stream ） Stored in the application , For example, you may want to save the cover image of the product .Masa The framework provides the function of object storage , And abstract the function , The benefits of abstraction :

Storage independence （ Alibaba cloud does not care about the storage platform OSS Or Tencent cloud COS）
It is cheaper to replace the storage platform （ Just change the provider of the storage , Low business infection ）
Support custom storage providers （ You only need to implement it yourself IClient）

Object storage provider

Alibaba cloud : stay Alibaba cloud OSS Storage service On storage

Currently, only Alibaba cloud storage is supported , In the future, we will gradually provide more cloud storage platform support , If you have other cloud storage platforms you like , Welcome to Suggest , Or implement it yourself and make it Masa Frame making contribution

How to make a custom storage program ?

Quick start

Masa.BuildingBlocks.Storage.ObjectStorage It is an abstract package of object storage services , You can use it in your project to write code , Last in Program.cs Select a storage provider to use

install .Net 6.0

newly build ASP.NET Core Empty item Assignment.OSS, And install Masa.Contrib.Storage.ObjectStorage.Aliyun

dotnet new web -o Assignment.OSS
cd Assignment.OSS
dotnet add package Masa.Contrib.Storage.ObjectStorage.Aliyun --version 0.5.0-preview.2

modify Program.cs

builder.Services.AddAliyunStorage();

#region  Or specify the incoming Alibaba cloud storage configuration information through code , There is no need to use a configuration file 
// builder.Services.AddAliyunStorage(new AliyunStorageOptions()
// {
//     AccessKeyId = "Replace-With-Your-AccessKeyId",
//     AccessKeySecret = "Replace-With-Your-AccessKeySecret",
//     Endpoint = "Replace-With-Your-Endpoint",
//     RoleArn = "Replace-With-Your-RoleArn",
//     RoleSessionName = "Replace-With-Your-RoleSessionName",
//     Sts = new AliyunStsOptions()
//     {
//         RegionId = "Replace-With-Your-Sts-RegionId",
//         DurationSeconds = 3600,
//         EarlyExpires = 10
//     }
// }, "storage1-test");
#endregion

modify appsettings.json, Add alicloud configuration

{
  "Aliyun": {
    "AccessKeyId": "Replace-With-Your-AccessKeyId",
    "AccessKeySecret": "Replace-With-Your-AccessKeySecret",
    "Sts": {
      "RegionId": "Replace-With-Your-Sts-RegionId",
      "DurationSeconds": 3600,
      "EarlyExpires": 10
    },
    "Storage": {
      "Endpoint": "Replace-With-Your-Endpoint",
      "RoleArn": "Replace-With-Your-RoleArn",
      "RoleSessionName": "Replace-With-Your-RoleSessionName",
      "TemporaryCredentialsCacheKey": "Aliyun.Storage.TemporaryCredentials",
      "Policy": "",
      "BucketNames" : {
        "DefaultBucketName" : "storage1-test"// Default BucketName, Not required , Use only IClientContainer You need to specify 
      }
    }
  }
}

New upload file service

app.MapPost("/upload", async (HttpRequest request, IClient client) =>
{
    var form = await request.ReadFormAsync();
    var formFile = form.Files["file"];
    if (formFile == null)
        throw new FileNotFoundException("Can't upload empty file");

    await client.PutObjectAsync("storage1-test", formFile.FileName, formFile.OpenReadStream());
});

Advanced

IClient

IClient It is the main interface for storing and reading objects , It can be passed anywhere in the project DI Get IClient To upload 、 Download or delete the specified BucketName The next object , It can also be used to judge whether the object exists , Obtain temporary vouchers, etc .

Upload object

app.MapPost("/upload", async (HttpRequest request, IClient client) =>
{
    var form = await request.ReadFormAsync();
    var formFile = form.Files["file"];
    if (formFile == null)
        throw new FileNotFoundException("Can't upload empty file");

    await client.PutObjectAsync("storage1-test", formFile.FileName, formFile.OpenReadStream());
});

Form Form submission ,key by file, The type is file upload

Delete object

public class DeleteRequest
{
    public string Key { get; set; }
}

app.MapDelete("/delete", async (IClient client, [FromBody] DeleteRequest request) =>
{
    await client.DeleteObjectAsync("storage1-test", request.Key);
});

Determines whether an object exists

app.MapGet("/exist", async (IClient client, string key) =>
{
    await client.ObjectExistsAsync("storage1-test", key);
});

Return the stream of object data

app.MapGet("/download", async (IClient client, string key, string path) =>
{
    await client.GetObjectAsync("storage1-test", key, stream =>
    {
        // Download the file to the specified path 
        using var requestStream = stream;
        byte[] buf = new byte[1024];
        var fs = File.Open(path, FileMode.OpenOrCreate);
        int len;
        while ((len = requestStream.Read(buf, 0, 1024)) != 0)
        {
            fs.Write(buf, 0, len);
        }
        fs.Close();
    });
});

Get a temporary voucher (STS)
```
app.MapGet("/GetSts", (IClient client) =>
{
    client.GetSecurityToken();
});
```
Alibaba cloud 、 Tencent cloud storage Wait for the platform to use STS To get temporary vouchers
Get a temporary voucher ( Temporary credentials of string type )
```
app.MapGet("/GetToken", (IClient client) =>
{
    client.GetToken();
});
```
Qiniuyun And other storage platforms

IBucketNameProvider

IBucketNameProvider It's for getting BucketName The interface of , adopt IBucketNameProvider You can get the specified storage space BucketName, by IClientContainer Provide BucketName Ability , Will not be used in business projects

IClientContainer

IClientContainer Object storage container , The main interface for storing and reading objects , There may be multiple management under one application BucketName, By using IClientContainer, Image management DbContext The same management is different Bucket The object of , There is no need to specify frequently in the project BucketName, In the same application , There is and only one default ClientContainer, Can pass DI obtain IClientContainer To use , for example ：

Upload object （ Upload to default Bucket）

app.MapPost("/upload", async (HttpRequest request, IClientContainer clientContainer) =>
{
    var form = await request.ReadFormAsync();
    var formFile = form.Files["file"];
    if (formFile == null)
        throw new FileNotFoundException("Can't upload empty file");

    await clientContainer.PutObjectAsync(formFile.FileName, formFile.OpenReadStream());
});

Upload to specified Bucket

[BucketName("picture")]
public class PictureContainer
{

}

builder.Services.Configure<StorageOptions>(option =>
{
    option.BucketNames = new BucketNames(new List<KeyValuePair<string, string>>()
    {
        new("DefaultBucketName", "storage1-test"),// Default BucketName
        new("picture", "storage1-picture")// Specify the alias as picture Of BucketName by storage1-picture
    });
});

app.MapPost("/upload", async (HttpRequest request, IClientContainer<PictureContainer> clientContainer) =>
{
    var form = await request.ReadFormAsync();
    var formFile = form.Files["file"];
    if (formFile == null)
        throw new FileNotFoundException("Can't upload empty file");

    await clientContainer.PutObjectAsync(formFile.FileName, formFile.OpenReadStream());
});

IClientFactory

IClientFactory Object storage provider factory , By designation BucketName, Create the specified IClientContainer

Create an object storage provider

Take adapting Tencent cloud storage as an example ：

The new library Masa.Contrib.Storage.ObjectStorage.Tencent
Choose Masa.Contrib.Storage.ObjectStorage.Tencent And create a new class DefaultStorageClient, And implement IClient
Because Tencent cloud storage provides Sts Provisional certificate , So we only need to realize GetSecurityToken The method can ,GetToken Method can throw unsupported exceptions , And explain it in the document

The new class ServiceCollectionExtensions, And to provide for IServiceCollection Extension method of AddTencentStorage, for example ：

public static IServiceCollection AddTencentStorage(
    this IServiceCollection services,
    TencentStorageOptions options,
    string? defaultBucketName = null)
{
    //todo:  Add Tencent cloud storage client 
    if (defaultBucketName != null)
    {
        services.Configure<StorageOptions>(option =>
        {
            option.BucketNames = new BucketNames(new List<KeyValuePair<string, string>>()
            {
                new(BucketNames.DEFAULT_BUCKET_NAME, defaultBucketName)
            });
        });
        services.TryAddSingleton<IClientContainer>(serviceProvider
            => new DefaultClientContainer(serviceProvider.GetRequiredService<IClient>(), defaultBucketName));
    }
    services.TryAddSingleton<IClientFactory, DefaultClientFactory>();
    services.TryAddSingleton<ICredentialProvider, DefaultCredentialProvider>();
    services.TryAddSingleton<IClient, DefaultStorageClient>();
    return services;
}

summary

Currently, object storage does not support multi tenancy 、 Multiple environments , Subsequently, we will gradually improve and increase multi tenancy according to the situation 、 Multi environment support , To suit different tenants 、 Objects in different environments are stored in the specified Bucket in