Getting started with Apache Shenyu

Introduce

I've been learning about gateway related things recently , Found out shenyu This asynchronous 、 High performance 、 Cross language 、 Responsive  API  gateway . It took a day or two to get started , Record here .

For details, you can go to the official website

Run locally

You need to understand two modules before starting locally :

1-shenyu-admin : Plug in and other information configuration management background , The startup class is as follows

ShenyuAdminBootstrap

2-shenyu-bootstrap : To start the project , The startup class is as follows

ShenyuBootstrapApplication

3- You need to configure it before starting db Information about , So here I'm going to choose mysql, modify shenyu-admin Below application.yml The contents in are as follows , Then configure application-mysql.yml The connection information in .

spring:
  profiles:
    active: mysql

4- Finally, initialize SQL Script :

incubator-shenyu/db/init/mysql/schema.sql

5- Run two startup classes Access address : http://localhost:9095/#/home User name, password admin 123456

At this point, the entire gateway service is started locally , But at this time, there is no access to our own services .

Getting started with service access

We can go straight to shenyu-examples Find the server you want to access demo. such as http,dubbo,motan,springmvc,springcloud wait .

Here I have chosen shenyu-examples-http To test it , It's really just a springboot project . Because we finally need to access through the gateway , The gateway needs to be aware of , So you need to do some configuration first (example Has been configured in , You can choose to modify , I modified it here contextPath and appName)

application.yml:
shenyu:
  register:
    registerType: http #zookeeper #etcd #nacos #consul
    serverLists: http://localhost:9095 #localhost:2181 #http://localhost:2379 #localhost:8848
    props:
      username: admin
      password: 123456
  client:
      http:
        props:
          contextPath: /api/test
          appName: testApp
          port: 8189

The above is mainly about how to configure the service we started to register with the gateway : registerType Representative types include http,zk,nacos etc. , The default here is http. client Some IDs of the current service in the gateway are configured in . Then you can start the application . Next you can use postman Call test . It can be downloaded from demo Medium HttpTestController Select an interface to test directly and through the gateway .

Direct access to the current application http://localhost:8189/test/payment:

Gateway based access http://localhost:9195/api/test/test/payment:

You can specify the above configuration by accessing the address contextPath What's the effect . If the path prefix of gateway based access is not configured by us contextPath The following error will be prompted : "message": "divide:Can not find selector, please check your configuration!"

Analysis of the principle of application access

After doing the most basic introduction above , I can't help but want to explore the principle behind it . So in the connected http example Of pom It is found in the file that it introduces a starter（springboot Medium starter No introduction ） Official address :shenyu.apache.org/zh/docs/des…

<dependency>
    <groupId>org.apache.shenyu</groupId>
    <artifactId>shenyu-spring-boot-starter-client-springmvc</artifactId>
    <version>${project.version}</version>
</dependency>

Then find this starter modular , Find out shenyu There are other starter, such as dubbo Of ,motan Of , You can make these RPC The framework is connected to our gateway .

Let's continue here shenyu-spring-boot-starter-client-springmvc. stay ShenyuSpringMvcClientConfiguration More than one is defined in bean, It mainly depends on

SpringMvcClientBeanPostProcessor
 Realized BeanPostProcessor Interface , stay bean Instantiation 、 Dependency injection 、 When initialization is completed, the execution will call postProcessAfterInitialization Method . The specific source code is as follows :

@Override
public Object postProcessAfterInitialization(@NonNull final Object bean, @NonNull final String beanName) throws BeansException {
    // Filter out is not controller out
    if (Boolean.TRUE.equals(isFull) || !hasAnnotation(bean.getClass(), Controller.class)) {
        return bean;
    }
    // Get path , First get ShenyuSpringMvcClient Comment on the , If not, get RequestMapping Upper 
    final ShenyuSpringMvcClient beanShenyuClient = AnnotationUtils.findAnnotation(bean.getClass(), ShenyuSpringMvcClient.class);
    final String superPath = buildApiSuperPath(bean.getClass());
    // Compatible with previous versions
    if (Objects.nonNull(beanShenyuClient) && superPath.contains("*")) {
        publisher.publishEvent(buildMetaDataDTO(beanShenyuClient, pathJoin(contextPath, superPath)));
        return bean;
    }
    
    // Get the method first ShenyuSpringMvcClient annotation , analysis path
    final Method[] methods = ReflectionUtils.getUniqueDeclaredMethods(bean.getClass());
    for (Method method : methods) {
        final RequestMapping requestMapping = AnnotatedElementUtils.findMergedAnnotation(method, RequestMapping.class);
        ShenyuSpringMvcClient methodShenyuClient = AnnotationUtils.findAnnotation(method, ShenyuSpringMvcClient.class);
        methodShenyuClient = Objects.isNull(methodShenyuClient) ? beanShenyuClient : methodShenyuClient;
        // the result of ReflectionUtils#getUniqueDeclaredMethods contains method such as hashCode, wait, toSting
        // add Objects.nonNull(requestMapping) to make sure not register wrong method
        //
        if (Objects.nonNull(methodShenyuClient) && Objects.nonNull(requestMapping)) {
            publisher.publishEvent(buildMetaDataDTO(methodShenyuClient, buildApiPath(method, superPath)));
        }
    }
    
    return bean;
}

// Finally, the above is to build the parsed annotation into MetaDataRegisterDTO, And pass publisher.publishEvent Send out 

private MetaDataRegisterDTO buildMetaDataDTO(@NonNull final ShenyuSpringMvcClient shenyuSpringMvcClient, final String path) {
    return MetaDataRegisterDTO.builder()
            .contextPath(contextPath)  //yml Configured 
            .appName(appName)          //yml Configured 
            .path(path)
            .pathDesc(shenyuSpringMvcClient.desc())
            .rpcType(RpcTypeEnum.HTTP.getName())
            .enabled(shenyuSpringMvcClient.enabled())
            .ruleName(StringUtils.defaultIfBlank(shenyuSpringMvcClient.ruleName(), path))
            .registerMetaData(shenyuSpringMvcClient.registerMetaData())
            .build();
}

ShenyuClientRegisterEventPublisher
 above publisher.publishEvent Is refers to ShenyuClientRegisterEventPublisher.
 He is based on Disruptor High performance queue to achieve a production consumption model .
 Provides publishEvent Method to produce messages 
 And it provides QueueConsumer For asynchronous consumption 
 Eventually it will be RegisterClientConsumerExecutor To spend 

private final ShenyuClientRegisterEventPublisher publisher = ShenyuClientRegisterEventPublisher.getInstance();


// Starting method , It specifies ShenyuClientMetadataExecutorSubscriber and ShenyuClientURIExecutorSubscriber, stay RegisterClientConsumerExecutor When consuming, I will use .
public void start(final ShenyuClientRegisterRepository shenyuClientRegisterRepository) {
    RegisterClientExecutorFactory factory = new RegisterClientExecutorFactory();
    factory.addSubscribers(new ShenyuClientMetadataExecutorSubscriber(shenyuClientRegisterRepository));
    factory.addSubscribers(new ShenyuClientURIExecutorSubscriber(shenyuClientRegisterRepository));
    providerManage = new DisruptorProviderManage<>(factory);
    providerManage.startup();
}

//ShenyuClientMetadataExecutorSubscriber The content of is where shenyu-admin register Metadata 了 .

//ShenyuClientRegisterRepository  Is in the starter As defined in bean, Here is an introduction , Anyway, we example What you get is HttpClientRegisterRepository

private final ShenyuClientRegisterRepository shenyuClientRegisterRepository;

/**
 * Instantiates a new shenyu client metadata executor subscriber.
 *
 * @param shenyuClientRegisterRepository the shenyu client register repository
 */
public ShenyuClientMetadataExecutorSubscriber(finalShenyuClientRegisterRepository shenyuClientRegisterRepository) {
    this.shenyuClientRegisterRepository = shenyuClientRegisterRepository;
}

@Override
public DataType getType() {
    return DataType.META_DATA;
}

// The message type is DataType.META_DATA Of , The consumer will eventually call this method to process the message 
@Override
public void executor(final Collection<MetaDataRegisterDTO> metaDataRegisterDTOList) {
    for (MetaDataRegisterDTO metaDataRegisterDTO : metaDataRegisterDTOList) {
        shenyuClientRegisterRepository.persistInterface(metaDataRegisterDTO);
    }
}

// The specific implementation is based on http The request will be metaData Registered to admin in 
@Override
public void doPersistInterface(final MetaDataRegisterDTO metadata) {
    doRegister(metadata, Constants.META_PATH, Constants.META_TYPE);
}

 Address of the interface :
String META_PATH = "/shenyu-client/register-metadata";

 We can do it in shenyu-admin Medium ShenyuClientHttpRegistryController Find the corresponding address in .
shenyu-admin How to receive new information changes will be explained later . Let's get to know .

ContextRegisterListener
 Go to... At startup publisher In the production URIRegisterDTO Type of message 

@Override
public void onApplicationEvent(@NonNull final ContextRefreshedEvent contextRefreshedEvent) {
    if (!registered.compareAndSet(false, true)) {
        return;
    }
    if (Boolean.TRUE.equals(isFull)) {
        publisher.publishEvent(buildMetaDataDTO());
    }
    try {
        final int mergedPort = port <= 0 ? PortUtils.findPort(beanFactory) : port;
        publisher.publishEvent(buildURIRegisterDTO(mergedPort));
    } catch (ShenyuException e) {
        throw new ShenyuException(e.getMessage() + "please config ${shenyu.client.http.props.port} in xml/yml !");
    }
}

private URIRegisterDTO buildURIRegisterDTO(final int port) {
    return URIRegisterDTO.builder()
        .contextPath(this.contextPath)
        .appName(appName)
        .protocol(protocol)
        .host(IpUtils.isCompleteHost(this.host) ? this.host : IpUtils.getHost(this.host))
        .port(port)
        .rpcType(RpcTypeEnum.HTTP.getName())
        .build();
}

ShenyuClientRegisterRepository
 Get the specific implementation according to the configuration , The default is http

/**
 * New instance shenyu client register repository.
 *
 * @param shenyuRegisterCenterConfig the shenyu register center config
 * @return the shenyu client register repository
 */
public static ShenyuClientRegisterRepository newInstance(final ShenyuRegisterCenterConfig shenyuRegisterCenterConfig) {
    if (!REPOSITORY_MAP.containsKey(shenyuRegisterCenterConfig.getRegisterType())) {
        //spi Mechanism to obtain concrete implementation , our demo Medium is HttpClientRegisterRepository
        ShenyuClientRegisterRepository result = ExtensionLoader.getExtensionLoader(ShenyuClientRegisterRepository.class).getJoin(shenyuRegisterCenterConfig.getRegisterType());
        result.init(shenyuRegisterCenterConfig);
        ShenyuClientShutdownHook.set(result, shenyuRegisterCenterConfig.getProps());
        REPOSITORY_MAP.put(shenyuRegisterCenterConfig.getRegisterType(), result);
        return result;
    }
    return REPOSITORY_MAP.get(shenyuRegisterCenterConfig.getRegisterType());
}

So far, our application has informed us of the information shenyu-admin.

shenyu-admin How to receive updated messages

shenyu-admin As the management background, the data will be stored in db in , And synchronize the data to the gateway service .

on top http example We already know that our service is based on ShenyuClientRegisterRepository Come on shenyu-admin To register MetaData Waiting for information .

ShenyuClientRegisterRepository There are multiple implementations instantiated according to our configuration . Such as http,nacos... Now let's see admin Here is how to receive registration messages .

be based on http The registration method of is based on ShenyuClientHttpRegistryController Inside the interface to receive messages to achieve registration

@PostMapping("/register-metadata")
@ResponseBody
public String registerMetadata(@RequestBody final MetaDataRegisterDTO metaDataRegisterDTO) {
    publisher.publish(metaDataRegisterDTO);
    return ShenyuResultMessage.SUCCESS;
}

You can also take a look at the nacos How to register , If it is based on nacos To register , be shenyu-admin Will depend on shenyu-register-client-server-nacos Module to listen for registration information .

//shenyu admin It will be initialized at startup bean, And register ShenyuClientRegisterRepository Echo each other 
@Bean(destroyMethod = "close")
public ShenyuClientServerRegisterRepository shenyuClientServerRegisterRepository(final ShenyuRegisterCenterConfig shenyuRegisterCenterConfig,
                                                                           final List<ShenyuClientRegisterService> shenyuClientRegisterService) {
    String registerType = shenyuRegisterCenterConfig.getRegisterType();
    ShenyuClientServerRegisterRepository registerRepository = ExtensionLoader.getExtensionLoader(ShenyuClientServerRegisterRepository.class).getJoin(registerType);
    RegisterClientServerDisruptorPublisher publisher = RegisterClientServerDisruptorPublisher.getInstance();
    Map<String, ShenyuClientRegisterService> registerServiceMap = shenyuClientRegisterService.stream().collect(Collectors.toMap(ShenyuClientRegisterService::rpcType, e -> e));
    publisher.start(registerServiceMap);
    registerRepository.init(publisher, shenyuRegisterCenterConfig);
    return registerRepository;
}

// be based on nacos Implementation class of 
NacosClientServerRegisterRepository
// The above statement bean It will be called when init Method . Will eventually call subscribe Method to monitor 

try {
    this.configService = ConfigFactory.createConfigService(nacosProperties);
    this.namingService = NamingFactory.createNamingService(nacosProperties);
} catch (NacosException e) {
    throw new ShenyuException(e);
}

subscribe();

//subscribe Method will eventually call , Is based on nacos Of API To listen to 

private void subscribeMetadata(final String serviceConfigName) {
    registerMetadata(readData(serviceConfigName));
    LOGGER.info("subscribe metadata: {}", serviceConfigName);
    try {
        configService.addListener(serviceConfigName, defaultGroup, new Listener() {

            @Override
            public Executor getExecutor() {
                return null;
            }

            @Override
            public void receiveConfigInfo(final String config) {
                registerMetadata(config);
            }
        });
    } catch (NacosException e) {
        throw new ShenyuException(e);
    }
}

// In the end, it will call publisher.publish, And based on http The interface of  /register-metadata  The final implementation is the same .
private void publishMetadata(final String data) {
    LOGGER.info("publish metadata: {}", data);
    publisher.publish(Lists.newArrayList(GsonUtis.getInstance().fromJson(data, MetaDataRegisterDTO.class)));
}

// there publisher  And... In the service access described above publisher The principle is the same 
 Is based on Disruptor High performance queue to achieve a production consumption model .

// Consumers will eventually invoke  MetadataExecutorSubscriber Medium 
shenyuClientRegisterService.register(metaDataRegisterDTO);

// here register  了 
public String register(final MetaDataRegisterDTO dto) {
    //handler plugin selector
    String selectorHandler = selectorHandler(dto);
    String selectorId = selectorService.registerDefault(dto, PluginNameAdapter.rpcTypeAdapter(rpcType()), selectorHandler);
    //handler selector rule
    String ruleHandler = ruleHandler();
    RuleDTO ruleDTO = buildRpcDefaultRuleDTO(selectorId, dto, ruleHandler);
    ruleService.registerDefault(ruleDTO);
    //handler register metadata
    registerMetadata(dto);
    //handler context path
    String contextPath = dto.getContextPath();
    if (StringUtils.isNotEmpty(contextPath)) {
        registerContextPath(dto);
    }
    return ShenyuResultMessage.SUCCESS;
}


 be based on http The implementation class is :ShenyuClientRegisterDivideServiceImpl

protected void registerMetadata(final MetaDataRegisterDTO dto) {
    if (dto.isRegisterMetaData()) {
        MetaDataService metaDataService = getMetaDataService();
        MetaDataDO exist = metaDataService.findByPath(dto.getPath());
        metaDataService.saveOrUpdateMetaData(exist, dto);
    }
}

// Final warehousing , And a eventPublisher.publishEvent operation , This operation is to synchronize information to the gateway . The details will be explained later .
public void saveOrUpdateMetaData(final MetaDataDO exist, final MetaDataRegisterDTO metaDataDTO) {
    DataEventTypeEnum eventType;
    MetaDataDO metaDataDO = MetaDataTransfer.INSTANCE.mapRegisterDTOToEntity(metaDataDTO);
    if (Objects.isNull(exist)) {
        Timestamp currentTime = new Timestamp(System.currentTimeMillis());
        metaDataDO.setId(UUIDUtils.getInstance().generateShortUuid());
        metaDataDO.setDateCreated(currentTime);
        metaDataDO.setDateUpdated(currentTime);
        metaDataMapper.insert(metaDataDO);
        eventType = DataEventTypeEnum.CREATE;
    } else {
        metaDataDO.setId(exist.getId());
        metaDataMapper.update(metaDataDO);
        eventType = DataEventTypeEnum.UPDATE;
    }
    // publish MetaData's event
    eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.META_DATA, eventType,
            Collections.singletonList(MetaDataTransfer.INSTANCE.mapToData(metaDataDO))));
}

Here we have a general understanding of the whole MetaData(URIRegister The principle is the same ) Data is registered from the service to shenyu-admin The whole process of , Then you can see how to synchronize data to the gateway . That's what I mentioned above :eventPublisher.publishEvent(new DataChangedEvent .....)

shenyu-admin Synchronize data to the gateway

above eventPublisher yes ApplicationEventPublisher, It is spring The built-in function of publishing and listening .

eventPublisher.publishEvent(new DataChangedEvent(ConfigGroupEnum.META_DATA, eventType,
        Collections.singletonList(MetaDataTransfer.INSTANCE.mapToData(metaDataDO))));

// Find the place to listen DataChangedEventDispatcher, The news will be released in onApplicationEvent Method 

public void onApplicationEvent(final DataChangedEvent event) {
    for (DataChangedListener listener : listeners) {
        switch (event.getGroupKey()) {
            case APP_AUTH:
                listener.onAppAuthChanged((List<AppAuthData>) event.getSource(), event.getEventType());
                break;
            case PLUGIN:
                listener.onPluginChanged((List<PluginData>) event.getSource(), event.getEventType());
                break;
            case RULE:
                listener.onRuleChanged((List<RuleData>) event.getSource(), event.getEventType());
                break;
            case SELECTOR:
                listener.onSelectorChanged((List<SelectorData>) event.getSource(), event.getEventType());
                applicationContext.getBean(LoadServiceDocEntry.class).loadDocOnSelectorChanged((List<SelectorData>) event.getSource(), event.getEventType());
                break;
            case META_DATA:
                listener.onMetaDataChanged((List<MetaData>) event.getSource(), event.getEventType());
                break;
            default:
                throw new IllegalStateException("Unexpected value: " + event.getGroupKey());
        }
    }
}
//listener There are multiple implementation classes , Which one is used 


// First look at it. DataSyncConfiguration Configuration class , It is configured to synchronize data to the gateway 
shenyu-admin Medium application.yml Find the configuration in :
 The default is websocket:
  sync:
      websocket:
         enabled: true
      messageMaxSize: 10240
#      zookeeper:
#        url: localhost:2181
#        sessionTimeout: 5000
#        connectionTimeout: 2000
#      http:
#        enabled: true

 also nacos etc. ...
 
 If it is websocket be listener The corresponding is WebsocketDataChangedListener 
 If it is http be listener The corresponding is HttpLongPollingDataChangedListener
nacos The corresponding is NacosDataChangedListener
 Others can be checked by yourself .

 If it is based on websocket,admin And the gateway service websocket Connect , Then send a message to the gateway .

shenyu-admin there DataChangedListener And the gateway SyncDataService Echo each other . such as WebsocketDataChangedListener  This corresponds to the gateway WebsocketSyncDataService.

 The functions of the gateway to synchronize data are focused on shenyu-sync-data-center Module , It also provides a variety of implementation correspondences admin How to synchronize data in .
 It is also based on the configuration to see which one to use SyncDataService. Here is websocket Configuration of :
@Configuration
@ConditionalOnClass(WebsocketSyncDataService.class)
@ConditionalOnProperty(prefix = "shenyu.sync.websocket", name = "urls")

Gateway call service

 About how the gateway calls to the service , This is mainly based on ShenyuWebHandler To process requests .
 We haven't done any further research on this , Be prepared to put it behind you and continue to learn .