Nifi 1.16.3 cluster setup +kerberos+ user authentication

After looking at the documents on the Internet, most of them are based on earlier versions , Share my experience , I hope you don't step on the pit , in addition nifi1.15.3 Later versions must add TLS , If you don't need permission management, you can use 1.15.2 edition ,

Building documents is this link

be based on Nifi 1.15.2 Cluster building _Mumunu- The blog of -CSDN Blog _nifi Cluster building

nifi I won't repeat what it is  

First download the installation package

• nifi-1.16.3-bin.tar.gz 
• nifi-toolkit-1.16.3-bin.tar.gz 

Unzip to the right place  

Then deploy one in the right place zk Must be 3.6 The new version above .

zk Deployment is simple .. I won't go into that

Get into nifi-toolkit-1.16.3  Prepare to build TLS Of key And configuration files  

bin/tls-toolkit.sh standalone -C 'CN=test, OU=NIFI' -n 'nifi-node1,nifi-node2,nifi-node3,nifiregistry-node' --keyPassword hhh123 --keyStorePassword hhh123 --trustStorePassword hhh123 -o 'targat'

Explain the meaning of the option

standalone Generate a certification authority in a command 、 Keystore 、 Truststore and nifi.properties file . In the same way Client/Server   This option is to use a certification authority server , The server accepts certificate signing requests from clients , Sign it , Then send the generated certificate back to . Both the client and the server verify each other's identity through the shared key , Not need

-C Generate a suitable for the specified DN The client certificate used in the browser Inside are some tag configurations , Fill in according to your own needs , such as CN=prod

-n   Generated certificate node name It can be separated by commas Generate a bunch of

-n 'nifi-node1,nifi-node2,nifi-node3,nifiregistry-node'

This name is very important , Is the domain name you use to log in . You need to use this name to log in to this node . Otherwise, the report will be wrong

For example, configured nifi-node1, You need to be able to parse the corresponding... Locally ip   use hosts Or domain name resolution dns Will do , If you use ip. Then write ip Address such as 172.31.255.125,172.31.255.126 such  

Here are the passwords of some certificates Fill in as needed

Last -o Is the output directory   It's OK not to But it will be output in the current directory It's a mess

Enter generated targat Catalog You can see the generated file

Send the node directory to each node

External documents are available, but not , It's all some https Of key It will be more comfortable to use , Too lazy to use

Directly into the directory  

There are 3 File  

keystore.jks  nifi.properties  truststore.jks

Distribute to nodes , Cover the original , Configuration file in conf Next , There is not much information to modify

nifi.zookeeper.connect.string=10.0.2.2:2181,10.0.2.3:2181,10.0.2.4:2181

nifi.cluster.is.node=true

Yes kerberos Add one more

nifi.kerberos.krb5.file=/etc/krb5.conf

modify state-management.xml

<property name="Connect String">10.0.2.2:2181,10.0.2.3:2181,10.0.2.4:2181</property>  #  add to zk The address of 

modify  bootstrap.conf

> java.arg.2=-Xms10240m

> java.arg.3=-Xmx10240m

Default 512  Too small , Modify as needed

 

nifi.cluster.load.balance.host= 

This configuration is empty in the generated configuration file , I don't know what it's for I didn't fill in either

Then update the inside of the cluster key

bin/nifi.sh set-sensitive-properties-key  xxxxxxxxx

Just customize here   The length needs to exceed 12 position

Each node of the cluster version needs to be set ,key The content of needs to be the same

If it is the first time to build, this meeting can be started

If the configuration is added after building

You need to remove conf Under the flow.xml.gz, These are some of the original cache files , Delete every time you change the configuration

Each node of the cluster version needs to be deleted

Start after setting  

Don't log in after startup , Set an initial account secret

bin/nifi.sh set-single-user-credentials <username> <password>

The password length needs to be greater than 12 position  
Then restart   Sign in https:// domain name :9443

This domain name is the node name configured above such as ip Namely https://172.31.255.213:9443

Using the host name is https://nifi-node1:9443

Be sure to bring https

You will be prompted that your certificate is incorrect   Confirm login   You can get your own certificate here I'm too lazy to do

Then you can see the login page Just log in by account secret Remind you that if the account secret is invalid, reset it   Then restart Basically done  

The basic account secret is completed

nifi In fact, there is a perfect authority system Update next time