JVM shutdown hook details

What is? JVM Shutdown Hook

Shutdown Hook It is a special structure , Generally used in JVM Before closing , When you need to perform some operations . common , For example, when you need to do some cleaning work when your program exits , You can consider using it Shutdown Hook. But if yours JVM Abnormal exit , For example, received SIGKILL, There is no guarantee at this time shutdown hook Be able to perform normally .

How to use Shutdown Hook

Java In fact, the corresponding interfaces have been provided to implement , It's also very convenient to use . Here are a few simple examples ：

public class ShutDownHookTest
{
    
  public static void main(String[] args)
  {
    
  
    Runtime.getRuntime().addShutdownHook(new Thread()
    {
    
      public void run()
      {
    
        System.out.println("Shutdown Hook now!");
      }
    });
    System.out.println("Going to exit");
  }
}

Output :

Going to exit
Shutdown Hook now!

You can see the above example , We just need to give addShutdownHook Pass in one Thread object , stay run() Write the operations to be performed in .

class MyThread extends Thread {
         
    public void run() {
             
        System.out.println("Execute done");
        System.out.println("Shutdown hook now!");
    }
}
  
class ShutDownHookTest {
    
      
    public static void main(String[] args) {
    
        Runtime.getRuntime().addShutdownHook(new MyThread());
  
        for(int i = 1; i < 4; i++) 
            System.out.println("Execute：" + i);
    }
}

Output :

Execute：1
Execute：2
Execute：3
Execute done
Shutdown hook now!

The above example clearly shows , When the main program is finished , Will start to execute Shutdown Hook, JVM You're going to quit .

Shutdown Hook Characteristics of

Shutdown Hook There is no guarantee that

If JVM crashe 了 , Shutdown Hook There is no guarantee that . For example, if you receive SIGKILL When , The program will immediately terminate execution ,JVM Quit now , Resulting in no opportunity to execute Shutdown Hook. If called Runtime.halt() Under the circumstances , Can also lead to JVM No execution at Shutdown Hook Exit directly when . Of course , If a program ends normally , Will be in JVM Exit to call Shutdown Hook. If JVM Because the user requests to interrupt or receives SIGTERM, Also called Shutdown Hook Of .

Shutdown Hook Can be forcibly interrupted

Even if the execution has started Shutdown Hook, It can also be interrupted , For example, when receiving SIGTERM, however shutdown hook Not completed within a certain time , Will also be forced to interrupt , Lead to shutdown hook Not fully implemented . So in general Shutdown hook All operations in should be completed quickly , It shouldn't be some long time The task of .

Shutdown Hook There can be multiple

You can register more than one in a program shutdown hook, however JVM perform shutdown hook Is an arbitrary order , also JVM perform hook Is concurrent .

Shutdown hook Can't regist/unregister shutdown hook

If so ,JVM Flip a IllegalStateException.

Shutdown hook Can be stopped

If one Shutdown It's been implemented , Except for example SIGKILL Such external intervention , It is necessary and only through Runtime.halt() interrupt .

Shutdown hook Security permissions are required

Shutdown hook stay Spark An example of application in

Here we use Spark Medium ApplicationMaster To do an example analysis .
ApplicationMaster.scala

// This shutdown hook should run *after* the SparkContext is shut down.
      val priority = ShutdownHookManager.SPARK_CONTEXT_SHUTDOWN_PRIORITY - 1
      ShutdownHookManager.addShutdownHook(priority) {
     () =>
        val maxAppAttempts = client.getMaxRegAttempts(sparkConf, yarnConf)
        val isLastAttempt = appAttemptId.getAttemptId() >= maxAppAttempts

        if (!finished) {
    
          // The default state of ApplicationMaster is failed if it is invoked by shut down hook.
          // This behavior is different compared to 1.x version.
          // If user application is exited ahead of time by calling System.exit(N), here mark
          // this application as failed with EXIT_EARLY. For a good shutdown, user shouldn't call
          // System.exit(0) to terminate the application.
          finish(finalStatus,
            ApplicationMaster.EXIT_EARLY,
            "Shutdown hook called before final status was reported.")
        }

        if (!unregistered) {
    
          // we only want to unregister if we don't want the RM to retry
          if (finalStatus == FinalApplicationStatus.SUCCEEDED || isLastAttempt) {
    
            unregister(finalStatus, finalMsg)
            cleanupStagingDir(new Path(System.getenv("SPARK_YARN_STAGING_DIR")))
          }
        }

In this program , When application At the end, you need to delete stagingDir The files under the . This is what we mentioned above shutdown hook Common use .

But in reality Spark application in , Sometimes application By kill Or abnormal exit , This time to observe stagingDir Will find ,application It was not deleted after the end . Because when application By kill Or exit abnormally ,Yarn Will send SIGTERM, And then send SIGKILL.shutdown hook It needs to be done in the middle of these two operations , If not completed in time , It will lead to shutdown hook Delete in stagingDir The operation of is not completed . This time can pass yarn.nodemanager.slepp-delay-before-sigkill.ms To set up .

Reference resources

http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/Runtime.html#addShutdownHook(java.lang.Thread)