IQ debugging of Hisilicon platform ISP and image (1)

One 、 What is the ISP What is the IQ debugging

1、ISP Conceptual analysis

(1)ISP: image signal process, Image signal processing （ Nowadays, it generally refers to digital image signal processing ）
    image signal processor Image signal processor

  ISP(Image Signal Processing) Image signal processing . It is mainly used to process the output signal of the front-end image sensor , To match image sensors from different manufacturers . Image processor for camera ISP（Image Signal Processor）.
(2) reflection ： What is an image signal ？

   The image signal is the electrical signal generated by photoelectric devices and scanning to represent the brightness information of the image .

(3) Why do I need to deal with ？

   The device or environment is not ideal, resulting in the image is not ideal , For example, overexposure 、 Not transparent 、 Bad point

(4) How to deal with it ？

   The original image data is processed with a targeted algorithm . Congenital ( device 、 Environmental Science ) no way , The day after tomorrow ( Algorithm ) To mend

(5)ISP The essence of ： Using the operation and processing of the collected digital image to repair the front-end is not ideal , Try to improve the image quality

(6)ISP processor ： The core is the ability to calculate numbers

2、 Where to do ISP

(1) The lens ->sensor->ISP->VIU->…

(2)ISP The unit can be in sensor in , Or independence , Or integrated into AP SoC in , Now most of them are in AP(Application processor) in

(3)HI3518E Built in ISP unit

3、 What is the IQ

(1)image quality, image quality

(2) Shallow image quality problems ： Bad point 、 distortion 、 stripe 、 Partially unclear, etc

(3) High level image quality problems ： permeability 、 Sharpness 、 Color fullness 、 Color reduction degree, etc

 permeability ： The clarity of the picture , If the permeability is low, it looks fuzzy ,
 Sharpness ： The physical outline in the picture , If it is very low, the outline will be unclear 
 Color saturation ： The brightness of the color 
 Color reduction ： The authenticity of color 

4、ISP and IQ summary

(1)IQ It is purpose ,ISP It's the means , Debugging is a method

(2) master ISP and IQ The key to debugging ： Relevant concepts 、 Hayes MPP in ISP The implementation of the 、PQTool Tools

Two 、 Hayes MPP in ISP The implementation framework of

1、 Official documents

(1)《HiISP Development reference 》 And 《ISP_3A Development of guidelines 》

 link ：https://pan.baidu.com/s/1KZqnKOG7h0TjOJLWLuogkg 
 Extraction code ：q0o6 
-- From Baidu network disk super member V5 The share of 

(2) The key is ISP Framework architecture and software process

  ISP Through a series of digital image processing algorithm to complete the effect of digital image processing . It mainly includes 3A(AE Auto exposure AF Autofocus AWB Automatic white balance )、 Bad point correction 、 Denoise 、 Strong light suppression 、 Backlight compensation 、 Color enhancement 、 Lens shadow correction and other processing . ISP Including the logical part and the running on it firmware.

  lens Project light signals onto sensor In the light-sensitive area of , sensor After photoelectric conversion , take Bayer The original image is sent to ISP, ISP After algorithmic processing , Output RGB The image in the spatial domain is sent to the video capture unit in the back end . In the process , ISP By running on it firmware Yes ISP Logic , lens and sensor Control accordingly , And then complete the automatic aperture 、 Auto exposure 、 Automatic white balance and other functions . among , firmware The operation of the system is driven by the interrupt of the video capture unit . PQ Tools Tools through the network port or serial port to complete ISP On line image quality adjustment .

  ISP from ISP Logic and what runs on it Firmware form , In addition to completing part of the algorithm processing, the logic unit , It can also calculate the real-time information of the current image . Firmware By acquiring ISP Logical image statistics , Recalculate , Feedback control lens、 sensor and ISP Logic , In order to achieve the purpose of automatic adjustment of image quality .

 Overexposure ： Too much light 
lens： Lens distance sensor Distance of , But the lens used in my development board is fixed , The focal length is fixed ,
 immutable 

(3)ISP As the front-end acquisition part , You need a video acquisition unit （ VIU） Working together . ISP After initialization and basic configuration , need VIU Interface timing matching . One is to match different sensor Input timing of , The second is for ISP Configure the correct input timing . After the timing configuration is completed , ISP You can start it Run To adjust the dynamic image quality . At this time, the output image is VIU collection , And then send it to display or code .

  PQ Tools The tool is mainly completed in PC Dynamic image quality adjustment is carried out at the end , Multiple factors that affect image quality can be adjusted , Such as denoising intensity 、 Color conversion matrix 、 Saturation, etc .

   If the user debugs the image effect , have access to PQ Tools The configuration file saving function provided by the tool saves the configuration parameters . The system can use... The next time it starts up PQ Tools The configuration file loading function provided by the tool loads the adjusted image parameters .

2、sample in ISP Read the relevant part again

(1) The core code is SAMPLE_COMM_ISP_Init Function (mpp/sample/common/sample_comm_isp.c)

   Analysis of the source code used in previous articles can choose any one , I won't repeat it .

(2) Pay attention to understanding the code against the architecture described above

3、 Hayes MPP in ISP The implementation framework of 3

Reference learning ：
Images sensor Features and driver analysis (1)
Images sensor Features and driver analysis (2)

main
	SAMPLE_VENC_720P_CLASSIC
		SAMPLE_COMM_VI_StartVi
			SAMPLE_COMM_VI_StartIspAndVi
				SAMPLE_COMM_ISP_Init
					sensor_register_callback //sensor Register callback function 
						cmos_init_sensor_exp_function // Do some initialization , Interface docking 
						HI_MPI_ISP_SensorRegCallBack //sensor towards isp The basic algorithm library registers the callback function 
						cmos_init_ae_exp_function
						HI_MPI_AE_SensorRegCallBack//sensor towards AE Algorithm library notes 
						cmos_init_awb_exp_function
						HI_MPI_AWB_SensorRegCallBack//sensor towards AWB Algorithm library notes 
					HI_MPI_AE_Register //AE Algorithm library direction isp Basic algorithm library registration 
					HI_MPI_AWB_Register //AWB Algorithm library direction isp Basic algorithm library registration 
					HI_MPI_AF_Register //AF Algorithm library direction isp Basic algorithm library registration 
					HI_MPI_ISP_MemInit // initialization  ISP  External registers .
					HI_MPI_ISP_SetWDRMode // Set wide dynamic mode 
					HI_MPI_ISP_SetPubAttr // Set up  ISP  Public attribute 
					HI_MPI_ISP_Init // initialization  ISP firmware.
				SAMPLE_COMM_ISP_Run
					Test_ISP_Run
						HI_MPI_ISP_Run//  function  ISP firmware.

3、sensor Details of registration

HI_S32 cmos_init_sensor_exp_function(ISP_SENSOR_EXP_FUNC_S *pstSensorExpFunc)
{
    
    memset(pstSensorExpFunc, 0, sizeof(ISP_SENSOR_EXP_FUNC_S));

    pstSensorExpFunc->pfn_cmos_sensor_init = sensor_init;
    pstSensorExpFunc->pfn_cmos_sensor_exit = sensor_exit;
    pstSensorExpFunc->pfn_cmos_sensor_global_init = sensor_global_init;// Initialize some global parameters 
    pstSensorExpFunc->pfn_cmos_set_image_mode = cmos_set_image_mode;// towards sensor Set up image mode;
    pstSensorExpFunc->pfn_cmos_set_wdr_mode = cmos_set_wdr_mode;// operation sensor The register of 
    
    pstSensorExpFunc->pfn_cmos_get_isp_default = cmos_get_isp_default;// Read isp Default configuration parameters 
    pstSensorExpFunc->pfn_cmos_get_isp_black_level = cmos_get_isp_black_level;
    pstSensorExpFunc->pfn_cmos_set_pixel_detect = cmos_set_pixel_detect;
    pstSensorExpFunc->pfn_cmos_get_sns_reg_info = cmos_get_sns_regs_info;

    return 0;
}

static ISP_CMOS_DRC_S g_stIspDrc =// These parameters are also modified by us later 
{
    
    0,
    10,
    0,
    2,
    192,
    60,
    0,
    0,
    0,
    {
    1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024,1024}
};

HI_U32 cmos_get_isp_default(ISP_CMOS_DEFAULT_S *pstDef)
{
       
    if (HI_NULL == pstDef)
    {
    
        printf("null pointer when get isp default value!\n");
        return -1;
    }

    memset(pstDef, 0, sizeof(ISP_CMOS_DEFAULT_S));
	
    switch (genSensorMode)
    {
    
        default:
        case WDR_MODE_NONE:

            memcpy(&pstDef->stDrc, &g_stIspDrc, sizeof(ISP_CMOS_DRC_S));
            memcpy(&pstDef->stDemosaic, &g_stIspDemosaicLin, sizeof(ISP_CMOS_DEMOSAIC_S));
            memcpy(&pstDef->stGe, &g_stIspGeLin, sizeof(ISP_CMOS_GE_S));			

			pstDef->stNoiseTbl.stNrCaliPara.u8CalicoefRow = DMNR_CALIB_CARVE_NUM_AR0130;
			pstDef->stNoiseTbl.stNrCaliPara.pCalibcoef    = (HI_FLOAT (*)[4])g_coef_calib_ar0130;
			//memcpy(&pstDef->stNoiseTbl.stNrCommPara, &g_stNrCommPara,sizeof(ISP_NR_COMM_PARA_S));
			memcpy(&pstDef->stNoiseTbl.stIsoParaTable[0], &g_stNrIsoParaTab[0],sizeof(ISP_NR_ISO_PARA_TABLE_S)*HI_ISP_NR_ISO_LEVEL_MAX);
			

			memcpy(&pstDef->stRgbSharpen, &g_stIspRgbSharpen, sizeof(ISP_CMOS_RGBSHARPEN_S));
			memcpy(&pstDef->stUvnr,       &g_stIspUVNR,       sizeof(ISP_CMOS_UVNR_S));
			memcpy(&pstDef->stDpc,       &g_stCmosDpc,       sizeof(ISP_CMOS_DPC_S));
        break;

    }
    pstDef->stSensorMaxResolution.u32MaxWidth  = 1280;
    pstDef->stSensorMaxResolution.u32MaxHeight = 720;

    return 0;
}

   You need to understand this part of the code deeply , You can only analyze the code in the data manual ！ Limited personal energy , I won't do this part of the work , I don't need to do the actual development at present , It can only be understood macroscopically , Focus on the big and let go of the small .

notes ： This article refers to 《 Mr. Zhu, Internet of things lecture hall 》 Course notes , Combined with their own actual development experience and other technical articles on the Internet , Comprehensive sorting results in . If there is any infringement , Contact deletion ！ Level co., LTD. , Welcome to communicate in the comments area .