TRTC zero foundation -- Video subscription on the code

This article will introduce TRTC Video streaming correlation API And considerations, plus sample code , Add some SDK API What is not on the document , Make the reader right TRTC I have some knowledge about Chinese video streaming .

TRTC Start a series of articles

• TRTC Zero basic video introduction
• TRTC Zero based video exception
• TRTC Video streaming on zero basis

Video streaming event callback

When used as the pull end , After pushing the flow at the far end, you can start to pull the flow , For the user experience , It is recommended to follow the specification requirements , Upon receipt of SDK Start streaming after the event callback of , And display the remote screen on the control

Sample code

//  Set up TRTC Event callback interface .
 mTRTCCloud.setListener(new TRTCCloudListener ());

matters needing attention

• Repeat settings SDK Event callback interface , The previous interface will be overwritten . If you need to keep the previous interface , You can use the proxy mode to distribute related events .
• It is recommended to initialize TRTC When to set the event callback interface .

1. onUserVideoAvailable（） Remote video streaming event callback

Callback event of available status of remote video stream , According to available To start or stop pulling remote streams ,SDK The video screen will be displayed on the control .

Parameter description ：

• userId The user ID of the remote user
• available Whether the user publishes （ Or unpublish ） The main road video picture is displayed ,true: Release ;false： Unpublish . Sample code

        @Override
        public void onUserVideoAvailable(String userId, boolean available) {
            Log.d(TAG, "onUserVideoAvailable  available " + available + " userId " + userId);
            if (available) { 
                TRTCCloudDef.TRTCRenderParams params = new TRTCCloudDef.TRTCRenderParams();
				//  Set the rendering mode 
                params.fillMode = TRTCCloudDef.TRTC_VIDEO_RENDER_MODE_FIT;
				// Set remote screen rendering parameters 
                mTRTCCloud.setRemoteRenderParams(userId,TRTCCloudDef.TRTC_VIDEO_STREAM_TYPE_BIG,params);
				//  Start pulling remote video stream , And display on the control 
                mTRTCCloud.startRemoteView(userId, TRTCCloudDef.TRTC_VIDEO_STREAM_TYPE_BIG, mTxcvvAnchorPreviewView);
            } else { 
				//  Stop pulling remote video stream 
                mTRTCCloud.stopRemoteView(userId, TRTCCloudDef.TRTC_VIDEO_STREAM_TYPE_BIG);
            }
        }

matters needing attention

• Remote publishing or stopping video streaming , Will call back this event
• Remote pause and resume video streaming , This event will also be called back

2. onUserSubStreamAvailable（） Remote auxiliary video streaming event callback

Callback event of available status of auxiliary video stream , Generally used for screen sharing , According to available To start or stop pulling remote streams ,SDK The video screen will be displayed on the control .

Parameter description

• userId The user ID of the remote user
• available Whether the user publishes （ Or unpublish ） The main road video picture is displayed ,true: Release ;false： Unpublish .

matters needing attention

• The secondary channel is a video stream independent of the primary channel , Separate streaming is required , Generally used for screen sharing
• There can only be one auxiliary flow in the room , Pushing two auxiliary routes at the same time will lead to failure .

3. onFirstVideoFrame（） First frame picture callback event

SDK Start rendering the first frame of your local or remote user. Event callback .

Parameter description

• userId Local or remote user id , If userId A null value indicates that the first frame of your local image has arrived ,userId If it is not empty, it means that the first frame of the remote user has arrived .
• streamType Video stream type ： Main road （Main） It is generally used to carry camera pictures , Side Rd （Sub） It is generally used to carry screen sharing pictures .
• width The width of the picture .
• height Picture height .

usage

• Obtain the resolution of the current video stream to calculate the picture size , Rearrange the display layout .

matters needing attention

• Only when you call startLocalPreview or startScreenCapture after , Will trigger its own local first frame event callback .
• Only when you call startRemoteView or startRemoteSubStreamView after , The first frame event callback of the remote user will be triggered .

4. onRemoteVideoStatusUpdated（） Event callback of remote video state change

For a better user experience , You can obtain the playback status of each remote channel （ Include Playing、Loading and Stopped Three states ）, So as to carry out the corresponding UI Exhibition .

Parameter description

• userId User ID
• streamType Video stream type ： Main road （Main） It is generally used to carry camera pictures , Side Rd （Sub） It is generally used to carry screen sharing pictures .
• status Video status ： Include Playing、Loading and Stopped Three states .
• reason The reason why the video status changes
• extrainfo Additional information

Video streaming control

After the distal end of the room , Local users can subscribe to remote streams or pause pulling streams as needed .TRTC SDK The following are provided API Control remote user video screen .

1. startRemoteView（） Subscribe to video streaming from remote users , And bind the video rendering control

usage

• If you already know the users who have video streaming in the room userid, Can be called directly startRemoteView Subscribe to the user's screen .
• If you don't know which users in the room are posting videos , You can go to enterRoom Then wait for the message from onUserVideoAvailable The notice of .

matters needing attention

• SDK Support watching a userid The big picture and the auxiliary road picture , Or watch something at the same time userid Small screen and auxiliary road screen , However, it does not support viewing large and small pictures at the same time .
• Only if specified userid adopt enableEncSmallVideoStream Turn on the two-way coding , To view the user's small screen .
• When the specified userid When the small picture of does not exist ,SDK Switch to the large screen of the user by default .

2. updateRemoteView（） Dynamically update video rendering controls for remote users

This interface can be used to update the rendering control of the remote video picture , It is often used in interactive scenes where the display area is switched

Sample code

mRemoteVideoView.addVideoView(new TextureView(context));
mTRTCCloud.updateRemoteView(remoteUserId, TRTCCloudDef.TRTC_VIDEO_STREAM_TYPE_BIG,  mRemoteVideoView);
	   

matters needing attention

• Android needs to add TextureView It works , The reference sample .

3. stopRemoteView（） Stop subscribing to video streams from remote users , And release the rendering controls

Calling this interface will make SDK Stop receiving the user's video stream , And release the decoding and rendering resources of the video stream .

usage

• If the remote user's screen is no longer required to be displayed locally , You can call this method , If you just temporarily stop the display , Recommend calling muteRemoteVideoStream（） Interface
• When the remote user exits the room or pauses video streaming , Call this method to stop pulling the video stream .

matters needing attention

• If the local user exits the room ,SDK The subscription will be automatically stopped , There is no need to call... For every user in the room stopRemoteView.

4. stopAllRemoteView () Stop subscribing to video streams from all remote users , And release all rendering resources

Calling this interface will make SDK Stop receiving all video streams from the remote end , And release all decoding and rendering resources .

5. muteRemoteVideoStream（） Pause / Resume subscribing to video streams from remote users

This interface only pauses / Resume receiving the video stream of the specified user , But it does not release display resources , The video screen will be frozen to the last frame when the interface is called , It is suitable for scenes that do not display the screen for a short time .

Parameter description

• userId Specifies the of the remote user ID.
• streamType To pause / Recovered video stream type （ Support only TRTCVideoStreamTypeBig and TRTCVideoStreamTypeSub）.
• mute Whether to pause receiving .

matters needing attention

• This interface enables you to enter the room （enterRoom） Pre invocation , The pause state will occur when you exit the room （exitRoom） It will be reset later

6. muteAllRemoteVideoStreams（） Pause / Resume subscribing to video streams of all remote users

This interface only pauses / Resume receiving video streams from all users , But it does not release display resources

Video subscription settings

1. setRemoteVideoStreamType（） Switch the size screen of the specified remote user

After a certain anchor has enabled two-way encoding , Other users in the room pass startRemoteView By default, the subscribed screen will be 【 HD big picture 】. Use this interface to select whether you want to subscribe to a large screen or a small screen .

matters needing attention

• The target user must have passed enableEncSmallVideoStream The two-way encoding mode is enabled in advance .
• The interface is in startRemoteView Both previous and subsequent calls can take effect

2. snapshotVideo（） Screenshot of the video

Capture the local video picture through this interface , The main route screen of the remote user and the auxiliary route of the remote user （ Screen sharing ） The picture

matters needing attention

• userId If it is specified to be empty, it means to intercept the local video picture .
• Windows The platform currently only supports interception TRTCSnapshotSourceTypeStream Source video .
• The captured video stream picture is clearer than the video rendered picture

3. setDefaultStreamRecvMode（） Set subscription mode

In most scenarios , After entering the room, users will subscribe to the audio and video streams of all anchors in the room , therefore TRTC Automatic subscription mode is adopted by default , To get the best “ Second open experience ”. If each room in your application scenario has many audio and video streams being released at the same time , And each user only wants to selectively subscribe to 1-2 road , It is recommended to use “ Manual subscription ” Mode to save traffic costs

matters needing attention

• TRTC The default is automatic subscription mode , You still need to pass startRemoteView Interface binding rendering control .
• Need to enter the room （enterRoom） Call this interface before , Settings can only take effect .
• In automatic subscription mode , If the user does not call after entering the room {@startRemoteView} Subscribe to video streaming ,SDK Will automatically stop subscribing to the video stream , In order to achieve the purpose of saving traffic .

Remote video picture settings

1. setRemoteRenderParams Remote screen parameter settings

Sample code

//  Picture rendering parameters  
TRTCCloudDef.TRTCRenderParams renderParams = new TRTCCloudDef.TRTCRenderParams();
...
    //  Set remote screen parameters 
    mTRTCCloud.setRemoteRenderParams(remoteUserId , TRTCCloudDef.TRTC_VIDEO_STREAM_TYPE_BIG , params);

2. fillMode Remote picture rendering mode

SDK Two picture rendering modes are supported ： Fill and Fit, The former will spread the picture regardless of the original resolution View Components , The extra picture will be cut ; The latter will display the full video picture , The extra part View The area will be blank .

Sample code

    TRTCCloudDef.TRTCRenderParams params = new TRTCCloudDef.TRTCRenderParams();
	//  Fill mode 
    params.fillMode = TRTCCloudDef.TRTC_VIDEO_RENDER_MODE_FILL;
	//  Tile mode 
    params.fillMode = TRTCCloudDef.TRTC_VIDEO_RENDER_MODE_FIT;
    ...
    //  Set remote screen parameters 
    mTRTCCloud.setRemoteRenderParams(remoteUserId , TRTCCloudDef.TRTC_VIDEO_STREAM_TYPE_BIG , params);
	

3. rotation Rotation angle of the remote screen

    TRTCCloudDef.TRTCRenderParams params = new TRTCCloudDef.TRTCRenderParams();
	//  Rotate the picture 90 degree 
    params.rotation = TRTCCloudDef.TRTC_VIDEO_ROTATION_90;
    ...
        //  Set remote screen parameters 
    mTRTCCloud.setRemoteRenderParams(remoteUserId , TRTCCloudDef.TRTC_VIDEO_STREAM_TYPE_BIG , params);
	

4. mirrorType The image strategy of the remote screen

Sample code

public void onRenderVideoFrame(String userId, int streamType, final TRTCCloudDef.TRTCVideoFrame frame) {
    mEglCore.makeCurrent();
    GLES20.glViewport(0, 0, mSurfaceSize.width, mSurfaceSize.height);
    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
    GLES20.glClearColor(0, 0, 0, 1.0f);
    GLES20.glClear(GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);
    mNormalFilter.onDraw(frame.texture.textureId, mGLCubeBuffer, mGLTextureBuffer);
    mEglCore.swapBuffer();
}

Custom screen rendering

1. setRemoteVideoRenderListener（）

Sample code 、

public void onRenderVideoFrame(String userId, int streamType, final TRTCCloudDef.TRTCVideoFrame frame) {
    mEglCore.makeCurrent();
    GLES20.glViewport(0, 0, mSurfaceSize.width, mSurfaceSize.height);
    GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, 0);
    GLES20.glClearColor(0, 0, 0, 1.0f);
    GLES20.glClear(GLES20.GL_DEPTH_BUFFER_BIT | GLES20.GL_COLOR_BUFFER_BIT);
    mNormalFilter.onDraw(frame.texture.textureId, mGLCubeBuffer, mGLTextureBuffer);
    mEglCore.swapBuffer();
}

Author's brief introduction

Pengliming , done Window Applications , done Java The back-end service , done Flash Streaming services , Worked as a front-end post , Later, he focused on Android development 、 Good at graphics API、 Audio and video . Like music 、 Like riding , I like to go sightseeing , Yearn for the carefree blue sky and white clouds . Self perception , thole 、 Be good at solving problems Imaginative 、 Accustomed to divergent thinking Strong learning ability , Rich professional skills

Reference documents

Code example

Video quality

Screen sharing