WebRTC在进行带宽评估后会通过OnNetworkChanged方法回调到BitrateAllocator类,音频,视频等媒体的码率分配都是在此类中进行;
音频和视频在其(audio/audio_send_stream.cc)AudioSendStream的ConfigureBitrateObserver和(video/video_send_stream_impl.cc)VideoSendStreamImpl的StartupVideoSendStream方法的调用 bitrate_allocator_->AddObserver注册监听和参数设置; 在BitrateAllocator::AddObserver中会判断是不是已经加入过如果没有加入过则push到bitrate_observer_configs_中,后续的码率分配都是以bitrate_observer_configs_循环分配。
先看一段WebRTC的日志输出:
[2020-10-22 15:57:46] (bitrate_allocator.cc:468): BitrateAllocator::NormalRateAllocation() bitrate = 544470 sum_min_bitrates = 112000 [2020-10-22 15:57:46](bitrate_allocator.cc:485): BitrateAllocator::NormalRateAllocation(第一次按照设置的最小码率分配后,再进行分配需要优先分配的流 observers_capacities) bitrate = 432470 priority_margin = -12000 allocation[observer_config.observer] = 12000 [2020-10-22 15:57:46][003:126] [116227] (bitrate_allocator.cc:485): BitrateAllocator::NormalRateAllocation(分配后 observers_capacities) bitrate = 432470 priority_margin = -100000 allocation[observer_config.observer] = 100000 [2020-10-22 15:57:46][003:127] [116227] (bitrate_allocator.cc:639): BitrateAllocator::DistributeBitrateRelatively(for...) capacity_bps = 52000 observer_config.bitrate_priority = 1 allocation[observer_config.observer] = 12000 [2020-10-22 15:57:46][003:127] [116227] (bitrate_allocator.cc:639): BitrateAllocator::DistributeBitrateRelatively(for...) capacity_bps = 400000 observer_config.bitrate_priority = 1 allocation[observer_config.observer] = 100000 // 排序后分配码率 [2020-10-22 15:57:46][003:127] [116227] (bitrate_allocator.cc:657): BitrateAllocator::DistributeBitrateRelatively(for...) observer_share = 0.5 enough_bitrate = 1 [2020-10-22 15:57:46][003:128] [116227] (bitrate_allocator.cc:664): BitrateAllocator::DistributeBitrateRelatively(for...) allocation->at(priority_rate_observer.allocation_key) = 64000 bitrate_priority_sum = 1 [2020-10-22 15:57:46][003:128] [116227] (bitrate_allocator.cc:657): BitrateAllocator::DistributeBitrateRelatively(for...) observer_share = 1 enough_bitrate = 0 [2020-10-22 15:57:46][003:128] [116227] (bitrate_allocator.cc:666): BitrateAllocator::DistributeBitrateRelatively() remaining_bitrate = 380470 // 剩余的码率分配 [2020-10-22 15:57:46][003:129] [116227] (bitrate_allocator.cc:675): BitrateAllocator::DistributeBitrateRelatively(for...)allocation->at(priority_rate_observer.allocation_key) = 480470 //最终分配的码率 [2020-10-22 15:57:46][003:134] [116227] (bitrate_allocator.cc:135): BitrateAllocator::OnNetworkChanged() config : target_bitrate = 544470 update.link_capacity = 64000 update.target_bitrate = 64000 last_rtt_ = 0 idx = 1 [2020-10-22 15:57:46][003:136] [116227] (bitrate_allocator.cc:135): BitrateAllocator::OnNetworkChanged() config : target_bitrate = 544470 update.link_capacity = 480470 update.target_bitrate = 480470 last_rtt_ = 0 idx = 2AllocateBitrates 方法中码率的分配根据用户设置的值分为三种情况:
如果评估的码率小于用户设置的最小码率(所有的媒体流码率的和);调用LowRateAllocation如果用户设置的最大码率(所有流设置的码率和)大于评估码率,调用NormalRateAllocation;其他情况使用MaxRateAllocation分配码率;一般大多数情况下都是调用MaxRateAllocation和NormalRateAllocation;下面主要分析这两个方法的具体实现;
NormalRateAllocation分配码率
先按照设置的最小码率给每一个媒体分配码率(allocation),同时计算每一路媒体设置的最大码率与最小码率的差值(observers_capacities);给优先的媒体分配码率,在优先媒体(prioritized streams)之间实现公平共享,目前以先到先得的原则对它们进行处理;调试下observer_config.priority_bitrate_bps的值默认0;如果有剩余的码率没有分配,则调用DistributeBitrateRelatively分配; 首先按照capacity_bps / bitrate_priority的结果按照从小到大的顺序排序全部的媒体,按照一定的规则(WebRTC里定义的一种计算方式),如果有足够的比特率的则给该路流累加对应的capacity_bps;剩下的比特率,按照比例量分配给未分配其最大容量的媒体MaxRateAllocation 分配码率
[2020-10-22 15:57:46](bitrate_allocator.cc:507): BitrateAllocator::MaxRateAllocation() bitrate = 837000 sum_max_bitrates = 564000 // 给所有的媒体 分配其设置的最大码率 [2020-10-22 15:57:46](bitrate_allocator.cc:512): BitrateAllocator::MaxRateAllocation() observer_config.max_bitrate_bps = 64000 bitrate = 773000 [2020-10-22 15:57:46](bitrate_allocator.cc:512): hosten BitrateAllocator::MaxRateAllocation() observer_config.max_bitrate_bps = 500000 bitrate = 273000 // 调用DistributeBitrateEvenly分配剩余的码率 [2020-10-22 15:57:46](bitrate_allocator.cc:573): BitrateAllocator::DistributeBitrateEvenly() total_allocation= 128000 max_multiplier * it->first = 128000 [2020-10-22 15:57:46](bitrate_allocator.cc:573): BitrateAllocator::DistributeBitrateEvenly() total_allocation= 709000 max_multiplier * it->first = 1000000 // 最终分配结果 [2020-10-22 15:57:46] (bitrate_allocator.cc:135): BitrateAllocator::OnNetworkChanged() config : target_bitrate = 837000 update.link_capacity = 128000 update.target_bitrate = 128000 last_rtt_ = 0 idx = 1 [2020-10-22 15:57:46](channel_send.cc:834): audio ChannelSend::OnBitrateAllocation() configured_bitrate_bps_ = 64000 [2020-10-22 15:57:46](bitrate_allocator.cc:135): BitrateAllocator::OnNetworkChanged() config : target_bitrate = 837000 update.link_capacity = 709000 update.target_bitrate = 709000 last_rtt_ = 0 idx = 2 给所有的媒体 分配其设置的最大码率;调用DistributeBitrateEvenly分配剩余的码率; 按照设置的最大码率排序按照一定规则分配追加每一个媒体的码率音频参数的大部分设置都在webrtc_voice_engine.cc文件里(media/engine/目录下);音频码率的两个参数min_bitrate_bps & max_bitrate_bps是webrtc::AudioSendStream::Config结构体实例config_的两个属性;其赋值是在WebRtcAudioSendStream的构造函数里调用: WebRtcAudioSendStream --> UpdateSendCodecSpec --> UpdateAllowedBitrateRange
void UpdateAllowedBitrateRange() { RTC_DCHECK(worker_thread_checker_.IsCurrent()); const bool is_opus = config_.send_codec_spec && absl::EqualsIgnoreCase(config_.send_codec_spec->format.name, kOpusCodecName); if (is_opus) { // 1 默认值 const int kDefaultBitrateBps = 32000; config_.min_bitrate_bps = kDefaultBitrateBps; config_.max_bitrate_bps = kDefaultBitrateBps; // 2 编码器规范的目标码率 if (config_.send_codec_spec && config_.send_codec_spec->target_bitrate_bps) { config_.min_bitrate_bps = *config_.send_codec_spec->target_bitrate_bps; config_.max_bitrate_bps = *config_.send_codec_spec->target_bitrate_bps; } // 3 参数设置的码率 if (rtp_parameters_.encodings[0].min_bitrate_bps) { config_.min_bitrate_bps = *rtp_parameters_.encodings[0].min_bitrate_bps; } if (rtp_parameters_.encodings[0].max_bitrate_bps) { config_.max_bitrate_bps = *rtp_parameters_.encodings[0].max_bitrate_bps; } } }在上述函数中先判断是不是opus编码;如果是才设置相关参数,配置参数级别从最低到最高的优先顺序是:
最小/最大是32kbps的合理默认值;既定编解码器规范的目标比特率;rtp_parameter编码设置中配置的比特率; 如果用户设置值,则以最终用户设置的值作为码率;(pc/rtp_sender.cc)RtpSenderBase::SetParameters --> RtpSenderBase::SetParametersInternal --> (media/engine/webrtc_voice_engine.cc)WebRtcVoiceMediaChannel::SetRtpSendParameters –> SetRtpParameters --> UpdateAllowedBitrateRange; 码率分配参数的设置流程和(1)后续一致;调用追踪日志记录:
[2020-10-22 09:54:30][000:997] [144131] (rtp_sender.cc:205):RtpSenderBase::SetParameters() parameters.max_bitrate_bps = 64000 parameters.encodings[0].bitrate_priority = 1 [2020-10-22 09:54:30][000:997] [144131] (rtp_sender.cc:172): RtpSenderBase::SetParametersInternal() parameters.max_bitrate_bps = 64000 parameters.encodings[0].bitrate_priority = 1 [2020-10-22 09:54:30][000:999] [144387] (webrtc_voice_engine.cc:1386): hosten WebRtcVoiceMediaChannel::SetRtpSendParameters() parameters.max_bitrate_bps = 64000 parameters.encodings[0].bitrate_priority = 1 [2020-10-22 09:54:30][001:000] [144387] (webrtc_voice_engine.cc:998): WebRtcVoiceMediaChannel::UpdateAllowedBitrateRange(end) config_ = {rtp: {ssrc: 1175076980,省略。。。 media_transport_config: {media_transport: null}, min_bitrate_bps: 12000, max_bitrate_bps: 64000, send_codec_spec: {nack_enabled: false, transport_cc_enabled: true, cng_payload_type: <unset>, payload_type: 111, format: {name: opus, clockrate_hz: 48000, num_channels: 2, parameters: {maxplaybackrate: 48000, minptime: 10, sprop-maxcapturerate: 48000, usedtx: 1, useinbandfec: 1}}}} rtp_parameters_.encodings[0].min_bitrate_bps = 12000 [2020-10-22 09:54:30][001:001] [144387] (audio_send_stream.cc:230): AudioSendStream::ConfigureStream: {rtp: {ssrc: 1175076980, 。。。 media_transport_config: {media_transport: null}, min_bitrate_bps: 12000, max_bitrate_bps: 64000,。。。}}} [2020-10-22 09:54:30][001:001] [144387] (audio_encoder_opus.cc:819): Set Opus bitrate to 64000 bps.Call::CreateAudioSendStream –> (audio/audio_send_stream.cc)AudioSendStream::AudioSendStream –> (call/bitrate_allcoator.cc)BitrateAllocator::AddObserver –>
在上述流程的AddObserver中会将每一个加入的数据存入bitrate_observer_configs_(如果之前没有存储过);然后就是带宽评估的结果通知里调用具体流程如下: BitrateAllocator::OnNetworkChanged –> BitrateAllocator::AllocateBitrates –> BitrateAllocator::NormalRateAllocation –> ‘BitrateAllocator::DistributeBitrateEvenly’
上述流程的NormalRateAllocation是所有observe设置的最大bits的和大于评估带宽时候才调用;在AllocateBitrates方法里如果参数里最小bits的和大于评估带宽或者每个观察着的平均估计带宽小于该观察者所需的最小比特率(包括观察者处于暂停状态时的启用滞后),调用的是LowRateAllocation;其他情况调用MaxRateAllocation;
