想做一个网站,个人业务网站建设,做网站+广告费+步骤,沈阳市于洪区建设局网站quic协议
1、网络通信时#xff0c;为了确保数据不丢包#xff0c;早在几十年前就发明了tcp协议#xff01;然而此一时非彼一时#xff0c;随着技术进步和业务需求增多#xff0c;tcp也暴露了部分比较明显的缺陷#xff0c;比如: 建立连接的3次握手延迟大#xff1b; T…quic协议
1、网络通信时为了确保数据不丢包早在几十年前就发明了tcp协议然而此一时非彼一时随着技术进步和业务需求增多tcp也暴露了部分比较明显的缺陷比如: 建立连接的3次握手延迟大 TLS需要至少需要2个RTT延迟也大 协议缺陷可能导致syn反射类的DDOS攻击 tcp协议紧耦合到了操作系统升级需要操作系统层面改动无法快速、大面积推广升级补丁包 对头阻塞数据被分成sequence一旦中间的sequence丢包后面的sequence也不会处理 中转设备僵化路由器、交换机等设备“认死理”比如只认80、443等端口其他端口一律丢弃
为了解决这些问题牛逼plus的google早在10年前也就是2012年发布了基于UDP的quic协议为啥不基于tcp了因为tcp有上述5条缺陷的嘛所以干脆“另起炉灶”重新开搞
2、正式介绍前先看一张图quci在右边底层用了udp的协议自生实现了Multistreaming、tls、拥塞控制然后支撑了上层的http/2所以我个人理解quic是一个夹在应用层和传输层之间的协议 上面“数落”了tcp协议的5点不是quic又是怎么基于udp解决这些问题的了quic 是基于 UDP 实现的协议而 UDP 是不可靠的面向报文的协议这和 TCP 基于 IP 层的实现并没有什么本质上的不同都是 底层只负责尽力而为的以 packet 为单位的传输; 上层协议实现更关键的特性如可靠有序安全等。
1由于quic并未改造udp而是直接使用udp所以不需要改动现有的操作系统也兼容了现有的网络中转设备这些都不需要做任何改动所以quic部署的改造成本相对较低但是quic毕竟是新的协议在哪部署和使用了只有应用层了这个和操作系统是解耦的全靠3环的app自己想办法实现和之前介绍的协程是不是类似了google已经开源了算法下载连接见文章末尾的参考5PS微软也实现了QUIC协议名称叫MsQuic源码在这https://github.com/microsoft/msquic
这里多说几句应用层app能操作的最底层协议就是传输层了。大家在用libc库编写通信代码时可以对指定的ip地址和端口收发数据没法改自己的mac地址吧也没法改自己的ip地址吧这些都是操作系统内核封装的app的开发人员是不需要、也是没法改变的所以站在安全防护的角度部分大厂基于传输层自研了类似quic的通信协议逆向时需要人工挨个分析协议字段的含义了现成的fiddler/charles/burpsuit等https/http的抓包工具是无效的用wireshark这类工具抓包也无法自动解析这些厂家自研的协议
2TCP连接需要3次握手tls最少需要2次RTT两个加起来一共要耗费5个RTT究其原因一方面是 TCP 和 TLS 分层设计导致的分层的设计需要每个逻辑层次分别建立自己的连接状态。另一方面是 TLS 的握手阶段复杂的密钥协商机制导致的quic又是怎么改进的了quic建立握手的步骤如下 客户端判断本地是否已有服务器的全部配置参数证书配置信息如果有则直接跳转到(5)否则继续 。 客户端向服务器发送 inchoate client hello(CHLO) 消息请求服务器传输配置参数。 服务器收到 CHLO回复 rejection(REJ) 消息其中包含服务器的部分配置参数 客户端收到 REJ提取并存储服务器配置参数跳回到 (1) 。 客户端向服务器发送 full client hello 消息开始正式握手消息中包括客户端选择的公开数。此时客户端根据获取的服务器配置参数和自己选择的公开数可以计算出初始密钥 K1。 服务器收到 full client hello如果不同意连接就回复 REJ同(3)如果同意连接根据客户端的公开数计算出初始密钥 K1回复 server hello(SHLO) 消息 SHLO 用初始密钥 K1 加密并且其中包含服务器选择的一个临时公开数。 客户端收到服务器的回复如果是 REJ 则情况同(4)如果是 SHLO则尝试用初始密钥 K1 解密提取出临时公开数。 客户端和服务器根据临时公开数和初始密钥 K1各自基于 SHA-256 算法推导出会话密钥 K2。 双方更换为使用会话密钥 K2 通信初始密钥 K1 此时已无用QUIC 握手过程完毕。之后会话密钥 K2 更新的流程与以上过程类似只是数据包中的某些字段略有不同。这里为啥不继续使用key1而是要重新生成key2来加密了核心是为了前向安全万一key1泄漏之前用key1加密的数据全都被解密。所以为了前向安全每次通信时会重新生成key2加密 总的来说 udp本身就不是面向连接的协议所以省略了tcp 3次握手连接的耗时直接通过事先内置的服务器参数发起通信请求 既然不是面向连接的怎么确保所有的数据都能到达了通过stream id和stream offset确保数据包不会丢失接收方能收到完整的全量数据 第一次用DH算法计算对称加密的密钥需要1个RTT后续每次都用这个缓存的密钥加密又省了一个RTT本质上是把tcp的打招呼、握手还有tls交换密钥的工作在1个RTT中全做了这就是相比于tcp实现的tls效率高的根本原因
注意通信双方用于密钥交换的DH算法无法防止中间人攻击所以仅通过密钥交换是无法防止被抓包的所以还要通过证书等其他方式验证身份x音就是通过libboringssl.sogoogle开源的一个openssl分支SSL_CTX_set_custom_verify函数验证客户端是否是原来的client而不是抓包软件
相关视频推荐
Linux C/C开发后端/音视频/游戏/嵌入式/高性能网络/存储/基础架构/安全https://link.zhihu.com/?targethttps%3A//ke.qq.com/course/417774%3FflowToken%3D1013300
需要C/C Linux服务器架构师学习资料加qun812855908获取资料包括C/CLinuxgolang技术NginxZeroMQMySQLRedisfastdfsMongoDBZK流媒体CDNP2PK8SDockerTCP/IP协程DPDKffmpeg等免费分享 3拥塞控制QUIC 使用可插拔的拥塞控制相较于 TCP它能提供更丰富的拥塞控制信息。比如对于每一个包不管是原始包还是重传包都带有一个新的序列号(seq)这使得 QUIC 能够区分 ACK 是重传包还是原始包从而避免了 TCP 重传模糊的问题。QUIC 同时还带有收到数据包与发出 ACK 之间的时延信息。这些信息能够帮助更精确的计算 RTT同时因为quic不依赖操作系统而是在应用层实现所以开发人员对于quic有非常强的操控能力完全可以根据不同的业务场景实现和配置不同的拥塞控制算法以及参数比如Google 提出的 BBR 拥塞控制算法与 CUBIC 是思路完全不一样的算法在弱网和一定丢包场景BBR 比 CUBIC 更不敏感性能也更好
4队头阻塞TCP 为了保证可靠性使用了基于字节序号的 Sequence Number 及 Ack 来确认消息的有序到达一旦中间某个sequence的包丢失哪怕是这个sequence后面的数据已经到达接收端操作系统也不会立即把数据发给上层的应用来接受处理而是一直等待发送端重新发送丢失的sequence包举例如下 应用层可以顺利读取 stream1 中的内容但由于 stream2 中的第三个 segment 发生了丢包TCP 为了保证数据的可靠性需要发送端重传第 3 个 segment 才能通知应用层读取接下去的数据。所以即使 stream3、stream4 的内容已顺利抵达应用层仍然无法读取只能等待 stream2 中丢失的包进行重传。在弱网环境下HTTP2 的队头阻塞问题在用户体验上极为糟糕quic是怎么既确保数据传输可靠不丢失又解决队头阻塞的这个问题的了
对于数据包的传输肯定是要编号的否则接受方在拼接这些数据包的时候怎么知道顺序了quic协议用Packet Number 代替了 TCP 的 Sequence Number不同之处在于 每个 Packet Number 都严格递增也就是说就算 Packet N 丢失了重传的 Packet N 的 Packet Number 已经不是 N而是一个比 N 大的值比如Packet NM 数据包支持乱序确认不再要求 TCP 那样必须有序确认
当数据包 Packet N 丢失后只要有新的已接收数据包确认当前窗口就会继续向右滑动。待发送端获知数据包 Packet N 丢失后会将需要重传的数据包放到待发送队列重新编号比如数据包 Packet NM 后重新发送给接收端对重传数据包的处理跟发送新的数据包类似这样就不会因为丢包重传将当前窗口阻塞在原地从而解决了队头阻塞问题但是问题又来了怎么确认Package NM就是重传PackageN的数据包了这就涉及到quic另一个重要的特性了多路复用比如用户访问某个网页这个页面有两个文件分别是index.htm和index.js可以同时、分别传输这两个文件每个传输的stream都有各自的id所以可以通过id确认是哪个stream超时丢包了但包的Packet 编号是NM怎么进一步确认就是重传的Packet N包了这就需要另一个重要的变量了offset怎么样单从英语是不是就能猜到这个变量的作用了每个数据包都有个offset字段用于标识在stream id中的偏移接收方完全可以根据offset来拼接收到的数据包
总结quic协议可以在乱序发送的情况下任然可靠不丢失靠的就是每个数据包的offset字段再搭配上stream id字段接收方完全可以在乱序的情况下无误拼接收到的数据包了
4除了以上通过stream id和stream offset确保数据不丢失外quic还采用了另一个叫向前纠错 (Forward Error CorrectionFEC)的校验方式即每个数据包除了它本身的内容之外还包括了部分其他数据包的数据因此少量的丢包可以通过其他包的冗余数据直接组装而无需重传。向前纠错牺牲了每个数据包可以发送数据的上限但是减少了因为丢包导致的数据重传因为数据重传将会消耗更多的时间(包括确认数据包丢失、请求重传、等待新数据包等步骤的时间消耗)这个原理和纠删码没有本质区别
5通信双方不论使用何种协议发送的数据必须事前约定好格式否则接受方怎么从数据包本质就是一段字符串中解析和提取关键的信息了quic协议的格式如下 数据包中除了个别报文比如 PUBLIC_RESET 和 CHLO所有报文头部上图红色部分都是经过认证的哈希散列值报文 Body 上图绿色部分都是经过加密的这样只要对 QUIC 报文任何修改接收端都能够及时发现每个字段的含义如下 Flags用于表示 Connection ID 长度、Packet Number 长度等信息 Connection ID客户端随机选择的最大长度为64位的无符号整数用于标识连接如果app更换了ip地址比如wifi和4G之间切换了仍然可以通过这个id和服务端在0 RTT下通信 QUIC VersionQUIC 协议的版本号32 位的可选字段。如果 Public Flag FLAG_VERSION ! 0这个字段必填。客户端设置 Public Flag 中的 Bit0 为1并且填写期望的版本号。如果客户端期望的版本号服务端不支持服务端设置 Public Flag 中的 Bit0 为1并且在该字段中列出服务端支持的协议版本0或者多个并且该字段后不能有任何报文 Packet Number长度取决于 Public Flag 中 Bit4 及 Bit5 两位的值最大长度 6 字节。发送端在每个普通报文中设置 Packet Number。发送端发送的第一个包的序列号是 1随后的数据包中的序列号的都大于前一个包中的序列号 Stream ID用于标识当前数据流属于哪个资源请求用于消除队头阻塞 Offset标识当前数据包在当前 Stream ID 中的字节偏移量用于消除队头阻塞。
6为了便于理解和记忆这里把quic的要点做了总结如下 3、正式因为quic有这么多优点国内很多互联网一、二线厂商都开始采用其中比较著名的app就是x音了lib库中有个libsscronet.so就支持quic协议
quic协议核心源码
quic协议最早是google提出来的所以狗家的源码肯定是最“正宗”的
1、quic相比tcp实现的tls前面省略了3~4个RTT根因就是发起连接请求时就发送自己的公钥给对方让对方利用自己的公钥计算后续对称加密的key这就是所谓的handshake在libquic-master\src\net\quic\core\quic_crypto_client_stream.cc中有具体实现握手的代码先看DoHandshakeLoop函数
void QuicCryptoClientStream::DoHandshakeLoop(const CryptoHandshakeMessage* in) {QuicCryptoClientConfig::CachedState* cached crypto_config_-LookupOrCreate(server_id_);QuicAsyncStatus rv QUIC_SUCCESS;do {CHECK_NE(STATE_NONE, next_state_);const State state next_state_;next_state_ STATE_IDLE;rv QUIC_SUCCESS;switch (state) {case STATE_INITIALIZE:DoInitialize(cached);break;case STATE_SEND_CHLO:DoSendCHLO(cached);return; // return waiting to hear from server.case STATE_RECV_REJ:DoReceiveREJ(in, cached);break;case STATE_VERIFY_PROOF:rv DoVerifyProof(cached);break;case STATE_VERIFY_PROOF_COMPLETE:DoVerifyProofComplete(cached);break;case STATE_GET_CHANNEL_ID:rv DoGetChannelID(cached);break;case STATE_GET_CHANNEL_ID_COMPLETE:DoGetChannelIDComplete();break;case STATE_RECV_SHLO:DoReceiveSHLO(in, cached);break;case STATE_IDLE:// This means that the peer sent us a message that we werent expecting.CloseConnectionWithDetails(QUIC_INVALID_CRYPTO_MESSAGE_TYPE,Handshake in idle state);return;case STATE_INITIALIZE_SCUP:DoInitializeServerConfigUpdate(cached);break;case STATE_NONE:NOTREACHED();return; // We are done.}} while (rv ! QUIC_PENDING next_state_ ! STATE_NONE);
}
只要quic的状态不是pending并且下一个状态不是NONE就根据不同的状态调用不同的处理函数具体发送handshake小的函数是DoSendCHLO代码如下
/*发送client hello消息*/
void QuicCryptoClientStream::DoSendCHLO(QuicCryptoClientConfig::CachedState* cached) {if (stateless_reject_received_) {//如果收到了server拒绝的消息// If weve gotten to this point, weve sent at least one hello// and received a stateless reject in response. We cannot// continue to send hellos because the server has abandoned state// for this connection. Abandon further handshakes.next_state_ STATE_NONE;if (session()-connection()-connected()) {session()-connection()-CloseConnection(//关闭连接QUIC_CRYPTO_HANDSHAKE_STATELESS_REJECT, stateless reject received,ConnectionCloseBehavior::SILENT_CLOSE);}return;}// Send the client hello in plaintext.//注意这是client hello消息没必要加密session()-connection()-SetDefaultEncryptionLevel(ENCRYPTION_NONE);encryption_established_ false;if (num_client_hellos_ kMaxClientHellos) {//握手消息已经发送了很多不能再发了CloseConnectionWithDetails(QUIC_CRYPTO_TOO_MANY_REJECTS,base::StringPrintf(More than %u rejects, kMaxClientHellos).c_str());return;}num_client_hellos_;//开始构造握手消息了CryptoHandshakeMessage out;DCHECK(session() ! nullptr);DCHECK(session()-config() ! nullptr);// Send all the options, regardless of whether were sending an// inchoate or subsequent hello./*填充握手消息的各个字段*/session()-config()-ToHandshakeMessage(out);// Send a local timestamp to the server.out.SetValue(kCTIM,session()-connection()-clock()-WallNow().ToUNIXSeconds());if (!cached-IsComplete(session()-connection()-clock()-WallNow())) {crypto_config_-FillInchoateClientHello(server_id_, session()-connection()-supported_versions().front(),cached, session()-connection()-random_generator(),/* demand_x509_proof */ true, crypto_negotiated_params_, out);// Pad the inchoate client hello to fill up a packet.const QuicByteCount kFramingOverhead 50; // A rough estimate.const QuicByteCount max_packet_size session()-connection()-max_packet_length();if (max_packet_size kFramingOverhead) {DLOG(DFATAL) max_packet_length ( max_packet_size ) has no room for framing overhead.;CloseConnectionWithDetails(QUIC_INTERNAL_ERROR,max_packet_size too smalll);return;}if (kClientHelloMinimumSize max_packet_size - kFramingOverhead) {DLOG(DFATAL) Client hello wont fit in a single packet.;CloseConnectionWithDetails(QUIC_INTERNAL_ERROR, CHLO too large);return;}// TODO(rch): Remove this when we remove:// FLAGS_quic_use_chlo_packet_sizeout.set_minimum_size(static_castsize_t(max_packet_size - kFramingOverhead));next_state_ STATE_RECV_REJ;/*做hash签名接收方会根据hash验证消息是否完整*/CryptoUtils::HashHandshakeMessage(out, chlo_hash_);//发送消息SendHandshakeMessage(out);return;}// If the server nonce is empty, copy over the server nonce from a previous// SREJ, if there is one. if (FLAGS_enable_quic_stateless_reject_support crypto_negotiated_params_.server_nonce.empty() cached-has_server_nonce()) {crypto_negotiated_params_.server_nonce cached-GetNextServerNonce();DCHECK(!crypto_negotiated_params_.server_nonce.empty());}string error_details;/*继续填充client hello消息*/QuicErrorCode error crypto_config_-FillClientHello(server_id_, session()-connection()-connection_id(),session()-connection()-version(),session()-connection()-supported_versions().front(), cached,session()-connection()-clock()-WallNow(),//这个随机数会被server用来计算生成对称加密的keysession()-connection()-random_generator(), channel_id_key_.get(),//保存了nonce、key、token相关信息后续对称加密的方法是CTR需要NONCE值crypto_negotiated_params_,out, error_details);if (error ! QUIC_NO_ERROR) {// Flush the cached config so that, if its bad, the server has a// chance to send us another in the future.cached-InvalidateServerConfig();CloseConnectionWithDetails(error, error_details);return;}/*继续对消息做hash便于server验证收到的消息是否完整*/CryptoUtils::HashHandshakeMessage(out, chlo_hash_);channel_id_sent_ (channel_id_key_.get() ! nullptr);if (cached-proof_verify_details()) {proof_handler_-OnProofVerifyDetailsAvailable(*cached-proof_verify_details());}next_state_ STATE_RECV_SHLO;SendHandshakeMessage(out);// Be prepared to decrypt with the new server write key.session()-connection()-SetAlternativeDecrypter(ENCRYPTION_INITIAL,crypto_negotiated_params_.initial_crypters.decrypter.release(),true /* latch once used */);// Send subsequent packets under encryption on the assumption that the// server will accept the handshake.session()-connection()-SetEncrypter(ENCRYPTION_INITIAL,crypto_negotiated_params_.initial_crypters.encrypter.release());session()-connection()-SetDefaultEncryptionLevel(ENCRYPTION_INITIAL);// TODO(ianswett): Merge ENCRYPTION_REESTABLISHED and// ENCRYPTION_FIRST_ESTABLSIHEDencryption_established_ true;session()-OnCryptoHandshakeEvent(QuicSession::ENCRYPTION_REESTABLISHED);
}
个人觉得最核心的代码就是FillClientHello函数了这里会生成随机数后续server会利用这个随机数生成对称加密的key部分通信的参数也会通过这个函数的执行保存在crypto_negotiated_params_对象中client发送了hello包接下来该server处理这个包了代码在libquic-master\src\net\quic\core\quic_crypto_server_stream.cc和quic_crypto_server_config.cc中代码如下核心功能是生成自己的公钥还有后续对称加密的key
QuicErrorCode QuicCryptoServerConfig::ProcessClientHello(const ValidateClientHelloResultCallback::Result validate_chlo_result,bool reject_only,QuicConnectionId connection_id,const IPAddress server_ip,const IPEndPoint client_address,QuicVersion version,const QuicVersionVector supported_versions,bool use_stateless_rejects,QuicConnectionId server_designated_connection_id,const QuicClock* clock,QuicRandom* rand,//发送给client用于计算对称keyQuicCompressedCertsCache* compressed_certs_cache,QuicCryptoNegotiatedParameters* params,QuicCryptoProof* crypto_proof,QuicByteCount total_framing_overhead,QuicByteCount chlo_packet_size,CryptoHandshakeMessage* out,DiversificationNonce* out_diversification_nonce,string* error_details) const {DCHECK(error_details);const CryptoHandshakeMessage client_hello validate_chlo_result.client_hello;const ClientHelloInfo info validate_chlo_result.info;QuicErrorCode valid CryptoUtils::ValidateClientHello(client_hello, version, supported_versions, error_details);if (valid ! QUIC_NO_ERROR)return valid;StringPiece requested_scid;client_hello.GetStringPiece(kSCID, requested_scid);const QuicWallTime now(clock-WallNow());scoped_refptrConfig requested_config;scoped_refptrConfig primary_config;{base::AutoLock locked(configs_lock_);if (!primary_config_.get()) {*error_details No configurations loaded;return QUIC_CRYPTO_INTERNAL_ERROR;}if (!next_config_promotion_time_.IsZero() next_config_promotion_time_.IsAfter(now)) {SelectNewPrimaryConfig(now);DCHECK(primary_config_.get());DCHECK_EQ(configs_.find(primary_config_-id)-second, primary_config_);}// Use the config that the client requested in order to do key-agreement.// Otherwise give it a copy of |primary_config_| to use.primary_config crypto_proof-config;requested_config GetConfigWithScid(requested_scid);}if (validate_chlo_result.error_code ! QUIC_NO_ERROR) {*error_details validate_chlo_result.error_details;return validate_chlo_result.error_code;}out-Clear();if (!ClientDemandsX509Proof(client_hello)) {*error_details Missing or invalid PDMD;return QUIC_UNSUPPORTED_PROOF_DEMAND;}DCHECK(proof_source_.get());string chlo_hash;CryptoUtils::HashHandshakeMessage(client_hello, chlo_hash);// No need to get a new proof if one was already generated.if (!crypto_proof-chain !proof_source_-GetProof(server_ip, info.sni.as_string(),primary_config-serialized, version, chlo_hash,crypto_proof-chain, crypto_proof-signature,crypto_proof-cert_sct)) {return QUIC_HANDSHAKE_FAILED;}StringPiece cert_sct;if (client_hello.GetStringPiece(kCertificateSCTTag, cert_sct) cert_sct.empty()) {params-sct_supported_by_client true;}if (!info.reject_reasons.empty() || !requested_config.get()) {BuildRejection(version, clock-WallNow(), *primary_config, client_hello,info, validate_chlo_result.cached_network_params,use_stateless_rejects, server_designated_connection_id, rand,compressed_certs_cache, params, *crypto_proof,total_framing_overhead, chlo_packet_size, out);return QUIC_NO_ERROR;}if (reject_only) {return QUIC_NO_ERROR;}const QuicTag* their_aeads;const QuicTag* their_key_exchanges;size_t num_their_aeads, num_their_key_exchanges;if (client_hello.GetTaglist(kAEAD, their_aeads, num_their_aeads) !QUIC_NO_ERROR ||client_hello.GetTaglist(kKEXS, their_key_exchanges,num_their_key_exchanges) ! QUIC_NO_ERROR ||num_their_aeads ! 1 || num_their_key_exchanges ! 1) {*error_details Missing or invalid AEAD or KEXS;return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;}size_t key_exchange_index;if (!QuicUtils::FindMutualTag(requested_config-aead, their_aeads,num_their_aeads, QuicUtils::LOCAL_PRIORITY,params-aead, nullptr) ||!QuicUtils::FindMutualTag(requested_config-kexs, their_key_exchanges,num_their_key_exchanges,QuicUtils::LOCAL_PRIORITY,params-key_exchange, key_exchange_index)) {*error_details Unsupported AEAD or KEXS;return QUIC_CRYPTO_NO_SUPPORT;}if (!requested_config-tb_key_params.empty()) {const QuicTag* their_tbkps;size_t num_their_tbkps;switch (client_hello.GetTaglist(kTBKP, their_tbkps, num_their_tbkps)) {case QUIC_CRYPTO_MESSAGE_PARAMETER_NOT_FOUND:break;case QUIC_NO_ERROR:if (QuicUtils::FindMutualTag(requested_config-tb_key_params, their_tbkps, num_their_tbkps,QuicUtils::LOCAL_PRIORITY, params-token_binding_key_param,nullptr)) {break;}default:*error_details Invalid Token Binding key parameter;return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;}}StringPiece public_value;/*提取client hello数据包发送的公钥server要用来生成对称加密的key*/if (!client_hello.GetStringPiece(kPUBS, public_value)) {*error_details Missing public value;return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;}const KeyExchange* key_exchange requested_config-key_exchanges[key_exchange_index];if (!key_exchange-CalculateSharedKey(public_value,params-initial_premaster_secret)) {*error_details Invalid public value;return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;}if (!info.sni.empty()) {std::unique_ptrchar[] sni_tmp(new char[info.sni.length() 1]);memcpy(sni_tmp.get(), info.sni.data(), info.sni.length());sni_tmp[info.sni.length()] 0;params-sni CryptoUtils::NormalizeHostname(sni_tmp.get());}string hkdf_suffix;//client hello消息序列化便于提取const QuicData client_hello_serialized client_hello.GetSerialized();/*根据一个原始密钥材料用hkdf算法推导出指定长度的密钥这里明显是要根据client hello的数据生成对称加密的密钥了*/hkdf_suffix.reserve(sizeof(connection_id) client_hello_serialized.length() requested_config-serialized.size());hkdf_suffix.append(reinterpret_castchar*(connection_id),sizeof(connection_id));hkdf_suffix.append(client_hello_serialized.data(),client_hello_serialized.length());hkdf_suffix.append(requested_config-serialized);DCHECK(proof_source_.get());if (crypto_proof-chain-certs.empty()) {*error_details Failed to get certs;return QUIC_CRYPTO_INTERNAL_ERROR;}hkdf_suffix.append(crypto_proof-chain-certs.at(0));StringPiece cetv_ciphertext;if (requested_config-channel_id_enabled client_hello.GetStringPiece(kCETV, cetv_ciphertext)) {CryptoHandshakeMessage client_hello_copy(client_hello);client_hello_copy.Erase(kCETV);client_hello_copy.Erase(kPAD);const QuicData client_hello_copy_serialized client_hello_copy.GetSerialized();string hkdf_input;hkdf_input.append(QuicCryptoConfig::kCETVLabel,strlen(QuicCryptoConfig::kCETVLabel) 1);hkdf_input.append(reinterpret_castchar*(connection_id),sizeof(connection_id));hkdf_input.append(client_hello_copy_serialized.data(),client_hello_copy_serialized.length());hkdf_input.append(requested_config-serialized);CrypterPair crypters;if (!CryptoUtils::DeriveKeys(params-initial_premaster_secret, params-aead,info.client_nonce, info.server_nonce,hkdf_input, Perspective::IS_SERVER,CryptoUtils::Diversification::Never(),crypters, nullptr /* subkey secret */)) {*error_details Symmetric key setup failed;return QUIC_CRYPTO_SYMMETRIC_KEY_SETUP_FAILED;}char plaintext[kMaxPacketSize];size_t plaintext_length 0;const bool success crypters.decrypter-DecryptPacket(kDefaultPathId, 0 /* packet number */,StringPiece() /* associated data */, cetv_ciphertext, plaintext,plaintext_length, kMaxPacketSize);if (!success) {*error_details CETV decryption failure;return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;}std::unique_ptrCryptoHandshakeMessage cetv(CryptoFramer::ParseMessage(StringPiece(plaintext, plaintext_length)));if (!cetv.get()) {*error_details CETV parse error;return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;}StringPiece key, signature;if (cetv-GetStringPiece(kCIDK, key) cetv-GetStringPiece(kCIDS, signature)) {if (!ChannelIDVerifier::Verify(key, hkdf_input, signature)) {*error_details ChannelID signature failure;return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;}params-channel_id key.as_string();}}string hkdf_input;size_t label_len strlen(QuicCryptoConfig::kInitialLabel) 1;hkdf_input.reserve(label_len hkdf_suffix.size());hkdf_input.append(QuicCryptoConfig::kInitialLabel, label_len);hkdf_input.append(hkdf_suffix);string* subkey_secret params-initial_subkey_secret;CryptoUtils::Diversification diversification CryptoUtils::Diversification::Never();if (version QUIC_VERSION_32) {rand-RandBytes(out_diversification_nonce-data(),out_diversification_nonce-size());diversification CryptoUtils::Diversification::Now(out_diversification_nonce);}if (!CryptoUtils::DeriveKeys(params-initial_premaster_secret, params-aead,info.client_nonce, info.server_nonce, hkdf_input,Perspective::IS_SERVER, diversification,params-initial_crypters, subkey_secret)) {*error_details Symmetric key setup failed;return QUIC_CRYPTO_SYMMETRIC_KEY_SETUP_FAILED;}string forward_secure_public_value;if (ephemeral_key_source_.get()) {params-forward_secure_premaster_secret ephemeral_key_source_-CalculateForwardSecureKey(key_exchange, rand, clock-ApproximateNow(), public_value,forward_secure_public_value);} else {std::unique_ptrKeyExchange forward_secure_key_exchange(key_exchange-NewKeyPair(rand));forward_secure_public_value forward_secure_key_exchange-public_value().as_string();/*生成共享密钥*/if (!forward_secure_key_exchange-CalculateSharedKey(public_value, params-forward_secure_premaster_secret)) {*error_details Invalid public value;return QUIC_INVALID_CRYPTO_MESSAGE_PARAMETER;}}string forward_secure_hkdf_input;label_len strlen(QuicCryptoConfig::kForwardSecureLabel) 1;forward_secure_hkdf_input.reserve(label_len hkdf_suffix.size());forward_secure_hkdf_input.append(QuicCryptoConfig::kForwardSecureLabel,label_len);forward_secure_hkdf_input.append(hkdf_suffix);string shlo_nonce;shlo_nonce NewServerNonce(rand, info.now);out-SetStringPiece(kServerNonceTag, shlo_nonce);/*生成密钥*/if (!CryptoUtils::DeriveKeys(params-forward_secure_premaster_secret, params-aead,info.client_nonce,shlo_nonce.empty() ? info.server_nonce : shlo_nonce,forward_secure_hkdf_input, Perspective::IS_SERVER,CryptoUtils::Diversification::Never(),params-forward_secure_crypters, params-subkey_secret)) {*error_details Symmetric key setup failed;return QUIC_CRYPTO_SYMMETRIC_KEY_SETUP_FAILED;}out-set_tag(kSHLO);QuicTagVector supported_version_tags;for (size_t i 0; i supported_versions.size(); i) {supported_version_tags.push_back(QuicVersionToQuicTag(supported_versions[i]));}out-SetVector(kVER, supported_version_tags);out-SetStringPiece(kSourceAddressTokenTag,NewSourceAddressToken(*requested_config.get(), info.source_address_tokens,client_address.address(), rand, info.now, nullptr));QuicSocketAddressCoder address_coder(client_address);out-SetStringPiece(kCADR, address_coder.Encode());/*server hello包中设置server的公钥后续client会利用这个生成对称加密的key*/out-SetStringPiece(kPUBS, forward_secure_public_value);return QUIC_NO_ERROR;
}
这里用了不同的方法来生成对称加密的key。这里以椭圆曲线为例计算对称加密key的代码如下这是直接调用了openssl/curve25519.h的接口计算出来的。一旦双方都生成了对称密钥后续就可以通过对称加密通信了
bool Curve25519KeyExchange::CalculateSharedKey(StringPiece peer_public_value,string* out_result) const {if (peer_public_value.size() ! crypto::curve25519::kBytes) {return false;}uint8_t result[crypto::curve25519::kBytes];if (!crypto::curve25519::ScalarMult(private_key_,reinterpret_castconst uint8_t*(peer_public_value.data()), result)) {return false;}out_result-assign(reinterpret_castchar*(result), sizeof(result));return true;
}
bool ScalarMult(const uint8_t* private_key,const uint8_t* peer_public_key,uint8_t* shared_key) {return !!X25519(shared_key, private_key, peer_public_key);
}
通信时给packet加密的方法
bool AeadBaseEncrypter::EncryptPacket(QuicPathId path_id,QuicPacketNumber packet_number,StringPiece associated_data,StringPiece plaintext,char* output,size_t* output_length,size_t max_output_length) {size_t ciphertext_size GetCiphertextSize(plaintext.length());if (max_output_length ciphertext_size) {return false;}// TODO(ianswett): Introduce a check to ensure that we dont encrypt with the// same packet number twice.const size_t nonce_size nonce_prefix_size_ sizeof(packet_number);ALIGNAS(4) char nonce_buffer[kMaxNonceSize];memcpy(nonce_buffer, nonce_prefix_, nonce_prefix_size_);uint64_t path_id_packet_number QuicUtils::PackPathIdAndPacketNumber(path_id, packet_number);memcpy(nonce_buffer nonce_prefix_size_, path_id_packet_number,sizeof(path_id_packet_number));/*这里用nonce给明文加密*/if (!Encrypt(StringPiece(nonce_buffer, nonce_size), associated_data,plaintext, reinterpret_castunsigned char*(output))) {return false;}*output_length ciphertext_size;return true;
}
最后server hello消息是从这里发出去的并且在某些情况下server hello已经用server新生成的key加密了如下
void QuicCryptoServerStream::FinishProcessingHandshakeMessage(const ValidateClientHelloResultCallback::Result result,std::unique_ptrProofSource::Details details) {const CryptoHandshakeMessage message result.client_hello;// Clear the callback that got us here.DCHECK(validate_client_hello_cb_ ! nullptr);validate_client_hello_cb_ nullptr;if (use_stateless_rejects_if_peer_supported_) {peer_supports_stateless_rejects_ DoesPeerSupportStatelessRejects(message);}CryptoHandshakeMessage reply;DiversificationNonce diversification_nonce;string error_details;QuicErrorCode error /*server处理client的hello消息重点是生成对称加密key、自己的公钥和nonce同时生成给client回复的消息*/ProcessClientHello(result, std::move(details), reply,diversification_nonce, error_details);if (error ! QUIC_NO_ERROR) {CloseConnectionWithDetails(error, error_details);return;}if (reply.tag() ! kSHLO) {if (reply.tag() kSREJ) {DCHECK(use_stateless_rejects_if_peer_supported_);DCHECK(peer_supports_stateless_rejects_);// Before sending the SREJ, cause the connection to save crypto packets// so that they can be added to the time wait list manager and// retransmitted.session()-connection()-EnableSavingCryptoPackets();}SendHandshakeMessage(reply);//给client发server helloif (reply.tag() kSREJ) {DCHECK(use_stateless_rejects_if_peer_supported_);DCHECK(peer_supports_stateless_rejects_);DCHECK(!handshake_confirmed());DVLOG(1) Closing connection session()-connection()-connection_id() because of a stateless reject.;session()-connection()-CloseConnection(QUIC_CRYPTO_HANDSHAKE_STATELESS_REJECT, stateless reject,ConnectionCloseBehavior::SILENT_CLOSE);}return;}// If we are returning a SHLO then we accepted the handshake. Now// process the negotiated configuration options as part of the// session config.//代码到这里已经给client发送了client hello表示server已经准备好接受数据了//这里保存一些双方协商好的通信配置QuicConfig* config session()-config();OverrideQuicConfigDefaults(config);error config-ProcessPeerHello(message, CLIENT, error_details);if (error ! QUIC_NO_ERROR) {CloseConnectionWithDetails(error, error_details);return;}session()-OnConfigNegotiated();config-ToHandshakeMessage(reply);// Receiving a full CHLO implies the client is prepared to decrypt with// the new server write key. We can start to encrypt with the new server// write key. 可以开始用服务端新生成的key解密数据了//// NOTE: the SHLO will be encrypted with the new server write key./*既然在server已经生成了对称加密的key这里可以用这个key加密server hello消息*/session()-connection()-SetEncrypter(ENCRYPTION_INITIAL,crypto_negotiated_params_.initial_crypters.encrypter.release());session()-connection()-SetDefaultEncryptionLevel(ENCRYPTION_INITIAL);// Set the decrypter immediately so that we no longer accept unencrypted// packets.session()-connection()-SetDecrypter(ENCRYPTION_INITIAL,crypto_negotiated_params_.initial_crypters.decrypter.release());if (version() QUIC_VERSION_32) {session()-connection()-SetDiversificationNonce(diversification_nonce);}SendHandshakeMessage(reply);//发送server hellosession()-connection()-SetEncrypter(ENCRYPTION_FORWARD_SECURE,crypto_negotiated_params_.forward_secure_crypters.encrypter.release());session()-connection()-SetDefaultEncryptionLevel(ENCRYPTION_FORWARD_SECURE);session()-connection()-SetAlternativeDecrypter(ENCRYPTION_FORWARD_SECURE,crypto_negotiated_params_.forward_secure_crypters.decrypter.release(),false /* dont latch */);encryption_established_ true;handshake_confirmed_ true;session()-OnCryptoHandshakeEvent(QuicSession::HANDSHAKE_CONFIRMED);
}
2为了防止tcp的队头阻塞quic在前面丢包的情况下任然继续发包丢的包用新的packet number重新发怎么区别这个新包是以往丢包的重发了核心是每个包都有stream id和stream offset字段根据这两个字段定位包的位置而不是packet number。整个包结构定义的类在这里
struct NET_EXPORT_PRIVATE QuicStreamFrame {QuicStreamFrame();QuicStreamFrame(QuicStreamId stream_id,bool fin,QuicStreamOffset offset,base::StringPiece data);QuicStreamFrame(QuicStreamId stream_id,bool fin,QuicStreamOffset offset,QuicPacketLength data_length,UniqueStreamBuffer buffer);~QuicStreamFrame();NET_EXPORT_PRIVATE friend std::ostream operator(std::ostream os,const QuicStreamFrame s);QuicStreamId stream_id;bool fin;QuicPacketLength data_length;const char* data_buffer;QuicStreamOffset offset; // Location of this data in the stream.// nullptr when the QuicStreamFrame is received, and non-null when sent.UniqueStreamBuffer buffer;private:QuicStreamFrame(QuicStreamId stream_id,bool fin,QuicStreamOffset offset,const char* data_buffer,QuicPacketLength data_length,UniqueStreamBuffer buffer);DISALLOW_COPY_AND_ASSIGN(QuicStreamFrame);
};
收到后自然要把payload取出来拼接成完整的数据stream id和stream offset必不可少拼接和处理的逻辑在这里里面涉及到很多duplicate冗余去重的动作都是依据offset来判断的
QuicErrorCode QuicStreamSequencerBuffer::OnStreamData(QuicStreamOffset starting_offset,base::StringPiece data,QuicTime timestamp,size_t* const bytes_buffered,std::string* error_details) {*bytes_buffered 0;QuicStreamOffset offset starting_offset;size_t size data.size();if (size 0) {*error_details Received empty stream frame without FIN.;return QUIC_EMPTY_STREAM_FRAME_NO_FIN;}// Find the first gap not ending before |offset|. This gap maybe the gap to// fill if the arriving frame doesnt overlaps with previous ones.std::listGap::iterator current_gap gaps_.begin();while (current_gap ! gaps_.end() current_gap-end_offset offset) {current_gap;}DCHECK(current_gap ! gaps_.end());// duplication: might duplicate with data alread filled,but also might// overlap across different base::StringPiece objects already written.// In both cases, dont write the data,// and allow the caller of this method to handle the result.if (offset current_gap-begin_offset offset size current_gap-begin_offset) {DVLOG(1) Duplicated data at offset: offset length: size;return QUIC_NO_ERROR;}if (offset current_gap-begin_offset offset size current_gap-begin_offset) {// Beginning of new data overlaps data before current gap.*error_details string(Beginning of received data overlaps with buffered data.\n) New frame range RangeDebugString(offset, offset size) with first 128 bytes: string(data.data(), data.length() 128 ? data.length() : 128) \nCurrently received frames: ReceivedFramesDebugString() \nCurrent gaps: GapsDebugString();return QUIC_OVERLAPPING_STREAM_DATA;}if (offset size current_gap-end_offset) {// End of new data overlaps with data after current gap.*error_details string(End of received data overlaps with buffered data.\n) New frame range RangeDebugString(offset, offset size) with first 128 bytes: string(data.data(), data.length() 128 ? data.length() : 128) \nCurrently received frames: ReceivedFramesDebugString() \nCurrent gaps: GapsDebugString();return QUIC_OVERLAPPING_STREAM_DATA;}// Write beyond the current range this buffer is covering.if (offset size total_bytes_read_ max_buffer_capacity_bytes_) {*error_details Received data beyond available range.;return QUIC_INTERNAL_ERROR;}if (current_gap-begin_offset ! starting_offset current_gap-end_offset ! starting_offset data.length() gaps_.size() kMaxNumGapsAllowed) {// This frame is going to create one more gap which exceeds max number of// gaps allowed. Stop processing.*error_details Too many gaps created for this stream.;return QUIC_TOO_MANY_FRAME_GAPS;}size_t total_written 0;size_t source_remaining size;const char* source data.data();// Write data block by block. If corresponding block has not created yet,// create it first.// Stop when all data are written or reaches the logical end of the buffer.while (source_remaining 0) {const size_t write_block_num GetBlockIndex(offset);const size_t write_block_offset GetInBlockOffset(offset);DCHECK_GT(blocks_count_, write_block_num);size_t block_capacity GetBlockCapacity(write_block_num);size_t bytes_avail block_capacity - write_block_offset;// If this write meets the upper boundary of the buffer,// reduce the available free bytes.if (offset bytes_avail total_bytes_read_ max_buffer_capacity_bytes_) {bytes_avail total_bytes_read_ max_buffer_capacity_bytes_ - offset;}if (reduce_sequencer_buffer_memory_life_time_ blocks_ nullptr) {blocks_.reset(new BufferBlock*[blocks_count_]());for (size_t i 0; i blocks_count_; i) {blocks_[i] nullptr;}}if (blocks_[write_block_num] nullptr) {// TODO(danzh): Investigate if using a freelist would improve performance.// Same as RetireBlock().blocks_[write_block_num] new BufferBlock();}const size_t bytes_to_copy minsize_t(bytes_avail, source_remaining);char* dest blocks_[write_block_num]-buffer write_block_offset;DVLOG(1) Write at offset: offset length: bytes_to_copy;memcpy(dest, source, bytes_to_copy);source bytes_to_copy;source_remaining - bytes_to_copy;offset bytes_to_copy;total_written bytes_to_copy;}DCHECK_GT(total_written, 0u);*bytes_buffered total_written;UpdateGapList(current_gap, starting_offset, total_written);frame_arrival_time_map_.insert(std::make_pair(starting_offset, FrameInfo(size, timestamp)));num_bytes_buffered_ total_written;return QUIC_NO_ERROR;
}
3为了精准测量RTTquic协议的数据包编号都是单调递增的哪怕是重发的包的编号都是增加的这部分的控制代码在WritePacket函数里面函数开头就判断数据包编号。一旦发现编号比最后一次发送包的编号还小说明出错了这时就关闭连接退出函数
bool QuicConnection::WritePacket(SerializedPacket* packet) {/*如果数据包号比最后一个发送包的号还小说明顺序错了直接关闭连接*/if (packet-packet_number sent_packet_manager_-GetLargestSentPacket(packet-path_id)) {QUIC_BUG Attempt to write packet: packet-packet_number after: sent_packet_manager_-GetLargestSentPacket(packet-path_id);CloseConnection(QUIC_INTERNAL_ERROR, Packet written out of order.,ConnectionCloseBehavior::SEND_CONNECTION_CLOSE_PACKET);return true;}/*没有连接、没有加密的包是不能发的*/if (ShouldDiscardPacket(*packet)) {stats_.packets_discarded;return true;}.........................
}
4为啥quic协议要基于udp了应用层现成的协议很复杂改造的难度大传输层只有tcp和udp两种协议tcp的缺点不再赘述udp的优点就是简单只提供最原始的发包功能完全不管对方有没有收到quic就是利用了udp这种最基础的send package发包能力在此之上完成了tls保证数据安全、拥塞控制保证链路被塞满、多路复用保证数据不丢失等应用层的功能
