Update spec and test for ECN

CMakeLists.txt

@@ -32,12 +32,10 @@ include_directories(include lib tests
     ${Picoquic_INCLUDE_DIRS} ${PTLS_INCLUDE_DIRS} ${OPENSSL_INCLUDE_DIR})
 
 set (C4_LIBRARY_FILES
-    src/c4.c
     src/register_cc_algo.c
 )
 
 set (C4_LIBRARY_HEADERS
-    src/c4.h
     src/picoquic_register_cc_algo.h
 )
 

c4_lib/c4_lib.vcxproj

@@ -144,14 +144,12 @@
   </ItemDefinitionGroup>
   <ItemGroup>
     <ClInclude Include="..\pico_sim_vs\pico_sim_vs\getopt.h" />
-    <ClInclude Include="..\src\c4.h" />
     <ClInclude Include="..\src\picoquic_register_cc_algo.h" />
     <ClInclude Include="framework.h" />
     <ClInclude Include="pch.h" />
   </ItemGroup>
   <ItemGroup>
     <ClCompile Include="..\pico_sim_vs\pico_sim_vs\getopt.c" />
-    <ClCompile Include="..\src\c4.c" />
     <ClCompile Include="..\src\register_cc_algo.c" />
   </ItemGroup>
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />

c4_lib/c4_lib.vcxproj.filters

@@ -21,9 +21,6 @@
     <ClInclude Include="pch.h">
       <Filter>Header Files</Filter>
     </ClInclude>
-    <ClInclude Include="..\src\c4.h">
-      <Filter>Source Files</Filter>
-    </ClInclude>
     <ClInclude Include="..\src\picoquic_register_cc_algo.h">
       <Filter>Source Files</Filter>
     </ClInclude>
@@ -32,9 +29,6 @@
     </ClInclude>
   </ItemGroup>
   <ItemGroup>
-    <ClCompile Include="..\src\c4.c">
-      <Filter>Source Files</Filter>
-    </ClCompile>
     <ClCompile Include="..\src\register_cc_algo.c">
       <Filter>Source Files</Filter>
     </ClCompile>

doc/c4-design.md

@@ -85,7 +85,9 @@ informative:
 
    I-D.irtf-iccrg-ledbat-plus-plus:
 
+   RFC9330:
    RFC9331:
+   I-D.briscoe-iccrg-prague-congestion-control:
    ICCRG-LEO:
     target: https://datatracker.ietf.org/meeting/122/materials/slides-122-iccrg-mind-the-misleading-effects-of-leo-mobility-on-end-to-end-congestion-control-00
     title: "Mind the Misleading Effects of LEO Mobility on End-to-End Congestion Control"
@@ -544,7 +546,7 @@ if measured_rate > nominal_rate and not congested:
 In our early experiments, we observed a "congestion bounce"
 that happened as follow:
 
-* congestion is detected, the nomnal rate is reduced, and
+* congestion is detected, the nominal rate is reduced, and
   C4 enters recovery.
 * packets sent at the data rate that caused the congestion
   continue to be acknowledged during recovery.
@@ -596,6 +598,43 @@ ack delay and the send delay as a divider is sufficient
 for stable operation, and does not cause the response
 delays that filtering would.
 
+## Early Congestion Modification
+
+We want C4 to handle Early Congestion Notification in a manner
+compatible with the L4S design. For that, we monitor
+the evolving ratio of CE marks that the L4S specification
+designates as `alpha`
+(we use `ecn_alpha` here to avoid confusion),
+and we detect congestion if the ratio grows over a threshold.
+
+We did not find a recommended algorithm for computing `ecn_alpha`
+in either {{RFC9330}} or {{RFC9331}}, but we could get some
+concrete suggestions in {{I-D.briscoe-iccrg-prague-congestion-control}}.
+That draft, now obsolete, suggests updating the ratio once per
+RTT, as the exponential weighted average of the fraction of
+CE marks per packet:
+
+~~~
+frac = nb_CE / (nb_CE + nb_ECT1)
+ecn_alpha += (frac - ecn_alpha)/16
+~~~
+
+This kind of averaging introduces a reaction delay. The draft suggests mitigating that
+delay by preempting the averaging if the fraction is large:
+
+~~~
+if frac > 0.5:
+    ecn_alpha = frac
+~~~
+
+We followed that design, but decided to update the coefficient after
+each acknowledgement, instead of after each RTT. This is in line with
+our implementation of "delayed acknowledgements" in QUIC, which
+results in a small number of acknowledgements per RTT.
+
+The reaction of C4 to an excess of CE marks is similar to the
+reaction to excess delays or to packet losses, see {{congestion}}.
+
 # Competition with other algorithms
 
 We saw in {{vegas-struggle}} that delay based algorithms required
@@ -802,7 +841,7 @@ connection using either Cubic, BBR or C4. We had to design a response,
 and we first turned to making the response to excess delay or
 packet loss a function of the data rate of the flow.
 
-## Introducing a sensitivity curve
+## Introducing a sensitivity curve {#sensitivity-curve}
 
 In our second design, we attempted to fix the unfairness and
 shutdowns effect by introducing a sensitivity curve,
@@ -831,6 +870,13 @@ For the loss threshold, the rule is:
 loss_threshold = 0.02 + 0.50 * (1-sensitivity);
 ~~~
 
+For the CE mark threshold, the rule is:
+
+~~~
+loss_threshold = 1/32 + 1/32 * (1-sensitivity);
+~~~
+
+
 This very simple change allowed us to stabilize the results. In our
 competition tests we see sharing of resource almost equitably between
 C4 connections, and reasonably between C4 and Cubic or C4 and BBR.
@@ -858,7 +904,8 @@ This means we would only double the bandwidth after about 68 RTT, or increase
 from 10 to 65 Mbps after 185 RTT -- by which time the LEO station might
 have connected to a different orbiting satellite. To go faster, we implement
 a "cascade": if the previous pushing at 6.25% was successful, the next
-pushing will use 25% (see {{variable-pushing}}). If three successive pushings
+pushing will use 25% (see {{variable-pushing}}), or an intermediate
+value if the observed ratio of ECN marks is greater than 0. If three successive pushings
 all result in increases of the
 nominal rate, C4 will reenter the "startup" mode, during which each RTT
 can result in a 100% increase of rate and CWND.
@@ -947,6 +994,12 @@ We manage that compromise by adopting a variable pushing rate:
   the next pushing will happen at 25%, otherwise it will
   remain at 6.25%
 
+If the observed ratio of ECN-CE marks is greater than zero, we will
+use it to modulate the amount of pushing. We leave the pushing rate
+at 6.25% if the previous pushing attempt was not successful, but
+otherwise we pick a value intermediate between 25% (if 0 ECN marks)
+and 6.25% (if the ratio of ECN marks approaches the threshold).
+
 As explained in {{cascade}}, if three consecutive pushing attempts
 result in significant increases, C4 detects that the underlying network
 conditions have changed, and will reenter the startup state.

doc/c4-spec.md

@@ -302,7 +302,7 @@ diagram.
 
 ~~~
 
-## Setting pacing rate, congestion window and quantum
+## Setting pacing rate, congestion window and quantum {#set_pace}
 
 If the nominal rate or the nominal max RTT are not yet
 assessed, C4 sets pacing rate, congestion window and
@@ -320,7 +320,6 @@ and on a coefficient `alpha_current`:
 ~~~
 pacing_rate = alpha_current_ * nominal_rate
 cwnd = max (pacing_rate * nominal_max_rtt, 2*MTU)
-quantum = max ( min (cwnd / 4, 64KB), 2*MTU)
 ~~~
 
 The coefficient `alpha` for the different states is:
@@ -332,6 +331,18 @@ Recovery | 15/16 |
 Cruising | 1 |
 Pushing | 5/4 or 17/16 | see {{c4-pushing}} for rules on choosing 5/4 or 17/16
 
+Setting the pacing quantum is a tradeoff between two requirements.
+Using a large quantum enables applications to send large batches of
+packets in a single transaction, which improves performance. But
+sending large batches of packets creates "instant queues" and
+causes some Active Queue Management mechanisms to mark packets as
+ECN/CE, or drop them. As a compromise, we set the quantum to
+4 milliseconds worth of transmission.
+
+~~~
+quantum = max ( min (pacing_rate*4_milliseconds, 64KB), 2*MTU)
+~~~
+
 ## Initial state {#c4-initial}
 
 When the flow is initialized, it enters the Initial state,
@@ -432,10 +443,21 @@ a congestion signal is received.
 
 ## Pushing state {#c4-pushing}
 
-The Pushing state is entered from the Cruising state. 
-The coefficient `alpha_current` is set to 5/4 if the previous
+The Pushing state is entered from the Cruising state.
+
+The coefficient `alpha_current` depend on whether the
+previous
 pushing attempt was successful (see {{c4-recovery}}),
-or 17/16 if it was not.
+and also of the current value of `ecn_alpha`
+(see {{process-ecn}}):
+
+~~~
+   if not previous_attempt_successful:
+       alpha_current = 17/16
+   else:
+       alpha_current = 17/16 +
+          17/16 * (1 - ecn_alpha / ecn_threshold)
+~~~
 
 C4 exits the pushing state after one era, or if a congestion
 signal is received before that. In an exception to
@@ -536,13 +558,53 @@ PTO timeouts. When testing in "high jitter" conditions, we realized that we shou
 not change the state of C4 for losses detected solely based on timer, and
 only react to those losses that are detected by gaps in acknowledgements.
 
-## Detecting Excessive CE Marks
+## Detecting Excessive CE Marks {#process-ecn}
+
+When the path supports ECN marking, C4 monitors the arrival of ECN/CE and
+ECN/ECT(1) marks by computing the ratio `ecn_alpha`. Congestion is detected
+when that ratio exceeds `ecn_threshold`, which varies depending on the
+sensitivity coefficient:
+
+~~~
+ecn_threshold = (2-sensitivity)*3/32
+~~~
+
+The ratio `ecn_alpha` is
+updated each time an acknowledgement is received, as follow:
+
+~~~
+delta_ce = increase in the reported CE marks
+delta_ect1 = increase in the reported ECT(1) marks
+frac = delta_ce / (delta_ce + delta_ect1)
+
+if frac >= 0.5:
+    ecn_alpha = frac
+else:
+    ecn_alpha += (frac - ecn_alpha)/16
+
+if ecn_alpha > ecn_threshold:
+    report congestion
+~~~
+
+Congestion detection causes C4 to enter recovery. The
+ration `ecn_alpha` is set to zero on exit of recovery.
+
+## Applying congestion signals
+
+On congestion signal, if C4 was not in recovery state, it
+will enter recovery.
 
-TBD. The plan is to mimic the L4S specification.
+As stated in {{c4-initial}} and {{c4-pushing}}, detecting
+a congestion in the Initial or Pushing state does not cause
+a change in the `nominal_rate` or `nominal_max_RTT`, because
+the pacing rate in these states is larger than the
+`nominal_rate`. Rate reduction only happens if recovery
+was entered from the Cruising state.
 
-## Rate Reduction on Congestion
+### Rate Reduction on Congestion {#rate-reduction}
 
-On entering recovery, C4 reduces the `nominal_rate` by the factor "beta"
+On entering recovery from the cruising state, C4 reduces the
+`nominal_rate` by the factor "beta"
 corresponding to the congestion signal:
 
 ~~~
@@ -560,13 +622,16 @@ the acceptable margin, capped to `1/4`:
 ~~~
     beta = min(1/4,
               (rtt_sample - (nominal_max_rtt + delay_threshold)/
-               delay_threshod))
+               delay_threshold))
+~~~
+
+If the signal is an ECN/CE rate, the coefficient is proportional
+to the difference between `ecn_alpha` and `ecn_threshold`, capped to '1/4':
+
+~~~
+    beta = min(1/4, (ecn_alpha - ecn_threshold)/ ecn_threshold))
 ~~~
 
-If the signal is an ECN/CE rate, this is still TBD. We could
-use a proportional reduction coefficient in line with
-{{RFC9331}}, but we should use the sensitivity coefficient to
-modulate that signal.
 
 # Security Considerations
 
@@ -586,6 +651,32 @@ This document has no IANA actions.
 
 TODO acknowledge.
 
+# Changes since previous versions
+
+This section should be deleted before publication as an RFC
+
+## Changes since draft-huitema-ccwg-c4-spec-00
+
+Added the specification of reaction to ECN in {{process-ecn}}
+and in {{rate-reduction}}. Update section {{c4-pushing}} to
+modulate pushing rate based on observed rate of ECN/CE marks.
+
+In {{set_pace}}, the computation of the "quantum" changed
+from:
+
+~~~
+quantum = max ( min (cwnd / 4, 64KB), 2*MTU)
+~~~
+
+to:
+
+~~~
+quantum = max ( min (pacing_rate*4_milliseconds, 64KB), 2*MTU)
+~~~
+
+The old formula caused long bursts of packets that would
+trigger packet drops or ECN/CE marking by active queue management
+algorithms.