bpf: fix SO_MAX_PACING_RATE to support TCP internal pacing

If sch_fq packet scheduler is not used, TCP can fallback to
internal pacing, but this requires sk_pacing_status to
be properly set.

Fixes: 8c4b4c7e9ff0 ("bpf: Add setsockopt helper function to bpf")
Signed-off-by: Yuchung Cheng <[email protected]>
Signed-off-by: Eric Dumazet <[email protected]>
Cc: Lawrence Brakmo <[email protected]>
Acked-by: Martin KaFai Lau <[email protected]>
Signed-off-by: Daniel Borkmann <[email protected]>

diff --git a/net/core/filter.c b/net/core/filter.c
index d9ea51b47f3585409080dd40e263d4dc23b18ab1..dab10d21cae848b60b6ea0ab6f37b4a672b78b29 100644 (file)
--- a/net/core/filter.c
+++ b/net/core/filter.c
@@ -4119,6 +4119,10 @@ BPF_CALL_5(bpf_setsockopt, struct bpf_sock_ops_kern *, bpf_sock,
                        sk->sk_sndbuf = max_t(int, val * 2, SOCK_MIN_SNDBUF);
                        break;
                case SO_MAX_PACING_RATE: /* 32bit version */
+                       if (val != ~0U)
+                               cmpxchg(&sk->sk_pacing_status,
+                                       SK_PACING_NONE,
+                                       SK_PACING_NEEDED);
                        sk->sk_max_pacing_rate = (val == ~0U) ? ~0UL : val;
                        sk->sk_pacing_rate = min(sk->sk_pacing_rate,
                                                 sk->sk_max_pacing_rate);