diff options
| author | Jason Xing <kerneljasonxing@gmail.com> | 2025-02-20 15:29:38 +0800 |
|---|---|---|
| committer | Martin KaFai Lau <martin.lau@kernel.org> | 2025-02-20 14:29:48 -0800 |
| commit | c9525d240c8117de35171ae705058ddf9667be27 (patch) | |
| tree | 6dc65b46fda900064c633445963488eadb803d76 | |
| parent | b3b81e6b009dd8f85cd3b9c65eb492249c2649a8 (diff) | |
bpf: Add BPF_SOCK_OPS_TSTAMP_SENDMSG_CB callback
This patch introduces a new callback in tcp_tx_timestamp() to correlate
tcp_sendmsg timestamp with timestamps from other tx timestamping
callbacks (e.g., SND/SW/ACK).
Without this patch, BPF program wouldn't know which timestamps belong
to which flow because of no socket lock protection. This new callback
is inserted in tcp_tx_timestamp() to address this issue because
tcp_tx_timestamp() still owns the same socket lock with
tcp_sendmsg_locked() in the meanwhile tcp_tx_timestamp() initializes
the timestamping related fields for the skb, especially tskey. The
tskey is the bridge to do the correlation.
For TCP, BPF program hooks the beginning of tcp_sendmsg_locked() and
then stores the sendmsg timestamp at the bpf_sk_storage, correlating
this timestamp with its tskey that are later used in other sending
timestamping callbacks.
Signed-off-by: Jason Xing <kerneljasonxing@gmail.com>
Signed-off-by: Martin KaFai Lau <martin.lau@kernel.org>
Reviewed-by: Willem de Bruijn <willemb@google.com>
Link: https://patch.msgid.link/20250220072940.99994-11-kerneljasonxing@gmail.com
| -rw-r--r-- | include/uapi/linux/bpf.h | 5 | ||||
| -rw-r--r-- | net/ipv4/tcp.c | 4 | ||||
| -rw-r--r-- | tools/include/uapi/linux/bpf.h | 5 |
3 files changed, 14 insertions, 0 deletions
diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index cc10104f34a0..defa5bb881f4 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -7049,6 +7049,11 @@ enum { * when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TSTAMP_SENDMSG_CB, /* Called when every sendmsg syscall + * is triggered. It's used to correlate + * sendmsg timestamp with corresponding + * tskey. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c index 2171e2f045bb..298d1da05bee 100644 --- a/net/ipv4/tcp.c +++ b/net/ipv4/tcp.c @@ -496,6 +496,10 @@ static void tcp_tx_timestamp(struct sock *sk, struct sockcm_cookie *sockc) if (tsflags & SOF_TIMESTAMPING_TX_RECORD_MASK) shinfo->tskey = TCP_SKB_CB(skb)->seq + skb->len - 1; } + + if (cgroup_bpf_enabled(CGROUP_SOCK_OPS) && + SK_BPF_CB_FLAG_TEST(sk, SK_BPF_CB_TX_TIMESTAMPING) && skb) + bpf_skops_tx_timestamping(sk, skb, BPF_SOCK_OPS_TSTAMP_SENDMSG_CB); } static bool tcp_stream_is_readable(struct sock *sk, int target) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index cc10104f34a0..defa5bb881f4 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -7049,6 +7049,11 @@ enum { * when SK_BPF_CB_TX_TIMESTAMPING * feature is on. */ + BPF_SOCK_OPS_TSTAMP_SENDMSG_CB, /* Called when every sendmsg syscall + * is triggered. It's used to correlate + * sendmsg timestamp with corresponding + * tskey. + */ }; /* List of TCP states. There is a build check in net/ipv4/tcp.c to detect |