summaryrefslogtreecommitdiff
path: root/drivers/infiniband/ulp/rtrs
AgeCommit message (Collapse)Author
2021-07-15RDMA/rtrs: Add error messages for failed operations.Jack Wang
It could help debugging in case of error happens. Link: https://lore.kernel.org/r/20210712060750.16494-2-jinpu.wang@ionos.com Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Reviewed-by: Aleksei Marov <aleksei.marov@ionos.com> Reviewed-by: Gioh Kim <gi-oh.kim@ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-21rnbd/rtrs-clt: Query and use max_segments from rtrs-clt.Jack Wang
With fast memory registration on write request, rnbd-clt can do bigger IO without split. rnbd-clt now can query rtrs-clt to get the max_segments, instead of using BMAX_SEGMENTS. BMAX_SEGMENTS is not longer needed, so remove it. Link: https://lore.kernel.org/r/20210621055340.11789-6-jinpu.wang@ionos.com Cc: Jens Axboe <axboe@kernel.dk> Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@cloud.ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-21RDMA/rtrs-clt: Raise MAX_SEGMENTSJack Wang
As we can do fast memory registration on write, we can increase the max_segments, default to 512K. Link: https://lore.kernel.org/r/20210621055340.11789-5-jinpu.wang@ionos.com Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@cloud.ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-21RDMA/rtrs_clt: Alloc less memory with write path fast memory registrationJack Wang
With write path fast memory registration, we need less memory for each request. With fast memory registration, we can reduce max_send_sge to save memory usage. Also convert the kmalloc_array to kcalloc. Link: https://lore.kernel.org/r/20210621055340.11789-4-jinpu.wang@ionos.com Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@cloud.ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-21RDMA/rtrs-clt: Write path fast memory registrationJack Wang
With fast memory registration in write path, we can reduce the memory consumption by using less max_send_sge, support IO bigger than 116 KB (29 segments * 4 KB) without splitting, and it also make the IO path more symmetric. To avoid some times MR reg failed, waiting for the invalidation to finish before the new mr reg. Introduce a refcount, only finish the request when both local invalidation and io reply are there. Link: https://lore.kernel.org/r/20210621055340.11789-3-jinpu.wang@ionos.com Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com> Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Dima Stepanov <dmitrii.stepanov@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-21RDMA/rtrs: Introduce head/tail wrJack Wang
Introduce tail wr, we can send as the last wr, we want to send the local invalidate wr after rdma wr in later patch. While at it, also fix coding style issue. Link: https://lore.kernel.org/r/20210621055340.11789-2-jinpu.wang@ionos.com Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@cloud.ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-21RDMA: Fix kernel-doc warnings about wrong commentLeon Romanovsky
Compilation with W=1 produces warnings similar to the below. drivers/infiniband/ulp/ipoib/ipoib_main.c:320: warning: This comment starts with '/**', but isn't a kernel-doc comment. Refer Documentation/doc-guide/kernel-doc.rst All such occurrences were found with the following one line git grep -A 1 "\/\*\*" drivers/infiniband/ Link: https://lore.kernel.org/r/e57d5f4ddd08b7a19934635b44d6d632841b9ba7.1623823612.git.leonro@nvidia.com Reviewed-by: Jack Wang <jinpu.wang@ionos.com> #rtrs Reviewed-by: Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com> Signed-off-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-18RDMA/rtrs: Check device max_qp_wr limit when create QPJack Wang
Currently we only check device max_qp_wr limit for IO connection, but not for service connection. We should check for both. So save the max_qp_wr device limit in wr_limit, and use it for both IO connections and service connections. While at it, also remove an outdated comments. Link: https://lore.kernel.org/r/20210614090337.29557-6-jinpu.wang@ionos.com Suggested-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Reviewed-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-18RDMA/rtrs: Rename cq_size/queue_size to cq_num/queue_numGuoqing Jiang
Those variables are passed to create_cq, create_qp, rtrs_iu_alloc and rtrs_iu_free, so these *_size means the num of unit. And cq_size also means number of cq element. Also move the setting of cq_num to common path. Link: https://lore.kernel.org/r/20210614090337.29557-5-jinpu.wang@ionos.com Signed-off-by: Guoqing Jiang <guoqing.jiang@cloud.ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@cloud.ionos.com> Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com> Reviewed-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-18RDMA/rtrs: RDMA_RXE requires more number of WRMd Haris Iqbal
When using rdma_rxe, post_one_recv() returns ENOMEM error due to the full recv queue. This patch increase the number of WR for receive queue to support all devices. Link: https://lore.kernel.org/r/20210614090337.29557-4-jinpu.wang@ionos.com Signed-off-by: Md Haris Iqbal <haris.iqbal@cloud.ionos.com> Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Reviewed-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-18RDMA/rtrs-clt: Use minimal max_send_sge when create qpJack Wang
We use device limit max_send_sge, which is suboptimal for memory usage. We don't need that much for User Con, 1 is enough. And for IO con, sess->max_segments + 1 is enough Link: https://lore.kernel.org/r/20210614090337.29557-3-jinpu.wang@ionos.com Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com> Signed-off-by: Guoqing Jiang <guoqing.jiang@cloud.ionos.com> Reviewed-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-06-18RDMA/rtrs-srv: Set minimal max_send_wr and max_recv_wrJack Wang
Currently rtrs when create_qp use a coarse numbers (bigger in general), which leads to hardware create more resources which only waste memory with no benefits. For max_send_wr, we don't really need alway max_qp_wr size when creating qp, reduce it to cq_size. For max_recv_wr, cq_size is enough. With the patch when sess_queue_depth=128, per session (2 paths) memory consumption reduced from 188 MB to 65MB When always_invalidate is enabled, we need send more wr, so treat it special. Fixes: 9cb837480424e ("RDMA/rtrs: server: main functionality") Link: https://lore.kernel.org/r/20210614090337.29557-2-jinpu.wang@ionos.com Signed-off-by: Jack Wang <jinpu.wang@cloud.ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@cloud.ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Reviewed-by: Leon Romanovsky <leonro@nvidia.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-31RDMA/rtrs: Avoid Wtautological-constant-out-of-range-compareJack Wang
drivers/infiniband/ulp/rtrs/rtrs-clt.c:1786:19: warning: result of comparison of constant 'MAX_SESS_QUEUE_DEPTH' (65536) with expression of type 'u16' (aka 'unsigned short') is always false [-Wtautological-constant-out-of-range-compare] To fix it, limit MAX_SESS_QUEUE_DEPTH to u16 max, which is 65535, and drop the check in rtrs-clt, as it's the type u16 max. Link: https://lore.kernel.org/r/20210531122835.58329-1-jinpu.wang@ionos.com Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-clt: Fix memory leak of not-freed sess->stats and stats->pcpu_statsGioh Kim
sess->stats and sess->stats->pcpu_stats objects are freed when sysfs entry is removed. If something wrong happens and session is closed before sysfs entry is created, sess->stats and sess->stats->pcpu_stats objects are not freed. This patch adds freeing of them at three places: 1. When client uses wrong address and session creation fails. 2. When client fails to create a sysfs entry. 3. When client adds wrong address via sysfs add_path. Fixes: 215378b838df0 ("RDMA/rtrs: client: sysfs interface functions") Link: https://lore.kernel.org/r/20210528113018.52290-21-jinpu.wang@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-clt: Check if the queue_depth has changed during a reconnectionMd Haris Iqbal
The queue_depth is a module parameter for rtrs_server. It is used on the client side to determing the queue_depth of the request queue for the RNBD virtual block device. During a reconnection event for an already mapped device, in case the rtrs_server module queue_depth has changed, fail the reconnect attempt. Also stop further auto reconnection attempts. A manual reconnect via sysfs has to be triggerred. Fixes: 6a98d71daea18 ("RDMA/rtrs: client: main functionality") Link: https://lore.kernel.org/r/20210528113018.52290-20-jinpu.wang@ionos.com Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-srv: Fix memory leak when having multiple sessionsJack Wang
Gioh notice memory leak below unreferenced object 0xffff8880acda2000 (size 2048): comm "kworker/4:1", pid 77, jiffies 4295062871 (age 1270.730s) hex dump (first 32 bytes): 00 20 da ac 80 88 ff ff 00 20 da ac 80 88 ff ff . ....... ...... 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<00000000e85d85b5>] rtrs_srv_rdma_cm_handler+0x8e5/0xa90 [rtrs_server] [<00000000e31a988a>] cma_ib_req_handler+0xdc5/0x2b50 [rdma_cm] [<000000000eb02c5b>] cm_process_work+0x2d/0x100 [ib_cm] [<00000000e1650ca9>] cm_req_handler+0x11bc/0x1c40 [ib_cm] [<000000009c28818b>] cm_work_handler+0xe65/0x3cf2 [ib_cm] [<000000002b53eaa1>] process_one_work+0x4bc/0x980 [<00000000da3499fb>] worker_thread+0x78/0x5c0 [<00000000167127a4>] kthread+0x191/0x1e0 [<0000000060802104>] ret_from_fork+0x3a/0x50 unreferenced object 0xffff88806d595d90 (size 8): comm "kworker/4:1H", pid 131, jiffies 4295062972 (age 1269.720s) hex dump (first 8 bytes): 62 6c 61 00 6b 6b 6b a5 bla.kkk. backtrace: [<000000004447d253>] kstrdup+0x2e/0x60 [<0000000047259793>] kobject_set_name_vargs+0x2f/0xb0 [<00000000c2ee3bc8>] dev_set_name+0xab/0xe0 [<000000002b6bdfb1>] rtrs_srv_create_sess_files+0x260/0x290 [rtrs_server] [<0000000075d87bd7>] rtrs_srv_info_req_done+0x71b/0x960 [rtrs_server] [<00000000ccdf1bb5>] __ib_process_cq+0x94/0x100 [ib_core] [<00000000cbcb60cb>] ib_cq_poll_work+0x32/0xc0 [ib_core] [<000000002b53eaa1>] process_one_work+0x4bc/0x980 [<00000000da3499fb>] worker_thread+0x78/0x5c0 [<00000000167127a4>] kthread+0x191/0x1e0 [<0000000060802104>] ret_from_fork+0x3a/0x50 unreferenced object 0xffff88806d6bb100 (size 256): comm "kworker/4:1H", pid 131, jiffies 4295062972 (age 1269.720s) hex dump (first 32 bytes): 00 00 00 00 ad 4e ad de ff ff ff ff 00 00 00 00 .....N.......... ff ff ff ff ff ff ff ff 00 59 4d 86 ff ff ff ff .........YM..... backtrace: [<00000000a18a11e4>] device_add+0x74d/0xa00 [<00000000a915b95f>] rtrs_srv_create_sess_files.cold+0x49/0x1fe [rtrs_server] [<0000000075d87bd7>] rtrs_srv_info_req_done+0x71b/0x960 [rtrs_server] [<00000000ccdf1bb5>] __ib_process_cq+0x94/0x100 [ib_core] [<00000000cbcb60cb>] ib_cq_poll_work+0x32/0xc0 [ib_core] [<000000002b53eaa1>] process_one_work+0x4bc/0x980 [<00000000da3499fb>] worker_thread+0x78/0x5c0 [<00000000167127a4>] kthread+0x191/0x1e0 [<0000000060802104>] ret_from_fork+0x3a/0x50 The problem is we increase device refcount by get_device in process_info_req for each path, but only does put_deice for last path, which lead to memory leak. To fix it, it also calls put_device when dev_ref is not 0. Fixes: e2853c49477d1 ("RDMA/rtrs-srv-sysfs: fix missing put_device") Link: https://lore.kernel.org/r/20210528113018.52290-19-jinpu.wang@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-srv: Fix memory leak of unfreed rtrs_srv_stats objectGioh Kim
When closing a session, currently the rtrs_srv_stats object in the closing session is freed by kobject release. But if it failed to create a session by various reasons, it must free the rtrs_srv_stats object directly because kobject is not created yet. This problem is found by kmemleak as below: 1. One client machine maps /dev/nullb0 with session name 'bla': root@test1:~# echo "sessname=bla path=ip:192.168.122.190 \ device_path=/dev/nullb0" > /sys/devices/virtual/rnbd-client/ctl/map_device 2. Another machine failed to create a session with the same name 'bla': root@test2:~# echo "sessname=bla path=ip:192.168.122.190 \ device_path=/dev/nullb1" > /sys/devices/virtual/rnbd-client/ctl/map_device -bash: echo: write error: Connection reset by peer 3. The kmemleak on server machine reported an error: unreferenced object 0xffff888033cdc800 (size 128): comm "kworker/2:1", pid 83, jiffies 4295086585 (age 2508.680s) hex dump (first 32 bytes): 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ backtrace: [<00000000a72903b2>] __alloc_sess+0x1d4/0x1250 [rtrs_server] [<00000000d1e5321e>] rtrs_srv_rdma_cm_handler+0xc31/0xde0 [rtrs_server] [<00000000bb2f6e7e>] cma_ib_req_handler+0xdc5/0x2b50 [rdma_cm] [<00000000e896235d>] cm_process_work+0x2d/0x100 [ib_cm] [<00000000b6866c5f>] cm_req_handler+0x11bc/0x1c40 [ib_cm] [<000000005f5dd9aa>] cm_work_handler+0xe65/0x3cf2 [ib_cm] [<00000000610151e7>] process_one_work+0x4bc/0x980 [<00000000541e0f77>] worker_thread+0x78/0x5c0 [<00000000423898ca>] kthread+0x191/0x1e0 [<000000005a24b239>] ret_from_fork+0x3a/0x50 Fixes: 39c2d639ca183 ("RDMA/rtrs-srv: Set .release function for rtrs srv device during device init") Link: https://lore.kernel.org/r/20210528113018.52290-18-jinpu.wang@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-srv: Duplicated session name is not allowedGioh Kim
If two clients try to use the same session name, rtrs-server generates a kernel error that it failed to create the sysfs because the filename is duplicated. This patch adds code to check if there already exists the same session name with the different UUID. If a client tries to add more session, it sends the UUID and the session name. Therefore it is ok if there is already same session name with the same UUID. The rtrs-server must fail only-if there is the same session name with the different UUID. Link: https://lore.kernel.org/r/20210528113018.52290-17-jinpu.wang@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Aleksei Marov <aleksei.marov@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs: Do not reset hb_missed_max after re-connectionGioh Kim
When re-connecting, it resets hb_missed_max to 0. Before the first re-connecting, client will trigger re-connection when it gets hb-ack more than 5 times. But after the first re-connecting, clients will do re-connection whenever it does not get hb-ack because hb_missed_max is 0. There is no need to reset hb_missed_max when re-connecting. hb_missed_max should be kept until closing the session. Fixes: c0894b3ea69d3 ("RDMA/rtrs: core: lib functions shared between client and server modules") Link: https://lore.kernel.org/r/20210528113018.52290-16-jinpu.wang@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-srv: convert scnprintf to sysfs_emitJack Wang
Link: https://lore.kernel.org/r/20210528113018.52290-15-jinpu.wang@ionos.com Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-srv: Replace atomic_t with percpu_ref for ids_inflightMd Haris Iqbal
ids_inflight is used to track the inflight IOs. But the use of atomic_t variable can cause performance drops and can also become a performance bottleneck. This commit replaces the use of atomic_t with a percpu_ref structure. The advantage it offers is, it doesn't check if the reference has fallen to 0, until the user explicitly signals it to; and that is done by the percpu_ref_kill() function call. After that, the percpu_ref structure behaves like an atomic_t and for every put call, checks whether the reference has fallen to 0 or not. rtrs_srv_stats_rdma_to_str shows the count of ids_inflight as 0 for user-mode tools not to be confused. Fixes: 9cb837480424e ("RDMA/rtrs: server: main functionality") Link: https://lore.kernel.org/r/20210528113018.52290-14-jinpu.wang@ionos.com Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-clt: Check state of the rtrs_clt_sess before reading its statsMd Haris Iqbal
When get_next_path_min_inflight is called to select the next path, it iterates over the list of available rtrs_clt_sess (paths). It then reads the number of inflight IOs for that path to select one which has the least inflight IO. But it may so happen that rtrs_clt_sess (path) is no longer in the connected state because closing or error recovery paths can change the status of the rtrs_clt_Sess. For example, the client sent the heart-beat and did not get the response, it would change the session status and stop IO processing. The added checking of this patch can prevent accessing the broken path and generating duplicated error messages. It is ok if the status is changed after checking the status because the error recovery path does not free memory and only tries to reconnection. And also it is ok if the session is closed after checking the status because closing the session changes the session status and flush all IO beforing free memory. If the session is being accessed for IO processing, the closing session will wait. Fixes: 6a98d71daea18 ("RDMA/rtrs: client: main functionality") Link: https://lore.kernel.org/r/20210528113018.52290-13-jinpu.wang@ionos.com Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com> Reviewed-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-clt: Remove redundant 'break'Guoqing Jiang
It is duplicated with the very next line Link: https://lore.kernel.org/r/20210528113018.52290-12-jinpu.wang@ionos.com Signed-off-by: Guoqing Jiang <guoqing.jiang@ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-srv: Kill __rtrs_srv_change_stateGuoqing Jiang
No need since the only user is rtrs_srv_change_state. Link: https://lore.kernel.org/r/20210528113018.52290-11-jinpu.wang@ionos.com Signed-off-by: Guoqing Jiang <guoqing.jiang@ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-clt: Kill rtrs_clt_disconnect_from_sysfsGuoqing Jiang
The function is just a wrapper of rtrs_clt_close_conns, let's call rtrs_clt_close_conns directly. Link: https://lore.kernel.org/r/20210528113018.52290-10-jinpu.wang@ionos.com Signed-off-by: Guoqing Jiang <guoqing.jiang@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-clt: Kill rtrs_clt_{start,stop}_hbGuoqing Jiang
The two wrappers are not needed since we can call rtrs_{start,stop}_hb directly. Link: https://lore.kernel.org/r/20210528113018.52290-9-jinpu.wang@ionos.com Signed-off-by: Guoqing Jiang <guoqing.jiang@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs: Use strscpy instead of strlcpyDima Stepanov
During checkpatch analyzing the following warning message was found: WARNING:STRLCPY: Prefer strscpy over strlcpy - see: https://lore.kernel.org/r/CAHk-=wgfRnXz0W3D37d01q3JFkr_i_uTL=V6A6G1oUZcprmknw@mail.gmail.com/ Fix it by using strscpy calls instead of strlcpy. Link: https://lore.kernel.org/r/20210528113018.52290-8-jinpu.wang@ionos.com Signed-off-by: Dima Stepanov <dmitrii.stepanov@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs: Define MIN_CHUNK_SIZEGioh Kim
Define MIN_CHUNK_SIZE to replace the hard-coding number. We need 4k for metadata, so MIN_CHUNK_SIZE should be at least 8k. Link: https://lore.kernel.org/r/20210528113018.52290-7-jinpu.wang@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs: Change MAX_SESS_QUEUE_DEPTHGioh Kim
Max IB immediate data size is 2^28 (MAX_IMM_PAYL_BITS) and the minimum chunk size is 4096 (2^12). Therefore the maximum sess_queue_depth is 65536 (2^16). Link: https://lore.kernel.org/r/20210528113018.52290-6-jinpu.wang@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Reported-by: kernel test robot <lkp@intel.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-srv: Clean up the code in __rtrs_srv_change_stateGuoqing Jiang
No need to use double switch to check the change of state everywhere, let's change them to "if" to reduce size. Link: https://lore.kernel.org/r/20210528113018.52290-5-jinpu.wang@ionos.com Signed-off-by: Guoqing Jiang <guoqing.jiang@ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-srv: Add error messages for cases when failing RDMA connectionMd Haris Iqbal
It was difficult to find out why it failed to establish RDMA connection. This patch adds some messages to show which function has failed why. Link: https://lore.kernel.org/r/20210528113018.52290-4-jinpu.wang@ionos.com Signed-off-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-clt: Remove MAX_SESS_QUEUE_DEPTH from rtrs_send_sess_infoGioh Kim
Client receives queue_depth value from server. There is no need to use MAX_SESS_QUEUE_DEPTH value. Link: https://lore.kernel.org/r/20210528113018.52290-3-jinpu.wang@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-28RDMA/rtrs-srv: Kill reject_w_econnreset labelGuoqing Jiang
We can goto reject_w_err label after initialize err with -ECONNRESET. Link: https://lore.kernel.org/r/20210528113018.52290-2-jinpu.wang@ionos.com Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Guoqing Jiang <guoqing.jiang@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-05-07Merge tag 'block-5.13-2021-05-07' of git://git.kernel.dk/linux-blockLinus Torvalds
Pull block fixes from Jens Axboe: - dasd spelling fixes (Bhaskar) - Limit bio max size on multi-page bvecs to the hardware limit, to avoid overly large bio's (and hence latencies). Originally queued for the merge window, but needed a fix and was dropped from the initial pull (Changheun) - NVMe pull request (Christoph): - reset the bdev to ns head when failover (Daniel Wagner) - remove unsupported command noise (Keith Busch) - misc passthrough improvements (Kanchan Joshi) - fix controller ioctl through ns_head (Minwoo Im) - fix controller timeouts during reset (Tao Chiu) - rnbd fixes/cleanups (Gioh, Md, Dima) - Fix iov_iter re-expansion (yangerkun) * tag 'block-5.13-2021-05-07' of git://git.kernel.dk/linux-block: block: reexpand iov_iter after read/write nvmet: remove unsupported command noise nvme-multipath: reset bdev to ns head when failover nvme-pci: fix controller reset hang when racing with nvme_timeout nvme: move the fabrics queue ready check routines to core nvme: avoid memset for passthrough requests nvme: add nvme_get_ns helper nvme: fix controller ioctl through ns_head bio: limit bio max size RDMA/rtrs: fix uninitialized symbol 'cnt' s390: dasd: Mundane spelling fixes block/rnbd: Remove all likely and unlikely block/rnbd-clt: Check the return value of the function rtrs_clt_query block/rnbd: Fix style issues block/rnbd-clt: Change queue_depth type in rnbd_clt_session to size_t
2021-05-03RDMA/rtrs: fix uninitialized symbol 'cnt'Gioh Kim
rtrs_clt_rdma_cq_direct returns an ninitialized value in cnt if there is no session. This patch makes rtrs_clt_rdma_cq_direct returns a negative value for block layer not to try again. Fixes: 2958a995edc94 ("block/rnbd-clt: Support polling mode for IO latency optimization") Reported-by: kernel test robot <lkp@intel.com> Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Link: https://lore.kernel.org/r/20210429092741.266533-1-gi-oh.kim@ionos.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-05-01Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdmaLinus Torvalds
Pull rdma updates from Jason Gunthorpe: "This is significantly bug fixes and general cleanups. The noteworthy new features are fairly small: - XRC support for HNS and improves RQ operations - Bug fixes and updates for hns, mlx5, bnxt_re, hfi1, i40iw, rxe, siw and qib - Quite a few general cleanups on spelling, error handling, static checker detections, etc - Increase the number of device ports supported beyond 255. High port count software switches now exist - Several bug fixes for rtrs - mlx5 Device Memory support for host controlled atomics - Report SRQ tables through to rdma-tool" * tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma: (145 commits) IB/qib: Remove redundant assignment to ret RDMA/nldev: Add copy-on-fork attribute to get sys command RDMA/bnxt_re: Fix a double free in bnxt_qplib_alloc_res RDMA/siw: Fix a use after free in siw_alloc_mr IB/hfi1: Remove redundant variable rcd RDMA/nldev: Add QP numbers to SRQ information RDMA/nldev: Return SRQ information RDMA/restrack: Add support to get resource tracking for SRQ RDMA/nldev: Return context information RDMA/core: Add CM to restrack after successful attachment to a device RDMA/cma: Skip device which doesn't support CM RDMA/rxe: Fix a bug in rxe_fill_ip_info() RDMA/mlx5: Expose private query port RDMA/mlx4: Remove an unused variable RDMA/mlx5: Fix type assignment for ICM DM IB/mlx5: Set right RoCE l3 type and roce version while deleting GID RDMA/i40iw: Fix error unwinding when i40iw_hmc_sd_one fails RDMA/cxgb4: add missing qpid increment IB/ipoib: Remove unnecessary struct declaration RDMA/bnxt_re: Get rid of custom module reference counting ...
2021-04-28Merge tag 'for-5.13/drivers-2021-04-27' of git://git.kernel.dk/linux-blockLinus Torvalds
Pull block driver updates from Jens Axboe: - MD changes via Song: - raid5 POWER fix - raid1 failure fix - UAF fix for md cluster - mddev_find_or_alloc() clean up - Fix NULL pointer deref with external bitmap - Performance improvement for raid10 discard requests - Fix missing information of /proc/mdstat - rsxx const qualifier removal (Arnd) - Expose allocated brd pages (Calvin) - rnbd via Gioh Kim: - Change maintainer - Change domain address of maintainers' email - Add polling IO mode and document update - Fix memory leak and some bug detected by static code analysis tools - Code refactoring - Series of floppy cleanups/fixes (Denis) - s390 dasd fixes (Julian) - kerneldoc fixes (Lee) - null_blk double free (Lv) - null_blk virtual boundary addition (Max) - Remove xsysace driver (Michal) - umem driver removal (Davidlohr) - ataflop fixes (Dan) - Revalidate disk removal (Christoph) - Bounce buffer cleanups (Christoph) - Mark lightnvm as deprecated (Christoph) - mtip32xx init cleanups (Shixin) - Various fixes (Tian, Gustavo, Coly, Yang, Zhang, Zhiqiang) * tag 'for-5.13/drivers-2021-04-27' of git://git.kernel.dk/linux-block: (143 commits) async_xor: increase src_offs when dropping destination page drivers/block/null_blk/main: Fix a double free in null_init. md/raid1: properly indicate failure when ending a failed write request md-cluster: fix use-after-free issue when removing rdev nvme: introduce generic per-namespace chardev nvme: cleanup nvme_configure_apst nvme: do not try to reconfigure APST when the controller is not live nvme: add 'kato' sysfs attribute nvme: sanitize KATO setting nvmet: avoid queuing keep-alive timer if it is disabled brd: expose number of allocated pages in debugfs ataflop: fix off by one in ataflop_probe() ataflop: potential out of bounds in do_format() drbd: Fix fall-through warnings for Clang block/rnbd: Use strscpy instead of strlcpy block/rnbd-clt-sysfs: Remove copy buffer overlap in rnbd_clt_get_path_name block/rnbd-clt: Remove max_segment_size block/rnbd-clt: Generate kobject_uevent when the rnbd device state changes block/rnbd-srv: Remove unused arguments of rnbd_srv_rdma_ev Documentation/ABI/rnbd-clt: Add description for nr_poll_queues ...
2021-04-20block/rnbd-clt: Remove max_segment_sizeJack Wang
We always map with SZ_4K, so do not need max_segment_size. Cc: Leon Romanovsky <leonro@nvidia.com> Cc: linux-rdma@vger.kernel.org Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Acked-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Leon Romanovsky <leonro@nvidia.com> Link: https://lore.kernel.org/r/20210419073722.15351-18-gi-oh.kim@ionos.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-20block/rnbd-srv: Remove unused arguments of rnbd_srv_rdma_evGioh Kim
struct rtrs_srv is not used when handling rnbd_srv_rdma_ev messages, so cleaned up rdma_ev function pointer in rtrs_srv_ops also is changed. Cc: Leon Romanovsky <leonro@nvidia.com> Cc: linux-rdma@vger.kernel.org Signed-off-by: Aleksei Marov <aleksei.marov@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com> Acked-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Leon Romanovsky <leonro@nvidia.com> Link: https://lore.kernel.org/r/20210419073722.15351-16-gi-oh.kim@ionos.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-20block/rnbd-clt: Support polling mode for IO latency optimizationGioh Kim
RNBD can make double-queues for irq-mode and poll-mode. For example, on 4-CPU system 8 request-queues are created, 4 for irq-mode and 4 for poll-mode. If the IO has HIPRI flag, the block-layer will call .poll function of RNBD. Then IO is sent to the poll-mode queue. Add optional nr_poll_queues argument for map_devices interface. To support polling of RNBD, RTRS client creates connections for both of irq-mode and direct-poll-mode. For example, on 4-CPU system it could've create 5 connections: con[0] => user message (softirq cq) con[1:4] => softirq cq After this patch, it can create 9 connections: con[0] => user message (softirq cq) con[1:4] => softirq cq con[5:8] => DIRECT-POLL cq Cc: Leon Romanovsky <leonro@nvidia.com> Cc: linux-rdma@vger.kernel.org Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Acked-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Leon Romanovsky <leonro@nvidia.com> Link: https://lore.kernel.org/r/20210419073722.15351-14-gi-oh.kim@ionos.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-20block/rnbd-clt: Replace {NO_WAIT,WAIT} with RTRS_PERMIT_{WAIT,NOWAIT}Gioh Kim
They are defined with the same value and similar meaning, let's remove one of them, then we can remove {WAIT,NOWAIT}. Also change the type of 'wait' from 'int' to 'enum wait_type' to make it clear. Cc: Leon Romanovsky <leonro@nvidia.com> Cc: linux-rdma@vger.kernel.org Signed-off-by: Guoqing Jiang <guoqing.jiang@ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Reviewed-by: Chaitanya Kulkarni <chaitanya.kulkarni@wdc.com> Acked-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Leon Romanovsky <leonro@nvidia.com> Link: https://lore.kernel.org/r/20210419073722.15351-9-gi-oh.kim@ionos.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
2021-04-13RDMA/rtrs-clt: Simplify error messageGioh Kim
Two error messages are only different message but have common code to generate the path string. Link: https://lore.kernel.org/r/20210406123639.202899-4-gi-oh.kim@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-04-13RDMA/rtrs-srv: More debugging info when fail to send replyGioh Kim
It does not help to debug if it only print error message without any debugging information which session and connection the error happened. Link: https://lore.kernel.org/r/20210406123639.202899-3-gi-oh.kim@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-04-13RDMA/rtrs-clt: Print more info when an error happensGioh Kim
Client prints only error value and it is not enough for debugging. 1. When client receives an error from server: the client does not only print the error value but also more information of server connection. 2. When client failes to send IO: the client gets an error from RDMA layer. It also print more information of server connection. Link: https://lore.kernel.org/r/20210406123639.202899-2-gi-oh.kim@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-04-13RDMA/rtrs-clt: New sysfs attribute to print the latency of each pathGioh Kim
It shows the latest latency that the client checked when sending the heart-beat. Link: https://lore.kernel.org/r/20210407113444.150961-3-gi-oh.kim@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-04-13RDMA/rtrs-clt: Add a minimum latency multipath policyGioh Kim
This patch adds new multipath policy: min-latency. Client checks the latency of each path when it sends the heart-beat. And it sends IO to the path with the minimum latency. Link: https://lore.kernel.org/r/20210407113444.150961-2-gi-oh.kim@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-04-12RDMA/rtrs-clt: destroy sysfs after removing session from active listGioh Kim
A session can be removed dynamically by sysfs interface "remove_path" that eventually calls rtrs_clt_remove_path_from_sysfs function. The current rtrs_clt_remove_path_from_sysfs first removes the sysfs interfaces and frees sess->stats object. Second it removes the session from the active list. Therefore some functions could access non-connected session and access the freed sess->stats object even-if they check the session status before accessing the session. For instance rtrs_clt_request and get_next_path_min_inflight check the session status and try to send IO to the session. The session status could be changed when they are trying to send IO but they could not catch the change and update the statistics information in sess->stats object, and generate use-after-free problem. (see: "RDMA/rtrs-clt: Check state of the rtrs_clt_sess before reading its stats") This patch changes the rtrs_clt_remove_path_from_sysfs to remove the session from the active session list and then destroy the sysfs interfaces. Each function still should check the session status because closing or error recovery paths can change the status. Fixes: 6a98d71daea1 ("RDMA/rtrs: client: main functionality") Link: https://lore.kernel.org/r/20210412084002.33582-1-gi-oh.kim@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Reviewed-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-04-01RDMA/rtrs-clt: Cap max_io_sizeJack Wang
Max io size is limited by both remote buffer size and the max fr pages per mr. Link: https://lore.kernel.org/r/20210325153308.1214057-20-gi-oh.kim@ionos.com Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Reviewed-by: Md Haris Iqbal <haris.iqbal@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-04-01RDMA/rtrs: Cleanup unused 's' variable in __alloc_sessJack Wang
Link: https://lore.kernel.org/r/20210325153308.1214057-18-gi-oh.kim@ionos.com Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
2021-04-01RDMA/rtrs-srv: Report temporary sessname for error messageGioh Kim
Before receiving the session name, the error message cannot include any information about which connection generates the error. This patch stores the addresses of source and target in the sessname field to show which generates the error. That field will be over-written when receiving the session name from client. Link: https://lore.kernel.org/r/20210325153308.1214057-17-gi-oh.kim@ionos.com Signed-off-by: Gioh Kim <gi-oh.kim@ionos.com> Signed-off-by: Jack Wang <jinpu.wang@ionos.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
generated by cgit (git 2.34.1) at 2025-07-18 11:55:35 +0000