diff options
| author | Tian Lan <tian.lan@twosigma.com> | 2023-05-22 17:05:55 -0400 |
|---|---|---|
| committer | Jens Axboe <axboe@kernel.dk> | 2023-05-23 11:10:16 -0600 |
| commit | 3e94d54e83cafd2b562bb6d15bb2f72d76200fb5 (patch) | |
| tree | 009ea8592672b715e904ad71645eaf25f2150f84 | |
| parent | 8a2b20a997a3779ae9fcae268f2959eb82ec05a1 (diff) | |
| download | linux-3e94d54e83cafd2b562bb6d15bb2f72d76200fb5.tar.gz | |
blk-mq: fix race condition in active queue accounting
If multiple CPUs are sharing the same hardware queue, it can
cause leak in the active queue counter tracking when __blk_mq_tag_busy()
is executed simultaneously.
Fixes: ee78ec1077d3 ("blk-mq: blk_mq_tag_busy is no need to return a value")
Signed-off-by: Tian Lan <tian.lan@twosigma.com>
Reviewed-by: Ming Lei <ming.lei@redhat.com>
Reviewed-by: Damien Le Moal <dlemoal@kernel.org>
Reviewed-by: John Garry <john.g.garry@oracle.com>
Link: https://lore.kernel.org/r/20230522210555.794134-1-tilan7663@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
| -rw-r--r-- | block/blk-mq-tag.c | 12 |
1 files changed, 8 insertions, 4 deletions
diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index d6af9d431dc63..dfd81cab57888 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -39,16 +39,20 @@ void __blk_mq_tag_busy(struct blk_mq_hw_ctx *hctx) { unsigned int users; + /* + * calling test_bit() prior to test_and_set_bit() is intentional, + * it avoids dirtying the cacheline if the queue is already active. + */ if (blk_mq_is_shared_tags(hctx->flags)) { struct request_queue *q = hctx->queue; - if (test_bit(QUEUE_FLAG_HCTX_ACTIVE, &q->queue_flags)) + if (test_bit(QUEUE_FLAG_HCTX_ACTIVE, &q->queue_flags) || + test_and_set_bit(QUEUE_FLAG_HCTX_ACTIVE, &q->queue_flags)) return; - set_bit(QUEUE_FLAG_HCTX_ACTIVE, &q->queue_flags); } else { - if (test_bit(BLK_MQ_S_TAG_ACTIVE, &hctx->state)) + if (test_bit(BLK_MQ_S_TAG_ACTIVE, &hctx->state) || + test_and_set_bit(BLK_MQ_S_TAG_ACTIVE, &hctx->state)) return; - set_bit(BLK_MQ_S_TAG_ACTIVE, &hctx->state); } users = atomic_inc_return(&hctx->tags->active_queues); |