1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
|
.\" -*- nroff -*-
.\"
.TH IBV_CREATE_CQ_EX 3 2016-05-08 libibverbs "Libibverbs Programmer's Manual"
.SH "NAME"
ibv_create_cq_ex \- create a completion queue (CQ)
.SH "SYNOPSIS"
.nf
.B #include <infiniband/verbs.h>
.sp
.BI "struct ibv_cq_ex *ibv_create_cq_ex(struct ibv_context " "*context" ",
.BI " struct ibv_create_cq_attr_ex " "*cq_attr" );
.fi
.SH "DESCRIPTION"
.B ibv_create_cq_ex()
creates a completion queue (CQ) for RDMA device context
.I context\fR.
The argument
.I cq_attr
is a pointer to struct ibv_create_cq_attr_ex as defined in <infiniband/verbs.h>.
.PP
.nf
struct ibv_create_cq_attr_ex {
.in +8
int cqe; /* Minimum number of entries required for CQ */
void *cq_context; /* Consumer-supplied context returned for completion events */
struct ibv_comp_channel *channel; /* Completion channel where completion events will be queued. May be NULL if completion events will not be used. */
int comp_vector; /* Completion vector used to signal completion events. Must be >= 0 and < context->num_comp_vectors. */
uint64_t wc_flags; /* The wc_flags that should be returned in ibv_poll_cq_ex. Or'ed bit of enum ibv_wc_flags_ex. */
uint32_t comp_mask; /* compatibility mask (extended verb). */
.in -8
};
enum ibv_wc_flags_ex {
IBV_WC_EX_WITH_BYTE_LEN = 1 << 0, /* Require byte len in WC */
IBV_WC_EX_WITH_IMM = 1 << 1, /* Require immediate in WC */
IBV_WC_EX_WITH_QP_NUM = 1 << 2, /* Require QP number in WC */
IBV_WC_EX_WITH_SRC_QP = 1 << 3, /* Require source QP in WC */
IBV_WC_EX_WITH_SLID = 1 << 4, /* Require slid in WC */
IBV_WC_EX_WITH_SL = 1 << 5, /* Require sl in WC */
IBV_WC_EX_WITH_DLID_PATH_BITS = 1 << 6, /* Require dlid path bits in WC */
IBV_WC_EX_WITH_COMPLETION_TIMESTAMP = 1 << 7, /* Require completion timestamp in WC /*
};
enum ibv_cq_init_attr_mask {
IBV_CQ_INIT_ATTR_MASK_FLAGS = 1 << 0,
};
enum ibv_create_cq_attr_flags {
IBV_CREATE_CQ_ATTR_SINGLE_THREADED = 1 << 0, /* This CQ is used from a single threaded, thus no locking is required */
};
.SH "Polling an extended CQ"
In order to poll an extended CQ efficiently, a user could use the following functions.
.TP
.B Completion iterator functions
.BI "int ibv_start_poll(struct ibv_cq_ex " "*cq" ", struct ibv_poll_cq_attr " "*attr")
.br
Start polling a batch of work completions.
.I attr
is given in order to make this function
easily extensible in the future. This function either returns 0 on success or an error code
otherwise. When no completions are available on the CQ, ENOENT is returned, but the CQ remains
in a valid state. On success, querying the completion's attribute could be done using the query
functions described below. If an error code is given, end_poll shouldn't be called.
.BI "int ibv_next_poll(struct ibv_cq_ex " "*cq")
.br
This function is called in order to get the next work completion. It has to be called after
.I start_poll
and before
.I end_poll
are called. This function either returns 0 on success or an error code
otherwise. When no completions are available on the CQ, ENOENT is returned, but the CQ remains
in a valid state. On success, querying the completion's attribute could be done using the query
functions described below. If an error code is given, end_poll should still be called,
indicating this is the end of the polled batch.
.BI "void ibv_end_poll(struct ibv_cq_ex " "*cq")
.br
This function indicates the end of polling batch of work completions. After calling this function, the user should start a new batch
by calling
.I start_poll.
.TP
.B Polling fields in the completion
Below members and functions are used in order to poll the current completion. The current completion is the completion which the iterator points to (start_poll and next_poll advances this iterator). Only fields that the user requested via wc_flags in ibv_create_cq_ex could be queried. In addition, some fields are only valid in certain opcodes and status codes.
.BI "uint64_t wr_id - Can be accessed directly from struct ibv_cq_ex".
.BI "enum ibv_wc_status - Can be accessed directly from struct ibv_cq_ex".
.BI "enum ibv_wc_opcode ibv_wc_read_opcode(struct ibv_cq_ex " "*cq"); \c
Get the opcode from the current completion.
.BI "uint32_t ibv_wc_read_vendor_err(struct ibv_cq_ex " "*cq"); \c
Get the vendor error from the current completion.
.BI "uint32_t ibv_wc_read_byte_len(struct ibv_cq_ex " "*cq"); \c
Get the vendor error from the current completion.
.BI "uint32_t ibv_wc_read_imm_data(struct ibv_cq_ex " "*cq"); \c
Get the immediate data field from the current completion.
.BI "uint32_t ibv_wc_read_qp_num(struct ibv_cq_ex " "*cq"); \c
Get the QP number field from the current completion.
.BI "uint32_t ibv_wc_read_src_qp(struct ibv_cq_ex " "*cq"); \c
Get the source QP number field from the current completion.
.BI "int ibv_wc_read_wc_flags(struct ibv_cq_ex " "*cq"); \c
Get the QP flags field from the current completion.
.BI "uint16_t ibv_wc_read_pkey_index(struct ibv_cq_ex " "*cq"); \c
Get the pkey index field from the current completion.
.BI "uint32_t ibv_wc_read_slid(struct ibv_cq_ex " "*cq"); \c
Get the slid field from the current completion.
.BI "uint8_t ibv_wc_read_sl(struct ibv_cq_ex " "*cq"); \c
Get the sl field from the current completion.
.BI "uint8_t ibv_wc_read_dlid_path_bits(struct ibv_cq_ex " "*cq"); \c
Get the dlid_path_bits field from the current completion.
.BI "uint64_t ibv_wc_read_completion_ts(struct ibv_cq_ex " "*cq"); \c
Get the completion timestamp from the current completion.
.SH "RETURN VALUE"
.B ibv_create_cq_ex()
returns a pointer to the CQ, or NULL if the request fails.
.SH "NOTES"
.B ibv_create_cq_ex()
may create a CQ with size greater than or equal to the requested
size. Check the cqe attribute in the returned CQ for the actual size.
.PP
CQ should be destroyed with ibv_destroy_cq.
.PP
.SH "SEE ALSO"
.BR ibv_create_cq (3),
.BR ibv_destroy_cq (3),
.BR ibv_resize_cq (3),
.BR ibv_req_notify_cq (3),
.BR ibv_ack_cq_events (3),
.BR ibv_create_qp (3)
.SH "AUTHORS"
.TP
Matan Barak <matanb@mellanox.com>
|
