[PATCH] doc: documentation for ioq

[PATCH] doc: documentation for ioq

[Eric Wong]

It's a queue that looks like a semaphore, so document it
in doc/queues.txt and provide pointers to perhaps-forgotten
documentation.
---
 doc/queues.txt | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/doc/queues.txt b/doc/queues.txt
index 8e3235e..d1de9da 100644
--- a/doc/queues.txt
+++ b/doc/queues.txt
@@ -91,3 +91,22 @@ must parallelize these syscalls through POSIX threads instead.
 
 Furthermore, no AIO implementation we've looked at is aware of nor
 designed to manage parallelism across multiple devices/filesystems.
+
+ioq (secondary queue)
+---------------------
+
+To manage parallelism across multiple devices/filesystems while
+being integrated into our existing thread pool architecture,
+'ioq' was implemented starting with v1.3.0.  Its goal is to
+prevent a single slow disk from tying up more than its fair
+share of disks.
+
+The API provided is semaphore-like; but compatible with our
+existing non-blocking architecture.  A lengthier description is
+in the 1.3.0 release notes:
+
+	https://bogomips.org/cmogstored.git/patch/?id=v1.3.0
+
+See also the ioq.c source for a diagram and implementation:
+
+	https://bogomips.org/cmogstored.git/plain/ioq.c
-- 
EW

[PATCH 2/1] doc: further comment updates around ioq

[Eric Wong]

Let's not forget about this queue, it is a useful design.
---
 cmogstored.h | 10 +++++-----
 ioq.c        |  4 ++--
 2 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/cmogstored.h b/cmogstored.h
index f5c682e..db87ac0 100644
--- a/cmogstored.h
+++ b/cmogstored.h
@@ -99,18 +99,18 @@ enum mog_next {
 struct mog_ioq {
 	unsigned cur;
 	unsigned max;
-	pthread_mutex_t mtx;
+	pthread_mutex_t mtx; /* protects cur, max, ioq_head */
 	SIMPLEQ_HEAD(ioq_head, mog_fd) ioq_head;
-	bool contended;
-	struct mog_svc *svc;
+	bool contended; /* hint, not protected */
+	struct mog_svc *svc; /* initialized once at creation */
 };
 
 struct mog_wbuf;
 struct mog_dev {
 	dev_t st_dev;
 	uint32_t devid;
-	struct mog_ioq ioq;
-	struct mog_ioq fsckq;
+	struct mog_ioq ioq; /* normal requests */
+	struct mog_ioq fsckq; /* low-priority for MogileFS fsck */
 };
 
 struct mog_rbuf {
diff --git a/ioq.c b/ioq.c
index c515a8b..9408786 100644
--- a/ioq.c
+++ b/ioq.c
@@ -37,7 +37,7 @@
  *                                 |                   /
  *                                 `---------<---------'
  *
- * mog_ioq_next is automatically called when releases a regular file.
+ * mog_ioq_next is automatically called when a thread releases a regular file.
  */
 __thread struct mog_ioq *mog_ioq_current;
 
@@ -140,7 +140,7 @@ void mog_ioq_next(struct mog_ioq *check_ioq)
 		mog_activeq_push(mog_ioq_current->svc->queue, mfd);
 	}
 	/*
-	 * We may not touch or use client_mfd here anymore.  Another
+	 * We may not touch mfd after mog_activeq_push.  Another
 	 * thread may already have it.  In the worst case, it's been
 	 * closed due to epoll/kqueue running out-of-space and another
 	 * system call (open/accept) may have already reused the FD
-- 
EW