[iwhd] This is the initial version of the Administration Guide. It documents basic terms and provides speci
by Pete Zaitcev
commit 24f06374f5d65608dcdc4a10f65512dd93a55d84
Author: Pete Zaitcev <zaitcev(a)yahoo.com>
Date: Wed Aug 31 13:33:44 2011 -0600
This is the initial version of the Administration Guide. It documents
basic terms and provides specific examples for conf.js. It is minimally
biased towards Fedora Linux, such as it lists the default locations for
configuration files.
doc/admin_guide.md | 120 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 120 insertions(+), 0 deletions(-)
---
diff --git a/doc/admin_guide.md b/doc/admin_guide.md
new file mode 100644
index 0000000..e66b8d8
--- /dev/null
+++ b/doc/admin_guide.md
@@ -0,0 +1,120 @@
+Image Warehouse Administration Guide
+====================================
+
+Introduction
+------------
+
+Image Warehouse manages so-called "images" by, firstly, storing and
+retrieving them under certain names, called "keys", and secondly, by
+associating images with "attributes". Keys are organized into "buckets",
+which function almost like filesystem directories, except they cannot
+be nested.
+
+Images themselves are stored either in the appropriate cloud, or in
+a local filesystem. Keys and attributes are stored in a MongoDB database.
+
+Clients access Image Warehouse through a RESTful api. This implies that
+any instance of Image Warehouse implements a webserver, access to which
+must be protected with a firewall.
+
+Almost all of Image Warehouse functions are implemented by a single
+daemon binary, *iwhd*. It only invokes additional programs when a
+"registration" function is requested.
+
+Step by step configuration
+--------------------------
+
+On Fedora Linux and Red Hat Enterprise Linux systems the Image Warehouse
+is installed from an "iwhd" RPM package. As of version 0.98, native
+systemd support is not yet available, but otherwise the iwhd is largely
+a conventional daemon.
+
+Before iwhd is configured, you have to bring up a dedicated MongoDB instance
+for the use by Image Warehouse. Sharing MongoDB with other applications is
+strongly discouraged. Set it up with no password and save its host and port
+if they are not the default (e.g. port 27017).
+
+Note that yum will permit you to install iwhd even if mongodb is not
+installed, with no warning. This is done in case the MongoDB instance
+is located on another node from iwhd.
+
+The second decision is to select where the images are to be kept. In some
+cases you may want iwhd to manage images that are stored in the cloud,
+such as Amazon S3 (or compatible), Rackspace CloudFiles (or OpenStack).
+Otherwise, let iwhd to keep images in a filesystem directory. Make sure
+the filesystem has enough space to store the images. Even small clouds
+quickly push into terabytes. The default location is /var/lib/iwhd,
+which is not likely to be anywhere large enough, unless your root is
+itself mounted from a cloud (using HekaFS, for example).
+
+Once the database is up and the image directory is created, fill out
+the /etc/sysconfig/iwhd and /etc/iwhd/conf.js. With iwhd 0.98, there are
+a couple of gotchas. First, the file in sysconfig is not installed at all,
+so you have to create it. Fortunately, the syntax is typical for such
+files: write IWHD_ARGS="-l /var/log/iwhd.log". Second, the start-up script
+force-adds the -c argument to IWHD_ARGS, so do not put it in there. However,
+it does not add any other arguments, such as -l.
+
+Consult iwhd(8) for the full list of command line options.
+
+The /etc/iwhd/conf.js uses JSON syntax, so you cannot add comments to it.
+As packaged, it describes one "provider" or "back-end" of the type "fs",
+for "filesystem". This type cannot be used for any registrations.
+See registration.md for examples of conf.js for various registration
+scenarios.
+
+If the installation did not come with a simple example, start with this:
+
+ [
+ {
+ "name": "primary",
+ "type": "fs",
+ "path": "/var/lib/iwhd"
+ }
+ ]
+
+Obviously, the "path" parameter has to point to the directory where
+images are to be stored. Other storage types require different paramenters.
+For "s3" type, start with the following:
+
+ [
+ {
+ "name": "mys3",
+ "type": "s3",
+ "host": "s3.amazon.com",
+ "port": 80,
+ "key": "0KA7CAPYP9Z5K3VWTKG2",
+ "secret": "BYLqimMYlUkqcq0xGPNNSJLAmPwsc9end7s3aCx+"
+ }
+ ]
+
+In S3 terms, "key" is actually a username and "secret" is a password,
+which you have to substitute. Host and port may change if a non-default
+Amazon region is used, or the images are stored in a private cloud
+with S3 interface (provided by Hail tabled, Eucalyptus Walrus, etc.).
+
+Once parameters are configured, on Fedora and similar, use the regular
+"service iwhd start" and "chkconfig iwhd on" to launch and enable iwhd.
+Use the hints in the chapter on troubleshooting to verify that iwhd is
+functioning properly.
+
+Troubleshooting
+---------------
+
+Troubleshooting of iwhd starts with accessing it through its RESTful API.
+The API is documented in warehouse_api_final.md, but for our purposes
+simply run a command like this:
+
+ curl http://localhost:9090/
+
+A properly functioning iwhd returns an XML-like document (albeit without
+a DTD). If that fails, you are going to need to read the error log.
+
+The iwhd writes a log by itself, without syslog. However, when installed
+from a package, the log is properly rotated using logrotate(8). The default
+location is /var/log/iwhd.log.
+
+The log format is somewhat arbitrary, but it is being improved.
+
+When troubleshooting, it is advised to add "-v" to IWHD_ARGS in
+/etc/sysconfig/iwhd and restart iwhd.
12 years, 9 months
[iwhd] A colleague of mine, Jason Ganovsky, found yet another RHEV-M, which seems generally compatible with
by Pete Zaitcev
commit edc051f651aef1b8c26d92154d4b0add27dbe3f0
Author: Pete Zaitcev <zaitcev(a)yahoo.com>
Date: Wed Aug 31 13:32:42 2011 -0600
A colleague of mine, Jason Ganovsky, found yet another RHEV-M, which
seems generally compatible with the ones we encountered before, but
not quite. This patch addresses the different way in which the import
status can be report in REST API.
NEWS | 3 +++
dc-rhev-image.c | 16 +++++++++++++++-
2 files changed, 18 insertions(+), 1 deletions(-)
---
diff --git a/NEWS b/NEWS
index b859a8b..c827c0e 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,9 @@ iwhd NEWS -*- outline -*-
** Bug fixes
+ Adapt to a slight variation in the status report of a template import
+ action in RHEV-M 2.x.
+
dc-rhev-image now allows a RHEV-M import to take 10 minutes.
Before, it would timeout after just 1 minute, which was often inadequate.
diff --git a/dc-rhev-image.c b/dc-rhev-image.c
index 6b74f35..1b86c43 100644
--- a/dc-rhev-image.c
+++ b/dc-rhev-image.c
@@ -1924,6 +1924,7 @@ static char *import_get_status(xmlNode *etparent)
char *status, *p;
const char *error;
xmlNode *et;
+ xmlNode *etstate;
xmlNode *ettext;
int rc;
@@ -1933,7 +1934,20 @@ static char *import_get_status(xmlNode *etparent)
exit(EXIT_FAILURE);
}
- ettext = et->children;
+ /*
+ * Some versions of RHEV-M 2.x return this:
+ * <status><state>complete</state></status>
+ * Most, including every RHEV-M 3 that we tested, return this:
+ * <status>COMPLETE</status>
+ * Adapt to either format.
+ */
+ etstate = xmlGetChild(et, "state");
+ if (etstate) {
+ ettext = etstate->children;
+ } else {
+ ettext = et->children;
+ }
+
if (!ettext || ettext->type != XML_TEXT_NODE || !ettext->content) {
fprintf(stderr, "ERROR Empty import status\n");
exit(EXIT_FAILURE);
12 years, 9 months
[iwhd] maint: give names to import-related sleep/timeout constants
by Jim Meyering
commit d1c95d665c76f4d83f51929a1181fd4b71a78e2c
Author: Jim Meyering <meyering(a)redhat.com>
Date: Mon Aug 22 11:06:39 2011 +0200
maint: give names to import-related sleep/timeout constants
* dc-rhev-image.c (IMPORT_SLEEP_SECONDS): Define.
(IMPORT_SLEEP_SECONDS): Define.
(import_tpl): Use those rather than hard-coded constants.
* NEWS (Bug fixes): Mention it.
NEWS | 3 +++
dc-rhev-image.c | 15 ++++++++++++---
2 files changed, 15 insertions(+), 3 deletions(-)
---
diff --git a/NEWS b/NEWS
index e122bcf..b859a8b 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,9 @@ iwhd NEWS -*- outline -*-
** Bug fixes
+ dc-rhev-image now allows a RHEV-M import to take 10 minutes.
+ Before, it would timeout after just 1 minute, which was often inadequate.
+
object deletion now succeeds for an object whose FS-backed file has been
removed behind iwhd's back.
diff --git a/dc-rhev-image.c b/dc-rhev-image.c
index 20a1d13..6b74f35 100644
--- a/dc-rhev-image.c
+++ b/dc-rhev-image.c
@@ -63,6 +63,15 @@
static const char NULID[] = "00000000-0000-0000-0000-000000000000";
+enum {
+ /* How long to sleep between import status queries. */
+ IMPORT_SLEEP_SECONDS = 3,
+
+ /* An import times out after this many seconds. 60 is too small.
+ 600 appears to be adequate in some real-life situations. */
+ IMPORT_MAXIMUM_DURATION_SECONDS = 600
+};
+
struct config {
char *image;
char *apiurl;
@@ -2238,8 +2247,8 @@ static void import_tpl(struct config *cfg, struct api_conn *connection,
}
if (c_strcasecmp(status, "PENDING") == 0 ||
c_strcasecmp(status, "IN_PROGRESS") == 0) {
- /* Yes, 10 minutes may be needed, 1 is way too small. */
- if (++i >= 200) {
+ if (i++ * IMPORT_SLEEP_SECONDS
+ >= IMPORT_MAXIMUM_DURATION_SECONDS) {
fprintf(stderr,
"ERROR import times out, status `%s'\n",
status);
@@ -2251,7 +2260,7 @@ static void import_tpl(struct config *cfg, struct api_conn *connection,
exit(EXIT_FAILURE);
}
- tm.tv_sec = 3;
+ tm.tv_sec = IMPORT_SLEEP_SECONDS;
tm.tv_nsec = 0;
nanosleep(&tm, NULL);
12 years, 10 months
[iwhd] Raise the RHEV-M import timeout
by Jim Meyering
commit 6f51c3b843b01ef3750fd073359b540c46c0b7c3
Author: Pete Zaitcev <zaitcev(a)redhat.com>
Date: Fri Aug 19 19:32:07 2011 -0600
Raise the RHEV-M import timeout
Early field experiments showed that I was too optimistic about the time
it takes to copy a typical image: 1 minute is definitely too short.
With this change in place both James Labocki and Richard Su report success.
dc-rhev-image.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
---
diff --git a/dc-rhev-image.c b/dc-rhev-image.c
index 00ee16c..20a1d13 100644
--- a/dc-rhev-image.c
+++ b/dc-rhev-image.c
@@ -2238,7 +2238,8 @@ static void import_tpl(struct config *cfg, struct api_conn *connection,
}
if (c_strcasecmp(status, "PENDING") == 0 ||
c_strcasecmp(status, "IN_PROGRESS") == 0) {
- if (++i >= 20) {
+ /* Yes, 10 minutes may be needed, 1 is way too small. */
+ if (++i >= 200) {
fprintf(stderr,
"ERROR import times out, status `%s'\n",
status);
12 years, 10 months
[iwhd] doc: mention the default log file name in generated man page
by Jim Meyering
commit 2140ad33195353863d7f1728a609eb2b12b02165
Author: Jim Meyering <meyering(a)redhat.com>
Date: Thu Aug 11 22:41:38 2011 +0200
doc: mention the default log file name in generated man page
* man/iwhd.x: New file, to create a "FILES" section.
* man/Makefile.am (EXTRA_DIST): Add it.
(iwhd.8): Use it.
man/Makefile.am | 6 ++++--
man/iwhd.x | 4 ++++
2 files changed, 8 insertions(+), 2 deletions(-)
---
diff --git a/man/Makefile.am b/man/Makefile.am
index 7cb842c..f7a5fa6 100644
--- a/man/Makefile.am
+++ b/man/Makefile.am
@@ -17,11 +17,13 @@
dist_man8_MANS = iwhd.8
MAINTAINERCLEANFILES = $(dist_man8_MANS)
+EXTRA_DIST = iwhd.x
# Depend on configure.ac to get version number changes.
-iwhd.8: $(top_srcdir)/configure.ac $(top_srcdir)/rest.c
+iwhd.8: $(top_srcdir)/configure.ac $(top_srcdir)/rest.c iwhd.x
$(AM_V_GEN) \
(echo '[NAME]' && sed 's@/\* *@@; s/-/\\-/; q' $(top_srcdir)/rest.c)|\
PATH=..$(PATH_SEPARATOR)$$PATH \
- $(HELP2MAN) --no-info -i - -S '$(PACKAGE) $(VERSION)' iwhd \
+ $(HELP2MAN) --include=$(srcdir)/iwhd.x --no-info -i - \
+ -S '$(PACKAGE) $(VERSION)' iwhd \
> $@-t && mv $@-t $@
diff --git a/man/iwhd.x b/man/iwhd.x
new file mode 100644
index 0000000..415ca29
--- /dev/null
+++ b/man/iwhd.x
@@ -0,0 +1,4 @@
+[FILES]
+.TP
+.I /var/log/iwhd.log
+Log file established by default /etc/init file.
12 years, 10 months
[iwhd] doc: mention in NEWS the delete-deleted fix
by Jim Meyering
commit 417858c9ffcd5345f4e04b92c0e54bfd054dfd10
Author: Jim Meyering <meyering(a)redhat.com>
Date: Thu Aug 11 22:20:12 2011 +0200
doc: mention in NEWS the delete-deleted fix
* NEWS (Bug fixes): Mention the fix from commit 2df124f0.
NEWS | 5 +++++
1 files changed, 5 insertions(+), 0 deletions(-)
---
diff --git a/NEWS b/NEWS
index 25b95ba..e122bcf 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,11 @@ iwhd NEWS -*- outline -*-
* Noteworthy changes in release ?.? (????-??-??) [?]
+** Bug fixes
+
+ object deletion now succeeds for an object whose FS-backed file has been
+ removed behind iwhd's back.
+
* Noteworthy changes in release 0.98 (2011-08-09) [stable]
12 years, 10 months
[iwhd] Let us delete objects if back-end lost them
by Pete Zaitcev
commit 2df124f0bbe3d66c21d1fe58f446f73d7822f93e
Author: Pete Zaitcev <zaitcev(a)yahoo.com>
Date: Thu Aug 11 14:06:16 2011 -0600
Let us delete objects if back-end lost them
If for some reason the back-end loses an object, it cannot be deleted.
This is a "should not happen" situation, which happens when a filesystem
checker unlinks something or when sysadmin makes a mistake.
The obious solution is to proceed with removal from the database even if
fails to find the object (a file in case of filesystem).
However, we cannot simply ignore errors, because this is how we
discover that a non-existing object was deleted, and return 404.
So, we make an extra lookup in the database, and only return an
error if there was nothing there.
But wait! There's more: why cannot we look up an object before even
reaching for the back-end? That is because we delete replicated objects
by first removing them from the database, and then requesting redundant
iwhd's to delete. They have to proceed with the deletion from their
back-ends first, and then report 404 if they like. This is a little
tricky, but works.
This patch also includes a test for this condition (t/delete-missing),
written by Jim Meyering.
Tracking ticket:
https://bugzilla.redhat.com/show_bug.cgi?id=710666
backend.c | 17 ++++++++++++-
rest.c | 3 ++
t/Makefile.am | 1 +
t/delete-missing | 64 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 83 insertions(+), 2 deletions(-)
---
diff --git a/backend.c b/backend.c
index efede7f..20a5d1c 100644
--- a/backend.c
+++ b/backend.c
@@ -1327,8 +1327,21 @@ fs_delete (const provider_t *prov, const char *bucket, const char *key,
(void)key;
if (unlink(url+1) < 0) {
- error (0, errno, _("%s: failed to unlink"), url+1);
- return MHD_HTTP_NOT_FOUND;
+ switch (errno) {
+ case ENOENT:
+ /* Mostly harmless consistency loss. */
+ DPRINTF("%s: already deleted\n", url+1);
+ return MHD_HTTP_OK;
+ case EPERM:
+ case EROFS:
+ case EACCES:
+ /* Possible correctable outside of iwhd by system tools. */
+ error (0, errno, _("%s: failed to unlink"), url+1);
+ return MHD_HTTP_FORBIDDEN;
+ default:
+ error (0, errno, _("%s: failed to unlink"), url+1);
+ return MHD_HTTP_NOT_FOUND;
+ }
}
return MHD_HTTP_OK;
diff --git a/rest.c b/rest.c
index a3aa834..eeae361 100644
--- a/rest.c
+++ b/rest.c
@@ -944,6 +944,7 @@ proxy_delete (void *cctx, struct MHD_Connection *conn, const char *url,
size_t *data_size, void **rctx)
{
my_state *ms = *rctx;
+ char *etag;
char *copied_url;
char *bucket;
char *key;
@@ -963,6 +964,8 @@ proxy_delete (void *cctx, struct MHD_Connection *conn, const char *url,
int rc = ms->thunk.prov->func_tbl->delete_func(main_prov,
ms->bucket,ms->key,url);
if (rc == MHD_HTTP_OK) {
+ if (meta_get_value(ms->bucket,ms->key,"_etag",&etag))
+ return do_resp(conn,ms,MHD_HTTP_NOT_FOUND,NULL,NULL);
copied_url = strdup(url);
assert (copied_url);
bucket = strtok_r(copied_url,"/",&stctx);
diff --git a/t/Makefile.am b/t/Makefile.am
index 6a668a1..ae2bdf3 100644
--- a/t/Makefile.am
+++ b/t/Makefile.am
@@ -18,6 +18,7 @@ TESTS = \
parse-test \
basic \
creation-code \
+ delete-missing \
exercise \
provider \
replication \
diff --git a/t/delete-missing b/t/delete-missing
new file mode 100644
index 0000000..9bc82d6
--- /dev/null
+++ b/t/delete-missing
@@ -0,0 +1,64 @@
+#!/bin/sh
+# Ensure that when an object has disappeared from a backend (say via
+# accidental deletion from an FS-backend), an attempt to delete it via
+# iwhd still removes it from the database.
+
+. "${srcdir=.}/init.sh"; path_prepend_ ..
+
+mkdir FS mongod iwhd || framework_failure_ mkdir failed
+
+m_port=$(get_port $mongo_base_port $lock_dir/m-) \
+ || fail_ "failed to get mongodb port"
+
+mongod --help > /dev/null || fail_ "the mongod program is not installed"
+
+mongod --port $m_port --pidfilepath mongod/pid --dbpath mongod > mongod.log 2>&1 &
+mongo_pid=$!
+cleanup_() { kill -9 $mongo_pid; }
+
+# Wait for up to 5 seconds for mongod to begin listening.
+wait_for .1 50 'mongo localhost:$m_port < /dev/null' \
+ || framework_failure_ mongod failed to start
+
+port=$(get_port 9095 $lock_dir/i-) || fail_ "failed to get iwhd port"
+
+printf '[{"path": "FS", "type": "fs", "name": "primary"}]\n' \
+ > iwhd.cfg || fail=1
+
+iwhd -v -p $port -c iwhd.cfg -d localhost:$m_port &
+iwhd_pid=$!
+cleanup_() { kill -9 $mongo_pid; kill $iwhd_pid; }
+
+# Wait for up to 5 seconds for iwhd to begin listening on $port.
+wait_for .1 50 "curl -s http://localhost:$port" \
+ || { echo iwhd failed to listen; Exit 1; }
+
+
+curl_w() { curl --write-out '%{http_code}' "$@"; }
+
+# Deleting an FS-removed-behind-our-back object should succeed.
+# With iwhd-0.98 and earlier, this would fail.
+delete_FS_deleted_evokes_200()
+{
+
+ # Create a bucket.
+ local b=http://localhost:$port/b9
+ curl_w -XPUT $b > http-code || fail=1
+ test "$(cat http-code)" = 201 || fail=1
+
+ # Create an object in that bucket.
+ local obj=$b/obj
+ curl_w -XPUT $obj > http-code || fail=1
+ test "$(cat http-code)" = 201 || fail=1
+
+ # Remove that object via the file system.
+ rm FS/b9/obj || fail=1
+
+ # Attempt to remove it via iwhd.
+ curl_w -f -X DELETE $b/obj > http-code || fail=1
+ test "$(cat http-code)" = 200 || fail=1
+}
+
+delete_FS_deleted_evokes_200
+
+Exit $fail
12 years, 10 months
[iwhd] Factor out Microhttpd response
by Pete Zaitcev
commit d63596ffc0e362030c0b08b504953c93b1e26d2a
Author: Pete Zaitcev <zaitcev(a)yahoo.com>
Date: Wed Aug 10 15:30:29 2011 -0600
Factor out Microhttpd response
This patch fell out from the authentication work: if one needs something
new done in every place a response is formed (such as to supply a header
for authentication), then we have too many places to do it. However, the
code seems more straightforward with it alone. Shorter, too.
rest.c | 315 ++++++++++++++++++----------------------------------------------
1 files changed, 87 insertions(+), 228 deletions(-)
---
diff --git a/rest.c b/rest.c
index 7bf8fe1..a3aa834 100644
--- a/rest.c
+++ b/rest.c
@@ -106,6 +106,8 @@ typedef struct {
MHD_AccessHandlerCallback handler;
} rule;
+static void simple_closer (void *ctx);
+
static unsigned short my_port = MY_PORT;
static LIST_HEAD(gc_list, _my_state) my_states; /* this keeps GC informed */
static pthread_mutex_t my_lock = PTHREAD_MUTEX_INITIALIZER;
@@ -168,6 +170,46 @@ validate_url (const char *url)
return !is_reserved(slash+1,reserved_name);
}
+static int
+do_resp (struct MHD_Connection *conn, my_state *ms,
+ unsigned int status_code, const char *etag, const char *datastr)
+{
+ struct MHD_Response *resp;
+ size_t datalen = datastr? strlen(datastr): 0;
+
+ resp = MHD_create_response_from_data(datalen,(void *)datastr,
+ MHD_NO,MHD_NO);
+ if (!resp) {
+ return MHD_NO;
+ }
+
+ if (etag) {
+ MHD_add_response_header(resp,"ETag",etag);
+ }
+
+ MHD_queue_response(conn,status_code,resp);
+ MHD_destroy_response(resp);
+ return MHD_YES;
+}
+
+static int
+do_resp_from_cb (struct MHD_Connection *conn, my_state *ms,
+ ssize_t (*func)(void *ctx, uint64_t pos, char *buf, size_t max))
+{
+ struct MHD_Response *resp;
+
+ resp = MHD_create_response_from_callback(MHD_SIZE_UNKNOWN,
+ CB_BLOCK_SIZE, func, ms, simple_closer);
+ if (!resp) {
+ log_msg(0,"MHD_crfc failed");
+ simple_closer(ms);
+ return MHD_NO;
+ }
+ MHD_queue_response(conn,MHD_HTTP_OK,resp);
+ MHD_destroy_response(resp);
+ return MHD_YES;
+}
+
/**********
* The proxy has MHD on one side and CURL on the other. The CURL side is
* always run in a child thread. Yes, there are both context switches
@@ -312,11 +354,7 @@ proxy_get_data (void *cctx, struct MHD_Connection *conn, const char *url,
conn, MHD_HEADER_KIND, "If-None-Match");
if (user_etag && !strcmp(user_etag,my_etag)) {
DPRINTF("ETag match!\n");
- resp = MHD_create_response_from_data(0,NULL,
- MHD_NO,MHD_NO);
- MHD_queue_response(conn,MHD_HTTP_NOT_MODIFIED,resp);
- MHD_destroy_response(resp);
- return MHD_YES;
+ return do_resp(conn,ms,MHD_HTTP_NOT_MODIFIED,NULL,NULL);
}
ms->from_master = 0;
}
@@ -324,11 +362,7 @@ proxy_get_data (void *cctx, struct MHD_Connection *conn, const char *url,
DPRINTF("%s/%s not found locally\n",ms->bucket,ms->key);
if (!master_host) {
DPRINTF(" that means it doesn't exist\n");
- resp = MHD_create_response_from_data(0,NULL,
- MHD_NO,MHD_NO);
- MHD_queue_response(conn,MHD_HTTP_NOT_FOUND,resp);
- MHD_destroy_response(resp);
- return MHD_YES;
+ return do_resp(conn,ms,MHD_HTTP_NOT_FOUND,NULL,NULL);
}
DPRINTF(" will fetch from %s:%u\n", master_host,master_port);
ms->from_master = 1;
@@ -424,7 +458,6 @@ proxy_put_data (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
- struct MHD_Response *resp;
my_state *ms = *rctx;
int rc;
@@ -437,14 +470,7 @@ proxy_put_data (void *cctx, struct MHD_Connection *conn, const char *url,
if (ms->state == MS_NEW) {
if (!validate_put(conn) || !validate_url(url)) {
DPRINTF("rejecting %s\n",url);
- resp = MHD_create_response_from_data(0,NULL,
- MHD_NO,MHD_NO);
- if (!resp) {
- return MHD_NO;
- }
- MHD_queue_response(conn,MHD_HTTP_FORBIDDEN,resp);
- MHD_destroy_response(resp);
- return MHD_YES;
+ return do_resp(conn,ms,MHD_HTTP_FORBIDDEN,NULL,NULL);
}
ms->state = MS_NORMAL;
ms->url = strdup(url);
@@ -479,14 +505,8 @@ proxy_put_data (void *cctx, struct MHD_Connection *conn, const char *url,
rc = pipe_prod_wait_init(&ms->pipe);
if (rc != 0) {
DPRINTF("producer wait failed\n");
- resp = MHD_create_response_from_data(0,NULL,
- MHD_NO,MHD_NO);
- if (!resp) {
- return MHD_NO;
- }
- MHD_queue_response(conn,MHD_HTTP_INTERNAL_SERVER_ERROR,
- resp);
- MHD_destroy_response(resp);
+ return do_resp(conn,ms,MHD_HTTP_INTERNAL_SERVER_ERROR,
+ NULL,NULL);
} else if (rc > 0) {
/*
* Note that we fail here even if 1 of N replicas fail.
@@ -495,14 +515,8 @@ proxy_put_data (void *cctx, struct MHD_Connection *conn, const char *url,
*/
DPRINTF("producer replicas failed (%u of %u)\n",
rc, ms->pipe.cons_total);
- resp = MHD_create_response_from_data(0,NULL,
- MHD_NO,MHD_NO);
- if (!resp) {
- return MHD_NO;
- }
- MHD_queue_response(conn,MHD_HTTP_INTERNAL_SERVER_ERROR,
- resp);
- MHD_destroy_response(resp);
+ return do_resp(conn,ms,MHD_HTTP_INTERNAL_SERVER_ERROR,
+ NULL,NULL);
} else {
DPRINTF("producer proceeding\n");
}
@@ -540,15 +554,7 @@ proxy_put_data (void *cctx, struct MHD_Connection *conn, const char *url,
recheck_replication(ms,NULL);
rc = MHD_HTTP_CREATED;
}
- resp = MHD_create_response_from_data(0,NULL,MHD_NO,MHD_NO);
- if (!resp) {
- return MHD_NO;
- }
- if (etag) {
- MHD_add_response_header(resp,"ETag",etag);
- }
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
+ return do_resp(conn,ms,rc,etag,NULL);
}
return MHD_YES;
@@ -581,14 +587,11 @@ proxy_list_attrs (void *cctx, struct MHD_Connection *conn, const char *url,
* means a request-specific routine (e.g. show_parts)
* created its own response. Therefore we shouldn't.
*/
- struct MHD_Response *resp
- = MHD_create_response_from_data(0,NULL, MHD_NO,MHD_NO);
- if (!resp) {
- log_msg(0,"MHD_crfd failed\n");
+ if (do_resp(conn,ms,rc,NULL,NULL) != MHD_YES) {
+ log_msg(0,"MHD_crfd failed");
return MHD_NO;
}
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
+ return MHD_YES;
}
}
@@ -600,10 +603,8 @@ proxy_get_attr (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
- struct MHD_Response *resp;
char *fixed;
my_state *ms = *rctx;
- int rc = MHD_HTTP_NOT_FOUND;
(void)cctx;
(void)method;
@@ -614,20 +615,11 @@ proxy_get_attr (void *cctx, struct MHD_Connection *conn, const char *url,
DPRINTF("PROXY GET ATTR %s\n",url);
if (meta_get_value(ms->bucket,ms->key,ms->attr,&fixed) == 0) {
- resp = MHD_create_response_from_data(strlen(fixed),fixed,
- MHD_NO,MHD_NO);
- rc = MHD_HTTP_OK;
+ return do_resp(conn,ms,MHD_HTTP_OK,NULL,fixed);
}
else {
- resp = MHD_create_response_from_data(0,NULL,MHD_NO,MHD_NO);
- }
- if (!resp) {
- return MHD_NO;
+ return do_resp(conn,ms,MHD_HTTP_NOT_FOUND,NULL,NULL);
}
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
-
- return MHD_YES;
}
static int
@@ -635,7 +627,6 @@ proxy_put_attr (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
- struct MHD_Response *resp;
my_state *ms = *rctx;
int send_resp = 0;
@@ -684,15 +675,7 @@ proxy_put_attr (void *cctx, struct MHD_Connection *conn, const char *url,
return MHD_NO;
}
if (is_reserved(ms->attr,reserved_attr)) {
- resp = MHD_create_response_from_data(
- 0,NULL,MHD_NO,MHD_NO);
- if (!resp) {
- return MHD_NO;
- }
- MHD_queue_response(conn,MHD_HTTP_BAD_REQUEST,
- resp);
- MHD_destroy_response(resp);
- return MHD_YES;
+ return do_resp(conn,ms,MHD_HTTP_BAD_REQUEST,NULL,NULL);
}
meta_set_value(ms->bucket,ms->key,ms->attr,ms->pipe.data_ptr);
/*
@@ -705,12 +688,7 @@ proxy_put_attr (void *cctx, struct MHD_Connection *conn, const char *url,
}
if (send_resp) {
- resp = MHD_create_response_from_data(0,NULL,MHD_NO,MHD_NO);
- if (!resp) {
- return MHD_NO;
- }
- MHD_queue_response(conn,MHD_HTTP_CREATED,resp);
- MHD_destroy_response(resp);
+ return do_resp(conn,ms,MHD_HTTP_CREATED,NULL,NULL);
/*
* TBD: check if the attribute was a replication policy, and
* start/stop replication activities as appropriate.
@@ -887,7 +865,6 @@ proxy_query (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
- struct MHD_Response *resp;
my_state *ms = *rctx;
(void)cctx;
@@ -932,21 +909,9 @@ proxy_query (void *cctx, struct MHD_Connection *conn, const char *url,
}
int rc = proxy_query_init(ms, ms->pipe.data_ptr);
if (rc != MHD_HTTP_OK) {
- resp = MHD_create_response_from_data(0,NULL,
- MHD_NO,MHD_NO);
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
- return MHD_YES;
- }
- resp = MHD_create_response_from_callback(MHD_SIZE_UNKNOWN,
- CB_BLOCK_SIZE, proxy_query_func, ms, simple_closer);
- if (!resp) {
- log_msg(0,"MHD_crfc failed");
- simple_closer(ms);
- return MHD_NO;
+ return do_resp(conn,ms,rc,NULL,NULL);
}
- MHD_queue_response(conn,MHD_HTTP_OK,resp);
- MHD_destroy_response(resp);
+ return do_resp_from_cb(conn,ms,proxy_query_func);
}
return MHD_YES;
@@ -958,7 +923,6 @@ proxy_list_objs (void *cctx, struct MHD_Connection *conn, const char *url,
size_t *data_size, void **rctx)
{
my_state *ms = *rctx;
- struct MHD_Response *resp;
(void)cctx;
(void)url;
@@ -969,22 +933,9 @@ proxy_list_objs (void *cctx, struct MHD_Connection *conn, const char *url,
int rc = proxy_query_init(ms, NULL);
if (rc != MHD_HTTP_OK) {
- resp = MHD_create_response_from_data(0,NULL, MHD_NO,MHD_NO);
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
- return MHD_YES;
+ return do_resp(conn,ms,rc,NULL,NULL);
}
- resp = MHD_create_response_from_callback(MHD_SIZE_UNKNOWN,
- CB_BLOCK_SIZE, proxy_query_func, ms, simple_closer);
- if (!resp) {
- log_msg(0,"MHD_crfc failed");
- simple_closer(ms);
- return MHD_NO;
- }
-
- MHD_queue_response(conn,MHD_HTTP_OK,resp);
- MHD_destroy_response(resp);
- return MHD_YES;
+ return do_resp_from_cb(conn,ms,proxy_query_func);
}
static int
@@ -993,7 +944,6 @@ proxy_delete (void *cctx, struct MHD_Connection *conn, const char *url,
size_t *data_size, void **rctx)
{
my_state *ms = *rctx;
- struct MHD_Response *resp;
char *copied_url;
char *bucket;
char *key;
@@ -1021,15 +971,7 @@ proxy_delete (void *cctx, struct MHD_Connection *conn, const char *url,
replicate_delete(url,ms);
}
- resp = MHD_create_response_from_data(0,NULL,MHD_NO,MHD_NO);
- if (!resp) {
- return MHD_NO;
- }
- log_msg(0, "DELETE BUCKET: rc=%d", rc);
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
-
- return MHD_YES;
+ return do_resp(conn,ms,rc,NULL,NULL);
}
/* TBD: get actual bucket list */
@@ -1135,8 +1077,6 @@ proxy_api_root (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
- struct MHD_Response *resp = NULL;
- unsigned int rc = MHD_HTTP_OK;
my_state *ms = *rctx;
(void)cctx;
@@ -1150,16 +1090,7 @@ proxy_api_root (void *cctx, struct MHD_Connection *conn, const char *url,
if (!ms->query) {
return MHD_NO;
}
- resp = MHD_create_response_from_callback(MHD_SIZE_UNKNOWN,
- CB_BLOCK_SIZE, root_blob_generator, ms, simple_closer);
- if (!resp) {
- return MHD_NO;
- }
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
-
- return MHD_YES;
-
+ return do_resp_from_cb(conn,ms,root_blob_generator);
}
static int
@@ -1331,8 +1262,8 @@ control_api_root (void *cctx, struct MHD_Connection *conn, const char *url,
}
if (len >= (int)sizeof(buf)) {
- len = 0;
- rc = MHD_HTTP_INTERNAL_SERVER_ERROR;
+ return do_resp(conn,ms,MHD_HTTP_INTERNAL_SERVER_ERROR,
+ NULL,NULL);
}
/* NB The last arg tells MHD to copy the arg and free it later. */
@@ -1351,7 +1282,6 @@ proxy_bucket_post (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
- struct MHD_Response *resp;
my_state *ms = *rctx;
char *key;
@@ -1400,13 +1330,10 @@ proxy_bucket_post (void *cctx, struct MHD_Connection *conn, const char *url,
else {
DPRINTF("A parameter is MISSING (fail)\n");
}
- resp = MHD_create_response_from_data(0,NULL,MHD_NO,MHD_NO);
- if (!resp) {
+ if (do_resp(conn,ms,rc,NULL,NULL) != MHD_YES) {
log_msg(0,"MHD_crfd failed");
return MHD_NO;
}
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
}
return MHD_YES;
@@ -1548,16 +1475,9 @@ show_parts (struct MHD_Connection *conn, my_state *ms)
return MHD_HTTP_NOT_FOUND;
}
- struct MHD_Response *resp;
- resp = MHD_create_response_from_callback(MHD_SIZE_UNKNOWN,
- CB_BLOCK_SIZE, parts_callback, ms, simple_closer);
- if (!resp) {
- log_msg(0,"MHD_crfc failed");
- simple_closer(ms);
+ if (do_resp_from_cb(conn,ms,parts_callback) != MHD_YES) {
return MHD_NO;
}
- MHD_queue_response(conn,MHD_HTTP_OK,resp);
- MHD_destroy_response(resp);
return MHD_HTTP_PROCESSING;
}
@@ -1566,7 +1486,6 @@ proxy_object_post (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
- struct MHD_Response *resp;
my_state *ms = *rctx;
char *op;
@@ -1627,14 +1546,10 @@ proxy_object_post (void *cctx, struct MHD_Connection *conn, const char *url,
* means a request-specific routine (e.g. show_parts)
* created its own response. Therefore we shouldn't.
*/
- resp = MHD_create_response_from_data(0,NULL,
- MHD_NO,MHD_NO);
- if (!resp) {
+ if (do_resp(conn,ms,rc,NULL,NULL) != MHD_YES) {
log_msg(0,"MHD_crfd failed");
return MHD_NO;
}
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
}
}
@@ -1828,7 +1743,6 @@ proxy_list_provs (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
- struct MHD_Response *resp;
my_state *ms = *rctx;
(void)cctx;
@@ -1838,17 +1752,7 @@ proxy_list_provs (void *cctx, struct MHD_Connection *conn, const char *url,
(void)data;
(void)data_size;
- resp = MHD_create_response_from_callback(MHD_SIZE_UNKNOWN,
- CB_BLOCK_SIZE, prov_list_generator, ms, simple_closer);
- if (!resp) {
- log_msg(0,"MHD_crfd failed");
- simple_closer(ms);
- return MHD_NO;
- }
- MHD_queue_response(conn,MHD_HTTP_OK,resp);
- MHD_destroy_response(resp);
-
- return MHD_YES;
+ return do_resp_from_cb(conn,ms,prov_list_generator);
}
static int
@@ -1887,6 +1791,7 @@ proxy_primary_prov (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
+ my_state *ms = *rctx;
(void)cctx;
(void)method;
(void)version;
@@ -1895,23 +1800,12 @@ proxy_primary_prov (void *cctx, struct MHD_Connection *conn, const char *url,
DPRINTF("PROXY GET PRIMARY PROVIDER (%s)\n", url);
// "/_providers/_primary" is the only one we accept for now.
- bool valid = strcmp (url, "/_providers/_primary") == 0;
- unsigned int rc = (valid ? MHD_HTTP_OK : MHD_HTTP_BAD_REQUEST);
- if (!valid)
+ if (strcmp (url, "/_providers/_primary") != 0) {
log_msg(0, _("invalid request: %s"), url);
-
- const char *name = get_main_provider()->name;
- struct MHD_Response *resp;
- resp = MHD_create_response_from_data(valid ? strlen (name) : 0,
- valid ? (void *) name : NULL,
- MHD_NO, MHD_NO);
- if (!resp) {
- return MHD_NO;
+ return do_resp(conn,ms,MHD_HTTP_BAD_REQUEST,NULL,NULL);
}
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
- return MHD_YES;
+ return do_resp(conn,ms,MHD_HTTP_OK,NULL,get_main_provider()->name);
}
static int
@@ -1919,6 +1813,7 @@ proxy_set_primary (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
+ my_state *ms = *rctx;
(void)cctx;
(void)method;
(void)version;
@@ -1956,16 +1851,7 @@ proxy_set_primary (void *cctx, struct MHD_Connection *conn, const char *url,
}
bad_set:;
-
- struct MHD_Response *resp;
- resp = MHD_create_response_from_data(0, NULL, MHD_NO, MHD_NO);
- if (!resp) {
- return MHD_NO;
- }
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
-
- return MHD_YES;
+ return do_resp(conn,ms,rc,NULL,NULL);
}
static int
@@ -1973,38 +1859,28 @@ proxy_delete_prov (void *cctx, struct MHD_Connection *conn, const char *url,
const char *method, const char *version, const char *data,
size_t *data_size, void **rctx)
{
- DPRINTF("PROXY DELETE PROVIDER %s\n",url);
+ my_state *ms = *rctx;
(void)cctx;
(void)method;
(void)version;
(void)data;
(void)data_size;
- struct MHD_Response *resp
- = MHD_create_response_from_data(0,NULL,MHD_NO,MHD_NO);
- if (!resp) {
- return MHD_NO;
- }
+ DPRINTF("PROXY DELETE PROVIDER %s\n",url);
char *prov_name = url_to_provider_name (url);
provider_t *prov = find_provider (prov_name);
+ if (!prov)
+ return do_resp(conn,ms,MHD_HTTP_NOT_FOUND,NULL,NULL);
+
// don't allow removal of current main provider.
if (prov == get_main_provider())
- prov = NULL;
-
- int rc = prov ? MHD_HTTP_OK : MHD_HTTP_NOT_FOUND;
-
- DPRINTF("PROXY DELETE PROVIDER prov=%s rc=%d\n", prov_name, rc);
+ return do_resp(conn,ms,MHD_HTTP_BAD_REQUEST,NULL,NULL);
- if (prov) {
- prov->deleted = 1;
- }
-
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
-
- return MHD_YES;
+ DPRINTF("PROXY DELETE PROVIDER prov=%s rc=%d\n", prov_name,MHD_HTTP_OK);
+ prov->deleted = 1;
+ return do_resp(conn,ms,MHD_HTTP_OK,NULL,NULL);
}
static int
@@ -2013,7 +1889,6 @@ proxy_add_prov (void *cctx, struct MHD_Connection *conn, const char *url,
size_t *data_size, void **rctx)
{
DPRINTF("PROXY ADD PROVIDER %s\n",url);
- struct MHD_Response *resp;
my_state *ms = *rctx;
(void)cctx;
@@ -2072,13 +1947,10 @@ proxy_add_prov (void *cctx, struct MHD_Connection *conn, const char *url,
}
add_fail:
- resp = MHD_create_response_from_data(0,NULL,MHD_NO,MHD_NO);
- if (!resp) {
+ if (do_resp(conn,ms,rc,NULL,NULL) != MHD_YES) {
log_msg(0,"MHD_crfd failed");
return MHD_NO;
}
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
}
return MHD_YES;
@@ -2101,15 +1973,10 @@ proxy_create_bucket (void *cctx, struct MHD_Connection *conn, const char *url,
/* curl -T moo.empty http://localhost:9090/_new by accident */
int rc = create_bucket(ms->bucket,ms);
- struct MHD_Response *resp
- = MHD_create_response_from_data(0,NULL, MHD_NO,MHD_NO);
- if (!resp) {
+ if (do_resp(conn,ms,rc,NULL,NULL) != MHD_YES) {
log_msg(0,"MHD_crfd failed");
return MHD_NO;
}
- MHD_queue_response(conn,rc,resp);
- MHD_destroy_response(resp);
-
return MHD_YES;
}
@@ -2227,7 +2094,6 @@ access_handler_0 (void *cctx, struct MHD_Connection *conn, const char *url,
{
unsigned int i;
url_type utype;
- struct MHD_Response *resp;
my_state *ms = *rctx;
log_check();
@@ -2273,14 +2139,7 @@ access_handler_0 (void *cctx, struct MHD_Connection *conn, const char *url,
}
log_msg(0,"bad request m=%s u=%s",method,url);
-
- resp = MHD_create_response_from_data(0,NULL,MHD_NO,MHD_NO);
- if (!resp) {
- return MHD_NO;
- }
- MHD_queue_response(conn,MHD_HTTP_NOT_FOUND,resp);
- MHD_destroy_response(resp);
- return MHD_YES;
+ return do_resp(conn,ms,MHD_HTTP_NOT_FOUND,NULL,NULL);
}
static int
12 years, 10 months
[iwhd] Created tag v0.98
by Jim Meyering
The signed tag 'v0.98' was created.
Tagger: Jim Meyering <meyering(a)redhat.com>
Date: Tue Aug 9 16:27:05 2011 +0200
iwhd 0.98
Changes since the last tag 'v0.97':
Jim Meyering (9):
post-release administrivia
avoid GC segfault/hang/etc: make gnulib's hash module use GC
diagnose command-line errors via error(..., not log_msg(...
diagnose invalid host and port command line arguments; don't modify ARGV
maint: don't include "xalloc.h" unless required
build: update gnulib submodule to latest
tests: update to newer framework from gnulib
maint: use gnulib's new largefile modulue
version 0.98
Pete Zaitcev (4):
Make the intent of the "path" clearer.
Merge branch 'master' of ssh://git.fedorahosted.org/git/iwhd
Add support for logrotate and timestamps
Fix a crash in GC due to thread-local *rctx
12 years, 10 months
[iwhd] post-release administrivia
by Jim Meyering
commit d6c4adc01b9011efc99500083e4ee0850099edf5
Author: Jim Meyering <meyering(a)redhat.com>
Date: Tue Aug 9 16:30:39 2011 +0200
post-release administrivia
* NEWS: Add header line for next release.
* .prev-version: Record previous version.
* cfg.mk (old_NEWS_hash): Auto-update.
.prev-version | 2 +-
NEWS | 3 +++
cfg.mk | 2 +-
3 files changed, 5 insertions(+), 2 deletions(-)
---
diff --git a/.prev-version b/.prev-version
index ea6844b..0f6db3d 100644
--- a/.prev-version
+++ b/.prev-version
@@ -1 +1 @@
-0.97
+0.98
diff --git a/NEWS b/NEWS
index c726737..25b95ba 100644
--- a/NEWS
+++ b/NEWS
@@ -1,5 +1,8 @@
iwhd NEWS -*- outline -*-
+* Noteworthy changes in release ?.? (????-??-??) [?]
+
+
* Noteworthy changes in release 0.98 (2011-08-09) [stable]
** New features
diff --git a/cfg.mk b/cfg.mk
index a88295e..e4e34f0 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -38,7 +38,7 @@ bootstrap-tools = autoconf,automake,gnulib
# Now that we have better tests, make this the default.
export VERBOSE = yes
-old_NEWS_hash = 2f77846f433127ee87d65f2fa88c623a
+old_NEWS_hash = 4bc259fb9e5fcb8fbdad180da1b65edb
sc_prohibit_echo_minus_en:
@prohibit='\<echo -[en]' \
12 years, 10 months