95 Commits

Author SHA1 Message Date
Joris Vink
f0a9c1fe73 Major improvements to HTTP/1.1 behaviour, error reporting and handling.
This comes with changes to http_response() (no longer returns a result).
2014-01-29 22:48:51 +01:00
Joris Vink
a80808d779 Add header as an option for authentication blocks 2014-01-22 23:11:52 +01:00
Joris Vink
9a8092bf41 Add authentication blocks for Kore.
Using authentication blocks one can define "authentication" mechanisms
in Kore for page handlers.

This can be used to require a session cookie (validated by your own validator)
for certain page handlers, and hopefully in the future provide a framework
for adding more authentication things (like HTTP Auth).

Right now only cookie checking is available.
2014-01-22 22:55:10 +01:00
Joris Vink
b64f674db2 Handle SPDY streams better when sending data. 2014-01-14 21:43:45 +01:00
Joris Vink
d22405cea7 Call the onload function whenever a module is loaded/reload.
Allows one to teardown whatever they setup properly when
the module gets a full reload. See example module for how it works.
2013-12-21 13:37:34 +01:00
Joris Vink
21b148e3a5 Allow Kore to load multiple modules at once. 2013-12-15 01:11:56 +01:00
Joris Vink
f2aa206f3b Add support for client certificates 2013-12-14 16:31:07 +01:00
Joris Vink
c7dcdbcd82 Rework the way validation and param extraction works.
- Parameter validation is now done only when http_process_*()
  is called and upon http_argument_add().
- You MUST have defined your params in a param block or they will
  be filtered out.
- http_argument_lookup() is dead, welcome http_argument_get() and
  its brothers and sisters:
	http_argument_get_string()
	http_argument_get_uint16()
	http_argument_get_int16()
	http_argument_get_uint32()
	http_argument_get_int32()

  They will automatically do bounds checking on integers for you
  and return proper integers or a NUL-terminated string.

- The http_argument_get* functions no longer create an additional
  copy of the string which you need to free. Easier going.
- http_multiple_args() is dead, byebye
- Make some stuff we don't want to share with the modules static.
2013-12-12 00:58:32 +01:00
Joris Vink
0f28b4a62d Enter versioning, starting at 1.0-current 2013-12-10 16:43:19 +01:00
Joris Vink
79aea48757 Don't stop passing the accept lock even when workers are very busy.
If a worker reached worker_max_connections and it was its turn to
grab the accept lock it would've gotten stuck and no new connections
would be handled even if other workers would be less busy.

Instead, we now skip the lock if we're too busy and pass it along
in the hopes other workers are less busy.
2013-11-27 23:02:15 +01:00
Joris Vink
afeb213260 Kore no longer requires root to be started. 2013-11-18 00:42:57 +01:00
Joris Vink
1e250c1473 Kore now supports GET parameters and automatic validation of GET/POST parameters.
Kore will automatically removes invalid parameters as a security measure.

See modules/examples/module.conf for an example of how this works.
2013-11-10 15:17:15 +01:00
Joris Vink
34c2f31a93 Add validators to kore, specified in the configuration using 'validator' keyword.
Example:
	validator	v_id	function	v_id_function
	validator	v_url	regex		^/url/path/[a-z]*$

You can then call these using kore_validator_run(char *, char *), example:

	if (!kore_validator_run("v_url", req->path))
		[req->path is bad];
2013-11-09 16:21:52 +01:00
Joris Vink
f17242262b Improve on sending data to the client.
- Attempt to chain as much as we can in the send netbufs
  (keeps down the SSL_write calls, silly seeing it go out with 8 bytes)
- Change NETBUF_SEND_PAYLOAD_MAX to 4K
- Call SSL_write() with max NETBUF_SEND_PAYLOAD_MAX in size
- Go back to flushing the send buffers after each request
- No more need for a fixed pool for nb->buf, go back to the heap for now
- Disable Nagle, we're doing the chaining now anyway
2013-10-26 00:48:09 +02:00
Joris Vink
4247d3139c Chain our sending netbufs together. Gives us a lot less SSL_write() calls. 2013-10-25 14:22:29 +02:00
Joris Vink
4922171d96 Change net_send_queue() in preparation for improving send performance.
No longer takes callbacks, flags, or *out arguments.

Update rest of the code that called these callbacks whenever sending
was completed, instead call them right away now.
2013-10-25 11:10:03 +02:00
Joris Vink
ac025adaa7 Make kore_worker_acceptlock_release() available to loaded modules.
This allows us to move the accept lock manually to another Kore worker
in case we own it and are about to do some heavy lifting on the current
Kore worker.
2013-10-24 09:05:46 +02:00
Joris Vink
8b47863cd4 Add http_hsts_enable (enabled by default with max-age=31536000) to Kore's configuration file.
If enabled Kore adds the HSTS header to every response.

- Additionally, fix some typos in the example configuration.
- Change default SSL cipher list again, no more RC4 and almost PFS for all browsers.
2013-10-15 10:44:56 +02:00
Joris Vink
d2447a8848 Better defaults for ssl_cipher, was already in example config but not here. 2013-10-14 09:26:48 +02:00
Joris Vink
1fb3c013ff Use kore_mem_find() in kore_buf_replace_string(), fixes certain crashes
that could occur when calling kore_buf_replace_string() with patterns
that would be found at the end or start of the buffer.
2013-09-26 16:49:44 +02:00
Joris Vink
88c3a3eb98 Add http_header_max and http_postbody_max configuration variables.
- http_header_max:
	Maximum size of HTTP headers (in non SPDY connections).

- http_postbody_max:
	Maximum size of an HTTP POST body (both in SPDY and HTTP mode).

Right now Kore will simply DC the client, ideally we want to send
a 413 (entity too large) to the client however.

See modules/examples/module.conf for more.
2013-09-22 20:05:24 +02:00
Joris Vink
25e8f93331 Add support for multipart forms.
New API functions (docs need to be updated):
	- http_file_lookup()
	- http_file_add()
	- http_argument_add()
	- kore_strip_chars()
	- kore_mem_find()

- Add an example under the example module on how files can be read.
2013-09-10 11:02:59 +02:00
Joris Vink
c9d4f70298 - Add SPDY RST control frame handler.
- Keep HTTP requests in connection, so we can delete them if the connection
  ends before the requests do (this way we don't leak them).
- When spdy_stream_close() is called, delete the attached http request.
  (This shouldn't hurt to do, so hopefully won't cause major fallout).
- When parsing HTTP, find the first occurence of end-of-headers so uploads
  with multipart/form-data can succeed properly.
- Add a test upload page to the example module.
2013-09-09 10:59:56 +02:00
Joris Vink
ee3fd3c039 Allow the user defined callback to run on workers as well. 2013-09-03 08:41:09 +02:00
Joris Vink
95c8b8e126 Add a callback that Kore can call in your module every given interval.
The callback is run from the parent process (which runs as root).

Adds kore_cb and kore_cb_interval configuration options.
2013-09-02 08:52:16 +02:00
Joris Vink
acc34e2d51 Change kore_buf_append() and kore_buf_replace_string() to take a void pointer. 2013-08-22 10:06:39 +02:00
Joris Vink
ecefdb18ab cb_extra -> hdlr_extra and add one to struct connection as well for persistent data across requests. 2013-08-14 15:56:44 +02:00
Joris Vink
bbb245654d Pass the base for strtoll() to kore_strtonum(), breakage ensues if we depend on the "auto" detection that happens when we pass 0 to strtoll() as base. 2013-08-13 16:13:43 +02:00
Joris Vink
af5f416e6d and as always, remove debug. 2013-08-13 14:19:57 +02:00
Joris Vink
3075162855 Add http_argument_urldecode() which takes a string and decodes any url encoding done to it.
Change kore_strtonum() to pass 0 to strtoll by default so we can use it to convert hex numbers (prepended with 0x) as well.
2013-08-13 14:18:47 +02:00
Joris Vink
bbfbfc4c61 add ssl_no_compression option to allow one to disable OpenSSL compression. 2013-08-07 16:59:45 +02:00
Joris Vink
04ee544982 Add support for ephemeral key exchange mechanisms, ssl_dhparam configuration option must be set (and point to a file containing a generated DH key). 2013-08-07 16:51:39 +02:00
Joris Vink
bb9f37f029 remove KORE_DEBUG define that sneaked in 2013-08-07 14:59:59 +02:00
Joris Vink
db7ed69f2a Add kore_buf_replace_string().
kore_buf_replace_string allows you to replace occurances of a certain
string with something else.

Example:
	char	*username = "Joris";

	page = kore_buf_create(static_len_html_profile);
	kore_buf_append(page, static_html_profile, static_len_html_profile);
	kore_buf_replace_string(page, "%name%", username, strlen(username));
2013-08-07 14:56:14 +02:00
Joris Vink
ef814a677d Add http_argument_multiple_lookup() and http_argument_multiple_free().
Prototypes:
	int	http_argument_multiple_lookup(struct http_req *req,
		    struct http_arg *args);
	void	http_argument_multiple_free(struct http_arg *args);

These functions can be used to lookup arguments in a single call.

args points to an array of struct http_arg elements. Each of them
have the argument name set and its value set to NULL.

The array must have its last element name field set to NULL.

Upon return http_argument_multiple_lookup() gives the caller the
number of arguments that were successfully found. It makes their values
available under the value field in the struct http_arg array passed.

Example:
	int			v;
	struct http_args	args[4];

	memset(args, 0, sizeof(args));
	args[0].name = "email";
	args[1].name = "password1";
	args[2].name = "password2";
	args[3].name = NULL;

	v = http_argument_multiple_lookup(req, args);
	if (v != 3) {
		kore_debug("argument %s was not present", args[v].name);
	} else {
		for (v = 0; args[v].name != NULL; v++)
			kore_debug("%s -> %s", args[v].name, args[v].value);
	}

	http_argument_multiple_free(args);
2013-08-07 14:41:16 +02:00
Joris Vink
659e19f92f add IPv6 support and support for multiple listeners. 2013-07-27 20:56:15 +02:00
Joris Vink
20f02ced23 remove meminuse and list of allocated memory blocks, we don't need it. 2013-07-25 23:41:00 +02:00
Joris Vink
2c1352b226 remove versioning numbers, i hate them. 2013-07-16 15:30:20 +02:00
Joris Vink
29fa49ba83 Add fixed size memory pools and use them throughout Kore. 2013-07-15 11:06:36 +02:00
Joris Vink
c999bf5001 Kore can now disconnect SPDY session if they've been idle too long.
Configurable via spdy_idle_time in your configuration file.

Setting this to 0 will keep SPDY sessions open indefinately.
2013-07-13 20:19:01 +02:00
Joris Vink
f59e94a7b6 Add spdy_session_teardown() which can properly teardown a SPDY session.
Use this throughout the spdy code to propagate session errors (if any) occur.

At the same time fix BSD's missing CONN_WRITE_BLOCK
2013-07-13 19:56:38 +02:00
Joris Vink
bb09e8b2a9 add base64 encoder/decoder in prep for http/2.0 and websockets 2013-07-10 15:00:53 +02:00
Joris Vink
bb2d4903f2 remove NETBUF_RETAIN, no longer needed 2013-07-10 13:39:35 +02:00
Joris Vink
a17f93e896 Include sys/types.h and unistd.h for close(), getopt(), getpid(),
getuid(), setproctitle(), unlink() and declaration of optarg.

Makes Kore compatible with FreeBSD too.

From Ayan George <ayan@ayan.net>
2013-07-09 09:45:16 +02:00
Joris Vink
95bacb5690 Kore will now keep track of page handlers that cause workers to die.
This is useful to track down any issues you might have in your module.

A log entry with a page handler causing issues looks like:
Jul  7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11
Jul  7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone
Jul  7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
2013-07-07 14:48:32 +02:00
Joris Vink
c1723f2db5 Clean up header includes, based on a diff from Ewan Higgs via github.
And while we're messing in it, make sure bsd.c compiles again.
2013-07-06 20:55:22 +02:00
Joris Vink
0c08b57d3e - add idle timer for normal connections (ie: !SPDY), max 20 seconds.
- use idle timer to make sure we don't block SPDY connections indefinately
  when our window size has reached 0.
2013-07-01 12:08:51 +02:00
Joris Vink
51a9e4db9d Implement SPDY WINDOW_UPDATE and SETTINGS. 2013-07-01 11:30:18 +02:00
Joris Vink
4b2420097b - Better spread load between all worker processes.
- Introduce own memory management system on top of malloc to keep track
  of all our allocations and free's. Later we should introduce a pooling
  mechanism for fixed size allocations (http_request comes to mind).
- Introduce ssl_cipher in configuration.

Memory usage is kind of high right now, but it seems its OpenSSL
doing it rather then Kore.
2013-06-27 08:46:18 +02:00
Joris Vink
b4a0330a96 - Better spread load between all worker processes.
- Introduce own memory management system on top of malloc to keep track
  of all our allocations and free's. Later we should introduce a pooling
  mechanism for fixed size allocations (http_request comes to mind).
- Introduce ssl_cipher in configuration.

Memory usage is kind of high right now, but it seems its OpenSSL
doing it rather then Kore.
2013-06-27 08:43:07 +02:00