291 Commits

Author SHA1 Message Date
Joris Vink
1ecb777d41 only remove the pgsql if it was scheduled.
fixes #172.
2017-01-29 10:34:53 +01:00
Joris Vink
c06ecf1c70 Teach single binaries about SIGHUP.
Make sure kore_onload() and kore_preload() can be called
either in native or python runtime.
2017-01-26 13:26:55 +01:00
Joris Vink
dcb5fd842f Expose get/post params via req.argument for python. 2017-01-26 12:01:28 +01:00
Joris Vink
da13e1e768 undef _POSIX_C_SOURCE and _XOPEN_SOURCE before Python.h 2017-01-25 22:41:57 +01:00
Joris Vink
f56938283d Merge branch 'master' of github.com:jorisvink/kore 2017-01-25 22:23:34 +01:00
Joris Vink
620cc231dc validators now work in python. 2017-01-25 21:17:12 +01:00
Joris Vink
bbcdec82fc Add initial python support.
Based on work done by Stanislav Yudin.
2017-01-24 20:18:12 +01:00
Yorick de Wid
86d85dd10b Message shutdown request (#168)
* Set server version

* Message shutdown request

* Update messaging example

* Set server verion in generic example

* Allow shutdown with NOHTTP=1
2017-01-17 18:18:01 +01:00
Yorick de Wid
565bf5bdec Set server version (#157) 2017-01-13 13:03:47 +01:00
Manuel Kniep
ec5e93010f add missing includes (#163)
to make includes order independent
2017-01-07 23:07:34 +01:00
Joris Vink
31d14d028a add compile-time configurable mime types.
these types are used for the new builtin asset_serve_* functions.

must be defined in conf/build.conf.
2016-12-26 23:37:05 +01:00
Joris Vink
d783a1d22d Add auto generated serving functions for assets.
These functions are created by the cli tool when building
and follow the naming format: asset_serve_<name>_<ext>().

Those serving functions can be used directly in handlers and
callthrough to a http_serveable() function that uses the SHA1
of the asset as its ETag and automatically checks for if-none-match.
2016-12-26 21:15:03 +01:00
Tobias Kortkamp
7eced6f035 Fix #153 (#155)
Compiling with Clang 3.9 on FreeBSD raises the following error (see [1]):
```
src/pgsql.c:222:17: error: passing an object that undergoes default argument promotion to 'va_start'
      has undefined behavior [-Werror,-Wvarargs]
        va_start(args, count);
                       ^
src/pgsql.c:217:45: note: parameter of type 'u_int8_t' (aka 'unsigned char') is declared here
    const char *query, int result, u_int8_t count, ...)
```
More information about this warning can be found on [2].

To solve this we can change the type of `count` to `int` in
`kore_pgsql_query_params()` and `kore_pgsql_v_query_params()`. This
also matches the signatures of both `PQexecParams()` [3] and
`PQsendQueryParams()` [4].

[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=214639
[2] https://www.securecoding.cert.org/confluence/display/cplusplus/EXP58-CPP.+Pass+an+object+of+the+correct+type+to+va_start
[3] https://www.postgresql.org/docs/9.2/static/libpq-exec.html
[4] https://www.postgresql.org/docs/9.2/static/libpq-async.html
2016-11-19 12:56:51 +01:00
Joris Vink
5b379a9185 add a kore_preload() call for single binaries.
This is run on the parent before workers are spawned.
2016-11-17 10:05:16 +01:00
Joris Vink
552d59248c constify delim in kore_split_string(). 2016-10-06 16:50:41 +02:00
Joris Vink
8b723479ec bump 2016-08-01 15:01:19 +02:00
Joris Vink
441cc34f31 bump to release 2016-08-01 09:38:40 +02:00
Raphaël Monrouzeau
016dc27346 JSONRPC Made request destruction explicit
Caller has now to destroy jsonrpc_request after use. This permits them
to read / inspect it after having responded.
2016-07-15 13:08:08 +02:00
Raphaël Monrouzeau
8c78b28be3 JSONRPC Removed upload size limit check
A true application dependant limit check would require stream parsing.
As the limit enforcement was done, it added nothing of value compared
to HTTP request limit check, which is in Kore already.
2016-07-15 13:08:08 +02:00
Raphaël Monrouzeau
3366ec6573 Required params to be structured as per spec 2016-07-15 13:08:08 +02:00
Raphaël Monrouzeau
db02e990ea JSON-RPC support for Kore.
The API surface is very limited. Jsonrpc support reads request from HTTP
body and such can't be activated if NOHTTP=1. At the moment there is no
websocket support either (which is a shame). It depends upon the
third-party Yajl library.

Errors can be emitted using jsonrpc_error() and results using
jsonrpc_result(), for the later you'll have to provide a callback which
will write the inner of the result object.

If errors happen during the response write process, no further error
output will be attempted and an HTTP error 500 will be returned.

Read the provided example for getting a better idea of the API.
2016-07-15 13:08:08 +02:00
Joris Vink
00b64033ca More buf improvements.
- make sure kore_buf_alloc() and kore_buf_init() buffers behave
  the same way in regards to calling kore_buf_release() and what not.
2016-07-14 12:41:39 +02:00
Joris Vink
b28b60c2ff buf API changes.
- rename kore_buf_destroy() to kore_buf_cleanup().
- rename kore_buf_create() to kore_buf_alloc().
2016-07-14 12:33:13 +02:00
Joris Vink
7cf73b5fa5 Merge branch 'kore_buf_noalloc' of https://github.com/raphaelmonrouzeau/kore into raphaelmonrouzeau-kore_buf_noalloc 2016-07-14 12:28:51 +02:00
Joris Vink
33c0b3c753 Make pools thread-safe if built with TASKS.
Use CAS to implement primitive spinlock ontop of the pool
if we have been built with TASKS=1.

This allows tasks to safely use kore_malloc() and kore_free()
since those are now backed by pools.
2016-07-12 16:30:57 +02:00
Joris Vink
3c43d1203c Move towards size_t in the pool API. 2016-07-12 14:01:02 +02:00
Joris Vink
4ad50caa29 Large changes to the memory subsystem in kore.
- Change pools to use mmap() for allocating regions.
- Change kore_malloc() to use pools for commonly sized objects.
  (split into multiple of 2 buckets, starting at 8 bytes up to 8192).
- Rename kore_mem_free() to kore_free().

The preallocated pools will hold up to 128K of elements per block size.

In case a larger object is to be allocated kore_malloc() will use
malloc() instead.
2016-07-12 13:54:14 +02:00
Raphaël Monrouzeau
db3cf28d22 Added kore_buf no alloc functions.
kore_buf_create and kore_buf_free do use kore_malloc. But sometimes you
may embed a kore_buf inside a structure and would like a single way to
initialize / destroy it.
2016-07-09 12:30:24 +02:00
Joris Vink
d30921103b Code cleanup, several API breaking changes in here 2016-07-08 10:03:41 +02:00
Joris Vink
a8917e355a Add prototype for kore_onload(). 2016-07-07 13:18:41 +02:00
Joris Vink
39a5f21986 Allow "kore build" to produce single binaries.
Producing single binaries can now be done with building with
"kore build". To get started edit your build.conf and add the
following directives:

single_binary = yes
kore_source = /path/to/kore

optionally you can add kore_flavor to instruct how kore should
be built:

kore_flavor = NOTLS=1

When doing this your build.conf must also include the correct
linking options as the linking is now done fully by kore build.

The binary produced will include your configuration and takes
over a few of kore its command line flags (such as -f, -n or -r).
2016-07-06 16:16:15 +02:00
Joris Vink
27acc51241 Improve kore_strlcpy().
Make it return the original length of the input string so the caller
can check for truncation. Also guard against len being 0 as this would
not do anything with the destination string (not even NUL terminate it).
2016-07-04 11:41:37 +02:00
Joris Vink
ae31ec01ac Separate private keys from worker processes.
Kore will now isolate RSA private keys to a separate process (keymgr).

Worker processes that require RSA signing for TLS connections will
communicate with this keymgr process in order to do so.

This behaviour cannot be disabled and is always turned on.
2016-06-08 13:56:38 +02:00
Raphaël Monrouzeau
159de3960d Added new function kore_pgsql_v_query_params().
Same as kore_pgsql_query_params but takes a va_list as last parameter
(non-v version takes a variable list of parameters).

Lets people write easier to call wrappers around the query calls. I use
it in a wrapper that takes next states (error, current, continue) as
arguments in a handler with multiple async queries.
2016-06-07 13:12:31 +02:00
Joris Vink
2dfd22a79a Change kore_buf_stringify() a bit.
Takes a size_t pointer as its second argument now, if not
NULL this will be populated with the length of the string
that is being returned.
2016-06-02 07:08:19 +02:00
Matt Thompson
43a5703456 changed needle length type to size_t in kore_mem_find(). updated function prototype in kore.h. Use tab instead of space indentation 2016-04-01 17:54:10 +00:00
Ángel González
1bda217d57 Based on #115 pull request, adds http_method_text, similar to http_status_text 2016-03-21 15:12:27 +01:00
Joris Vink
0c0393fc64 constify 2016-02-13 14:33:06 +01:00
Joris Vink
fcc2010117 Make debug builds build again. 2016-02-13 14:20:31 +01:00
Daniel Fahlgren
be1c6f7155 Remove ; from end of do { ... } while(0) macro
With the ; the following types of usage doesn't work:

if (a == 1)
  COPY_ARG_TYPE(...);
else
  foo(...);
2016-02-12 08:56:10 +01:00
Joris Vink
40f69924bb Move shared file reading code to kore_read_line() 2016-02-01 22:10:10 +01:00
Joris Vink
c8484ee9ab Redo config parsing a bit.
No longer just call kore_string_split() on the line
but separate out the configuration directive and let
the appropriate callbacks parse things on their own.
2016-02-01 21:33:51 +01:00
Stig Telfer
7963a2deaa Added cleanup for the module handlers and validators 2016-02-01 12:13:22 +00:00
Stig Telfer
2ac6e7d41d Merge branch 'master' into oneswig 2016-01-24 13:46:38 +00:00
Stig Telfer
ec73c35952 Add missing C library headers 2016-01-22 11:08:13 +00:00
Joris Vink
fcb86ddb8b Massive rework of HTTP layer.
This commit is a flag day, your old modules will almost certainly
need to be updated in order to build properly with these changes.

Summary of changes:

- Offload HTTP bodies to disk if they are large (inspired by #100).
  (disabled by default)
- The http_argument_get* macros now takes an explicit http_request parameter.
- Kore will now throw 404 errors almost immediately after an HTTP request
  has come in instead of waiting until all data has arrived.

API changes:

- http_argument_get* macros now require an explicit http_request parameter.
  (no more magic invokations).
- http_generic_404() is gone
- http_populate_arguments() is gone
- http_body_bytes() is gone
- http_body_text() is gone
- http_body_read() has been added
- http_populate_post() has been added
- http_populate_get() has been added
- http_file_read() has been added
- http_file_rewind() has been added
- http_file_lookup() no longer takes name, fname, data and len parameters.
- http_file_lookup() now returns a struct http_file pointer.
- http_populate_multipart_form() no longer takes an secondary parameter.

New configuration options:

- http_body_disk_offload:
	Number of bytes after which Kore will offload the HTTP body to
	disk instead of retaining it in memory. If 0 this feature is
	disabled. (Default: 0)

- http_body_disk_path:
	The path where Kore will store temporary HTTP body files.
	(this directory does not get created if http_body_disk_offload is 0).

New example:

The upload example has been added, demonstrating how to deal with file
uploads from a multipart form.
2016-01-18 11:30:22 +01:00
Joris Vink
ce874f09ea Cut memory usage for incoming HTTP bodies in half.
Kore pre-allocates a kore_buf for the full size of the
incoming HTTP body ... but also was passing the full
size to the net_recv_reset() function.

Instead of this, properly read smaller chunks from the
network and append them to the body buffer as they roll in.
2016-01-08 20:56:21 +01:00
Joris Vink
0647901ef5 Improve http_body_max directive a bit.
Allow setting it to 0 which will disable HTTP requests
that have a body (POST/PUT).

Reduce default http_body_max to 1MB by default, 10MB seems large.

Revisit to this code inspired by #100.
2016-01-08 17:54:40 +01:00
Joris Vink
07ed037a00 Reduce memory footprint for NOTLS builds. 2016-01-07 09:20:09 +01:00
Stig Telfer
18d3cc032d rename *_fini to *_cleanup 2016-01-04 21:40:14 +00:00