Before kore needed to be built with NOTLS=1 to be able to do non TLS
connections. This has been like this for years.
It is time to allow non TLS listeners without having to rebuild Kore.
This commit changes your configuration format and will break existing
applications their config.
Configurations now get listener {} contexts:
listen default {
bind 127.0.0.1 8888
}
The above will create a listener on 127.0.0.1, port 8888 that will serve
TLS (still the default).
If you want to turn off TLS on that listener, specify "tls no" in that
context.
Domains now need to be attached to a listener:
Eg:
domain * {
attach default
}
For the Python API this kills kore.bind(), and kore.bind_unix(). They are
replaced with:
kore.listen("name", ip=None, port=None, path=None, tls=True).
- Kore can now fully be configured via Python code if one wants nothing to
do with configuration files.
- Kore can now start single python files and no longer requires them to be
inside a module directory.
- Pass all regex capture groups to the handler methods, allowing you to
get access to them immediately.
- Change python websocket_handshake to take callable objects directly.
- Added a new deployment configuration option. If set to "dev" or
"development" Kore will automatically foreground, no chroot / etc.
If set to "production" Kore *will* chroot, drop privs, etc.
- Many more..
These are all backported from a project that I was working on a while
ago. I decided these should go back into mainline Kore.
The HTTP layer used to make a copy of each incoming header and its
value for a request. Stop doing that and make HTTP headers zero-copy
all across the board.
This change comes with some api function changes, notably the
http_request_header() function which now takes a const char ** rather
than a char ** out pointer.
This commit also constifies several members of http_request, beware.
Additional rework how the worker processes deal with the accept lock.
Before:
if a worker held the accept lock and it accepted a new connection
it would release the lock for others and back off for 500ms before
attempting to grab the lock again.
This approach worked but under high load this starts becoming obvious.
Now:
- workers not holding the accept lock and not having any connections
will wait less long before returning from kore_platform_event_wait().
- workers not holding the accept lock will no longer blindly wait
an arbitrary amount in kore_platform_event_wait() but will look
at how long until the next lock grab is and base their timeout
on that.
- if a worker its next_lock timeout is up and failed to grab the
lock it will try again in half the time again.
- the worker process holding the lock will when releasing the lock
double check if it still has space for newer connections, if it does
it will keep the lock until it is full. This prevents the lock from
bouncing between several non busy worker processes all the time.
Additional fixes:
- Reduce the number of times we check the timeout list, only do it twice
per second rather then every event tick.
- Fix solo worker count for TLS (we actually hold two processes, not one).
- Make sure we don't accidentally miscalculate the idle time causing new
connections under heavy load to instantly drop.
- Swap from gettimeofday() to clock_gettime() now that MacOS caught up.
doing this allows us to get rid of the validator reload
and handler reload as well as fixing websocket runtime
callbacks which were never being resolved upon module reloads.
- Change pools to use mmap() for allocating regions.
- Change kore_malloc() to use pools for commonly sized objects.
(split into multiple of 2 buckets, starting at 8 bytes up to 8192).
- Rename kore_mem_free() to kore_free().
The preallocated pools will hold up to 128K of elements per block size.
In case a larger object is to be allocated kore_malloc() will use
malloc() instead.
Producing single binaries can now be done with building with
"kore build". To get started edit your build.conf and add the
following directives:
single_binary = yes
kore_source = /path/to/kore
optionally you can add kore_flavor to instruct how kore should
be built:
kore_flavor = NOTLS=1
When doing this your build.conf must also include the correct
linking options as the linking is now done fully by kore build.
The binary produced will include your configuration and takes
over a few of kore its command line flags (such as -f, -n or -r).
Kore will now isolate RSA private keys to a separate process (keymgr).
Worker processes that require RSA signing for TLS connections will
communicate with this keymgr process in order to do so.
This behaviour cannot be disabled and is always turned on.
- Build with -O2 unless NOOPT is set to 1.
- Hide -g behind DEBUG instead of always building with it.
- Explicitely set the standard used to c99, use pedantic.
This basically turns off the HTTP layer for Kore. It does not
compile in anything for HTTP.
This allows Kore to be used as a network application platform as well.
Added an example for this called nohttp.
Other changes that sneaked in while hacking on this:
* Use calloc(), kill pendantic malloc option.
* Killed off SPDY/3.1 support completely, will be superseded by http2
Note that comes with massive changes to a lot of the core API
functions provided by Kore, these might break your application.
At times it seems relevant that worker their modules should not
be reloaded when receiving a SIGHUP. Developers can now control
this by returning anything else but KORE_RESULT_OK from their
initialization methods.
The parent module will always be reloaded.
After revisiting why this exists in Kore I decided it
does not belong in this platform and instead of letting
it sit there staring at me I rather just kill it.
Using authentication blocks one can define "authentication" mechanisms
in Kore for page handlers.
This can be used to require a session cookie (validated by your own validator)
for certain page handlers, and hopefully in the future provide a framework
for adding more authentication things (like HTTP Auth).
Right now only cookie checking is available.
Example:
validator v_id function v_id_function
validator v_url regex ^/url/path/[a-z]*$
You can then call these using kore_validator_run(char *, char *), example:
if (!kore_validator_run("v_url", req->path))
[req->path is bad];
This is useful to track down any issues you might have in your module.
A log entry with a page handler causing issues looks like:
Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (18193)-> status 11
Jul 7 14:44:30 devbook kore[18191]: [parent]: worker 1 (pid: 18193) (hdlr: 0x242d9c0) gone
Jul 7 14:44:30 devbook kore[18191]: [parent]: hdlr serve_intro has caused 2 error(s)
- Introduce own memory management system on top of malloc to keep track
of all our allocations and free's. Later we should introduce a pooling
mechanism for fixed size allocations (http_request comes to mind).
- Introduce ssl_cipher in configuration.
Memory usage is kind of high right now, but it seems its OpenSSL
doing it rather then Kore.
