1796 Commits

Author SHA1 Message Date
Joris Vink
2576427dc0 Simplify if a worker needs to be added to msg.
Just look if the kore_worker data structure its ps member is non NULL.
If it is, it was started by the parent process at some point.
2021-09-14 15:45:34 +02:00
Joris Vink
a2d48feeb7 Lets make sure milliseconds are formatted nicely. 2021-09-14 09:36:33 +02:00
Joris Vink
41511c1683 Log timestamps in UTC for, add milliseconds.
This is when using the normal foreground logs or a specified logfile.
2021-09-14 09:30:17 +02:00
Joris Vink
b6ec4081d5 Oops, remove debug. 2021-09-13 22:55:09 +02:00
Joris Vink
824d6421d5 Use correct format for strftime(). 2021-09-13 22:54:05 +02:00
Joris Vink
983f5a03f5 Initial mem and log earlier.
Kill the kodev mention.
2021-09-13 15:33:42 +02:00
Joris Vink
450aabbea1 Add timestamp prefix to log when not using syslog. 2021-09-13 15:07:43 +02:00
Joris Vink
77848e0708 Always use logfile when set, even if !foreground. 2021-09-12 15:19:37 +02:00
Joris Vink
4b4767ac39 Missed committing http.h from previous commit. 2021-09-12 15:12:48 +02:00
Joris Vink
862bf1a5f6 Add http_response_json().
Since its an HTTP response function it functions like http_response() but
takes a kore_json_item pointer that it will automatically convert to a kore_buf
and send/free using http_response_stream().

While here fix a problem with http_response_stream() which could end up
not calling the cb() in case of HTTP_METHOD_HEAD. Since the behaviour is
that it should call cb() when done it should do so immediately.
2021-09-12 15:10:06 +02:00
Joris Vink
cdd681d602 Let http_response_header() handle duplicates.
If a response header was previously by an application for an HTTP request
http_response_header() will now overwrite the previous value.
2021-09-12 14:30:33 +02:00
Joris Vink
eb0b8f21e3 Add http_response_close() to the C API.
This is the same as http_response() except it will automatically
close the connection after the response is sent.

This is a bit easier than setting CONN_CLOSE_EMPTY yourself manually.
2021-09-12 14:13:24 +02:00
Joris Vink
b77d727f72 Add a logfile configuration option.
This will log all output from Kore processes to the specified file.
2021-09-10 13:34:57 +02:00
Joris Vink
ff6bae6513 move startup log back into kore_server_start(). 2021-09-07 23:26:36 +02:00
Joris Vink
a6677b873f On linux, keep track of seccomp tracing properly.
With the new process startup code we must handle the SIGSTOP
from the processes if seccomp_tracing is enabled. Otherwise
they just hang indefinitely and we assume they failed to start,
which is somewhat true.
2021-09-07 23:05:25 +02:00
Joris Vink
1af0772ebe Don't add acme to msg framework if not needed. 2021-09-07 22:27:02 +02:00
Joris Vink
116f935e10 use the correct name for acme. 2021-09-07 22:19:21 +02:00
Joris Vink
9fd30db598 Change timeout for worker startup a bit.
Also give some feedback we are waiting for process startup.
2021-09-07 22:14:28 +02:00
Joris Vink
bbae4be6f1 remove unused kore_progname. 2021-09-07 22:06:18 +02:00
Joris Vink
3b20cda11c Rework worker startup/privsep config.
Starting with the privsep config, this commit changes the following:

- Removes the root, runas, keymgr_root, keymgr_runas, acme_root and
  acme_runas configuration options.

  Instead these are now configured via a privsep configuration context:

  privsep worker {
      root /tmp
      runas nobody
  }

  This is also configurable via Python using the new kore.privsep() method:

      kore.privsep("worker", root="/tmp", runas="nobody", skip=["chroot"])

Tied into this we also better handle worker startup:

- Per worker process, wait until it signalled it is ready.
- If a worker fails at startup, display its last log lines more clearly.
- Don't start acme process if no domain requires acme.
- Remove each process its individual startup log message in favour
  of a generalized one that displays its PID, root and user.
- At startup, log the kore version and built-ins in a nicer way.
- The worker processes now check things they need to start running
  before signaling they are ready (such as access to CA certs for
  TLS client authentication).
2021-09-07 21:59:22 +02:00
Joris Vink
1c33ce01d0 Add kore_build_date to version.c 2021-09-07 21:58:53 +02:00
Joris Vink
c8c9a24d99 Only set initial python deployment if needed.
We should only be setting this if an actual module was specified
on the command-line that will be loaded.
2021-09-07 21:15:17 +02:00
Joris Vink
599835e7fd Python: Only use parameters if needed.
We always called kore_pgsql_query_param_fields() regardless if the
params keyword was specified or not, instead only use it if actual
parameters have been given.

Otherwise use the kore_pgsql_query() function directly to execute the query.
2021-09-06 15:39:38 +02:00
Joris Vink
0ac54eb48d Add a kore.config.skipchroot to the Python API.
If set to True, will skip the chroot() of the Kore workers.

This can be handy in case you want to set your deployment target to
production or docker so you get user changes but you don't want
to chroot the processes.
2021-09-06 14:35:04 +02:00
Joris Vink
00ef837d62 call explicit fflush() on stdout 2021-09-06 14:16:09 +02:00
Joris Vink
06991d22d5 remove norwegian debug 2021-09-06 13:40:33 +02:00
Joris Vink
7f56c7dbf2 Change how worker processes do logging.
Before each worker process would either directly print to stdout if
Kore was running in foreground mode, or syslog otherwise.

With this commit the workers will submit their log messages to the
parent process who will either put it onto stdout or syslog.

This change in completely under the hood and users shouldn't care about it.
2021-09-06 13:28:38 +02:00
Joris Vink
8661aee2f4 only log key info if !kore_quiet 2021-09-06 13:26:54 +02:00
Joris Vink
ebe090833d add Matthew to BEERS 2021-09-05 18:04:54 +02:00
Joris Vink
5f11f796a8 Allow configuration to pickup values from environment.
Eg:

certfile $CERTFILE

will pickup the value from the set $CERTFILE environment variable.

This works for _any_ Kore configuration option.
2021-09-05 17:53:09 +02:00
Joris Vink
3c4acd9ac3 Allow curlopt keyword at httpclient request level.
Now you can set curlopt on kore.httpclient at both the
global httpclient object level and individual requests.

Eg:

client = kore.httpclient("https://kore.io",
    curlopt={
        kore.CURLOPT_VERBOSE: 1
    }
)

status, body = await client.get(
    curlopt={
        kore.CURLOPT_VERBOSE: 0
    }
)
2021-08-27 10:42:40 +02:00
Joris Vink
3eff4b9790 whitespace fixes 2021-08-27 10:12:11 +02:00
Joris Vink
355cf87b93 use correct format specifier. 2021-08-27 10:06:45 +02:00
Joris Vink
55aaef875d Add support for setting curlopts in kore.httpclient.
Much of the work done by Matthew Norström with minor cleanup by me.
2021-08-27 10:05:30 +02:00
Joris Vink
89085246e5 style nits 2021-07-10 10:03:01 +02:00
Joris Vink
6d7c774ba2 remove support for older openssl apis in keymgr. 2021-07-10 10:02:53 +02:00
Joris Vink
90056dbdcb make python_module_init() non static 2021-07-10 10:02:46 +02:00
Joris Vink
95139925ec Add query string support to the Python validator API.
Now you can specify the qs keyword in a route which can contain
validators for the query string.

Eg:

@kore.route("/", methods=["post"], qs={"id": "^[0-9]+$"})
def index:
    ...
2021-06-18 13:00:57 +02:00
Joris Vink
f1a65ef236 Small improvement to the Python kore.timer() api.
Do not allow kore.timer() to be called from the parent process
as it shouldn't be run there.

This makes Kore fail more gracefully.
2021-06-03 14:03:45 +02:00
Joris Vink
f39919e98c Add readlinkat to seccomp whitelist. 2021-05-10 23:05:58 +02:00
Joris Vink
d4a78de5fc Add madvise to keymgr seccomp rules. 2021-05-10 14:51:30 +02:00
Joris Vink
12909b98bb be explicit 2021-05-10 10:32:54 +02:00
Joris Vink
61c06291b6 Add a docker python kore.config.deployment setting.
This keeps kore in the foreground will still doing privsep.
Useful with upcoming official kore docker images.
2021-05-10 10:27:32 +02:00
Joris Vink
0abc9b19ff simplify the python-async example 2021-05-02 16:25:46 +02:00
Joris Vink
66dd856bdc and add the echo.py file of course.. 2021-05-02 16:08:32 +02:00
Joris Vink
5e84ebdab2 Simplify the echo example, kore can just run the file nowadays. 2021-05-02 16:07:41 +02:00
Frederic Cambus
582e18d2ec Stop hardcoding HTTP error codes in http_response() calls.
Use predefined HTTP_STATUS_* macros instead.
2021-05-02 16:03:18 +02:00
Joris Vink
fb335e1e0c Major Python API improvements.
1) Add @kore.route as a decorator for Python.

This decorator can be used on non-class methods to automatically
declare their route and parameters.

Takes the same arguments as the kore.domain.route function that
exists today.

Provides a nice clean way of setting up Kore if you dont want
a whole class based approach.

2) Remove the requirement for the name for kore.server() and the
kore.domain(attach=) keywords.

Instead of no name was given, the name "default" is used in both
places resulting in less boilerplating.

3) Allow multiple routes to be defined for the same URI as long
as the methods are different. So you can have one method for GET /
and another for POST /.

All changes combined condense the initial experience of getting
a Kore Python app up and running:

eg:

import kore

kore.server(ip="127.0.0.1", port="8888", tls=False)
kore.domain("*")

@kore.route("/", methods=["get"])
async def index(req):
    req.response(200, b'get method')

@kore.route("/", methods=["post"])
async def index_post(req)
    req.response(200, b'post method')
2021-05-02 00:32:47 +02:00
Joris Vink
c4a60c54bb resolve tls_dhparam after configure. 2021-04-21 22:39:35 +02:00
Joris Vink
c77ec598e7 Remove dh parameters from kodev. 2021-04-21 10:52:02 +02:00