326 Commits

Author SHA1 Message Date
Joris Vink
40c93e66a2 Add memory tagging.
This allows modules that have global pointers to upon reload repopulate
those with the addresses of when they were first created.

Meaning it now is easier to write modules that can be reloaded if those
modules kept global state one way or the other.

This should only be used at module init/reload time and is very simple:

if ((ptr = kore_mem_lookup(MY_ID_VALUE)) == NULL) {
	ptr = kore_malloc_tagged(length, MY_ID_VALUE);
	/* initialize for the first time. */
}

If we were in a reload the kore_mem_lookup() will return the original address
returned by the initial kore_malloc_tagged() call for MY_ID_VALUE.
2017-08-30 11:47:26 +02:00
Joris Vink
4893a030a6 small pgsql fixes.
- make sure conn_count per pgsqldb structure is initialized to 0.
- allow pgsql_conn_max to be 0, meaning just create a new connection
  if none was free.
2017-08-21 14:25:09 +02:00
Joris Vink
fcc55453c7 massively simplify base64 encoding. 2017-07-24 08:19:03 +02:00
Joris Vink
7f1a9b8092 Several postgresql improvements.
- Make pgsql_conn_count count per database rather then globally.
  This means you now define the number of clients *per* database registered
  rather then the number of clients in total of all databases.

- In case a connection is in failed transaction state Kore will now
  automatically rollback the transaction before placing that connection
  back in the connection pool.
2017-07-11 15:11:13 +02:00
Stanislav Yudin
b73343aea4 add HTTP_METHOD_OPTIONS as another supported http method. (#186) 2017-04-04 09:37:19 +02:00
Joris Vink
59f7e85f45 Decouple pgsql from the http layer.
When the pgsql layer was introduced it was tightly coupled with the
http layer in order to make async work fluently.

The time has come to split these up and follow the same method we
used for tasks, allowing either http requests to be tied to a pgsql
data structure or a simple callback function.

This also reworks the internal queueing of pgsql requests until
connections to the db are available again.

The following API functions were changes:
	- kore_pgsql_query_init() -> kore_pgsql_setup()
		no longer takes an http_request parameter.
	- NEW kore_pgsql_init()
		must be called before operating on an kore_pgsql structure.
	- NEW kore_pgsql_bind_request()
		binds an http_request to a kore_pgsql data structure.
	- NEW kore_pgsql_bind_callback()
		binds a callback to a kore_pgsql data structure.

With all of this you can now build kore with PGSQL=1 NOHTTP=1.

The pgsql/ example has been updated to reflect these changes and
new features.
2017-03-24 12:53:07 +01:00
Joris Vink
add6d724e3 expose connection address to python. 2017-03-24 12:42:40 +01:00
Joris Vink
ed9a34ce95 cleanup net layer a bit.
this change was long overdue and finally gets rid of the legacy crap
we had laying around to appease openssl's read/write functions.
2017-03-16 09:54:46 +01:00
Joris Vink
ffbabed7d6 Move websocket_send() to pyconnection.
While here convert all instances of y* for incoming binary data to y#
since we won't be changing it at all and avoids some unneeded copying.
2017-03-13 13:30:57 +01:00
Joris Vink
0b84a9c1d4 expose c.disconnect() and c.fd to Python. 2017-03-13 12:22:50 +01:00
Joris Vink
ec901d0339 Make http_body_rewind() public.
Also let this function reset offset and lengths for http_body_read().

Make sure of this function in the python code so req.body can be called
multiple times in succession.
2017-03-13 11:17:55 +01:00
Joris Vink
3ae9bb7ae9 change type of maxage. 2017-03-10 14:36:51 +01:00
Joris Vink
4db51d7846 screw it, rework interface for cookies.
The only reason you would want to directly modify the cookie
after creating it should be to unset the HTTPONLY or SECURE flags
if that is what you *really* want to do.

Change http_response_cookie() to take all required parameters instead
of having to marshall those in yourself after.

Now you set a sane default cookie in one shot:

http_response_cookie(req, "key", "value", "/", 0, -1, NULL);

Which would create a session cookie key=value for / under the current domain.
2017-03-10 14:31:08 +01:00
Joris Vink
3d24b65268 Change default http_cookie behaviour.
We now default to httponly & secure for newly created cookies.

This should've been the default all along.

The http_response_cookie() no longer returns a pointer to http_cookie
but rather takes it as a parameter and will populate the pointer with
the newly created http_cookie if not NULL.

Additionally http_response_cookie() automatically sets the domain
based on the http_request passed into the function.
2017-03-10 14:20:40 +01:00
Joris Vink
f57ca7dcc2 Let workers fetch entropy from keymgr.
At bootup and every 1800 seconds after that the worker processes will
ask the keymgr for new entropy that they will seed into their PRNG.

Additionally once received the worker calls RAND_poll() to grab
more entropy from the system to be mixed in.
2017-02-27 21:28:35 -08:00
Joris Vink
6ba7390cc9 Add rand_file configuration option for keymgr.
This option allows the user to specify a file to be used for
seeding the PRNG initially and to write random bytes at exit.

The option is only available if kore has TLS enabled (by default).

If you enable this option Kore will refuse to start if there is
a problem with the file specified (not found, not a file, invalid size, etc).

While here let the keymgr process call RAND_poll() every half hour
to grab more system entropy and seed it into the PRNG.
2017-02-27 20:58:04 -08:00
Joris Vink
7f11e37161 Add kore_sockopt().
Use it where we before were using setsockopt().
2017-02-22 13:23:30 -08:00
Joris Vink
c6ca68f3f2 export http_body_path via python 2017-02-22 17:52:57 +01:00
Joris Vink
fc6b3bf740 Split up kore cli tools into new binary.
Having the create, build, run tools baked into the kore binary
made things harder then they had to be for multiple projects with
each different build flavors.

So move away this functionality into a new "kodev" (name may change)
binary that is installed next to kore.

The new build tools will automatically pick up the correct flavors
the kore binary it points to is installed with. Or for single builds
what flavors where enabled.

The new tool also will honor looking into PREFIX for the kore binary
when doing a `kodev run`.

Additionally add a new command "info" that shows some basic info
about your project and how it will be built. For example it will
show you the flavors of the kore binary installed on the system
or the flavors you configured for a single binary build.

Obligitory, hacking on a plane comment.
2017-02-19 00:52:29 -05:00
Joris Vink
a0c545884f even more ssl to tls terminology. 2017-02-11 21:33:09 +01:00
Joris Vink
98f0c41f38 more ssl -> tls wording 2017-02-11 21:30:38 +01:00
Joris Vink
06ae4644e0 spacing & style 2017-02-07 23:04:24 +01:00
Joris Vink
e9b4f966c2 expose new cookie stuff to python.
req.populate_cookies()
value = req.cookie("name")
2017-02-07 22:54:42 +01:00
Joris Vink
8b9ea825eb cookie improvements.
- split up writing of cookies into its own function.
- turn maxage into a signed int and use -1 for it not being set.
- lots of style fixes
- remove HTTP_COOKIE_DEFAULT, just pass 0 if you don't want flags.
2017-02-07 22:03:06 +01:00
Stanislav Yudin
f4ac8c2955 Cookies and arguments parsing improvements (#166)
Add new cookie API for handling of cookies.
2017-02-07 21:49:10 +01:00
Joris Vink
ace8c4e80c Add asynchronous pgsql query support to python.
This commit adds the ability to use python "await" to suspend
execution of your page handler until the query sent to postgresql
has returned a result.

This is built upon the existing asynchrous query framework Kore had.

With this you can now write stuff like:

async def page(req):
	result = await req.pgsql("db", "SELECT name FROM table");
	req.response(200, json.dumps(result).encode("utf-8"))

The above code will fire off a query and suspend itself so Kore can
take care of business as usual until the query is successful at which
point Kore will jump back into the handler and resume.

This does not use threading, it's purely based on Python's excellent
coroutines and generators and Kore its built-in pgsql support.
2017-02-06 11:42:53 +01:00
Joris Vink
de7a6d4855 pgsql improvements.
- adds new cleanup function that workers will call.
- adds kore_pgsql_nfields() to return number of fields in result.
- add kore_pgsql_fieldname() to return name of a given field.

This commit also changes the behaviour of pgsql_conn_release() in
that it will now cancel the active query before releasing the connection.

This makes sure that if long running queries are active they are hopefully
cancelled if an http request is removed while such queries are still running.
2017-02-06 11:40:33 +01:00
Joris Vink
76cb532de9 mark kore_log with printf attribute.
'bout time.
2017-02-06 11:39:30 +01:00
Joris Vink
6578fc6589 expose http_file interface to python. 2017-02-01 21:23:56 +01:00
Joris Vink
3d8e0dabc0 expose kore_server_bind() and fatal() to python 2017-02-01 17:12:52 +01:00
Joris Vink
0c0a9371bd Change kore_preload() and kore_onload().
Renamed both of them:
	kore_preload -> kore_parent_configure
	kore_onload  -> kore_worker_configure

These functions will now always be called if they are defined in any module
regardless of your application being built as a single binary or not.
2017-02-01 17:12:11 +01:00
Joris Vink
8b26ecdf14 more websocket improvements.
- send a CLOSE opcode if we haven't yet when removing a ws client.
- ignore unsolicited PONG.
2017-01-30 22:21:57 +01:00
Joris Vink
d5a32c2dab use the correct opcode for PONG.
djees i can't hex.
2017-01-30 22:17:47 +01:00
Joris Vink
30666618f8 expose kore_websocket_send() to python. 2017-01-30 22:00:03 +01:00
Joris Vink
7510834968 initial fudging to add websockets to python 2017-01-29 22:57:34 +01:00
Joris Vink
1ecb777d41 only remove the pgsql if it was scheduled.
fixes #172.
2017-01-29 10:34:53 +01:00
Joris Vink
c06ecf1c70 Teach single binaries about SIGHUP.
Make sure kore_onload() and kore_preload() can be called
either in native or python runtime.
2017-01-26 13:26:55 +01:00
Joris Vink
dcb5fd842f Expose get/post params via req.argument for python. 2017-01-26 12:01:28 +01:00
Joris Vink
da13e1e768 undef _POSIX_C_SOURCE and _XOPEN_SOURCE before Python.h 2017-01-25 22:41:57 +01:00
Joris Vink
f56938283d Merge branch 'master' of github.com:jorisvink/kore 2017-01-25 22:23:34 +01:00
Joris Vink
620cc231dc validators now work in python. 2017-01-25 21:17:12 +01:00
Joris Vink
bbcdec82fc Add initial python support.
Based on work done by Stanislav Yudin.
2017-01-24 20:18:12 +01:00
Yorick de Wid
86d85dd10b Message shutdown request (#168)
* Set server version

* Message shutdown request

* Update messaging example

* Set server verion in generic example

* Allow shutdown with NOHTTP=1
2017-01-17 18:18:01 +01:00
Yorick de Wid
565bf5bdec Set server version (#157) 2017-01-13 13:03:47 +01:00
Manuel Kniep
ec5e93010f add missing includes (#163)
to make includes order independent
2017-01-07 23:07:34 +01:00
Joris Vink
31d14d028a add compile-time configurable mime types.
these types are used for the new builtin asset_serve_* functions.

must be defined in conf/build.conf.
2016-12-26 23:37:05 +01:00
Joris Vink
d783a1d22d Add auto generated serving functions for assets.
These functions are created by the cli tool when building
and follow the naming format: asset_serve_<name>_<ext>().

Those serving functions can be used directly in handlers and
callthrough to a http_serveable() function that uses the SHA1
of the asset as its ETag and automatically checks for if-none-match.
2016-12-26 21:15:03 +01:00
Tobias Kortkamp
7eced6f035 Fix #153 (#155)
Compiling with Clang 3.9 on FreeBSD raises the following error (see [1]):
```
src/pgsql.c:222:17: error: passing an object that undergoes default argument promotion to 'va_start'
      has undefined behavior [-Werror,-Wvarargs]
        va_start(args, count);
                       ^
src/pgsql.c:217:45: note: parameter of type 'u_int8_t' (aka 'unsigned char') is declared here
    const char *query, int result, u_int8_t count, ...)
```
More information about this warning can be found on [2].

To solve this we can change the type of `count` to `int` in
`kore_pgsql_query_params()` and `kore_pgsql_v_query_params()`. This
also matches the signatures of both `PQexecParams()` [3] and
`PQsendQueryParams()` [4].

[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=214639
[2] https://www.securecoding.cert.org/confluence/display/cplusplus/EXP58-CPP.+Pass+an+object+of+the+correct+type+to+va_start
[3] https://www.postgresql.org/docs/9.2/static/libpq-exec.html
[4] https://www.postgresql.org/docs/9.2/static/libpq-async.html
2016-11-19 12:56:51 +01:00
Joris Vink
5b379a9185 add a kore_preload() call for single binaries.
This is run on the parent before workers are spawned.
2016-11-17 10:05:16 +01:00
Joris Vink
552d59248c constify delim in kore_split_string(). 2016-10-06 16:50:41 +02:00