Context
-------
The overall goal of this commit is to refactor the external plugins
implementation, with the following goals in mind:
- Make plugin server code more approachable to unfamiliar engineers and
easier to evolve with confidence
- Harden configuration; ensure configuration defects are caught before
Kong is started
- Extend testing coverage
This is related to ongoing work on the Go PDK, with similar goals in
mind.
Summary
-------
This commit implements the following overall changes to plugin server
code:
- Move configuration related code into conf loader, so that configuration
loading and validation happens at startup time, rather than lazily, when
plugin data is loaded or pluginservers are started. Add tests for
current behavior.
- Move process-management code - for starting up plugin servers as well
as querying external plugins info - into the `process.lua` module.
- Introduce a `kong.runloop.plugin_servers.rpc` module that encapsulates
RPC initialization and protocol-specific implementations. This further
simplifies the main plugin server main module.
- Factor exposed API and phase handlers bridging code into a new `plugin`
module, which encapsulates an external plugin representation, including
the expected fields for any Kong plugin, plus external plugin-specific
bits, such as the RPC instance. Part of this external plugin-specific part
is the instance life cycle management. With this structure, the `kong.runloop.plugin_servers`
main module contains only general external plugin code, including a list
of loaded external plugins, and associated start/stop functions for
plugin servers.
Testing
-------
This commit also implements the following improvements to tests:
- Restructure fixtures to accommodate new external plugin servers --
namely, targeting for now in the existing Python and Javascript
- Add new test cases for external plugins:
* External plugin configuration: add test cases for current behavior;
in particular:
- Fail if no `query_cmd` is provided;
- Warn if no `start_cmd` is provided - this is by design, as
external plugins servers can be managed outside of Kong
* Plugin server start / stop - for both Go and Python plugins
* External plugin info querying for both Go and Python plugins
* External plugin execution - for both Go and Python plugins
Internal flow
-------------
`.plugin_servers.init:` loads all external plugins, by calling .plugin_servers.process and `.plugin_servers.plugin`
`.plugin_servers.process`: queries external plugins info with the command specified in `_query_cmd` proeprties
`.plugin_servers.plugin`: with info obtained as described above, `.plugin:new` returns a kong-compatible representation
of an external plugin, with phase handlers, PRIORITY, and wrappers to the PDK. Calls
`.plugin_servers.rpc` to create an RPC through which Kong communicates with the plugin process
`.plugin_servers.rpc`: based on info contained in the plugin (protocol field), creates the correct RPC for the
given external plugin
`.plugin_servers.rpc.pb_rpc`: protobuf rpc implementation - used by Golang
`.plugin_servers.rpc.mp.rpc`: messagepack rpc implementation - used by JS and Python
`.plugin_servers.init`: calls `.plugin_servers.process` to start external plugin servers
`.plugin_servers.process`: optionally starts all external plugin servers (if a `_start_cmd` is found)
uses the resty pipe API to manage the external plugin process
* Begin removing gojira from upgrade tests
* Can compare two numbered versions
* Rework upgrade tests without gojira
* trigger test
* remove testing branch spec
* Address review comments
* Remove symbolic link that is not needed after all
* remove test trigger
* chore(prerelease) ability for us to release prerelease builds of Kong
* chore(release) merge the jenkins daily release with release stages
* chore(prerelease) commit the prerelease script [WIP]
* chore(automation) remove uneeded steps for prerelease
* chore(release) adjust alpha release script per team feedback
* chore(ci) adjustments so the Makeifle logic is more approachable
* chore(ci) use kong-build-tools to build, package and release Kong
* misc(travis) testing -- setting up travis-ci to build daily images using kong-build-tools
* misc(travis) testing -- setting up travis-ci to build daily images using kong-build-tools
* misc(travis) testing -- setting up travis-ci to build daily images using kong-build-tools
* misc(travis) testing -- setting up travis-ci to build daily images using kong-build-tools
* misc(travis) testing -- setting up travis-ci to build daily images using kong-build-tools
* misc(travis) testing -- setting up travis-ci to build daily images using kong-build-tools
* misc(travis) testing -- setting up travis-ci to build daily images using kong-build-tools
* misc(travis) testing -- setting up travis-ci to build daily images using kong-build-tools
* misc(travis) testing -- setting up travis-ci to build daily images using kong-build-tools
* misc(travis) testing -- setting up travis-ci to build daily images using kong-build-tools
* fix(release) let .travis.yml set the kong version
* fix(ci) adjust how default vs ENV overrides works in the Makefile
* feat(travis) have travis build additional package types
* feat(travis) have travis build additional package types
* feat(travis) have travis build additional package types
* feat(travis) have travis build additional package types
* feat(travis) have travis-ci build the alpine apk package
* fix(travis) need the if statement on every deploy task
* fix(travis) need the if statement on every deploy task
* fix(release) use current working directory name to determine package name
* misc(ci) make the daily build only run on cron tasks
* misc(openssl) bump openssl to 1.1.1a
* fix(ci) override bintray naming convention on daily releases
* chore(ci) pin the kong-build-tools version
* feat(ci) remove previous daily build process
* feat(ci) adjust daily build process
* chore(ci) bump the kong-build-tools version
* fix(build) we dont need java in our CI anymore
