[build] Trim down Dockerfile

Remove the unnecessary packages and the ones that are already installed
as part of dependencies and meta-packages.

Also compile whatever we explicitly link against.