Skip to content

0.15.1 / 2019-05-14

Latest
Latest
Compare
Choose a tag to compare
@GUI GUI released this 14 May 06:05
v0.15.1
8df1f6d
This commit was signed with the committer’s verified signature. The key has expired.
GUI Nick Muerdter
GPG key ID: 7C517C5A1A14988A
Expired
Learn about vigilant mode.

👋 Long time no release! Sorry for the long gap since our last formal release, but we have a sizable upgrade ready that fixes various bugs, and makes a lot of internal improvements. Upgrading is recommended.

Upgrade Instructions

If you're upgrading a previous API Umbrella version, you may upgrade the api-umbrella package using your package manager.

Fixed

  • Fix filtering admin analytics on the "HTTP Method" field: Analytics filters for the "HTTP Method" field were not working. (api.data.gov#401, #389)
  • Fix admins without admin management permissions accessing their own account page: If an admin account didn't also have "Admin Accounts - View & Manage" permissions, the admin wasn't able to access their own admin account page. (api.data.gov#451, api.data.gov#443)
  • Fix admin navigation links not hiding based on admin account permissions: Admin accounts with limited permissions were still being shown links to all the possible admin pages, even if they didn't have permissions to those pages. This is fixed so there are only navigation links to the permitted admin areas now. (api.data.gov#432, api.data.gov#394)
  • Fix admins with limited permissions not able to publish website backends: Only superuser admins were able to publish website backend changes. (9091de9, 0356c6b)
  • Fix CSV download for admin drilldown analytics: The CSV download link in the API Drilldown part of the admin analytics wasn't working. (api.data.gov#410)
  • Fix missing column headers in admin analytics "Filter Logs" CSVs: Some of the last columns of data in this CSV were missing the associated column headers. (api.data.gov#480)
  • Fix out-of-memory issues potentially leading to outage: In the event the API backend configuration exceeds the allocated memory for this configuration in nginx (configured via nginx.shared_dicts.active_config.size), the API backend configuration could become unloaded leading to an API outage. This is now fixed so that the new API backend configuration will only get published if there's enough available memory (otherwise, the old configuration will remain in place, and a warning will be logged). The default memory size for this configuration has also been increased to allow for 750-1000 API backends by default (up from the previous default allowing 150-300 API backends). (cb5e2c1, 3af5700, api.data.gov#385)
  • Fix URL handling for query strings containing "api_key": It was possible that API Umbrella was stripping the string "api_key" from inside URLs before passing requests to the API backend in some unexpected cases. The api_key query parameter should still be stripped, but other instances of "api_key" elsewhere in the URL (for example as a value, like ?foo=api_key), are now retained. (de3e207)
  • Fix behavior of drilldown chart in admin analtyics: The behavior of the drilldown chart in the analtyics area could sporadically be incorrect and render the wrong data in the chart. (api.data.gov#433)
  • Fix redirect rewriting from API backends: When an API backend returns a redirect, there were some situations where the rewritten redirect would be incorrect (if API Umbrella was running on a custom HTTP or HTTPS port, or in situations where the API backend has multiple URL prefix matches, or if the API backend returns an already rewritten path). (735212b, 4d5cc3f)
  • Fix configuration settings to extend the default HTTP timeout: Fix the nginx.proxy_read_timeout and nginx.proxy_connect_timeout settings for use with API backends that are slower to respond. (#441, 17bc65c)
  • Fix empty 404 and 500 error pages served from web-app: If the web-app returned 404 or 500 errors, these were returned with an empty response body in v0.14.0+. (a6fb68e)
  • Fix memory leaks: Remove background task to periodically reload nginx due to unexpected memory growth which is now fixed. (09b3f74)
  • Fix admin logouts when API Umbrella is restarted: The randomized secret token used for session encryption could be regenerated on API Umbrella restarts, which could lead to admins needing to login again. (c65ea2f, f88a2c0)
  • Fix admin analytics when no indices for the date range are present: If querying the analytics for date ranges where no analytics indices were present, ensure that the API still responds successfully (with 0 values). (c743e79)
  • Fix nginx warnings: Fix warnings generated in the nginx log files. (04e8c9c, 08b59e7)
  • Fix edge case with seeded API keys having the same key: It was possible that the API keys created during startup for internal usage could end up having duplicate, colliding API key values. This likely only affected the test environment when repeated, rapid reload were performed. (a725342, 8fd99e3)
  • Fix edge cases to handle MongoDB replicaset changes more gracefully: Better handle errors during MongoDB replicaset changes to retry queries. (a808feb)
  • Improve keepalive handling: Fix possibility of 502 Bad Gateway responses in cases where an API backend closes a keepalive connection to API Umbrella. (833e3de, api.data.gov#446)
  • Fix edge case with rapid reloads causing config data to go missing: If rapidly reloading the API Umbrella process, the config could go missing. This likely only affected our test suite which performs rapid reloads. (e274d86)

Security

  • Prevent API URLs and contact URLs from linking to unknown domains in API key signup e-mails: Someone could trigger an API key signup e-mail to a user with links to unexpected locations for the example API URL or "contact us" link. Thanks to @nuke11 for the bug bounty report. (api.data.gov#460)
  • XSS issue in flash error messages from external login providers: Error messages from external login providers (eg, Google) could contain a cross-site scripting (XSS) vulnerability. (469572c)
  • Prevent admin groups from having analytics permissions: If an admin account belonged to only admin groups that didn't have any analytics permissions, then it was possible they admin could inadvertently view all analytics data. (a4569a6)

Added

  • Added packages for Ubuntu 18.04 and Debian 9: Pre-packaged binaries are now available for the latest Debian and Ubuntu LTS releases. (#432, #444)
  • Elasticsearch V5, V6, and V7 compatibility: If using an external Elasticsearch database, API Umbrella now support Elasticsearch versions 5, 6, and 7. The elasticsearch.api_version must be adjusted accordingly. (#393)
  • Elasticsearch SSL support: You can now point to an Elasticsearch URL over HTTPS. (a201220, a5a403f, d89960f)
  • AWS Elasticsearch signing for IAM access control: There is an extra proxy layer to support using AWS Elasticsearch when using IAM for access control. (9ddce5e)
  • Option to log all output to stdout/stderr: The log.destination: console option can be used to log all output to stdout/stderr instead of log files. This makes API Umbrella easier to run in containerized environments. (#376)
  • Options to parse client IPs from different HTTP headers: If other proxies are present in front of API Umbrella, additional options have been added to parse the original client's IP address from a different HTTP header. (api.data.gov#429, #431)
  • Option to perform HTTPS redirects on specific API URLs: The router.api_backend_required_https_regex_default configuration option can be used to force certain API URLs to redirect to HTTPS based on a regex. (api.data.gov#457)
  • Configurable API user validation regexes: Regexes to validate e-mail addresses, first names, and last names for API key signups are now configurable (web.api_user.email_regex, web.api_user.first_name_exclude_regex, web.api_user.last_name_exclude_regex). (15f14f3, 1566eef)
  • Configurable nginx log levels: Allow the log level of the nginx processes to be configurable (nginx.error_log_level). (2b0c8ac)
  • Configurable log levels for nginx rate limiting: Allow the log level used for nginx rate limit messages to be configurable (router.global_rate_limits.ip_connections_log_level and router.global_rate_limits.ip_rate_log_level, a804e0c)
  • Docker development environment: Add a Docker-based development environment for easier development setup.
  • Experimental support for integrating automatic SSL certificate registration: Integrate lua-resty-auto-ssl for automatically handling SSL certificates. (2f6c5b5)

Changed

  • Route API backend requests directly from Traffic Server: Routing to API backends has been simplified so it occurs directly from Traffic Server, instead of routing back through an extra nginx hop. This should improve efficiency, simplifies routing, and eliminates DNS-related code. (#410)
  • Admin UI Upgrades: Upgrade the admin UI project from Ember 2.8 to Ember 3.9 and Bootstrap 3 to Bootstrap 4. This switch also moves all dependencies into NPM instead of Bower, and better uses ES6 syntax throughout the admin UI code. Integration tests have also been switched from PhantomJS to Selenium tests using headless Chrome. (#429, api.data.gov#434)
  • Upgrade to GeoIP2 database: The legacy GeoIP data previously being used has been discontinued, so GeoIP2 is now being used for geo-locating IP addresses. (8f17dae, #440)
  • Redirect all website content to HTTPS by default: All website requests now redirect to HTTPS by default. (b3a8abc, #407, api.data.gov#430)
  • Improve HTTPS requirements for API requests to error earlier: When making an insecure API request, return an error about HTTPS being required before the API key requirement error. (api.data.gov#454)
  • Improve filesystem permissions and use more restrictive umask: Ensure that the files generated by API Umbrella are only readable by the needed users, and not readable by other users that may have access to the server. (2e595ce)
  • Increase size of allowed HTTP response header lengths: If an API backend returned very long HTTP headers, it could generate 502 errors. This increases the default size of allowed HTTP headers, and also makes this length configurable. (api.data.gov#461, #398)
  • Improve the build process for better caching: The build process has been revamped to allow for better caching of the dependencies. (#409) #414
  • Output Traffic Server logs as text logs: Output Traffic Server's access log as a text log file, instead of a binary log. (bd7f9fa)
  • Preload Lua modules in nginx to improve memory usage: Lua modules are now preloaded in the nginx master process to improve memory usage by the nginx workers. (031620a)
  • Add more validations on API user names: Add additional validations to ensure a valid first and last name have been entered to help prevent signup spam.
  • Improve Cache-Control responses for admin content: Use stricter Cache-Control settings for admin responses that should not be cached, and improve caching of asset files. (api.data.gov#425)
  • Disable animations of admin analytics charts: Disable the animations of the charts in the admin analytics to improve responsiveness. (92c9351)
  • Switch to Lua code for generating request IDs: Switch from ngx_txid to lua-resty-txid for generating the request IDs to reduce need for custom nginx modules. (9d2ebd4)
  • Require multi-factor authentication for MAX.gov admin logins: If using MAX.gov for admin logins, multi-factor authentication is required by default. (api.data.gov#435)
  • Upgrade bundled software dependencies:
    • Elasticsearch 2.4.5 -> 2.4.6
    • MongoDB 3.2.15 -> 3.2.22
    • OpenResty 1.11.2.4 -> 1.13.6.2
    • OpenSSL 1.0.2l -> 1.0.2r
    • Rails 4.2.9 -> 4.2.11.1
    • Rsyslog 8.27.0 -> 8.1904.0
    • Ruby 2.3.4 -> 2.4.6
    • Traffic Server 5.3.2 -> 8.0.3

Removed

  • Removed Ubuntu 12.04, Ubuntu 14.014 and Debian 7 packages: Removed packages for unsupported distributions.
  • Removed references to request_ip_location: Removed defunct references to the analytics request_ip_location field that was removed in v0.14.0. (c783e1c)
  • Removed experimental analytics: Removed code related to experimental analytics backend. (77d50d0)
  • Removed Vagrant development environment: Removed the Vagrant-based development environment in favor of the Docker-based development environment.
Assets 2