Skip to content
This repository has been archived by the owner on Aug 18, 2023. It is now read-only.
/ gateway Public archive

A proxy to buffer and forward metrics, events, and traces.

License

Notifications You must be signed in to change notification settings

signalfx/gateway

Repository files navigation

⚠️ End of Support (EoS) Notice

The SignalFx Gateway has reached End of Support.

We recommend using our Splunk Distribution of OpenTelemetry Collector going forward, which offers the same capabilities and fully supports the OpenTelemetry standard.

ℹ️  SignalFx was acquired by Splunk in October 2019. See Splunk SignalFx for more information.

Gateway Circle CI

The SignalFx Gateway lets you aggregate metrics and send them to SignalFx. It is a multilingual datapoint demultiplexer that can accept time series data from the carbon (Graphite), collectd or SignalFx protocols and emit those datapoints to a series of servers using the carbon, collectd or SignalFx protocols. We recommend placing the gateway either on the same server as another existing metrics aggregator or on a central server that is already receiving datapoints, such as Graphite's carbon database.

The SignalFx Gateway also acts as a simple proxy and forwarder for SignalFx's Events Ingest and Trace Ingest APIs.

Usage and Configuration

Please refer to the usage and configuration documentation located here for more information on operating the SignalFx Gateway.

Development

If you want to submit patches for the gateway, make sure your code passes travis_check.sh with exit code 0. For help setting up your development environment, it should be enough to mirror the install steps of .travis.yml. You may need to make sure your GOPATH env variable is set correctly.

Code layout

You only need to read this if you want to develop the gateway or understand the gateway's code.

The gateway is divided into two main components: forwarder and listener. The forwarder and listener are glued together by the demultiplexer.

When a listener receives a datapoint, it converts the datapoint into a basic datapoint type. This core datapoint type is then sent to the multiplexer that will send a pointer to that datapoint to each forwarder.

Sometimes there is a loss of fidelity during transmission if a listener and forwarder don't support the same options. While it's impossible to make something understand an option it does not, we don't want to forget support for this option when we translate a datapoint through the multiplexer. We work around this by sometimes encoding the raw representation of the datapoint into the Datapoint object we forward. For example, points from carbon are not only translated into our core datapoint format, but also support ToCarbonLine which allows us to directly convert the abstract datapoint into what it looked like for carbon, which allows us to forward the point to another carbon database exactly as we received it.

All message passing between forwarders, multiplexer, and listeners happen on golang's built in channel abstraction.

Config file format

See the example config file for an example of how configuration looks. Configuration is a JSON file with two important fields: ListenFrom and ForwardTo.

License

The SignalFx Gateway is released under the Apache 2.0 license. See LICENSE for more details.

Dependencies

Note: this list of dependencies applies to both the SignalFx Gateway and its Smart Gateway variant.

Name License
github.com/apache/thrift/lib/go/thrift Apache Software License v2.0
github.com/beorn7/perks/quantile MIT
github.com/coreos/bbolt MIT
github.com/coreos/etcd Apache Software License v2.0
github.com/coreos/go-semver/semver Apache Software License v2.0
github.com/coreos/go-systemd/journal Apache Software License v2.0
github.com/coreos/pkg/capnslog Apache Software License v2.0
github.com/davecgh/go-spew/spew ISC
github.com/dgrijalva/jwt-go MIT
github.com/ghodss/yaml MIT
github.com/go-logfmt/logfmt MIT
github.com/go-stack/stack MIT
github.com/gobwas/glob MIT
github.com/gogo/protobuf BSD 3-Clause
github.com/golang/groupcache/lru Apache Software License v2.0
github.com/golang/protobuf BSD 3-Clause
github.com/golang/snappy BSD 3-Clause
github.com/google/btree Apache Software License v2.0
github.com/google/go-cmp BSD 3-Clause
github.com/gopherjs/gopherjs/js BSD 2-Clause
github.com/gorilla/context BSD 3-Clause
github.com/gorilla/mux BSD 3-Clause
github.com/gorilla/websocket BSD 2-Clause
github.com/grpc-ecosystem/go-grpc-middleware Apache Software License v2.0
github.com/grpc-ecosystem/go-grpc-prometheus Apache Software License v2.0
github.com/grpc-ecosystem/grpc-gateway BSD 3-Clause
github.com/inconshreveable/mousetrap Apache Software License v2.0
github.com/jaegertracing/jaeger/thrift-gen/jaeger Apache Software License v2.0
github.com/jonboulle/clockwork Apache Software License v2.0
github.com/jtolds/gls MIT
github.com/konsorten/go-windows-terminal-sequences MIT
github.com/kr/logfmt MIT
github.com/mailru/easyjson MIT
github.com/matttproud/golang_protobuf_extensions/pbutil Apache Software License v2.0
github.com/mdubbyap/timespan MIT
github.com/opentracing/opentracing-go Apache Software License v2.0
github.com/pkg/errors BSD 2-Clause
github.com/pmezard/go-difflib/difflib BSD 3-Clause
github.com/prometheus/client_golang/prometheus Apache Software License v2.0
github.com/prometheus/client_model/go Apache Software License v2.0
github.com/prometheus/common Apache Software License v2.0
github.com/prometheus/procfs Apache Software License v2.0
github.com/prometheus/prometheus/prompb Apache Software License v2.0
github.com/signalfx/com_signalfx_metrics_protobuf Apache Software License v2.0
github.com/signalfx/embetcd/embetcd Apache Software License v2.0
github.com/signalfx/go-distribute Apache Software License v2.0
github.com/signalfx/go-metrics BSD 3-Clause
github.com/signalfx/gohistogram MIT
github.com/signalfx/gohelpers Apache Software License v2.0
github.com/signalfx/golib Apache Software License v2.0
github.com/signalfx/ondiskencoding Apache Software License v2.0
github.com/mdubbyap/tdigest Apache Software License v2.0
github.com/signalfx/xdgbasedir Apache Software License v2.0
github.com/sirupsen/logrus MIT
github.com/smartystreets/assertions MIT
github.com/smartystreets/goconvey MIT
github.com/soheilhy/cmux Apache Software License v2.0
github.com/spaolacci/murmur3 BSD 3-Clause
github.com/spf13/cobra Apache Software License v2.0
github.com/spf13/pflag BSD 3-Clause
github.com/stretchr/testify/assert MIT
github.com/tmc/grpc-websocket-proxy/wsproxy MIT
github.com/uber/tchannel-go MIT
github.com/ugorji/go MIT
github.com/xiang90/probing MIT
go.uber.org/atomic MIT
go.uber.org/multierr MIT
go.uber.org/zap MIT
golang.org/x/crypto BSD 3-Clause
golang.org/x/lint BSD 3-Clause
golang.org/x/net BSD 3-Clause
golang.org/x/sys BSD 3-Clause
golang.org/x/text BSD 3-Clause
golang.org/x/time BSD 3-Clause
golang.org/x/tools BSD 3-Clause
google.golang.org/googleapis Apache Software License v2.0
google.golang.org/genproto Apache Software License v2.0
google.golang.org/grpc Apache Software License v2.0
gopkg.in/logfmt.v0 MIT
gopkg.in/natefinch/lumberjack.v2 MIT
gopkg.in/stack.v1 MIT
gopkg.in/yaml.v2 Apache Software License v2.0
gotest.tools Apache Software License v2.0
stathat.com/c/consistent MIT