Add custom build support for lambdacomponents

Author: its-felix

README.md

@@ -25,6 +25,8 @@ Some layers include the corresponding OTel language SDK for the Lambda. This all
 
 * **What exporters/receivers/processors are included from the OpenTelemetry Collector?**
     > You can check out [the stripped-down collector's imports](https://github.com/open-telemetry/opentelemetry-lambda/blob/main/collector/lambdacomponents/default.go#L18) in this repository for a full list of currently included components.
+  
+    > Self-built binaries of the collector have **experimental** support for a custom set of connectors/exporters/receivers/processors. For more information, see [(Experimental) Customized collector build](#experimental-customized-collector-build)
 * **Is the Lambda layer provided or do I need to build it and distribute it myself?**
     > This repository provides pre-built Lambda layers, their ARNs are available in the [Releases](https://github.com/open-telemetry/opentelemetry-lambda/releases). You can also build the layers manually and publish them in your AWS account. This repo has files to facilitate doing that. More information is provided in [the Collector folder's README](collector/README.md).
 
@@ -99,6 +101,45 @@ The following are runtimes which are no longer or not yet supported by this repo
 [3]: https://github.com/open-telemetry/semantic-conventions/blob/main/docs/faas/faas-spans.md#outgoing-invocations
 [4]: https://github.com/open-telemetry/semantic-conventions/blob/main/docs/faas/faas-metrics.md
 
+## (Experimental) Customized collector build
+The collector can be built with a customized set of connectors/exporters/receivers/processors. This feature is **experimental** and is only supported for self-built binaries of the collector.
+
+### Build Tags
+The build-tag `lambdacomponents.custom` must always be provided to opt-in for a custom build.
+Once this build-tag is present, you need provide additional build-tags to include your desired components in the resulting binary:
+
+- `lambdacomponents.all` includes all available components
+- `lambdacomponents.connector.all` includes all available connectors
+- `lambdacomponents.exporter.all` includes all available exporters
+- `lambdacomponents.extension.all` includes all available extensions
+- `lambdacomponents.processor.all` includes all available processors
+- `lambdacomponents.receiver.all` includes all available receivers
+
+Each available component can also be included explicitly by using its specific build-tag. For a full-list of available components, have a look into the [lambdacomponents](./collector/lambdacomponents) package.
+
+As an example, the full build command including the following components:
+- All receivers
+- All processors
+- No extensions
+- Only the otlphttp exporter
+- Only the spanmetrics connector
+
+would be the following:
+```shell
+go build -tags "lambdacomponents.custom,lambdacomponents.receiver.all,lambdacomponents.processor.all,lambdacomponents.exporter.otlphttp,lambdacomponents.connector.spanmetrics"
+```
+
+### Adding additional options
+To add more options for a customized build, you can add your desired component to the [lambdacomponents](./collector/lambdacomponents) package.
+Make sure to always restrict your addition using the appropriate build-tags.
+
+For example, if you want to add the extension `foo`, the file providing this extension should be located in the [extension](./collector/lambdacomponents/extension) directory have to following build restriction:
+```
+//go:build lambdacomponents.custom && (lambdacomponents.all || lambdacomponents.extension.all || lambdacomponents.extension.foo)
+```
+
+You can provide your addition as a pull-request to this repository. Before doing so, please also read through the details of [Contributing](#contributing) to this project.
+
 ## Contributing
 
 See the [Contributing Guide](CONTRIBUTING.md) for details.

collector/lambdacomponents/connector/pkg.go

@@ -0,0 +1,19 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package connector
+
+import "go.opentelemetry.io/collector/connector"
+
+var Factories []func(extensionId string) connector.Factory

collector/lambdacomponents/connector/spanmetrics.go

@@ -0,0 +1,28 @@
+//go:build lambdacomponents.custom && (lambdacomponents.all || lambdacomponents.connector.all || lambdacomponents.connector.spanmetrics)
+
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package connector
+
+import (
+	"github.com/open-telemetry/opentelemetry-collector-contrib/connector/spanmetricsconnector"
+	"go.opentelemetry.io/collector/connector"
+)
+
+func init() {
+	Factories = append(Factories, func(extensionId string) connector.Factory {
+		return spanmetricsconnector.NewFactory()
+	})
+}

collector/lambdacomponents/custom.go

@@ -0,0 +1,83 @@
+//go:build lambdacomponents.custom
+
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package lambdacomponents
+
+import (
+	custom_connector "github.com/open-telemetry/opentelemetry-lambda/collector/lambdacomponents/connector"
+	custom_exporter "github.com/open-telemetry/opentelemetry-lambda/collector/lambdacomponents/exporter"
+	custom_extension "github.com/open-telemetry/opentelemetry-lambda/collector/lambdacomponents/extension"
+	custom_processor "github.com/open-telemetry/opentelemetry-lambda/collector/lambdacomponents/processor"
+	custom_receiver "github.com/open-telemetry/opentelemetry-lambda/collector/lambdacomponents/receiver"
+
+	"go.opentelemetry.io/collector/component"
+	"go.opentelemetry.io/collector/connector"
+	"go.opentelemetry.io/collector/exporter"
+	"go.opentelemetry.io/collector/extension"
+	"go.opentelemetry.io/collector/otelcol"
+	"go.opentelemetry.io/collector/processor"
+	"go.opentelemetry.io/collector/receiver"
+
+	"go.uber.org/multierr"
+)
+
+func Components(extensionID string) (otelcol.Factories, error) {
+	var errs []error
+
+	receivers, err := makeFactoryMap(custom_receiver.Factories, receiver.MakeFactoryMap, extensionID)
+	if err != nil {
+		errs = append(errs, err)
+	}
+
+	processors, err := makeFactoryMap(custom_processor.Factories, processor.MakeFactoryMap, extensionID)
+	if err != nil {
+		errs = append(errs, err)
+	}
+
+	exporters, err := makeFactoryMap(custom_exporter.Factories, exporter.MakeFactoryMap, extensionID)
+	if err != nil {
+		errs = append(errs, err)
+	}
+
+	extensions, err := makeFactoryMap(custom_extension.Factories, extension.MakeFactoryMap, extensionID)
+	if err != nil {
+		errs = append(errs, err)
+	}
+
+	connectors, err := makeFactoryMap(custom_connector.Factories, connector.MakeFactoryMap, extensionID)
+	if err != nil {
+		errs = append(errs, err)
+	}
+
+	factories := otelcol.Factories{
+		Receivers:  receivers,
+		Processors: processors,
+		Exporters:  exporters,
+		Extensions: extensions,
+		Connectors: connectors,
+	}
+
+	return factories, multierr.Combine(errs...)
+}
+
+func makeFactoryMap[F any](factories []func(extensionId string) F, fn func(...F) (map[component.Type]F, error), extensionId string) (map[component.Type]F, error) {
+	preprocessedFactories := make([]F, len(factories))
+	for i, f := range factories {
+		preprocessedFactories[i] = f(extensionId)
+	}
+
+	return fn(preprocessedFactories...)
+}

collector/lambdacomponents/default.go

@@ -1,3 +1,5 @@
+//go:build !lambdacomponents.custom
+
 // Copyright The OpenTelemetry Authors
 //
 // Licensed under the Apache License, Version 2.0 (the "License");

collector/lambdacomponents/exporter/logging.go

@@ -0,0 +1,28 @@
+//go:build lambdacomponents.custom && (lambdacomponents.all || lambdacomponents.exporter.all || lambdacomponents.exporter.logging)
+
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package exporter
+
+import (
+	"go.opentelemetry.io/collector/exporter"
+	"go.opentelemetry.io/collector/exporter/loggingexporter"
+)
+
+func init() {
+	Factories = append(Factories, func(extensionId string) exporter.Factory {
+		return loggingexporter.NewFactory()
+	})
+}

collector/lambdacomponents/exporter/otlp.go

@@ -0,0 +1,28 @@
+//go:build lambdacomponents.custom && (lambdacomponents.all || lambdacomponents.exporter.all || lambdacomponents.exporter.otlp)
+
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package exporter
+
+import (
+	"go.opentelemetry.io/collector/exporter"
+	"go.opentelemetry.io/collector/exporter/otlpexporter"
+)
+
+func init() {
+	Factories = append(Factories, func(extensionId string) exporter.Factory {
+		return otlpexporter.NewFactory()
+	})
+}

collector/lambdacomponents/exporter/otlphttp.go

@@ -0,0 +1,28 @@
+//go:build lambdacomponents.custom && (lambdacomponents.all || lambdacomponents.exporter.all || lambdacomponents.exporter.otlphttp)
+
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package exporter
+
+import (
+	"go.opentelemetry.io/collector/exporter"
+	"go.opentelemetry.io/collector/exporter/otlphttpexporter"
+)
+
+func init() {
+	Factories = append(Factories, func(extensionId string) exporter.Factory {
+		return otlphttpexporter.NewFactory()
+	})
+}

collector/lambdacomponents/exporter/pkg.go

@@ -0,0 +1,19 @@
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package exporter
+
+import "go.opentelemetry.io/collector/exporter"
+
+var Factories  []func(extensionId string) exporter.Factory

collector/lambdacomponents/exporter/prometheusremotewrite.go

@@ -0,0 +1,28 @@
+//go:build lambdacomponents.custom && (lambdacomponents.all || lambdacomponents.exporter.all || lambdacomponents.exporter.prometheusremotewrite)
+
+// Copyright The OpenTelemetry Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package exporter
+
+import (
+	"github.com/open-telemetry/opentelemetry-collector-contrib/exporter/prometheusremotewriteexporter"
+	"go.opentelemetry.io/collector/exporter"
+)
+
+func init() {
+	Factories = append(Factories, func(extensionId string) exporter.Factory {
+		return prometheusremotewriteexporter.NewFactory()
+	})
+}