Fix ExpandedValue sanitization on struct collection by theomagellan · Pull Request #14413 · open-telemetry/opentelemetry-collector (original) (raw)
and others added 2 commits
… unmarshalling a confmap
useExpandedValue would resolve ExpandedValues after checking if a
collection's type was "stringy." For collections of structs, isStringy
always returns false, causing all ExpandedValues to be sanitized to their
parsed values and breaking decoding of string fields if the parsed
value was not string.
Now, useExpandedValue checks if it's dealing with a collection of
structs; in which case, skip sanitization and rely on mapstructure's
per-field decoding.
- revert to old behavior when feature gate disabled, bug included
- test unmarshalling with both behaviors and expect error when feature gate disabled
- renamed featuregate
- added test with a stringy collection
- added changelog entry Co-authored-by: Jade Guiton jade.guiton@datadoghq.com
The e2e test module was not recording coverage for the parent
confmap/internal package that it tests. This caused test coverage
to appear as 0% even though the tests exercise the code.
This adds the parent package to COVER_PKGS so coverage of
confmap/internal is properly tracked for e2e tests.
Co-authored-by: Pablo Baeyens pbaeyens31+github@gmail.com
Merged via the queue into open-telemetry:main with commit 246e428
60 of 61 checks passed
github-merge-queue Bot pushed a commit that referenced this pull request
Description
The e2e test module was not recording coverage for the parent
confmap/internal package that it tests. This caused test coverage to
appear as 0% even though the tests exercise the code.
This adds the parent package to COVER_PKGS so coverage of
confmap/internal is properly tracked for e2e tests.
When looking at overall coverage of confmap/internal/decoder.go, it
goes from 91.4% to 96.3% of statements.
Link to tracking issue
Issue was found in PR #14413 (review).
Testing
Running make gotest-with-cover without the fix, then looking at
coverage in percentage with
go tool covdata percent -i=./coverage/unit -pkg=go.opentelemetry.io/collector/confmap/internalRunning these commands again with the fix added shows the improvement.
The e2e tests can also be run individually to verify they're tracking parent package coverage:
make -C confmap/internal/e2e test-with-cover
# [...] coverage: 45.3% of statements in go.opentelemetry.io/collector/confmap/internal, go.opentelemetry.io/collector/confmap/internal/e2e, )
TimoBehrendt pushed a commit to TimoBehrendt/tracebasedlogsampler that referenced this pull request
…ocessortest to v0.147.0 (#38)
This PR contains the following updates:
| Package | Change | Age | Confidence |
|---|---|---|---|
| go.opentelemetry.io/collector/processor/processortest | v0.144.0 → v0.147.0 |
Release Notes
open-telemetry/opentelemetry-collector (go.opentelemetry.io/collector/processor/processortest)
v0.147.0
💡 Enhancements 💡
exporter/debug: Output bucket counts for exponential histogram data points in normal verbosity. (#10463)pkg/exporterhelper: Addmetadata_keysconfiguration tosending_queue.batch.partitionto partition batches by client metadata (#14139) Themetadata_keysconfiguration option is now available in thesending_queue.batch.partitionsection for all exporters. When specified, batches are partitioned based on the values of the listed metadata keys, allowing separate batching per metadata partition. This feature is automatically configured when usingexporterhelper.WithQueue().
🧰 Bug fixes 🧰
cmd/builder: Fix duplicate error output when CLI command execution fails in the builder tool. (#14436)cmd/mdatagen: Fix duplicate error output when CLI command execution fails in the mdatagen tool. (#14436)cmd/mdatagen: Fix semconv URL validation for metrics with underscores in their names (#14583) Metrics likesystem.disk.io_timenow correctly validate against semantic convention URLs containing underscores in the anchor tag.extension/memory_limiter: Use ChainUnaryInterceptor instead of UnaryInterceptor to allow multiple interceptors. (#14634) If multiple extensions that use the UnaryInterceptor are set the binary panics at start time.extension/memory_limiter: Add support for streaming services. (#14634)pkg/config/configmiddleware: Add context.Context to HTTP middleware interface constructors. (#14523) This is a breaking API change for components that implement or use extensionmiddleware.pkg/confmap: Fix another issue where configs could fail to decode when using interpolated values in string fields. (#14034) For example, a resource attribute can be set via an environment variable to a string that is parseable as a number, e.g.1234.(A similar bug was fixed in a previous release: that one was triggered when the field was nested in a struct, whereas this one is triggered when the field internally has type "pointer to string" rather than "string".)
pkg/otelcol: The featuregate subcommand now rejects extra positional arguments instead of silently ignoring them. (#14554)pkg/queuebatch: Fix data race in partition_batcher where resetTimer() was called outside mutex, causing concurrent timer.Reset() calls and unpredictable batch flush timing under load. (#14491)pkg/scraperhelper: Log scrapers now emit log-appropriate receiver telemetry (#14654) Log scrapers previously emitted the same receiver telemetry as metric scrapers, such as the otelcol_receiver_accepted_metric_points metric (instead of otelcol_receiver_accepted_log_records), or spans named receiver/myreceiver/MetricsReceived (instead of receiver/myreceiver/LogsReceived).This did not affect scraper-specific spans and metrics.
processor/batch: Fixes a bug where the batch processor would not copySchemaUrlmetadata from resource and scope containers during partial batch splits. (#12279, #14620)
v0.146.1
v0.146.0
🛑 Breaking changes 🛑
all: Increase minimum Go version to 1.25 (#14567)
🚩 Deprecations 🚩
pdata/pprofile: Declare removed aggregation elements as deprecated. (#14528)
💡 Enhancements 💡
all: Add detailed failure attributes to exporter send_failed metrics at detailed telemetry level. (#13956) Theotelcol_exporter_send_failed_{spans,metric_points,log_records}metrics now include failure attributes when telemetry level is Detailed:error.type(OpenTelemetry semantic convention describing the error class) anderror.permanent(indicates if error is permanent/non-retryable). Theerror.typeattribute captures gRPC status codes (e.g., "Unavailable", "ResourceExhausted"), standard Go context errors (e.g., "canceled", "deadline_exceeded"), and collector-specific errors (e.g., "shutdown"). This enables better alerting and debugging by providing standardized error classification.cmd/builder: Introduce new experimentalinitsubcommand (#14530) The newinitsubcommand initializes a new custom collectorcmd/builder: Add "telemetry" field to allow configuring telemetry providers (#14575) Most users should not need to use this, this field should only be set if you intend to provide your own OpenTelemetry SDK.cmd/mdatagen: Introduce additional metadata (the version since the deprecation started, and the deprecation reason) for deprecated metrics. (#14113)cmd/mdatagen: Add optionalrelationshipsfield to entity schema in metadata.yaml (#14284)exporter/debug: Addoutput_pathsconfiguration option to control output destination whenuse_internal_loggeris false. (#10472) Whenuse_internal_loggeris set tofalse, the debug exporter now supports configuring the output destination via theoutput_pathsoption. This allows users to send debug exporter output tostdout,stderr, or a file path. The default value is["stdout"]to maintain backward compatibility.pkg/confmap: Add experimentalToStringMapRawfunction to decodeconfmap.Confinto a string map without losing internal types (#14480) This method exposes the internal structure of aconfmap.Confwhich may change at any time without prior notice
🧰 Bug fixes 🧰
cmd/mdatagen: Reset aggDataPoints during metric init to avoid index out of range panic across emit cycles when reaggregation is enabled. (#14569)cmd/mdatagen: Fix panic when mdatagen is run without arguments. (#14506)pdata/pprofile: Fix off-by-one issue in dictionary lookups. (#14534)pkg/config/confighttp: Fix high cardinality span name from request method from confighttp server internal telemetry (#14516) Follow spec to bound request method cardinality.pkg/otelcol: Ignore component aliases in theotelcol componentscommand (#14492)pkg/otelcol: Order providers and converters in alphabetical order in thecomponentssubcommand. (#14476)
v0.145.0
💡 Enhancements 💡
pkg/scraperhelper: ScraperID has been added to the logs for metrics, logs, and profiles (#14461)
🧰 Bug fixes 🧰
exporter/otlp_grpc: Fix the OTLP exporter balancer to use round_robin by default, as intended. (#14090)pkg/config/configoptional: FixUnmarshalmethods not being called when config is wrapped insideOptional(#14500) This bug notably manifested in the fact that thesending_queue::batch::sizerconfig for exporters stopped defaulting tosending_queue::sizer, which sometimes caused the wrong units to be used when configuringsending_queue::batch::min_sizeandmax_size.As part of the fix,
xconfmapexposes a newxconfmap.WithForceUnmarshaleroption, to be used in theUnmarshalmethods of wrapper types likeconfigoptional.Optionalto make sure theUnmarshalmethod of the inner type is called.The default behavior remains that calling
conf.Unmarshalon theconfmap.Confpassed as argument to anUnmarshalmethod will skip any top-levelUnmarshalmethods to avoid infinite recursion in standard use cases.pkg/confmap: Fix an issue where configs could fail to decode when using interpolated values in string fields. (#14413) For example, a header can be set via an environment variable to a string that is parseable as a number, e.g.1234pkg/service: Don't error on startup when process metrics are enabled on unsupported OSes (e.g. AIX) (#14307)
Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
- If you want to rebase/retry this PR, check this box
This PR has been generated by Renovate Bot.
Reviewed-on: https://gitea.t000-n.de/t.behrendt/tracebasedlogsampler/pulls/38 Reviewed-by: t.behrendt t.behrendt@noreply.localhost Co-authored-by: Renovate Bot renovate@t00n.de Co-committed-by: Renovate Bot renovate@t00n.de
TimoBehrendt pushed a commit to TimoBehrendt/tracebasedlogsampler that referenced this pull request
…1.54.0 (#33)
This PR contains the following updates:
| Package | Change | Age | Confidence |
|---|---|---|---|
| go.opentelemetry.io/collector/confmap | v1.50.0 → v1.54.0 |
Release Notes
open-telemetry/opentelemetry-collector (go.opentelemetry.io/collector/confmap)
v1.54.0
❗ Known Issues ❗
service: The collector's internal Prometheus metrics endpoint (:8888) now emits OTel service labels with underscore names (service_name,service_instance_id,service_version) instead of dot-notation names (service.name,service.instance.id,service.version). Users scraping this endpoint with the Prometheus receiver will see these renamed labels in resource and datapoint attributes. As a workaround, add the followingmetric_relabel_configsto your scrape config in prometheus receiver:
See #14814 for details and updates.metric_relabel_configs: - source_labels: [service_name] target_label: service.name - source_labels: [service_instance_id] target_label: service.instance.id - source_labels: [service_version] target_label: service.version - regex: service_name|service_instance_id|service_version action: labeldrop
🛑 Breaking changes 🛑
all: Change metric units to be singular to match OTel specification, e.g.{requests}->{request}(#14753)
💡 Enhancements 💡
cmd/mdatagen: Add deprecated_type field to allow specifying an alias for component types. (#14718)cmd/mdatagen: Generate entity-scoped MetricsBuilder API that enforces entity-metric associations at compile time (#14659)cmd/mdatagen: Skip generating reaggregation config options for metrics that have no aggregatable attributes. (#14689)pkg/service: The internal status reporter no longer drops repeated Ok and RecoverableError statuses (#14282) Status events can now carry metadata and there's value in allowing them to be emitted despite the status value itself not changing.
🧰 Bug fixes 🧰
cmd/builder: Add.exeto output binary names when building for Windows targets. (#12591)exporter/debug: Add printing of metric metadata in detailed verbosity. (#14667)exporter/otlp_grpc: Prevent nil pointer panic when push methods are called before the OTLP exporter initializes its gRPC clients. (#14663) When the sending queue and retry are disabled, calling ConsumeTraces, ConsumeMetrics, ConsumeLogs, or ConsumeProfiles before the OTLP exporter initializes its gRPC clients could cause a nil pointer dereference panic. The push methods now return an error instead of panicking.exporter/otlp_http: Show the actual destination URL in error messages when request URL is modified by middleware. (#14673) Unwraps the*url.Errorreturned byhttp.Client.Do()to prevent misleading error logs when a middleware extension dynamically updates the endpoint.pdata/pprofile: Switch the dictionary of dictionary tables entries only once when merging profiles (#14709) For dictionary table data, we used to switch their dictionaries when doing the switch for the data that uses them. However, when an entry is associated with multiple other data (several samples can use the same stack), we would have been switching the dictionaries of the entry multiple times.We now switch dictionaries for dictionary table data only once, before switching the resource profiles.
v1.53.0
💡 Enhancements 💡
exporter/debug: Output bucket counts for exponential histogram data points in normal verbosity. (#10463)pkg/exporterhelper: Addmetadata_keysconfiguration tosending_queue.batch.partitionto partition batches by client metadata (#14139) Themetadata_keysconfiguration option is now available in thesending_queue.batch.partitionsection for all exporters. When specified, batches are partitioned based on the values of the listed metadata keys, allowing separate batching per metadata partition. This feature is automatically configured when usingexporterhelper.WithQueue().
🧰 Bug fixes 🧰
cmd/builder: Fix duplicate error output when CLI command execution fails in the builder tool. (#14436)cmd/mdatagen: Fix duplicate error output when CLI command execution fails in the mdatagen tool. (#14436)cmd/mdatagen: Fix semconv URL validation for metrics with underscores in their names (#14583) Metrics likesystem.disk.io_timenow correctly validate against semantic convention URLs containing underscores in the anchor tag.extension/memory_limiter: Use ChainUnaryInterceptor instead of UnaryInterceptor to allow multiple interceptors. (#14634) If multiple extensions that use the UnaryInterceptor are set the binary panics at start time.extension/memory_limiter: Add support for streaming services. (#14634)pkg/config/configmiddleware: Add context.Context to HTTP middleware interface constructors. (#14523) This is a breaking API change for components that implement or use extensionmiddleware.pkg/confmap: Fix another issue where configs could fail to decode when using interpolated values in string fields. (#14034) For example, a resource attribute can be set via an environment variable to a string that is parseable as a number, e.g.1234.(A similar bug was fixed in a previous release: that one was triggered when the field was nested in a struct, whereas this one is triggered when the field internally has type "pointer to string" rather than "string".)
pkg/otelcol: The featuregate subcommand now rejects extra positional arguments instead of silently ignoring them. (#14554)pkg/queuebatch: Fix data race in partition_batcher where resetTimer() was called outside mutex, causing concurrent timer.Reset() calls and unpredictable batch flush timing under load. (#14491)pkg/scraperhelper: Log scrapers now emit log-appropriate receiver telemetry (#14654) Log scrapers previously emitted the same receiver telemetry as metric scrapers, such as the otelcol_receiver_accepted_metric_points metric (instead of otelcol_receiver_accepted_log_records), or spans named receiver/myreceiver/MetricsReceived (instead of receiver/myreceiver/LogsReceived).This did not affect scraper-specific spans and metrics.
processor/batch: Fixes a bug where the batch processor would not copySchemaUrlmetadata from resource and scope containers during partial batch splits. (#12279, #14620)
v1.52.0
v1.51.0
💡 Enhancements 💡
pkg/scraperhelper: ScraperID has been added to the logs for metrics, logs, and profiles (#14461)
🧰 Bug fixes 🧰
exporter/otlp_grpc: Fix the OTLP exporter balancer to use round_robin by default, as intended. (#14090)pkg/config/configoptional: FixUnmarshalmethods not being called when config is wrapped insideOptional(#14500) This bug notably manifested in the fact that thesending_queue::batch::sizerconfig for exporters stopped defaulting tosending_queue::sizer, which sometimes caused the wrong units to be used when configuringsending_queue::batch::min_sizeandmax_size.As part of the fix,
xconfmapexposes a newxconfmap.WithForceUnmarshaleroption, to be used in theUnmarshalmethods of wrapper types likeconfigoptional.Optionalto make sure theUnmarshalmethod of the inner type is called.The default behavior remains that calling
conf.Unmarshalon theconfmap.Confpassed as argument to anUnmarshalmethod will skip any top-levelUnmarshalmethods to avoid infinite recursion in standard use cases.pkg/confmap: Fix an issue where configs could fail to decode when using interpolated values in string fields. (#14413) For example, a header can be set via an environment variable to a string that is parseable as a number, e.g.1234pkg/service: Don't error on startup when process metrics are enabled on unsupported OSes (e.g. AIX) (#14307)
Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
- If you want to rebase/retry this PR, check this box
This PR has been generated by Renovate Bot.
Reviewed-on: https://gitea.t000-n.de/t.behrendt/tracebasedlogsampler/pulls/33 Reviewed-by: t.behrendt t.behrendt@noreply.localhost Co-authored-by: Renovate Bot renovate@t00n.de Co-committed-by: Renovate Bot renovate@t00n.de
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})