| Age | Commit message (Collapse) | Author |
|---|
|
Add statsd protos to platform_protos rule
BUG: 167962588
TEST: TH
TEST: Local build [ crosshatch-userdebug ]
Change-Id: I053f2a211ea28c2f181937af3d58ad16b235d096
|
|
- init new/replaced metrics
- guardrails for number of metrics/matchers/conditions/alerts
- report update to statsdstats
- update if the config is active
Test: atest statsd_test
Bug: 162323476
Change-Id: Ieaa2d18903de8020fcee5bb128124ab19f10a152
|
|
Partial config update for:
- atoms allowed from any uid
- no report metrics
- installer/version strings in uidmap
- hash strings
- annotations
- config ttl
- persisting configs locally
Mostly just reset these fields.
BYPASS_INCLUSIVE_LANGUAGE_REASON=proto field that cant be renamed
Test: atest statsd_test
Bug: 162323476
Change-Id: I5e636f8eb240bd3d44f08a87811b1dfefc428191
|
|
Completely resets the allowed log sources from scratch.
Also:
- renamed initLogSourceWhitelist to initAllowedLogSources.
- refactored tests to have cleaner asserts on data structures
Test: atest statsd_test
Bug: 162323471
Change-Id: Ie717da732eea82c764b5b1f272080a4d97a7a8eb
|
|
This is the same as initializing because alarms have no state to carry
over. We can just replace them all without losing anything.
Test: atest statsd_test
Bug: 162323547
Change-Id: Ia2fb33e9ac79476babce57e2e0fb1ca49091e260
|
|
Preserves alerts if the definition didn't change and the underlying
metric didnt change.
All alerts need to be readded to the metric b/c metrics will always
clear the alerts on config updates. In the existing code, the metric
producer created the AnomalyTracker (since duration metrics need
DurationAnomalyTrackers). Preserved alerts already have an
AnomalyTracker. Therefore, we need a second overload of
addAnomalyTracker within MetricProducer to add existing ones. Note that
we don't need to worry about the type of AnomlayTracker, since if the
alert is preserved, both the AnomalyTracker and metric must not have
changed, so they will both be the correct type.
Subscriptions are also redone - alerts will clear their subscriptions,
and all subscriptions will be readded.
For duration metric, we now need to propagate adding anomaly trackers to
the DurationTrackers since it's now possible to add anomaly trackers
after the durationTrackers have been created.
Test: atest statsd_test
Bug: 162323547
Change-Id: Id3afa04b252f3a3523114407a389be25b9171a97
|
|
Step 1. Determines if a metric needs to be updated
Step 2: performs the update. if the metric is replaced, it creates a new
metric. If the metric is preserved, we update the appropriate indices.
In both cases, the appropriate data structures that MetricsManager
relies on are updated.
Test: atest_statsd_test
Bug: 162322294
Change-Id: Iac24e02c70880bb8d0d42bd2da34acf84161ba16
|
|
Store metric and state hashes, for use when determining if the
metric/state changed.
Metric hashes include any metric activations, if present. The hash is
calculated by doing h(h(metric) | h(activation)).
Also added a getType function for metrics, since the metric will be
different if either the type, the hash, or any dependencies change.
Test: m statsd
Change-Id: I7184fef6d8e726168eabbc1c0597d8c4dfdc5fff
|
|
This cl is part of the effort to allow partial config updates in statsd.
As with matchers, there are two main steps:
1. Determine update status of each predicate
2. Perform the update
a. Copy over preserved conditions and create new ones for ones that
need to be replaced or are new.
b. Update the tracker and predicate indices of preserved conditions
c. Init all conditions to initialize new ones and determine the
current condition status
Added new unit tests, and use gmock to make container assertions more
readable.
Test: atest statsd_test
Change-Id: I74249431a9db2b409253d536044acc1fc3cf5c4c
|
|
CombinationLogMatchingTracker, LogTracker, and AtomMatcher naming consistent"
|
|
See https://source.android.com/setup/contribute/respectful-code for reference
BUG=161896447
Test: Build (all changes are comments + 1 error log message)
Change-Id: I26ef8e079bf13f19bdc3bbc8219f572a43b68090
|
|
CombinationLogMatchingTracker, LogTracker, and AtomMatcher naming consistent
Bug: 163421298
Test: `m statsd`
Test: `m statsd_test`
Test: `m`
Test: `atest statsd_test`
Change-Id: I626be9507b64959d3dcb7f260c62229f0f1d0140
|
|
There are two primary steps:
1. Determine which matchers to update
2. Perform the update
Added unit tests
Test: atest statsd_test
Bug: 162322018
Change-Id: I6dbaeb781eef362e2d024a560904497a3525cad2
|
|
This sets up a basic code path for implementing modular config updates.
Test: m statsd
Change-Id: I7db188fb0ac2f6a0e021f476f4c1ae5b752dc878
|
|
Remove kWhitelistedAtoms usage in MetricsManager and remove:
- atoms_info references from statsd
- libstatsmetadata usages from statsd
- libstatsmetadata library
- atoms_info_writer.h/.cpp
- references to atoms_info in rest of stats-log-api-gen
- allow_from_any_uid annotation
Fixes: 147600720
Fixes: 154856835
Fixes: 148993016
Test: m stats-log-api-gen
Test: m stats-log-api-gen-test &&
out/host/linux-x86/nativetest/stats-log-api-gen-test/stats-log-api-gen-test
Test: m statsd
Change-Id: I2e0579609ff257da934b95cdda397f3ca0ffa1f0
|
|
Bug: 158703584
Test: atest statsd_test
Change-Id: Ia6814c2cdb67dde2fd790ddc18fc785b1bba062c
|
|
Get whitelisted atom ids from StatsdConfig which can be logged from any
uid.
Fixes: 155521934
Test: statsd_test
Change-Id: Ib2db310d1cd8479d4996e70ac09d40e8dc45af48
|
|
|
|
Previously, MetricsManagers would unregister themselves as a
PullUidProvider for a given ConfigKey in the destructor. This caused all
pulls to fail after a config update because the new MetricsManager would
register itself before the old MetricsManager was destructed and
unregistered. This resulted in the old MetricsManager removing the new
config since they shared the same config key. The fix is for the
PullerManager to check that the PullUidProviders are equal in the
unregister function before actually erasing it.
Test: bit statsd_test:* (wrote a failing test that now passes).
Test: statsd_localdrive to manually update a config, ensured pulls still
worked.
Bug: 154544328
Change-Id: Id7af3b3b407e24bee74fc34bd1c2b9e0575e9c9e
|
|
Also clean up a bit of code on splitting on app upgrades
Piggy-backed off the app upgrade tests, adding parameterized tests to
also test boot complete event.
Refactored some value metric test code to increase code reuse and
assertions.
Fixed a broken value metric test that had assertions commented out.
Refactored NamedLatch into MultiConditionTrigger to avoid creating a
thread before necessary.
Test: atest statsd_test
Test: push a simple test config, reboot, wait, get data. Made sure
the bucket was split
Bug: 144099206
Bug: 154511974
Change-Id: I73858b5db08e8cda762bd8091b30da8738d1fd88
|
|
In MetricsManager::onLogEvent, we check whether the uid in
AppBreadcrumbReported is the same as the logger's uid. However, because
we map uids within LogEvents from isolated uids to host uids prior to
this, we should convert the logger uid to the host uid as well.
Test: m
Bug: 154124933
Change-Id: I9dba4327dd8f1102c651ca84b5ef7b48be9cf84d
|
|
* changes:
Load Metadata from Disk
Save metadata to Disk
|
|
Bug: 148280505
Test: bit statsd_test:*
Test: Manual - Added Config, Tested System Server Crash
Test: Manual - Added Config, Tested Device restart
Change-Id: Icd1b1e57bbf24e4a5676820a2b955f2034d4b37d
|
|
Bug: 148280505
Test: bit statsd_test:*
Change-Id: Ib9c6b9b4f22e7380717b480c7ae4a37bb3364619
|
|
Overall flow of implementation:
1. parsing the config in MetricsManager to store the uids per atom. It
follows the mAllowedLogSources logic very closely
2. MetricsManager register itself as a PullUidProvider with the
PullerManager.
3. Metrics pass the config key when pulling (for both registering
receivers and normal pulls) , and the puller manager gets
the allowed uids from the PullUidProvider for that config.
4. PullerManager keys receivers by <atomId, configKey> so that it can
look up the uids for that atom using the PullUidProvider as well.
5. Added shell subscriber support. Hardcode a default of AID_SYSTEM for
them and also allow packages per atom. This involved adding a second
interface to Pull that simply accepts the uids, since I didnt want to
make the ShellSubscriber a PullUidProvider as well.
6. Change adb shell cmd stats pull-source to allow users to specify a
package. Default to AID_SYSTEM as well.
Notes:
The feature is flagged off right now, since configs do not pass in the
desired package. Another approach could be to hardcode in the current
mapping, but that doesn't work for OEM pulled atoms.
Test: m statsd
Test: bit statsd_test:* with useUids = false
Test: bit statsd_test:* with useUids = true
Bug: 144099783
Bug: 151978258
Change-Id: I4a7481d7402a52b9beb4ea28b102803f9e50e79f
|
|
Bug: 148280505
Test: bit statsd_test:*
Change-Id: I7127dd7dac9f6c4f3d415534c339aaba28fa41f3
|
|
Test: m -j
Bug: 149781190
Change-Id: I765852b1719a3056c06a0965ea6a05a8e8410693
|
|
Delete libplatformprotos since its not used.
Test: m -j
Bug: 147599928
Change-Id: I80203756d714bf123978d26bbe81351a1abdc967
|
|
Clients don't need to know about statsd metadata. Extract out metadata
from statslog.h/cpp into atoms_info.h/cpp which is only used by statsd.
Generated atoms_info.h: https://paste.googleplex.com/6303016724463616
Generated atoms_info.cpp: https://paste.googleplex.com/5717940978581504
Test: m -j
Test: old metadata in statslog matches the metadata in
atoms_info
Test: Flashes successfully
Test: adb logcat "*:S statsd:*"
Change-Id: I56ef3cc4ea1fbd2cd0130d4e9576b242efb9f627
|
|
Currently, it is possible for two threads in statsd to concurrently
access/modify memory in ConditionTrackers since they do not have locks.
This happens when one thread is processing LogEvents (lock on
StatsLogProcessor mutex), while the other thread receives uidmap updates
and locks on the mutex in the MetricProducer. This Cl changes uidmap
updates to also go through the mutex in StatsLogProcessor.
Test: bit statsd_test:*
Test: atest CtsStatsdHostTestCases
Test: local test (ag/9725088) that forced the race condition now passes
Bug: 144373785
Change-Id: I04ae2f7ed025f5ce8bc4fdeb7f10717e20d76282
|
|
Changes in this CL compared to previous StateTracker version:
- MetricsManager parses StatsdConfig for State information and updates
MetricProducers accordingly
- MetricProducer implements StateListener
- MetricProducer has a list of atom ids of its slice_by_states and any
state maps from these states. MetricProducers are responsible for
handling the conversion from original state value to mapped state value.
- More unit tests added and e2e CountMetric test added
Test: bit statsd_test:*
Bug: 136566566
Change-Id: I07ce468733a1affcadf2be0206e081fddbbce4c5
|
|
Add logging to record whenever the 10 second guardrail for sending
active configs changed broadcast is met, so that we know when the
broadcast would not have been sent.
Add activation time for configs that are active when the device boots.
This will help determine whether or not statsd thinks configs are active
and will help debug issues like b/133809948
Bug: 133809948
Test: bit statsd_test:*
Test: manually rebooted to make sure activation time was logged
Change-Id: Ifa72202bc52485e4953e49d78dffea685cc3d1e7
|
|
Previously, all metrics/configs would deactivate on system server death.
Now, active status is restored.
Bug: 129717537
Test: bit statsd_test:*
Test: libprotoutil_test:ProtoOutputStream*
Change-Id: Idf372457f60a931a2d00176a5eab58c534a25e41
|
|
Also:
- rename time_to_live to ttl
- rename activation_ns to start_ns
Bug: 129719662
Fixes: 129719662
Test: statsd_test
Change-Id: I4069f85d0c1f5bd0885a9588d8a9157d94b2c587
|
|
Usually after config owner calls statsd to get data, statsd will remove that data
forever. This cl adds an option to save historical data locally so that when a bugreport
is taken, it contains historical statsd metrics for debugging.
+ All existing guardrail on storage still apply, and local history data has lower
priority when the guardrail is hit. On top of that, statsd will remove history
files that are more than 2 days old.
+ Also fixed issues inside StorageManager (e.g.,
using string::compare to sort by timestamp is problematic, redundant file size read, etc)
Test: statsd_test and manually
Fix: 126562025
Change-Id: I5238b9fd3069cbef5bcccabd2212bc4d6c734702
|
|
Cancel Metric activations triggered by atom matchers
Bug: 128218061
Test: statsd_test
Test: statsd_localdrive
Change-Id: I90a705d74725c2aa04025e18e1fa77ec4fefc522
|
|
We are currently dumping invalid data for pulled metrics. Pulled metrics
require a new pull when flushing a bucket. We should either do another
pull or invalidate the previous bucket.
There are cases where we cannot afford to do another pull, e.g. statsd
being killed. If we do not have enough time, we'll just invalidte the
bucket to make sure we have correct data.
Bug: 123866830
Test: atest statsd_test
Change-Id: I090127cace3b7265032ebb2c9bddae976c883771
|
|
Statsd now sends active configs changed broadcasts when needed per uid.
Also made an adb command to help debug.
More gts tests and unit tests required, will follow.
Test: GTS in topic
Bug: 123372077
Change-Id: Ib079018ded85d002581ffc2ba1240138ce7a54e7
|
|
from any source Test: stats-log-api-gen-test Bug: 119217680 Change-Id: Ia5faed04d696b59ba4ffaab13f5046f943d8a8b7"
|
|
Whitelisted atoms can be triggered from any source
Test: stats-log-api-gen-test
Bug: 119217680
Change-Id: Ia5faed04d696b59ba4ffaab13f5046f943d8a8b7
|
|
For metric activation that spans across boots, we need to persist active
metrics onto disk upon shutdown and load them on boot.
Bug: 123904359
Test: unit test
Change-Id: I5a4142a42595c8c132175fb574c3aa2ad30dcac0
|
|
am: 2296036a40
Change-Id: Ia99ea6c9633986e68340bc88983d06baa7a1c597
|
|
Bug: 30413223
Test: make with WITH_TIDY=1 DEFAULT_GLOBAL_TIDY_CHECKS=-*,performance*
Change-Id: Ie481e88025a7a1f3abde8ff63420d5ccd8577e52
|
|
|
|
* Creates an incident section for statsd data.
* Allows dump to output statsd data, in proto format.
* Hooks up two statsd outputs to bugreports:
-statsd report data in proto format
-statsd metadata (statsdstats) in text format
The incident section does not import stats_log.proto because that turns
out to be extremely difficult: stats_log.proto imports atoms.proto,
which imports more things and is enormous and causes all sorts of
problems. atoms.proto was purposefully never compiled in AOSP, so to
retain that feature, the incident section uses 'bytes' instead of an
actual message. Since this isn't ever read in AOSP (other than testing),
this should be fine.
Bug: 115678461
Test: take a bug report and confirm valid proto
Test: cts-tradefed run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.atom.HostAtomTests#testDumpsysStats
Change-Id: I1c370af7678d1dc7440ce299ea5ea4da6d33832b
|
|
Each config can choose to include version strings and installer with
each metrics report. This data may be useful in the cloud to filter
the app-specific data.
BUG: 115626330
Change-Id: I3972ff2a94e7f0347ac0cc8a443cf328c1731e13
Test: Modified unit-tests, verified on marlin-eng
|
|
Currently, once a metric config is pushed to statsD, it will
start to collect metrics immediately. This CL introduces the metric
activation logic. When metric needs an activation, the metric producer
will hold until the activation event is detected. Then the metric producer
starts metric generation until the TTL expires (timebomb).
This is to support Mainline where it wants to collect a few metrics for
a few hours when the binary push starts or flag flips.
Test: statsd test
BUG: b/117858835
Change-Id: I992ae98f4303d5b79932eb94eddf6c19ded3727e
|
|
+ Created bugs for those TODOs that are still relevant.
+ Remove obsolete TODOs.
Test: no code change.
Change-Id: I41c2a89a882f087817ee6cbc3f095e1d80e1928e
|
|
This is to be consistent with other patterns such as UidMap.
This also makes unit test simpler.
Change-Id: I1558cd609e470481f269ecf2ae616277a95cfbf0
Bug: 72722120
Test: unit test
|
|
Statsd hashes (using its own hashing function) raw strings to reduce the
upload data size when there are duplicate strings in the report. And in cloud,
the clearcut translator would backfill the strings.
In a few droidfood users, we find the translator was unable to do that. While
debugging the root cause, we first decided to provide an option to disable
the hashing from the cloud.
Test: statsd unit test, CTS test, tested manually
BUG: b/79943763
Change-Id: If0359c8cf3f3cf83a2938db9ebf95ea7906f0b0c
|
