summaryrefslogtreecommitdiff
path: root/cmds/statsd/src/metrics/MetricsManager.cpp
AgeCommit message (Collapse)Author
2018-05-07Merge "Fixes statsd returning too much data at once." into pi-devYang Lu
2018-05-06Re-enable the logging of APP_BREADCRUMB_REPORTED atomyro
Bug: 78613419 Test: manual, cts, unit tests Change-Id: I279158c8031eda3ee648053ae6a0d13fde7f1176
2018-05-04Fixes statsd returning too much data at once.David Chen
We observe a single ConfigMetricsReportList can be greater than the safe size for the binder transaction buffer since we only check the size of the current metrics in progress, but we also return the previous reports stored on disk. This change will attempt to send another ConfigMetricsReportList as soon as possible if there's already a report on disk. Also fixes a bug when trying to trigger data fetch before the client has registered the corresponding dataFetchOperation. Bug: 79201869 Test: Tested manually on marlin-eng Change-Id: I2d3677162804a27e7a7a95d482d80c46bd994a67
2018-05-03Turns DEBUG to false in statsd.Yangster-mac
Test: statsd test BUG: b/79161505 Change-Id: Ic6eee527d625b10aa86b2beb4b4c4fc05b051c7d
2018-05-02Merge "Reduce statsd log data size." into pi-devandroid-build-team Robot
2018-04-26Reduce statsd log data size.Yangster-mac
1. Hash the strings in metric dimensions. 2. Optimize the timestamp encoding in bucket. Use bucket num for full bucket and millis for partial bucket. 3. Encode the dimension path per metric and avoid deduping it across dimensons. Test: statsd test Change-Id: I18f69654de85edb21a9c835c73edead756295e05 BUG: b/77813755
2018-04-24Clear the past buckets for no-report metricsYangster-mac
Test: statsd tests. BUG: b/78503732 Change-Id: I3de855929ad05677769cffaae4fea2db0d2f32da
2018-04-10Flush the bucket when creating the metric producer.Yangster-mac
Use int64 for value field. E2e test for gauge/value metric. BUG: b/74445671 Test: statsd test. Change-Id: I823a0bade8f89834bdfb9cf48864852a47d7b63b
2018-04-05Flush the partial bucket when startd shuts down or config updated.Yangster-mac
Test: statsd test BUG: b/77556036 Change-Id: Ie4a04ace55e07c4529cdff5906ba874f8815f620
2018-04-04Statsd config TTLYangster-mac
Roughly check the config every hour to see whether the ttl expired. If so, read the config from disk and recreate the metric manager. Test: statsd test BUG: b/77274363 Change-Id: I16838afe5bbe966c3a0f638869751f9b59a5a259
2018-04-03Includes annotations with statsd reports.David Chen
It's tricky to determine the source of the metrics on a device currently since we can take the union of multiple configs and send only one giant statsd_config into statsd. We will use the int64 field to track the sub config id's and the int32 field to track the version for each sub config, but the fields are named more generically as annotations. The annotations are available in both the reports and metadata. Test: Check that all unit-tests pass on marlin-eng Bug: 77327261 Change-Id: Ic37c549c8b2991676f69948c515156765c9f5108
2018-04-02Move forward the alarm timestamp when config is added to statsd.Yangster-mac
Test: statsd test BUG: b/77344187 Change-Id: Ieacffaa29422829b8956f2b3fcb2c647c8c3eed9
2018-03-31Merge "E2e test for periodic alarm." into pi-devTreeHugger Robot
2018-03-27E2e test for periodic alarm.Yangster-mac
Test: new test BUG: b/76281156 Change-Id: I60cb28baaeec6996e946a7cb3358ec8e0aca80e5
2018-03-27Support sliced condition change in GaugeMetricYao Chen
TODO: We need CTS to verify the behavior. Bug: 73958484 Test: statsd_test Change-Id: I56406983ddede12bc6a2e12188693a0c51ccae5c
2018-03-15Deletes default allowed_log_sources in statsd.David Chen
We should fail right away if someone forgets to set this field instead of setting default values since this may lead to hard to explain bugs in the future (eg, why isn't systemui logging an atom to a config). Bug: 74608359 Test: Verified unit-tests still pass on marlin-eng. Change-Id: Ibd8a6ccbc5cc8d2dfb8a1577c64bf9b49822a2c3
2018-03-10Add wall clock timestamp for ConfigMetricsReport and gauge atoms.Yangster-mac
Fix the bug when serializing multiple atoms in gauge metric BUG: b/74159560 Test: new test for ALL_CONDITION_CHANGES sampling method. Change-Id: I6d33c1efbac92b6e13be2d64c323e090cb1f84aa
2018-03-05Use uint64_t instead of long long as API type for consistent reason.Yi Jin
Bug: 74118023 Test: manual Change-Id: Icd5f506c76d3a008a79cb6c9d2061962ca7fdd40
2018-03-05Merge "Add #alerts to StatsdStats printout"TreeHugger Robot
2018-03-02Add #alerts to StatsdStats printoutBookatz
When print-stats is called, now #alert=0 won't always be 0. Change-Id: If618c9a0890b45f7211f05853f4d97332d7f4cf1 Fixes: 73493830 Test: Manual confirmation
2018-03-01Add API to let metrics directly drop data without writing to an output.Yao Chen
+ Metrics will do flushIfNeeded() to correctly move the clock and informing AnomalyTracker the past bucket info, and then clear past buckets. + We will still keep the current bucket data for the validity of the future metrics. Bug: 70571383 Test: statsd_test Change-Id: Ib13c45574974e7b4e82bd8f305091dc93bda76f5
2018-02-27Merge "Alarm: wakes up statsd and notifies the subscribers."TreeHugger Robot
2018-02-27Alarm: wakes up statsd and notifies the subscribers.Yangster-mac
Test: manually tested it. Change-Id: Id796a68976aeb1611183023ba4e9c6a8b8c44bb8
2018-02-26Updates jank metrics in statsd to include uid.David Chen
We need the uid to easily know which app to blame for producing the frame with excessively long render time. Also updates the errors so it's more obvious if the error is in parsing versus the other checks. Test: Test that statsd builds and verified CTS test still passes. Change-Id: Ib6518f2d9fe6f9c78d548b6dcbdb67a0f211ff5c
2018-02-15Merge "More formatting fixes for statsd"TreeHugger Robot
2018-02-14More formatting fixes for statsdDavid Chen
Test: N/A Change-Id: I378cde8700655bfba26b112ae6df1b4347683bdb
2018-02-14Statsd guardrail for AlertsBookatz
Now, each config is allowed at most 100 Alerts in it. That is, for each uid, for each config id, the config's alert section can have at most 100 items in it. Bug: 73287046 Test: none yet Change-Id: I2c81bc647627e432337c359c0a76aa3fc08bdd23
2018-02-14Merge "Updates statsd atoms.proto with small changes."TreeHugger Robot
2018-02-13Use elapsed realtime instead of times based on wall clock, which can jump ↵Yangster-mac
around and go backwards. Test: statsd unit test passed Change-Id: Ib541df99231e171b3be2a24f75632693e36da90e
2018-02-13Updates statsd atoms.proto with small changes.David Chen
Renames the apphook atom to be more descriptive. Renames time suffixes to match more clear convention (ms and msec are ambiguous). Adds a field to CPU active time that's required for the metrics to be usable. Test: Test that statsd can still build. Change-Id: I94866510738db994e8d757260f30e599ba995dbd
2018-02-12Statsd CPU optimization.Yao Chen
The key change is to revamp how we parse/store/match a log event, especially how we match repeated field and attribution nodes, and how we construct dimensions and compare them. + We use a integer to encode the field of a log element. And also encode the FieldMatcher into an integer and a bit mask. The log matching becomes 2 integer operations. + Dimension is stored as encoded field and value pair. Checking if 2 dimensions are equal is then becoming checking if the underlying integers are equal. The integers are stored contiguously in memory, so it's much faster than previous tree structure. Start review from FieldValue.h Test: statsd_test + new unit tests Bug: 72659059 Change-Id: Iec8daeacdd3f39ab297c10ab9cd7b710a9c42e86
2018-02-08Turn off debug logging in statsdTej Singh
Sets DEBUG to false everywhere and replaces all ALOGD with VLOG so they do not print with DEBUG false. Leaves all ALOGI, ALOGW and ALOGE as is. Test: ran all CTS tests and checked "adb logcat -s statsd" to make sure it wasn't spammy Change-Id: Iaa8eb3a0a63723ffe40f94f2815f94df877fd432
2018-02-08Modifies statsd output for start and end times.David Chen
We include the start of when the last dump occurred and the current timestamp. These timestamps are shared across all metrics, so there's no advantage in duplicating these numbers across all metrics. Also, we should use elapsed realtime instead of times based on wall clock, which can jump around and go backwards. Test: Test that statsd can still build and adb shell cmd stats dump-report doesn't crash. Change-Id: I819e5643cee75dfa3e78a58f94c9d61ededa78d7
2018-02-06Merge "Partial buckets on app upgrade and fix duration."TreeHugger Robot
2018-02-05Statsd - use AID_STATSD in MetricsManagerBookatz
MetricsManager needs to know the uid of Statsd. Right now it fetches it, but it is actually a constant that it can access directly. Test: manual Change-Id: I36437136babaf99ecad1fac7c233f7aaf06f380d
2018-02-02Statsd - adb cmd for AppHook; long compare supportBookatz
1. Create an adb command for statsd to let the adb user write AppHook to the StatsLog buffer. This can be used in the CTS tests (instead of relying on screen state changes, etc. for conditioning), and for local testing. 2. Fixes the fact that AppHook loggers can spoof uids (they can put whatever uid they want and statsd doesn't validate it - now it will). 3. Allow FieldValueMatcher to compare longs (not just ints). Fix: 72266788 Fix: 72836157 Fix: 72872130 Fix: 72829733 Test: manually did the adb command. Test: run cts-dev -m CtsStatsdHostTestCases -t android.cts.statsd.alert.AnomalyDetectionTests Test: locally modified android.cts.statsd.alert.BroadcastSubscriberTests#testBroadcastSubscriber to have the app attempt both valid and invalid AppHook writes. Change-Id: I68931a71805bcfa6fe56e7a0a0d3f07290cb78d1
2018-02-02Partial buckets on app upgrade and fix duration.David Chen
Statsd will create partial buckets in all metrics producers when an app is upgraded so that we can separate metrics between different versions of an app. By looking at the uid map changes, we can tell which app versions belong to a bucket; for metrics that are not affected by an app version, we can instead join the buckets together. To simplify the logic, the ends of the full buckets are always aligned to when the metric producers were created. These boundaries are computed on the fly by using the bucket number and the metric producers' start times. We keep the anomaly trackers to only be given full buckets; we buffer the partial buckets within each metric producer. Duration metric's MAX_SPARSE is fixed to be implemented as such. In addition, after further discussion, we find anomaly detection on MAX_SPARSE to be unnecessary, so this functionality is removed. Test: Unit-tests added and modified, passed on marlin-eng. Change-Id: I5ff7a9c7f05c406e9faf400c6a39162970ded102
2018-02-01Atom: DaveyOccurredTej Singh
Atom for davey occurred Test: cts test accompanying && statsd unit tests && hwui unit tests Change-Id: I715ac213b09ef2b3ef1de75bc456a5edf7e0a244
2018-01-26Add more statsd's debugging info to dumpsys.Yao Chen
+ Bugreport will use the non-verbose mode + Reuse the log_msg object in LogReader + Add logd errors to StatsdStats Bug: 72383073 Test: manual + statsd_test Change-Id: Id5a8b103074d034f5ece3c9831c740d44a5df9cd
2018-01-04use only string type in the log source whitelist.Yao Chen
+ predefined "AID_X" will be provided as string type to statsd, and we will translate to integer uid using the static map. Test: statsd_test Change-Id: Ie47d8481e0c456457e6881ebb9cb4ce008e772b8
2018-01-03Merge "1/ Change all "name" to id in statsD. 2/ Handle Subscription for ↵TreeHugger Robot
alert. 3/ Support no_report_metric"
2018-01-031/ Change all "name" to id in statsD.Yangster-mac
2/ Handle Subscription for alert. 3/ Support no_report_metric Bug: 69522276 Test: all statsd unit tests passed. Change-Id: I851b235f2d149b8602b0cad632d5bf541962f40a
2018-01-03Merge "Checks in statsd that fields are within range."TreeHugger Robot
2018-01-011/ Support nested message and repeated fields in statsd.Yangster-mac
2/ Filter gauge fields by FieldMatcher. 3/ Wire up wakelock attribution chain. 4/ e2e test: wakelock duration metric with aggregated predicate dimensions. 5/ e2e test: count metric with multiple metric condition links for 2 predicates and 1 non-sliced predicate. Test: statsd unit test passed. Change-Id: I89db31cb068184a54e0a892fad710966d3127bc9
2017-12-28Checks in statsd that fields are within range.David Chen
We check that both the developer-chosen label and the state fields are within the expected ranges. Test: Test that statsd still builds. Change-Id: I9f76dfbf16edc4540b28a778385f416cb1306d4e
2017-12-20Add log source filtering in statsd to filter out spams.Yao Chen
+ Add log source whitelist in StatsdConfig + Some changes in UidMap API. Listener needs to be wp instead of sp. + Update dogfood app config to have log source + Increase the stats service thread pool size to 10 (9+1). TODO: add unit tests(b/70805664). This unit test takes some time to write. Test: statsd_test & manual Change-Id: I129b1cc13db5114db7417580962bd7cc4438519d
2017-12-17Merge "align metrics to 5min bundary We use one alarm clock for all pulled ↵TreeHugger Robot
atoms. If metrics from different configs are not aligned, the clock will be set to repeat at higher and higher frequency, and consume a lot of battery. Current implementation assumes a 5min minimum bucket size. New metric start time is set to be aligned to the start time of statsd in the next 5min. So it will ignore events up to 5min."
2017-12-16align metrics to 5min bundaryChenjie Yu
We use one alarm clock for all pulled atoms. If metrics from different configs are not aligned, the clock will be set to repeat at higher and higher frequency, and consume a lot of battery. Current implementation assumes a 5min minimum bucket size. New metric start time is set to be aligned to the start time of statsd in the next 5min. So it will ignore events up to 5min. align puller alarm to minute bundary Test: unit test Change-Id: I77ffa3c13de363c780b1000181b9a9b780dd0846
2017-12-14Further reduce statsd memory usage.Yao Chen
+ Remove the protobuf *Metric object from MetricProducers -- This saves ~150 bytes per metric. + Remove the StatsdConfig from ConfigManager -- This saves us xKB per config. + Also remove alerts from fake config to avoid crash (Bug: 70627390) + Other misc fixes too. Test: statsd_test & manual Change-Id: Ied4eb3fa31c50599817b3a5e1caf5077c487fad2
2017-12-13use running sum for ValueMetricProducer bucketChenjie Yu
simplify ValueMetricProducer logic for pulled data Test: unit test Change-Id: Ic0a21a543166cc5c34c1fa505dba08d1fc2f510a
generated by cgit v1.2.3 (git 2.25.1) at 2025-10-03 09:37:05 +0000