summaryrefslogtreecommitdiff
path: root/cmds/statsd/src/metrics/MetricsManager.cpp
AgeCommit message (Collapse)Author
2017-12-12Only create ProtoOutputStream when onGetData() is called.Yao Chen
The exception is EventMetricProducer. Each EventMetricProducer will still have a ProtoOutputStream Because LogEvent comes as a fixed 4K, it's more memory efficient to have an 8k ProtoOutputStream for storing the events. Also removed finish() api in MetricProducer, which was intended to use with Dropbox. Test: statsd_test & dogfood app Bug: 70393808 Change-Id: I2efe4ecc76a88060a9aa5eb49d1fa6ea60bc5da8
2017-12-04Rename a few things in statsd_config.proto.Stefan Lafon
This for frameworks/base/ SimpleLogEntryMatcher -> SimpleAtomMatcher. simple_log_entry_matcher -> simple_atom_matcher LogEntryMatcher -> AtomMatcher log_entry_matcher -> atom_matcher Test: Ran statsd_test. Change-Id: I7398758e3fab86235897c162884f5ceab3002510
2017-11-301/ Only expose thread-safe interfaces in metric producer.Yangster
2/ Simplify lock logic. 3/ Add test for duration metric producer. Test: all unit test passsed. Change-Id: If6ee2e69a17f12406f4b3ea3553b14642cd636d6
2017-11-27Statsd anomaly detection - fixesBookatz
Fixes a few items in AnomalyTracker, especially to do with what happens when an anomaly alarm fires. Test: unit tests still pass Change-Id: Ia89bd617442e952e587336b890c3ca67430b5e21
2017-11-27Add StatsdStats and guardrail.Yao Chen
+ StatsdStats is the global class that tracks the stats about statsd. + Added guardrail for classes that have a map which could potentially grow unboundedly with the number of logs. TODO: add unit tests & CTS for StatsdStats, and guardrail add stats for pulled atoms. Test: statsd_test Change-Id: I0ea562de4dd3f6162f7923a9c193420b482c1d51
2017-11-201/ Duration anomaly tracker with alarm.Yangster-mac
2/ Init anomaly from config based on the public language. 3/ Unit tests for anomaly detection in count/gauge producer. 4/ Revisit the duration tracker logic. Test: unit test passed. Change-Id: I2423c0e0f05b1e37626954de9e749303423963f2
2017-11-10make statsd less chatty.Yao Chen
Test: builds and adb logcat Change-Id: I06ee9cce1844b21ed3960bbaf76262fca3357bf4
2017-11-10Merge "Make SimpleCondition satisfactorily complicated."TreeHugger Robot
2017-11-10Make SimpleCondition satisfactorily complicated.Yao Chen
+ Support nested counting + Support StopAll + Added default_condition to SimpleCondition config + Some refactoring/clean up + Added unit tests Test: Added unit tests, statsd_test Change-Id: I6564ac2e068ce6810e8090c0818064c625c7847a
2017-11-08Finish migrating to use ProtoOutputStream. This change will take theyro
outputs of each MetricProducer's and merge the results into ConfigMetricsReport which holds ConfigKey, repeated field of StatsLogReport, and UidMap. The data will be represented as vector<uint8_t> which can be passed down to binder call for clients to pick up. Also, all unnecessary dependencies to stats_log proto have been removed. Test: statsd, statsd_test Change-Id: Ia69137cbc8613644a892e6be1e87b4858bd39fe3
2017-11-06Add unit tests for CountMetricProducer, EventMetricProducerYao Chen
And other miscellaneous fixes. + clang-format + 2 bug fixes, one in dump-report command, one in ResourcePowerManagerPuller Test: statsd_test Change-Id: Ibd164d948ad62adcc529d813df1210781e38be47
2017-11-03Gauge metric producer.Yangster
Test: manual tests passed for pushed device temperature. Change-Id: I7592a4c04666606b745cdb41db8f9d8a96a966da
2017-11-01reretry ValueMetric implementation and pulling mechanismChenjie Yu
Note: This is for value metric. The default operations is sum the diffs. The test uses kernel wake lock, which also needs dimension by kernel wake lock name. The test is a bit cumbersome as it needs StatsCompanionService to do the alarm, which is not exact alarm. The internal state of a slice of bucket would look something like this: 4:ipc0000005e_727_android.hardwar 0 0 4:SensorService_wakelock 40 64 4:ipc0000005c_727_android.hardwar ... Test: manual test on device. Change-Id: I2ed0ac7d3c5fcba8b7611d46f38a38ffd8bdc92a
2017-11-01Adding multi layer aggregation in DurationMetricYao Chen
Newly supported metrics examples: 1) Compute [Total|Max] duration of [an app] holding [ANY] wake lock while [this app] is in [background] and [screen off], bucket size 30seconds, and slice output by uid. 2) Compute [Total|Max] duration of [ANY app] holding [ANY] wake lock while [this app] is in [background] and [screen off], bucket size 30 seconds. + DurationMetric proto has a "what" which is a SimpleCondition. It defines the atom level start and stop of the duration timer, and it has its atom dimension. e.g., for wake locks, the atom dimensions wil be uid and wl name. + Now dimension is explicitly specified in SimpleCondition proto instead of inferred from the "link" + Added support for "Or" and "Max" through 2 layers of aggregation. TODO: (1) The way we track slicedCondition in duration metric is not efficient. optimize! (2) The output dimension should all use int32 instead of KeyMatcher. Fix in a future cl. Test: Added some unit tests using gmock. Will add more unit tests. Change-Id: I58a827624f01f9a54fcb80709c4de4ff94a8bc67
2017-10-30Implement a logic to rate limitted flush statsd with by checking theyro
size of each MetricsProducer's. The implementation of byteSize() method is still TBD as it depends on migration to ProtoOutputStream. Test: statsd, statsd_test Change-Id: I966606044d7cb814dabe94192bacecad91f28177
2017-10-23Anomaly detection for count reads from configBookatz
Count anomaly detection now reads in parameters from the config. Also adds refractory period. Test: Manually, using ConfigManager's fake config Change-Id: I5618c0c6dcd6ef35e14d32315d5ea75ba58f0031
2017-10-19Add support for dimension, and link with condition and added DurationMetricYao Chen
Now we support following metrics: <Duration> of [app holding a wake lock], while [*this app*] is [in background] [AND] [screen is off] [Slice] the output by [app name, wake lock name], with bucket size [30sec] + Also added onDumpReport() api in MetricsManager, it can be called from client to fetch the data from statsd + Also added command line tool to dump the StatsLogReport from all metrics for debugging. + Synced proto from google3. with a pending cl (cr/172359050) TODO: We need to add tons of tests to test the Metrics. I will work on it after this CL so people can be unblocked. I locally test the duration metric with wake lock with an app that generates StatsLog events. Test: statsd_test and manual test, and run: adb shell cmd stats dump-report We have a default config, which contains a metrics to count PROCESS_START event sliced by package name. Change-Id: I4838cc6cf025c143b7e84f43040703a78121fd25
2017-10-18Introduce a new wrapper for log_msg -- LogEventJoe Onorato
It stores all of the parsed fields in a single vector, and provides accessor methods to get at fields by index with their correct type. Test: statsd_test Change-Id: I4fa94e4ce52db3ac87f19b62f9c85398de6e8145
2017-10-18Clean up how we handle configurations, and other assorted cleanupJoe Onorato
- Add a ConfigManager class that tracks the configurations that have been passed to us. Configurations are now tracked by tuples of (uid,tag), where the tag is an app-defined string, in case a single uid has multiple configurations. - Move all of the initialization into StatsService. - Get rid of the ability to have multiple LogListeners. Raw events are now pushed directly into StatsService, which can distribute them to the interested parties (and will eventually be able to do the proper locking). - Add Log.h, which sets our LOG_TAG correctly. - Move some of the related files that I expect will grow some into their own subdirectories. Test: statsd_test Test: adb shell cmd stats config ... Test: adb shell dumpsys stats Change-Id: I79487603003d8a842d5bd319741f1ecbf72063d1
2017-10-10More complete implementation for condition and log matchers in statsd.Yao Chen
+ also synced proto from google3 to fix the LogEntryMatcher proto + MetricsManager represents StatsdConfig, it's responsible for initializing and managing all LogEntryMatcher, Condition, and Metrics. Start review from here. + Added more complete StatsdConfig initialization, including building the map for: LogEntryMatcher -> Metrics LogEntryMatcher -> Condition Condition -> Metrics. All the maps use index(int). The extra amount of memory for storing mappings help us quickly process log events. The StatsdConfig initialization process detects malformed config - Circle dependency - Missing definition etc. And once we detect ANY error, statsd will reject the config. And the resources related to this config will be released. Test: Added unit tests Change-Id: I2c4aefdbf3e2aa1701eacbb2fb5e653819ec1fbb
2017-09-29Add metric computation skeleton to statsd.Yao Chen
This cl is to let statsd understand statsd_config, and compute metrics defined in the config. + StatsLogProcessor is given a StatsdConfig (hard coded right now). We construct a MetricProducer for each of the metric, and the metrics share Condition and LogEntryMatchers + Added the CountMetricProducer type for CountMetric. We can now count times of SCREEN_ON events given a config. TODO: 1) conditions are not implemented. 2) slicings are not implemented in CountMetric 3) move the interaction to dropbox to a separate thread 4) decide how the in memory metrics would be used by anomaly detection Test: manual test. $ adb shell /system/bin/statsd $ cat config_file.dat | adb shell cmd stats config Change-Id: I38f4059c0dc5a827c338131d4a6fa7d4cbe865db