Statistics Tracking

[Xendergo]

Closes #50

MVP

• Rewrite my old statistics tracking code to only store data for the current day in memory; replacing the data whenever a new day is observed using atomic trickery (ce14147)
• Output the statistics data into a writer in CSV format (9262c9e)
• Research compression formats that supports being appended to and use one of them to write to a statistics file on disk

Important but maybe a future PR

• Allow people to download the data from the website
• Send a "weekly update" in the webring channel with statistics aggregated over the week

[codecov]

Codecov Report

❌ Patch coverage is 84.12698% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.35%. Comparing base (18eeb66) to head (3e5934c).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
src/stats/mod.rs	84.29%	18 Missing and 1 partial ⚠️
src/webring.rs	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master      #75      +/-   ##
==========================================
- Coverage   87.86%   87.35%   -0.51%     
==========================================
  Files           8        8              
  Lines        2430     2506      +76     
  Branches     2430     2506      +76     
==========================================
+ Hits         2135     2189      +54     
- Misses        261      282      +21     
- Partials       34       35       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Test Results

46 tests   46 ✅  2s ⏱️
 1 suites   0 💤
 1 files     0 ❌

Results for commit 3e5934c.

♻️ This comment has been updated with latest results.

[kdkasad]

Why not export metrics in OTel or Prometheus format so we can integrate with dashboards?

[Xendergo]

I don't know anything about either of those; could you point me to what I should read?

[kdkasad]

I don't know anything about either of those; could you point me to what I should read?

Both are systems for collecting/processing metrics from software.

Prometheus: https://prometheus.io/docs/introduction/overview/

OpenTelemetry (intro): https://opentelemetry.io/docs/what-is-opentelemetry/
OpenTelemetry (for devs): https://opentelemetry.io/docs/getting-started/dev/

Originally the plan was to add Prometheus metrics to collect standard performance/behavior metrics about the process, but it sounded like you wanted to collect other statistics (more like user behavior) so I left that up to you.

We did also discuss Prometheus a little in Discord: https://discord.com/channels/772576325897945119/1319140464812753009/1373470039478370385

Basically it boils down to a standard format for exporting metrics so we can use existing software to collect/process/display the data instead of having to re-implement all of that. There are also application-side libraries for Prometheus (and I assume OTel too) that make it easy to calculate and export more complex metrics like histograms of response latencies.

[Xendergo]

Ok sick, I will look into those, though I'm highkey locked in for spill so I may look at that over the winter

[socket-security]

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Action	Severity	Alert  (click "▶" to expand/collapse)
Warn		Obfuscated code: cargo hyper-util is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: ? → cargo/httpmock@0.8.3 → cargo/axum@0.8.9 → cargo/reqwest@0.13.4 → cargo/hyper-util@0.1.20 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/hyper-util@0.1.20. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Obfuscated code: cargo libc is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: ? → cargo/httpmock@0.8.3 → cargo/tempfile@3.27.0 → cargo/rand@0.10.1 → cargo/axum@0.8.9 → cargo/tokio@1.52.3 → cargo/sentry@0.48.2 → cargo/reqwest@0.13.4 → cargo/notify@8.2.0 → cargo/seize@0.5.1 → cargo/tera@1.20.1 → cargo/libc@0.2.186 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/libc@0.2.186. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Obfuscated code: cargo openssl is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: ? → cargo/sentry@0.48.2 → cargo/reqwest@0.13.4 → cargo/openssl@0.10.81 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/openssl@0.10.81. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Obfuscated code: cargo writeable is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: ? → cargo/httpmock@0.8.3 → cargo/sentry@0.48.2 → cargo/tower-http@0.6.11 → cargo/reqwest@0.13.4 → cargo/writeable@0.6.3 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/writeable@0.6.3. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.
Warn		Obfuscated code: cargo zerocopy is 90.0% likely obfuscated Confidence: 0.90 Location: Package overview From: ? → cargo/sentry@0.48.2 → cargo/reqwest@0.13.4 → cargo/zerocopy@0.8.52 ℹ Read more on: This package | This alert | What is obfuscated code? Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev. Suggestion: Packages should not obfuscate their code. Consider not using packages with obfuscated code. Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore cargo/zerocopy@0.8.52. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

View full report