Edge Telemetry Daemon

A small UDP telemetry writer written in Go.

It reads high-frequency sensor packets, keeps packet buffers ready ahead of time, and writes the raw bytes to disk.

Motivation & Problem Solved

At 1000Hz, even small delays can make UDP packets pile up and drop. A normal database is too much for this write path.

This daemon keeps the write path simple: read a packet, check it, append it to a file, then reuse the same buffer.

What this project does

• Listens for high-frequency UDP telemetry traffic.
• Reuses a fixed set of byte buffers.
• Rejects old packets using the packet timestamp.
• Appends raw binary data to data.bin.
• Includes a verification tool for packet shape, value bounds, and timestamp order.

Main goal

Keep UDP ingestion fast and boring on small machines.

Out of scope (for now)

This repository skips database features because the first job is only to save incoming data.

Features considered out of scope for this daemon:

• Query parsing: No SQL or PromQL.
• Compaction: Data is appended raw.
• Network retries: UDP is fire-and-forget.

Setup & Run Locally

1. Clone and tidy:

go mod tidy

1. Start the server:

go run cmd/server/main.go

1. Run the client: In a separate terminal, send 1000Hz test packets:

go run cmd/client/main.go

1. Verify the file: Once stopped, check packet shape and values:

go run cmd/verify/main.go

Performance & Benchmark Results

Measured on: Linux (amd64), AMD Ryzen 5 5600H Date: 2026-05-11

24-Hour Stress Test

Tested over a continuous 24-hour run:

=======================================
       PERFORMANCE AUTOPSY REPORT
=======================================
[EXECUTION METRICS]
Total Run Time       : 24h0m0.001219672s
Total Rows Saved     : 81368675

[DATA INGESTION]
Packets Received     : 16273735
Bytes Received       : 1627373500 (1551.98 MB)

[DISK STORAGE]
Packets Written      : 16273735
Bytes Written        : 1627373500 (1551.98 MB)

[PIPELINE INTEGRITY]
Internal Drops       : 0 (Perfect Transfer)

Heap Objects (Total) : 920 (Expect < 500 from startup)
Live Heap RAM        : 646.02 KB

[GARBAGE COLLECTION & MEMORY]
Total GC Cycles Run  : 0
>> RESULT: ZERO-ALLOCATION ACHIEVED <<
=======================================

Disk Writer Benchmark

Benchmark command used to measure the bufio disk-flush pipeline:

go test -bench=. -benchmem ./internal/writer

Observed benchmark output:

goos: linux
goarch: amd64
pkg: github.com/iamharshdabas/edge-telemetry-daemon/internal/writer
cpu: AMD Ryzen 5 5600H with Radeon Graphics
BenchmarkDiskWriter-12      7592066        166.5 ns/op  600.45 MB/s        0 B/op        0 allocs/op
PASS
ok   github.com/iamharshdabas/edge-telemetry-daemon/internal/writer 1.448s

How it works

flowchart TD
A[Hardware Sensors] -->|UDP Packets 1000Hz| B[OS UDP Read Buffer]
B --> C[reader.UDPReader]

subgraph fixed_buffers["Fixed Buffers"]
C -->|Gets byte buffer| D[buffer Pool]
C -->|Checks timestamp| E{Is Packet New?}
E -->|Yes| F[packetBuffer Channel]
E -->|No| G[Drop and reuse buffer]
end

F --> H[writer.DiskWriter]
H --> I[bufio.Writer 32KB]
I -->|Periodic Flush| J[(data.bin Binary Append)]
H -->|Returns buffer| D

Loading