The question
Can an encoding exploit relationships among irregular timestamps, instrument IDs, event types, prices and quantities while preserving exact values and fast selective reads?
This is a proposed research direction. No novel algorithm, completed benchmark or performance gain is claimed.
Why it matters
Market-data platforms and database engines could reduce storage and transfer costs without sacrificing access speed.
Search space
Block sizes, predictors, exception encodings, dictionaries and restart points.
What would count as progress?
- Exact reconstruction including timestamp precision and decimal scale
- bytes per event
- encode/decode throughput
- range-read latency.
A useful result improves a measured trade-off against strong baselines on held-out workloads, with reproducible code and documented limitations.
Baselines and boundaries
Compare delta/double-delta plus Zstandard or LZ4, dictionary encoding and applicable modern codecs. Hold out instruments, dates and market regimes; do not convert exact decimal amounts to lossy floating point.
Starting reference
Where you could contribute
Help define representative, legally shareable workloads; critique the correctness contract; implement baselines; propose candidates; or reproduce measurements. Negative results and failure cases are useful contributions.
A detailed research brief is being prepared. Access requests will become available when that brief and the request service are ready.