Skip to content

Instantly share code, notes, and snippets.

@R4wm
Last active February 2, 2026 18:44
Show Gist options
  • Select an option

  • Save R4wm/ecc454588b630aea80ea60f2b868e8ad to your computer and use it in GitHub Desktop.

Select an option

Save R4wm/ecc454588b630aea80ea60f2b868e8ad to your computer and use it in GitHub Desktop.
Data Lake Phase 1 - Event Prioritization Analysis (TVM-1698)

Data Lake Phase 1 - Event Prioritization Analysis

Epic: TVM-1698 Date: 2026-02-02 Author: Raymond Mintz


Executive Summary

This document analyzes the remaining events needed for the Data Lake Phase 1 launch, mapping them against the metrics specification and current implementation status.

Current Coverage

  • Creator Metrics: ~80% covered (content, engagement, video, audience growth)
  • Advertiser Metrics: N/A - No ad auction/billing infrastructure exists yet

Current Implementation Status

DONE / DEPLOYED

Ticket Description
TVM-1654 Core event infrastructure (raw_events table, Kafka)
TVM-1705 Single event endpoint /v1/events
TVM-1704 Feature toggle for events
TVM-1717 Batch events in payload
TVM-1727 Unauthenticated events + rate limiting
TVM-1699 Notification events (sent, delivered, failed)
TVM-1697 New user, new post events to Kafka
TVM-1718 Events.raw populated for development

IN CODE REVIEW

Ticket Description Assignee
TVM-1700 Display Events (content.display_viewable) Ron Pruitt
TVM-1701 Click Events (click.content, click.link, click.profile) Ron Pruitt
TVM-1731 Add 25 missing event types to enum Raymond Mintz
TVM-1729 Rename event/action fields Jairo Lopez

IN PROGRESS

Ticket Description Assignee
TVM-1732 Engagement Events (share, save, bookmark, hide/report) Raymond Mintz

BACKLOG

Ticket Description
TVM-1730 Consent screens (mobile/web)
TVM-1715 Automated testing for event verification

Events by Source

Backend Events (Server-side)

Category Events
Post post.view, post.create, post.delete, post.like, post.unlike, post.share, post.bookmark, post.unbookmark, post.report
Comment comment.create, comment.delete, comment.like, comment.unlike
Follow follow.create, follow.remove
Video (VOD) video.progress, video.complete
Livestream livestream.start, livestream.join, livestream.leave, livestream.heartbeat, livestream.vod_ready
Notification notification.sent, notification.delivered, notification.failed
Session session.start, session.end
User user.create, user.block

Client Events (Frontend-side)

Category Events
Feed feed.view, feed.scroll, feed.refresh
Video video.play, video.pause, video.seek, video.resume, video.sound_on, video.fullscreen, video.loop
Livestream livestream.mic_toggle, livestream.camera_toggle
Notification notification.receive, notification.click, notification.dismiss
Click click.content, click.link, click.profile
Content content.display_viewable, content.click, content.reacted
Profile profile.view
Search search.query, search.result_click, search.start
App Lifecycle app.installed, app.opened, app.backgrounded, app.foregrounded, app.terminated, app.updated, app.crash

Prioritization Matrix (Revised)

P0 - ACTUALLY DOABLE NOW (Backend, infrastructure exists)

Event Type Difficulty Hook Point Metrics Unlocked
creator.payout Backend Easy EarningsRepository Creator Earnings, Earnings per 1K Followers
boost.spend Backend Easy Boost feature (if exists) Boost ROI, Avg Boost Performance
api.error Backend Easy Exception handler middleware Error tracking
rate.limited Backend Easy Rate limit middleware Rate limit tracking
permission.denied Backend Easy Auth middleware Auth failure tracking
hashtag.applied Backend Easy Post creation flow Content tagging
ticker.applied Backend Easy Post creation flow Content tagging
tag.applied Backend Easy Post creation flow Content tagging

P1 - CLIENT WORK IN PROGRESS

Event Type Status Notes
content.display_viewable Client Code Review (TVM-1700) Ron's PR - MRC viewability
click.content/link/profile Client Code Review (TVM-1701) Ron's PR
feed.view/scroll/refresh Client In Enum Client-side tracking

P2 - CLIENT WORK NEEDED

Event Type Status Notes
app.installed/opened/backgrounded/foregrounded/terminated/updated/crash Client In Enum Need client SDK implementation
video.sound_on/fullscreen/loop Client In Enum Need client implementation
search.filter_applied, search.sort_changed Client Not started Search UX events

BLOCKED - NO INFRASTRUCTURE EXISTS

Per team feedback: These advertiser/auction metrics don't have supporting infrastructure yet.

Event Why Blocked Future Dependency
ad.spend No advertiser billing system Ad platform build
auction.entry No ad auction system Ad platform build
auction.win No ad auction system Ad platform build
auction.position No ad auction system Ad platform build
conversion.event No conversion tracking E-commerce/attribution build
transaction.amount No transaction system E-commerce build
transaction.customer_id No transaction system E-commerce build
ad.view No ad-specific tracking Ad platform build
ad.click No ad-specific tracking Ad platform build
touchpoint No attribution service Attribution build
relevance.signal No ML/quality scoring Quality scoring build
landing_page.signal No landing page analysis Quality scoring build

FUTURE - Lower Priority

Event Type Why Deferred
message.* Both Messaging feature not current priority
invitation.* Both Invitation feature scope
suspicious.behavior, clock.skew Backend Integrity/fraud detection - future

Creator Metrics Coverage

Category Coverage Notes
Content Performance 90% Views, reach, video metrics covered
Engagement 95% Like, comment, share, save covered
Video Performance 85% Progress, completion, loop covered
Audience Growth 90% Follow/unfollow, profile views covered
Monetization 20% Missing creator.payout, boost.spend

Recommended Action Plan

This Week: Complete Current Work

Merge PRs in Code Review:

  • TVM-1700 - Display Events (Ron)
  • TVM-1701 - Click Events (Ron)
  • TVM-1731 - Event Type Enum (Ray)
  • TVM-1729 - Field Renaming (Jairo)
  • TVM-1732 - Engagement Events (Ray)

Next: P0 Backend Easy Wins

Create Tickets:

  • CDP: Add Creator Monetization Events (creator.payout, boost.spend)
  • CDP: Add Error Events (api.error, rate.limited, permission.denied)
  • CDP: Add Content Tagging Events (hashtag.applied, ticker.applied, tag.applied)

Future: When Infrastructure Exists

  • Advertiser/auction events (requires ad platform)
  • Transaction/conversion events (requires e-commerce)
  • Attribution events (requires attribution service)

Infrastructure Notes

Schema Validation (from team meeting)

  • Need JSON schema registry for Kafka
  • Need DLQ (Dead Letter Queue) for malformed messages
  • ClickHouse consumer protection for invalid timestamps

Team Philosophy

"We're going to default to the backend everywhere we can. It's fewer calls... if we deem that the backend doesn't have the context that the client does, we can switch to client."


References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment