page-view

Description

A page view event is a record of a user visiting a page on the website with code snipped installed.

class PageEvent(BaseKafkaModel):
    _topic = 'page_events'
 
    def __init__(self, channel_id: int, timestamp: int = None, visit_count: int = None, 
                 browser: str = None, subscriber_uuid: str = None, **kwargs):
        super(PageEvent, self).__init__(**kwargs)
        self.channel_id = channel_id
        self.browser = browser or ''
        self.visit_count = visit_count or 0
        self.subscriber_uuid = subscriber_uuid or ''
        self.timestamp = timestamp or int(time.time())

Cleaning

import gc
 
from webpush.lib.kafka.clients import Consumer
from webpush.lib.kafka.models.page import PageEvent
 
i = 0
with Consumer([PageEvent], 'statistics', prepare_topics=False, with_manual_commits=False) as consumer:
    while True:
        print('.')
        gc.collect()
        for _ in range(2500):
            try:
                req = consumer.get(timeout=1)
            except TimeoutError:
                continue

PRODUCERS

manager-api

Produces event every time a user visits a page with a code snippet installed.

CONSUMERS

clickhouse

Collects the data in a form of several tables:

• page_events
• page_events_analytics

directory events listener

Checks if subscriber who initiated the event has any source parameters related. If so - pushes source parameter event to the Kafka.