System Design & Data Flow

System Design & Data Flow

The Merq platform is a multi-component system designed with clear separation of concerns. It consists of a central backend API, a web-based administration dashboard, an offline-first mobile application for field agents, and a set of supporting infrastructure services.

Architecture Diagram

This diagram illustrates the high-level relationship between the different parts of the Merq ecosystem.

flowchart TD
    subgraph Clients
        A("Web Dashboard\nReact, Vite")
        B("Mobile App\nReact Native")
    end

    subgraph Backend["Backend API (Go)"]
        C("Gin Web Server")
        D("Middleware\nAuth · RBAC · Workspace")
        E("Handlers")
        F("Services\nBusiness Logic")
        G("Repositories\nData Access")
    end

    subgraph Infrastructure
        H[("PostgreSQL\nPrimary Database")]
        I[("Redis\nCache")]
        J("Typesense\nSearch")
        K("Object Storage\nS3-Compatible")
        L("Firebase FCM\nPush Notifications")
    end

    A -->|HTTPS + JWT| C
    B -->|HTTPS + JWT| C
    C --> D
    D --> E
    E --> F
    F --> G
    G --> H
    G --> I
    F --> J
    F --> K
    F --> L

Component Responsibilities

1. Web Dashboard (merq-web):

   • Role: The primary interface for back-office and administrative users.
   • Interaction: Communicates exclusively with the Backend API via REST calls. It is a pure client-side application with no direct access to the database or other infrastructure.
   • Data Flow: Fetches data using TanStack Query, sends mutations for updates, and stores authentication tokens securely in the browser.

2. Mobile App (merq-mobile):

   • Role: The tool for field agents. It is designed to be functional offline.
   • Interaction: Synchronizes data with the Backend API when online.
   • Data Flow (Offline-First):
     • When online, TanStack Query fetches data from the API and caches it locally in MMKV.
     • When offline, the app reads exclusively from the local MMKV cache.
     • Mutations (writes) require an active network connection — there is no offline mutation queue.

3. Backend API (merq-backend):

   • Role: The single source of truth and the orchestrator of all business logic.
   • Internal Flow (Clean Architecture):
     1. Gin Server receives an HTTP request.
     2. Middleware intercepts the request to handle authentication (JWT validation), authorization (RBAC), and workspace scoping.
     3. The request is routed to a specific Handler, which parses and validates request parameters.
     4. The Handler calls a Service method, passing the validated data.
     5. The Service executes the core business logic and calls one or more Repositories.
     6. The Repository builds and executes the GORM query against PostgreSQL or Redis.
     7. The result flows back up the chain and the Handler returns the final HTTP response.

4. Infrastructure:

   • PostgreSQL: The primary relational database for all core business data.
   • Redis: High-speed cache for frequently accessed data (e.g., permissions, workspace settings).
   • Typesense: External search engine providing fast, typo-tolerant search, kept in sync by backend services.
   • Object Storage (S3): Stores binary files (images, documents) uploaded by field agents via pre-signed URLs.
   • Firebase (FCM): Used by the backend to send push notifications to mobile devices.

Backend Services

Regional Service Integration

Merq integrates with a separate Indonesian Regional service for administrative division data (province, city, district, village).

graph TB
    Web[Web Admin] --> Backend[merq-backend]
    Mobile[Mobile App] --> Backend
    Backend -->|HTTP Proxy| Regional[merq-indonesian-regional]
    Regional --> DB_Reg[(reg_* tables)]
    Backend --> DB_Main[(main DB)]
    
    style Regional fill:#fff3e0
    style DB_Reg fill:#ffe0b2

Key Points:

• Separate service with own PostgreSQL database
• HTTP proxy pattern from merq-backend
• Both services in same Docker network (Dokploy)
• Configured via REGIONAL_SERVICE_URL environment variable
• Graceful degradation: returns empty data if regional service unreachable

Use Cases:

• Outlet regional location (store_province_id, etc.)
• Employee address (province, city dropdowns)
• Reporting by region

See Regional Service API for details.

Mobile Architecture

Sales Order Flow

The mobile app supports barcode-enabled sales order creation with offline-first support.

sequenceDiagram
    participant User as Field Agent
    participant UI as Mobile UI
    participant Outbox as MMKV Outbox
    participant API as Backend API
    
    User->>UI: Scan barcode
    UI->>API: GET /products/barcode/:code
    API-->>UI: Product details
    User->>UI: Add to cart
    User->>UI: Select order/payment type
    User->>UI: Capture PO photos + signature
    User->>UI: Review & submit
    UI->>Outbox{Online?}
    Outbox->>API: POST /sales-orders
    API-->>Outbox: Success
    Outbox-->>UI: Synced
    Note over Outbox: If offline: queue

Key Components:

• CreateSalesOrderScreen — Cart management + order/payment type
• ReviewOrderScreen — Final review before submit
• useProductByBarcodeQuery — Barcode lookup with caching
• useCreateSalesOrderMutation — Offline-aware mutation

See Sales Order for details.

Key Data Flows

1. Outlet Enrichment Data Flow

graph LR
    Web[Web Admin] --> Backend[Backend API]
    Backend --> CG[ChannelGroup]
    Backend --> C[Channel]
    Backend --> A[Account]
    Backend --> SA[SubArea]
    Backend --> Regional[Regional Service]
    
    CG --> DB[(workspace-scoped)]
    C --> DB
    A --> DB
    SA --> DB
    Regional --> ExtDB[(reg_* tables)]

Flow:

1. Admin creates ChannelGroup → Channel → Account hierarchy
2. Admin creates/edits Outlet with enrichment fields
3. Backend stores IDs, resolves names from regional service
4. Mobile app displays enriched outlet data

2. Sales Order Data Flow

graph LR
    Mobile[Mobile App] -->|1. Scan| Barcode[Barcode Scan]
    Mobile -->|2. Lookup| Product[Product API]
    Mobile -->|3. Queue| Outbox[MMKV Outbox]
    Outbox -->|4. Sync| Backend[Backend API]
    Backend -->|5. Create| SO[Sales Order]
    Backend -->|6. Link| Visit[Outlet Visit]

3. Location Tracking Data Flow

graph LR
    GPS[Background GPS] --> App[Mobile App]
    App -->|Online| Upload[Immediate Upload]
    App -->|Offline| Buffer[MMKV Buffer]
    Buffer -->|Reconnect| Batch[Batch Upload]
    Batch --> Backend[Backend API]

Component Interactions

merq-indonesian-regional Service

Purpose: Centralized Indonesian administrative division data

Tables:

• reg_provinces (~34 rows)
• reg_cities (~514 rows)
• reg_districts (~7,200 rows)
• reg_villages (~83,000+ rows)

Integration:

• Backend proxies requests via HTTP
• No cross-DB foreign keys (IDs stored as plain integers)
• Name resolution at API response time

Configuration:

REGIONAL_SERVICE_URL=http://merq-indonesian-regional:8081

Idempotency System

Purpose: Prevent duplicate mutations on retry

Components:

• Client: UUID generator + header interceptor
• Server: Middleware + idempotency_keys table
• TTL: 24 hours (lazy eviction)

Flow:

1. Client generates UUID for mutation
2. Include in X-Idempotency-Key header
3. Server checks table for existing key
4. If exists: Return cached response (409)
5. If new: Process, store key, return success

See Idempotency for details.

Technology Stack

Backend

• Go 1.24 — Primary language
• Gin — HTTP web framework
• GORM — ORM for PostgreSQL
• PostgreSQL — Primary database
• Redis — Caching layer
• Typesense — Search engine
• Regional Service: Separate Go service for Indonesian regional data

Mobile

• React Native 0.80 — Mobile framework
• Paper MD3 — UI component library
• MMKV — Local storage
• TanStack Query — Data fetching with offline support
• MMKV Outbox: Offline mutation queue
• Location Buffer: GPS tracking buffer
• Idempotency: Duplicate prevention

Web

• React 18 — Frontend framework
• Vite — Build tool
• shadcn/ui — UI component library
• TanStack Query — Data fetching
• TypeScript — Type safety

Related Documentation

• Regional Service API
• Sales Order
• Location Tracking
• Idempotency
• Outlet Enrichment