System Architecture - System-Level Structure and Trade-offs

Choose a system topology that matches scale, team size, and operational maturity.

Quick Overview

• Modular Monolith: Single deployable with strong internal boundaries.
• Microservices: Independent services with separate deploys and data stores.
• Event-Driven Architecture: Services communicate via events and queues.
• API Gateway / BFF: Single entry point for clients, routing to backend services.
• Multi-Tenant SaaS: Shared infrastructure with tenant isolation at data layer.

Detailed Explanation

Modular Monolith

What it is: A single deployment with strict module boundaries (e.g., Orders, Pricing, Risk).

Pros:

• Simple deployment and debugging
• Strong consistency and transactions
• Clear boundaries without distributed complexity

Cons:

• Scaling is all-or-nothing
• Boundary violations creep in without discipline

Use it when: Team is small to medium and you want fast delivery with strong consistency.

Microservices Architecture

What it is: Independent services owning their data and deploy lifecycle.

                        [Keycloak IDP]
                              |
                         JWT Tokens
                              |
    [API Gateway] -----> Authentication
         |
    +----|----+---------+---------+
    |         |         |         |
[Identity] [Orders] [Content] [Menu]
    |         |         |         |
  [DB]      [DB]     [S3]      [DB]

Pros:

• Independent scaling and deployments
• Clear ownership boundaries
• Technology flexibility per service
• Team autonomy

Cons:

• Distributed systems complexity
• Data consistency challenges
• Higher operational overhead (observability, tracing, CI/CD)

Key Components:

Use it when: You have multiple teams, different scaling needs, and strong DevOps maturity.

Event-Driven Architecture

What it is: Services publish events; consumers react asynchronously.

[Order Service] ---> [Message Broker] ---> [Notification Service]
                           |
                           +-----------> [Analytics Service]
                           |
                           +-----------> [Billing Service]

Event Patterns:

Pros:

• Loose coupling between services
• Natural fit for streaming (price ticks, trade events)
• Resilient under bursts
• Easy to add new consumers

Cons:

• Eventual consistency
• Harder debugging without tracing
• Need idempotency and deduplication
• Event schema evolution challenges

Use it when: You need high throughput, asynchronous workflows, or streaming pipelines.

API Gateway Pattern

What it is: Single entry point that routes requests to appropriate microservices.

[Mobile App]  [Web App]  [Third Party]
      \          |          /
       \         |         /
        +--------+---------+
                 |
           [API Gateway]
           - Authentication
           - Rate Limiting
           - Request Routing
           - Response Aggregation
                 |
    +------------+------------+
    |            |            |
[Service A] [Service B] [Service C]

Responsibilities:

• Authentication/Authorization: Validate JWT tokens, enforce policies
• Rate Limiting: Protect services from abuse, per-tenant limits
• Request Routing: Route to appropriate backend service
• Response Aggregation: Combine multiple service responses (BFF pattern)
• Protocol Translation: REST to gRPC, WebSocket proxying
• Caching: Edge caching for common requests

Technologies:

• YARP (C# reverse proxy)
• Kong, AWS API Gateway
• Azure API Management
• Nginx, Envoy

Use it when: You have multiple backend services and need unified edge policies.

Multi-Tenant SaaS Architecture

What it is: Single application instance serving multiple tenants with data isolation.

Isolation Strategies:

Row-Level Implementation (EF Core):

// Automatic tenant filtering in DbContext
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity<Order>().HasQueryFilter(e =>
        _currentTenantService.TenantId == null || // SuperUser bypass
        e.TenantId == _currentTenantService.TenantId);
}

Tenant Resolution:

• JWT claims (tenant_id claim)
• Subdomain (tenant1.app.com)
• Header (X-Tenant-Id)
• Path segment (/api/tenants/{tenantId}/orders)

Key Considerations:

• Data Isolation: Prevent cross-tenant data access
• Resource Limits: Per-tenant quotas, rate limiting
• Configuration: Tenant-specific settings, feature flags
• Billing/Metering: Track usage per tenant
• Onboarding: Self-service vs manual provisioning

Comparison Table

Observability Requirements

Per Architecture Style

Essential Tools

Security Considerations

OWASP Top 10 for Distributed Systems

Multi-Tenant Security Checklist

• [ ] Tenant ID validated on every request
• [ ] Global query filters prevent cross-tenant access
• [ ] Separate encryption keys per tenant (optional)
• [ ] Rate limiting per tenant
• [ ] Audit logging with tenant context

Scalability Patterns

Horizontal Scaling

Scaling Checklist

• [ ] Services are stateless
• [ ] Database connections pooled
• [ ] Session state externalized (Redis)
• [ ] Health checks implemented
• [ ] Horizontal Pod Autoscaling configured

Why It Matters for Interviews

• You can explain trade-offs instead of repeating buzzwords.
• You can tie topology to operational maturity (observability, CI/CD, on-call).
• You can articulate data consistency and failure modes.
• You can discuss multi-tenant isolation strategies.

Common Pitfalls

• Jumping to microservices without operational readiness.
• Ignoring data ownership boundaries between services.
• Mixing synchronous calls with event-driven flows without clear SLAs.
• Treating a monolith as a microservice without modular boundaries.
• Missing API Gateway when services proliferate.
• Forgetting tenant isolation in multi-tenant systems.
• No distributed tracing in microservices (debugging nightmare).
• CORS configuration duplicated across services.

Quick Reference

• Modular Monolith: One deploy, clear boundaries, strong consistency.
• Microservices: Multiple deploys, independent scaling, higher ops cost.
• Event-Driven: Async workflows, eventual consistency, needs idempotency.
• API Gateway: Single entry point, unified auth, rate limiting.
• Multi-Tenant: Shared infra, data isolation, per-tenant configuration.

Sample Interview Q&A

• Q: When would you prefer a modular monolith over microservices?
• A: When you need speed, strong consistency, and your team/ops maturity is not ready for distributed complexity.

• Q: How do you avoid data inconsistency in microservices?
• A: Use clear data ownership, outbox/inbox patterns, idempotent consumers, and sagas for workflows.

• Q: What is the purpose of an API Gateway?
• A: Single entry point for clients providing unified authentication, rate limiting, routing, and protocol translation. Prevents clients from knowing about individual services.

• Q: How do you implement multi-tenant data isolation?
• A: Global query filters in EF Core that automatically filter by TenantId. Resolve tenant from JWT claims or headers. Ensure filters cannot be bypassed except by superuser roles.

• Q: What observability is required for microservices?
• A: Centralized logging with correlation IDs, distributed tracing (OpenTelemetry), metrics with service-level dashboards, and health check endpoints for orchestrators.

• Q: Why is event-driven architecture common in trading systems?
• A: Market data and trades are naturally streams of events, and async pipelines handle bursts without blocking request paths.