SYSTEM ARCHITECTURE

The Backend Engine

A high-performance orchestration layer transforming raw UIDAI logs into actionable intelligence.

Singleton Data Engine

The ChronosDataEngine class (api_server.py:236) uses a Singleton pattern to maintain the aggregated state in memory. By leveraging Polars' Lazy API, we optimize query plans before execution, ensuring sub-millisecond aggregation times even on millions of rows.

Zero-Copy Arrow Memory
Data stays in Apache Arrow format from disk to response buffer, eliminating serialization overhead.
Lazy Execution Strategy
Query optimizer pushes down predicates and filters before reading full dataset.
Parquet Compression
500MB CSVs → 45MB Snappy-compressed Parquet with dictionary encoding.

api_server.py: ChronosDataEngine.aggregate_by_state

def aggregate_by_state(self, month):
  # 1. Lazy Scan & Filter
  df = self.pulse_vectors.filter(
      pl.col("month") == month_val
  )
  
  # 2. Parallel GroupBy (Rust Backend)
  state_agg = df.group_by("state").agg([
      pl.col("growth_norm").mean(),
      pl.col("flow_norm").mean(),
      pl.col("labor_norm").mean()
  ])
  
  # 3. zero-copy collect() to PyArrow
  return [StateAggregation(**row) 
          for row in state_agg.iter_rows()]

src/ai_forecaster.py: AIForecaster.forecast_pin

def forecast_pin(self, history, samples=20):
  torch = _get_torch()
  
  # 1. Convert to Tensor
  context = torch.tensor(history, 
                         dtype=torch.float32)
  
  # 2. Chronos T5 Inference
  forecast = self.pipeline.predict(
      context, horizon=6, 
      num_samples=samples
  )
  
  # 3. Extract Uncertainty Quantiles
  low, median, high = np.quantile(
      forecast[0].numpy(), 
      [0.1, 0.5, 0.9], axis=0
  )
  
  return { "median": median.tolist(), ... }

The AI Forecaster

The AIForecaster class (src/ai_forecaster.py:56) wraps the Amazon Chronos T5 foundation model. Since Chronos is pre-trained on millions of time series, it can perform Zero-Shot Forecasting even with our limited 2-month hackathon dataset.

Input:torch.tensor([Nov_val, Dec_val])
Output:Quantiles [0.1, 0.5, 0.9] for uncertainty bounds
Horizon:Flexible (tuned to 6 months)

Hybrid Ensemble

We don't trust AI blindly. The HybridPredictor class blends T5 predictions with Spatial Parallel Search (Cosine Similarity) results.
Final = w1 * AI_Forecast + w2 * Parallel_Outcome

FastAPI Orchestrator

Connecting the AI brain to the frontend interface.

WebSocket Protocol

Real-time bidirectional communication for the Chronos Copilot.

ws://api/chat
-> { "query": "Analysis for Mumbai" }
<- { "type": "token", "content": "..." }

Context Injection

Dynamically injecting seasonality data (Rabi/Kharif harvests) via ContextEngine.

Prompt += "Season: Rabi Harvest..."

Geo-Resolver

Fuzzy logic engine converting "Banglore" to "560xxx" using SequenceMatcher.

difflib.SequenceMatcher(query, city)