torch.compile Optimization AI Prompt

Apply torch.compile to optimize this PyTorch model for both training and inference.

1. Basic compilation:
   - Apply torch.compile(model) with default settings
   - Measure speedup: run 50 warmup steps, then benchmark 200 steps
   - Expected speedup: 1.5–3× for well-supported models, less for models with dynamic shapes

2. Compilation modes — benchmark and recommend:
   - 'default': balanced compile time and runtime performance
   - 'reduce-overhead': minimize Python overhead, best for small batches
   - 'max-autotune': exhaustive kernel search, longest compile but best runtime
   - 'max-autotune-no-cudagraphs': use if CUDA graphs cause issues with dynamic shapes

3. Backend selection:
   - 'inductor' (default): best general performance
   - 'cudagraphs': lowest latency for fixed-size inputs
   - 'onnxrt': for ONNX-compatible subgraphs

4. Dynamic shapes:
   - Use dynamic=True if input shapes vary at runtime
   - Use torch._dynamo.mark_dynamic(tensor, dim) for specific dynamic dimensions
   - Static shapes (default) produce faster code but recompile on shape changes

5. Debugging compilation issues:
   - torch._dynamo.explain(model)(input) to see why a graph break occurs
   - Set TORCH_LOGS=recompiles to monitor recompilation events
   - Use torch._dynamo.disable decorator to exclude problematic submodules

6. Training vs inference:
   - Compile the model before wrapping with DDP
   - Compile the loss function separately if it is a significant cost

Return: compilation setup, mode comparison benchmark, dynamic shape handling, and debugging guide.