Spaces:
Running
on
Zero
fix: resolve MCP server tracking and risk metrics display issues
Browse filesResolved two critical UI display issues in the portfolio analysis application:
Issue 1: MCP Server Count Displaying as Zero
- Updated MCPCall model to accept both 'mcp' and 'mcp_server' field names
using Pydantic validation_alias for backwards compatibility
- Modified workflow to create proper MCPCall model instances instead of
plain dictionaries
- Added populate_by_name configuration to model for flexible field naming
Issue 2: Risk Metrics Showing Raw Dictionary Structures
- Implemented format_risk_metrics() function to properly handle nested
Pydantic model structures (VaRResult, CVaRResult, RiskMetrics)
- Added financial formatting following industry best practices:
* Currency values with commas and 2 decimal places
* Percentages with 2 decimal places
* Ratios with 3 decimal places
- Included user-friendly interpretations for VaR and CVaR metrics
- Added star rating system for Sharpe and Sortino ratios
Technical improvements:
- Applied Pydantic V2 best practices for model serialisation
- Added comprehensive test suite (test_fixes.py)
- Created documentation (FIXES_SUMMARY.md)
- Maintained backwards compatibility with existing data structures
Files modified:
- backend/models/agent_state.py: MCPCall model with validation_alias
- backend/agents/workflow.py: MCPCall instance creation
- app.py: Risk metrics formatting and display logic
- FIXES_SUMMARY.md +291 -0
- app.py +142 -5
- backend/agents/workflow.py +10 -9
- backend/models/agent_state.py +8 -3
- test_fixes.py +119 -0
|
@@ -0,0 +1,291 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Implementation Summary: MCP Tracking and Risk Metrics Fixes
|
| 2 |
+
|
| 3 |
+
## Overview
|
| 4 |
+
Successfully implemented two critical fixes to improve MCP server tracking and risk metrics display in the Portfolio Intelligence Platform.
|
| 5 |
+
|
| 6 |
+
## Fix 1: MCP Tracking Field Name Mismatch
|
| 7 |
+
|
| 8 |
+
### Problem
|
| 9 |
+
- Workflow created dicts with `"mcp"` key
|
| 10 |
+
- MCPCall model expected `"mcp_server"` field
|
| 11 |
+
- app.py checked for `"mcp_server"` in dicts
|
| 12 |
+
- Result: MCP server count always showed 0
|
| 13 |
+
|
| 14 |
+
### Solution Implemented
|
| 15 |
+
|
| 16 |
+
#### 1. Updated MCPCall Model (`backend/models/agent_state.py`)
|
| 17 |
+
```python
|
| 18 |
+
class MCPCall(BaseModel):
|
| 19 |
+
"""Record of an MCP tool call.
|
| 20 |
+
|
| 21 |
+
Accepts both 'mcp_server' and 'mcp' field names for backward compatibility.
|
| 22 |
+
"""
|
| 23 |
+
|
| 24 |
+
model_config = {"populate_by_name": True}
|
| 25 |
+
|
| 26 |
+
mcp_server: str = Field(..., validation_alias="mcp", description="MCP server name")
|
| 27 |
+
tool_name: str = Field(..., validation_alias="tool", description="Tool called")
|
| 28 |
+
# ... other fields
|
| 29 |
+
```
|
| 30 |
+
|
| 31 |
+
**Key Changes:**
|
| 32 |
+
- Added `model_config = {"populate_by_name": True}` to enable alias support
|
| 33 |
+
- Added `validation_alias="mcp"` to `mcp_server` field
|
| 34 |
+
- Added `validation_alias="tool"` to `tool_name` field
|
| 35 |
+
- Model now accepts both old (`"mcp"`) and new (`"mcp_server"`) field names
|
| 36 |
+
|
| 37 |
+
#### 2. Updated Workflow (`backend/agents/workflow.py`)
|
| 38 |
+
```python
|
| 39 |
+
# Import MCPCall
|
| 40 |
+
from backend.models.agent_state import AgentState, MCPCall
|
| 41 |
+
|
| 42 |
+
# Phase 1: Create proper MCPCall instances
|
| 43 |
+
state["mcp_calls"].extend([
|
| 44 |
+
MCPCall.model_validate({"mcp": "yahoo_finance", "tool": "get_quote"}).model_dump(),
|
| 45 |
+
MCPCall.model_validate({"mcp": "yahoo_finance", "tool": "get_historical_data"}).model_dump(),
|
| 46 |
+
MCPCall.model_validate({"mcp": "fmp", "tool": "get_company_profile"}).model_dump(),
|
| 47 |
+
MCPCall.model_validate({"mcp": "trading_mcp", "tool": "get_technical_indicators"}).model_dump(),
|
| 48 |
+
MCPCall.model_validate({"mcp": "fred", "tool": "get_economic_series"}).model_dump(),
|
| 49 |
+
])
|
| 50 |
+
|
| 51 |
+
# Phase 2: Create proper MCPCall instances
|
| 52 |
+
state["mcp_calls"].extend([
|
| 53 |
+
MCPCall.model_validate({"mcp": "portfolio_optimizer_mcp", "tool": "optimize_hrp"}).model_dump(),
|
| 54 |
+
MCPCall.model_validate({"mcp": "portfolio_optimizer_mcp", "tool": "optimize_black_litterman"}).model_dump(),
|
| 55 |
+
MCPCall.model_validate({"mcp": "portfolio_optimizer_mcp", "tool": "optimize_mean_variance"}).model_dump(),
|
| 56 |
+
MCPCall.model_validate({"mcp": "risk_analyzer_mcp", "tool": "analyze_risk"}).model_dump(),
|
| 57 |
+
])
|
| 58 |
+
```
|
| 59 |
+
|
| 60 |
+
**Key Changes:**
|
| 61 |
+
- Changed from plain dicts to `MCPCall.model_validate()` calls
|
| 62 |
+
- Model validates and converts `"mcp"` to `"mcp_server"`
|
| 63 |
+
- `model_dump()` returns dict with proper `"mcp_server"` key
|
| 64 |
+
- app.py can now correctly count unique MCP servers
|
| 65 |
+
|
| 66 |
+
### Expected Output
|
| 67 |
+
```
|
| 68 |
+
π§ MCP Servers Called
|
| 69 |
+
Used 6 MCP servers: fmp, fred, portfolio_optimizer_mcp, risk_analyzer_mcp, trading_mcp, yahoo_finance
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
## Fix 2: Risk Metrics Formatting
|
| 73 |
+
|
| 74 |
+
### Problem
|
| 75 |
+
- Generic key-value display for risk metrics
|
| 76 |
+
- Nested dict structures (VaR, CVaR) displayed as unhelpful text
|
| 77 |
+
- No formatting for monetary values or percentages
|
| 78 |
+
- No interpretations for risk metrics
|
| 79 |
+
|
| 80 |
+
### Solution Implemented
|
| 81 |
+
|
| 82 |
+
#### 1. Created `format_risk_metrics()` Function (`app.py`)
|
| 83 |
+
```python
|
| 84 |
+
def format_risk_metrics(risk_data: Dict[str, Any], portfolio_value: float) -> str:
|
| 85 |
+
"""Format risk analysis metrics for financial UI display.
|
| 86 |
+
|
| 87 |
+
Args:
|
| 88 |
+
risk_data: Risk analysis results from risk_analyzer_mcp
|
| 89 |
+
portfolio_value: Total portfolio value for percentage calculations
|
| 90 |
+
|
| 91 |
+
Returns:
|
| 92 |
+
Formatted markdown string with risk metrics
|
| 93 |
+
"""
|
| 94 |
+
# Handles nested VaR structure
|
| 95 |
+
# Handles nested CVaR structure
|
| 96 |
+
# Formats volatility, Sharpe ratio, Sortino ratio, max drawdown
|
| 97 |
+
# Adds star ratings for performance metrics
|
| 98 |
+
# Includes interpretations for each metric
|
| 99 |
+
```
|
| 100 |
+
|
| 101 |
+
**Key Features:**
|
| 102 |
+
- Proper handling of nested dict structures
|
| 103 |
+
- Monetary values formatted with $ and commas: `$1,119.58`
|
| 104 |
+
- Percentages formatted with 2 decimals: `3.90%`
|
| 105 |
+
- Star ratings for Sharpe/Sortino ratios: `βββ`
|
| 106 |
+
- User-friendly interpretations for each metric
|
| 107 |
+
|
| 108 |
+
#### 2. Created `get_rating_stars()` Helper Function (`app.py`)
|
| 109 |
+
```python
|
| 110 |
+
def get_rating_stars(value: float, metric_type: str = "sharpe") -> str:
|
| 111 |
+
"""Get star rating for Sharpe or Sortino ratio.
|
| 112 |
+
|
| 113 |
+
Sharpe Ratio Thresholds:
|
| 114 |
+
- βββββ >= 3.0 (Exceptional)
|
| 115 |
+
- ββββ >= 2.0 (Excellent)
|
| 116 |
+
- βββ >= 1.0 (Good)
|
| 117 |
+
- ββ >= 0.5 (Fair)
|
| 118 |
+
- β > 0 (Poor)
|
| 119 |
+
"""
|
| 120 |
+
```
|
| 121 |
+
|
| 122 |
+
#### 3. Updated Risk Analysis Section (`app.py`)
|
| 123 |
+
```python
|
| 124 |
+
# Risk Analysis
|
| 125 |
+
if state.get('risk_analysis'):
|
| 126 |
+
output.append("\n## β οΈ Risk Metrics\n")
|
| 127 |
+
risk = state['risk_analysis']
|
| 128 |
+
if isinstance(risk, dict):
|
| 129 |
+
# Calculate total portfolio value for percentage context
|
| 130 |
+
portfolio_value = sum(h.get('market_value', 0) for h in holdings)
|
| 131 |
+
formatted_risk = format_risk_metrics(risk, portfolio_value)
|
| 132 |
+
if formatted_risk:
|
| 133 |
+
output.append(formatted_risk)
|
| 134 |
+
else:
|
| 135 |
+
# Fallback to generic display if formatting returns nothing
|
| 136 |
+
# ... fallback code
|
| 137 |
+
```
|
| 138 |
+
|
| 139 |
+
### Expected Output
|
| 140 |
+
```
|
| 141 |
+
β οΈ Risk Metrics
|
| 142 |
+
|
| 143 |
+
**Value at Risk (VaR) - 95% Confidence, 1-Day Horizon**
|
| 144 |
+
- Maximum Expected Loss: $1,119.58 (3.90% of portfolio)
|
| 145 |
+
- *Interpretation: With 95% confidence, maximum expected loss over 1 day is $1,119.58*
|
| 146 |
+
|
| 147 |
+
**Conditional Value at Risk (CVaR) - 95% Confidence**
|
| 148 |
+
- Expected Shortfall: $1,409.97 (4.91% of portfolio)
|
| 149 |
+
- *Interpretation: If losses exceed VaR threshold, average expected loss is $1,409.97*
|
| 150 |
+
|
| 151 |
+
**Risk-Adjusted Performance**
|
| 152 |
+
- Annual Volatility: 39.30%
|
| 153 |
+
- Sharpe Ratio: 0.839 ββ
|
| 154 |
+
- Sortino Ratio: 1.256 βββ
|
| 155 |
+
- Maximum Drawdown: -39.10%
|
| 156 |
+
```
|
| 157 |
+
|
| 158 |
+
## Files Modified
|
| 159 |
+
|
| 160 |
+
1. **backend/models/agent_state.py**
|
| 161 |
+
- Added Pydantic validation_alias to MCPCall model
|
| 162 |
+
- Enabled populate_by_name in model config
|
| 163 |
+
- Supports both old and new field names
|
| 164 |
+
|
| 165 |
+
2. **backend/agents/workflow.py**
|
| 166 |
+
- Imported MCPCall model
|
| 167 |
+
- Changed from plain dicts to MCPCall instances
|
| 168 |
+
- Added get_historical_data to Phase 1 tracking
|
| 169 |
+
- Updated MCP server names for Phase 2
|
| 170 |
+
|
| 171 |
+
3. **app.py**
|
| 172 |
+
- Created format_risk_metrics() function
|
| 173 |
+
- Created get_rating_stars() helper function
|
| 174 |
+
- Updated risk analysis section to use new formatter
|
| 175 |
+
- Added fallback for edge cases
|
| 176 |
+
|
| 177 |
+
## Testing
|
| 178 |
+
|
| 179 |
+
Created comprehensive test suite (`test_fixes.py`) that verifies:
|
| 180 |
+
|
| 181 |
+
1. **MCPCall Validation Alias**
|
| 182 |
+
- Accepts both `"mcp"` and `"mcp_server"` field names
|
| 183 |
+
- model_dump() produces dicts with `"mcp_server"` key
|
| 184 |
+
- Backwards compatibility maintained
|
| 185 |
+
|
| 186 |
+
2. **Risk Metrics Formatting**
|
| 187 |
+
- Nested VaR/CVaR structures formatted correctly
|
| 188 |
+
- Monetary values use $ and commas
|
| 189 |
+
- Percentages formatted with proper precision
|
| 190 |
+
- Star ratings display correctly
|
| 191 |
+
- All key sections present in output
|
| 192 |
+
|
| 193 |
+
3. **Star Rating System**
|
| 194 |
+
- Sharpe ratio thresholds work correctly
|
| 195 |
+
- Sortino ratio thresholds work correctly
|
| 196 |
+
- Edge cases handled
|
| 197 |
+
|
| 198 |
+
### Test Results
|
| 199 |
+
```
|
| 200 |
+
β
All MCPCall validation tests passed!
|
| 201 |
+
β
All risk metrics formatting tests passed!
|
| 202 |
+
β
All star rating tests passed!
|
| 203 |
+
```
|
| 204 |
+
|
| 205 |
+
## Verification Steps
|
| 206 |
+
|
| 207 |
+
To verify the fixes work in the live application:
|
| 208 |
+
|
| 209 |
+
1. **Start the application:**
|
| 210 |
+
```bash
|
| 211 |
+
python app.py
|
| 212 |
+
```
|
| 213 |
+
|
| 214 |
+
2. **Enter a test portfolio:**
|
| 215 |
+
```
|
| 216 |
+
AAPL 50
|
| 217 |
+
TSLA 25
|
| 218 |
+
NVDA $5000
|
| 219 |
+
```
|
| 220 |
+
|
| 221 |
+
3. **Click "Analyse Portfolio"**
|
| 222 |
+
|
| 223 |
+
4. **Verify MCP Tracking:**
|
| 224 |
+
- Look for "π§ MCP Servers Called" section
|
| 225 |
+
- Should show: "Used 6 MCP servers: fmp, fred, portfolio_optimizer_mcp, risk_analyzer_mcp, trading_mcp, yahoo_finance"
|
| 226 |
+
- Previously showed: "Used 0 MCP servers"
|
| 227 |
+
|
| 228 |
+
5. **Verify Risk Metrics:**
|
| 229 |
+
- Look for "β οΈ Risk Metrics" section
|
| 230 |
+
- Should show formatted VaR with dollar amount and percentage
|
| 231 |
+
- Should show formatted CVaR with interpretation
|
| 232 |
+
- Should show star ratings for Sharpe and Sortino ratios
|
| 233 |
+
- Previously showed: generic key-value pairs with dict objects as strings
|
| 234 |
+
|
| 235 |
+
## Design Patterns Used
|
| 236 |
+
|
| 237 |
+
1. **Pydantic Validation Alias Pattern**
|
| 238 |
+
- Enables field name migration without breaking changes
|
| 239 |
+
- Allows gradual refactoring of codebase
|
| 240 |
+
- Maintains backwards compatibility
|
| 241 |
+
|
| 242 |
+
2. **Financial UI Formatting Pattern**
|
| 243 |
+
- Separates data structure from display format
|
| 244 |
+
- Handles nested structures gracefully
|
| 245 |
+
- Provides context and interpretation for users
|
| 246 |
+
- Follows financial industry best practices
|
| 247 |
+
|
| 248 |
+
3. **Defensive Programming**
|
| 249 |
+
- Fallback to generic display if formatting fails
|
| 250 |
+
- Type checking for nested structures
|
| 251 |
+
- Handles missing or unexpected data gracefully
|
| 252 |
+
|
| 253 |
+
## Follow-Up Recommendations
|
| 254 |
+
|
| 255 |
+
1. **MCP Tracking Enhancements:**
|
| 256 |
+
- Add duration_ms tracking for performance monitoring
|
| 257 |
+
- Add error tracking for failed MCP calls
|
| 258 |
+
- Consider adding result size tracking
|
| 259 |
+
|
| 260 |
+
2. **Risk Metrics Enhancements:**
|
| 261 |
+
- Add color coding for risk levels (red/yellow/green)
|
| 262 |
+
- Add comparison to benchmarks
|
| 263 |
+
- Add historical risk trends
|
| 264 |
+
- Consider adding risk score visualisations
|
| 265 |
+
|
| 266 |
+
3. **Testing:**
|
| 267 |
+
- Add integration tests with actual MCP calls
|
| 268 |
+
- Add UI snapshot tests for Gradio interface
|
| 269 |
+
- Add performance benchmarks
|
| 270 |
+
|
| 271 |
+
4. **Documentation:**
|
| 272 |
+
- Add user guide for interpreting risk metrics
|
| 273 |
+
- Add developer guide for adding new MCP servers
|
| 274 |
+
- Add architecture diagram showing data flow
|
| 275 |
+
|
| 276 |
+
## Compliance
|
| 277 |
+
|
| 278 |
+
All changes follow the project guidelines:
|
| 279 |
+
- British English spelling (analyse, optimise)
|
| 280 |
+
- Google-style docstrings with type hints
|
| 281 |
+
- No emojis in code (only in user-facing UI)
|
| 282 |
+
- Conventional commit style ready
|
| 283 |
+
- No Claude Code attribution in code
|
| 284 |
+
|
| 285 |
+
## Conclusion
|
| 286 |
+
|
| 287 |
+
Both fixes successfully implemented and tested. The application now:
|
| 288 |
+
1. Correctly tracks and displays MCP server usage
|
| 289 |
+
2. Provides professional, user-friendly risk metrics formatting
|
| 290 |
+
3. Maintains backwards compatibility
|
| 291 |
+
4. Follows financial UI best practices
|
|
@@ -119,6 +119,136 @@ async def run_analysis(portfolio_text: str) -> str:
|
|
| 119 |
return f"β **Error during analysis**: {str(e)}\n\nPlease check your API keys in .env file and try again."
|
| 120 |
|
| 121 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 122 |
def format_analysis_results(state: AgentState, holdings: List[Dict]) -> str:
|
| 123 |
"""Format workflow results for Gradio display.
|
| 124 |
|
|
@@ -163,11 +293,18 @@ def format_analysis_results(state: AgentState, holdings: List[Dict]) -> str:
|
|
| 163 |
output.append("\n## β οΈ Risk Metrics\n")
|
| 164 |
risk = state['risk_analysis']
|
| 165 |
if isinstance(risk, dict):
|
| 166 |
-
|
| 167 |
-
|
| 168 |
-
|
| 169 |
-
|
| 170 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 171 |
|
| 172 |
# Optimization Results
|
| 173 |
if state.get('optimisation_results'):
|
|
|
|
| 119 |
return f"β **Error during analysis**: {str(e)}\n\nPlease check your API keys in .env file and try again."
|
| 120 |
|
| 121 |
|
| 122 |
+
def format_risk_metrics(risk_data: Dict[str, Any], portfolio_value: float) -> str:
|
| 123 |
+
"""Format risk analysis metrics for financial UI display.
|
| 124 |
+
|
| 125 |
+
Args:
|
| 126 |
+
risk_data: Risk analysis results from risk_analyzer_mcp
|
| 127 |
+
portfolio_value: Total portfolio value for percentage calculations
|
| 128 |
+
|
| 129 |
+
Returns:
|
| 130 |
+
Formatted markdown string with risk metrics
|
| 131 |
+
"""
|
| 132 |
+
output = []
|
| 133 |
+
|
| 134 |
+
# VaR (Value at Risk)
|
| 135 |
+
if "var" in risk_data:
|
| 136 |
+
var_data = risk_data["var"]
|
| 137 |
+
if isinstance(var_data, dict):
|
| 138 |
+
confidence = var_data.get("confidence_level", 0.95) * 100
|
| 139 |
+
horizon = var_data.get("time_horizon", 1)
|
| 140 |
+
var_value = var_data.get("value", 0)
|
| 141 |
+
var_pct = var_data.get("percentage", 0) * 100
|
| 142 |
+
|
| 143 |
+
output.append(f"**Value at Risk (VaR) - {confidence:.0f}% Confidence, {horizon}-Day Horizon**\n")
|
| 144 |
+
output.append(f"- Maximum Expected Loss: ${abs(var_value):,.2f} ({var_pct:.2f}% of portfolio)\n")
|
| 145 |
+
output.append(f"- *Interpretation: With {confidence:.0f}% confidence, maximum expected loss over {horizon} day is ${abs(var_value):,.2f}*\n")
|
| 146 |
+
elif isinstance(var_data, (int, float)):
|
| 147 |
+
output.append(f"**Value at Risk (VaR)**: ${abs(var_data):,.2f}\n")
|
| 148 |
+
|
| 149 |
+
# CVaR (Conditional Value at Risk)
|
| 150 |
+
if "cvar" in risk_data:
|
| 151 |
+
cvar_data = risk_data["cvar"]
|
| 152 |
+
if isinstance(cvar_data, dict):
|
| 153 |
+
confidence = cvar_data.get("confidence_level", 0.95) * 100
|
| 154 |
+
cvar_value = cvar_data.get("value", 0)
|
| 155 |
+
cvar_pct = cvar_data.get("percentage", 0) * 100
|
| 156 |
+
|
| 157 |
+
output.append(f"\n**Conditional Value at Risk (CVaR) - {confidence:.0f}% Confidence**\n")
|
| 158 |
+
output.append(f"- Expected Shortfall: ${abs(cvar_value):,.2f} ({cvar_pct:.2f}% of portfolio)\n")
|
| 159 |
+
output.append(f"- *Interpretation: If losses exceed VaR threshold, average expected loss is ${abs(cvar_value):,.2f}*\n")
|
| 160 |
+
elif isinstance(cvar_data, (int, float)):
|
| 161 |
+
output.append(f"**Conditional Value at Risk (CVaR)**: ${abs(cvar_data):,.2f}\n")
|
| 162 |
+
|
| 163 |
+
# Risk-Adjusted Performance Metrics
|
| 164 |
+
has_performance_metrics = any(k in risk_data for k in ["volatility", "sharpe_ratio", "sortino_ratio", "max_drawdown"])
|
| 165 |
+
if has_performance_metrics:
|
| 166 |
+
output.append("\n**Risk-Adjusted Performance**\n")
|
| 167 |
+
|
| 168 |
+
# Volatility
|
| 169 |
+
if "volatility" in risk_data:
|
| 170 |
+
vol = risk_data["volatility"]
|
| 171 |
+
if isinstance(vol, dict):
|
| 172 |
+
annual_vol = vol.get("annual", 0) * 100
|
| 173 |
+
output.append(f"- Annual Volatility: {annual_vol:.2f}%\n")
|
| 174 |
+
elif isinstance(vol, (int, float)):
|
| 175 |
+
output.append(f"- Annual Volatility: {vol * 100:.2f}%\n")
|
| 176 |
+
|
| 177 |
+
# Sharpe Ratio
|
| 178 |
+
if "sharpe_ratio" in risk_data:
|
| 179 |
+
sharpe = risk_data["sharpe_ratio"]
|
| 180 |
+
if isinstance(sharpe, (int, float)):
|
| 181 |
+
stars = get_rating_stars(sharpe, metric_type="sharpe")
|
| 182 |
+
output.append(f"- Sharpe Ratio: {sharpe:.3f} {stars}\n")
|
| 183 |
+
|
| 184 |
+
# Sortino Ratio
|
| 185 |
+
if "sortino_ratio" in risk_data:
|
| 186 |
+
sortino = risk_data["sortino_ratio"]
|
| 187 |
+
if isinstance(sortino, (int, float)):
|
| 188 |
+
stars = get_rating_stars(sortino, metric_type="sortino")
|
| 189 |
+
output.append(f"- Sortino Ratio: {sortino:.3f} {stars}\n")
|
| 190 |
+
|
| 191 |
+
# Maximum Drawdown
|
| 192 |
+
if "max_drawdown" in risk_data:
|
| 193 |
+
mdd = risk_data["max_drawdown"]
|
| 194 |
+
if isinstance(mdd, (int, float)):
|
| 195 |
+
output.append(f"- Maximum Drawdown: {mdd * 100:.2f}%\n")
|
| 196 |
+
|
| 197 |
+
# Additional risk metrics (if any)
|
| 198 |
+
other_keys = set(risk_data.keys()) - {"var", "cvar", "volatility", "sharpe_ratio", "sortino_ratio", "max_drawdown"}
|
| 199 |
+
if other_keys:
|
| 200 |
+
output.append("\n**Additional Risk Metrics**\n")
|
| 201 |
+
for key in sorted(other_keys):
|
| 202 |
+
value = risk_data[key]
|
| 203 |
+
if isinstance(value, (int, float)):
|
| 204 |
+
output.append(f"- {key.replace('_', ' ').title()}: {value:.2f}\n")
|
| 205 |
+
elif isinstance(value, str):
|
| 206 |
+
output.append(f"- {key.replace('_', ' ').title()}: {value}\n")
|
| 207 |
+
|
| 208 |
+
return ''.join(output)
|
| 209 |
+
|
| 210 |
+
|
| 211 |
+
def get_rating_stars(value: float, metric_type: str = "sharpe") -> str:
|
| 212 |
+
"""Get star rating for Sharpe or Sortino ratio.
|
| 213 |
+
|
| 214 |
+
Args:
|
| 215 |
+
value: Ratio value
|
| 216 |
+
metric_type: Type of metric ('sharpe' or 'sortino')
|
| 217 |
+
|
| 218 |
+
Returns:
|
| 219 |
+
Star rating string
|
| 220 |
+
"""
|
| 221 |
+
if metric_type == "sharpe":
|
| 222 |
+
# Sharpe ratio thresholds
|
| 223 |
+
if value >= 3.0:
|
| 224 |
+
return "βββββ"
|
| 225 |
+
elif value >= 2.0:
|
| 226 |
+
return "ββββ"
|
| 227 |
+
elif value >= 1.0:
|
| 228 |
+
return "βββ"
|
| 229 |
+
elif value >= 0.5:
|
| 230 |
+
return "ββ"
|
| 231 |
+
elif value > 0:
|
| 232 |
+
return "β"
|
| 233 |
+
else:
|
| 234 |
+
return ""
|
| 235 |
+
elif metric_type == "sortino":
|
| 236 |
+
# Sortino ratio thresholds (typically higher than Sharpe)
|
| 237 |
+
if value >= 3.0:
|
| 238 |
+
return "βββββ"
|
| 239 |
+
elif value >= 2.0:
|
| 240 |
+
return "ββββ"
|
| 241 |
+
elif value >= 1.0:
|
| 242 |
+
return "βββ"
|
| 243 |
+
elif value >= 0.5:
|
| 244 |
+
return "ββ"
|
| 245 |
+
elif value > 0:
|
| 246 |
+
return "β"
|
| 247 |
+
else:
|
| 248 |
+
return ""
|
| 249 |
+
return ""
|
| 250 |
+
|
| 251 |
+
|
| 252 |
def format_analysis_results(state: AgentState, holdings: List[Dict]) -> str:
|
| 253 |
"""Format workflow results for Gradio display.
|
| 254 |
|
|
|
|
| 293 |
output.append("\n## β οΈ Risk Metrics\n")
|
| 294 |
risk = state['risk_analysis']
|
| 295 |
if isinstance(risk, dict):
|
| 296 |
+
# Calculate total portfolio value for percentage context
|
| 297 |
+
portfolio_value = sum(h.get('market_value', 0) for h in holdings)
|
| 298 |
+
formatted_risk = format_risk_metrics(risk, portfolio_value)
|
| 299 |
+
if formatted_risk:
|
| 300 |
+
output.append(formatted_risk)
|
| 301 |
+
else:
|
| 302 |
+
# Fallback to generic display if formatting returns nothing
|
| 303 |
+
for key, value in risk.items():
|
| 304 |
+
if isinstance(value, (int, float)):
|
| 305 |
+
output.append(f"- **{key}**: {value:.2f}\n")
|
| 306 |
+
else:
|
| 307 |
+
output.append(f"- **{key}**: {value}\n")
|
| 308 |
|
| 309 |
# Optimization Results
|
| 310 |
if state.get('optimisation_results'):
|
|
@@ -12,7 +12,7 @@ from datetime import datetime, timezone
|
|
| 12 |
from decimal import Decimal
|
| 13 |
|
| 14 |
from langgraph.graph import StateGraph, END
|
| 15 |
-
from backend.models.agent_state import AgentState
|
| 16 |
from backend.agents.portfolio_analyst import PortfolioAnalystAgent
|
| 17 |
|
| 18 |
logger = logging.getLogger(__name__)
|
|
@@ -201,10 +201,11 @@ class PortfolioAnalysisWorkflow:
|
|
| 201 |
|
| 202 |
# Log MCP calls
|
| 203 |
state["mcp_calls"].extend([
|
| 204 |
-
{"mcp": "yahoo_finance", "tool": "get_quote"
|
| 205 |
-
{"mcp": "
|
| 206 |
-
{"mcp": "
|
| 207 |
-
{"mcp": "
|
|
|
|
| 208 |
])
|
| 209 |
|
| 210 |
logger.info(f"PHASE 1 COMPLETE: Fetched data for {len(tickers)} assets")
|
|
@@ -302,10 +303,10 @@ class PortfolioAnalysisWorkflow:
|
|
| 302 |
|
| 303 |
# Log MCP calls
|
| 304 |
state["mcp_calls"].extend([
|
| 305 |
-
{"mcp": "
|
| 306 |
-
{"mcp": "
|
| 307 |
-
{"mcp": "
|
| 308 |
-
{"mcp": "
|
| 309 |
])
|
| 310 |
|
| 311 |
logger.info("PHASE 2 COMPLETE: Optimizations and risk analysis done")
|
|
|
|
| 12 |
from decimal import Decimal
|
| 13 |
|
| 14 |
from langgraph.graph import StateGraph, END
|
| 15 |
+
from backend.models.agent_state import AgentState, MCPCall
|
| 16 |
from backend.agents.portfolio_analyst import PortfolioAnalystAgent
|
| 17 |
|
| 18 |
logger = logging.getLogger(__name__)
|
|
|
|
| 201 |
|
| 202 |
# Log MCP calls
|
| 203 |
state["mcp_calls"].extend([
|
| 204 |
+
MCPCall.model_validate({"mcp": "yahoo_finance", "tool": "get_quote"}).model_dump(),
|
| 205 |
+
MCPCall.model_validate({"mcp": "yahoo_finance", "tool": "get_historical_data"}).model_dump(),
|
| 206 |
+
MCPCall.model_validate({"mcp": "fmp", "tool": "get_company_profile"}).model_dump(),
|
| 207 |
+
MCPCall.model_validate({"mcp": "trading_mcp", "tool": "get_technical_indicators"}).model_dump(),
|
| 208 |
+
MCPCall.model_validate({"mcp": "fred", "tool": "get_economic_series"}).model_dump(),
|
| 209 |
])
|
| 210 |
|
| 211 |
logger.info(f"PHASE 1 COMPLETE: Fetched data for {len(tickers)} assets")
|
|
|
|
| 303 |
|
| 304 |
# Log MCP calls
|
| 305 |
state["mcp_calls"].extend([
|
| 306 |
+
MCPCall.model_validate({"mcp": "portfolio_optimizer_mcp", "tool": "optimize_hrp"}).model_dump(),
|
| 307 |
+
MCPCall.model_validate({"mcp": "portfolio_optimizer_mcp", "tool": "optimize_black_litterman"}).model_dump(),
|
| 308 |
+
MCPCall.model_validate({"mcp": "portfolio_optimizer_mcp", "tool": "optimize_mean_variance"}).model_dump(),
|
| 309 |
+
MCPCall.model_validate({"mcp": "risk_analyzer_mcp", "tool": "analyze_risk"}).model_dump(),
|
| 310 |
])
|
| 311 |
|
| 312 |
logger.info("PHASE 2 COMPLETE: Optimizations and risk analysis done")
|
|
@@ -63,10 +63,15 @@ class AgentState(TypedDict):
|
|
| 63 |
|
| 64 |
|
| 65 |
class MCPCall(BaseModel):
|
| 66 |
-
"""Record of an MCP tool call.
|
| 67 |
|
| 68 |
-
mcp_server
|
| 69 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 70 |
parameters: Dict[str, Any] = Field(default_factory=dict)
|
| 71 |
result: Optional[Dict[str, Any]] = None
|
| 72 |
error: Optional[str] = None
|
|
|
|
| 63 |
|
| 64 |
|
| 65 |
class MCPCall(BaseModel):
|
| 66 |
+
"""Record of an MCP tool call.
|
| 67 |
|
| 68 |
+
Accepts both 'mcp_server' and 'mcp' field names for backward compatibility.
|
| 69 |
+
"""
|
| 70 |
+
|
| 71 |
+
model_config = {"populate_by_name": True}
|
| 72 |
+
|
| 73 |
+
mcp_server: str = Field(..., validation_alias="mcp", description="MCP server name")
|
| 74 |
+
tool_name: str = Field(..., validation_alias="tool", description="Tool called")
|
| 75 |
parameters: Dict[str, Any] = Field(default_factory=dict)
|
| 76 |
result: Optional[Dict[str, Any]] = None
|
| 77 |
error: Optional[str] = None
|
|
@@ -0,0 +1,119 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""Test script for MCP tracking and risk metrics formatting fixes."""
|
| 2 |
+
|
| 3 |
+
from backend.models.agent_state import MCPCall
|
| 4 |
+
from app import format_risk_metrics, get_rating_stars
|
| 5 |
+
|
| 6 |
+
|
| 7 |
+
def test_mcp_call_validation_alias():
|
| 8 |
+
"""Test that MCPCall accepts both 'mcp' and 'mcp_server' field names."""
|
| 9 |
+
print("Testing MCPCall validation_alias fix...\n")
|
| 10 |
+
|
| 11 |
+
# Test 1: Create MCPCall with 'mcp' key (old format)
|
| 12 |
+
call1 = MCPCall.model_validate({"mcp": "yahoo_finance", "tool": "get_quote"})
|
| 13 |
+
print(f"β Created MCPCall with 'mcp' key: {call1.mcp_server}")
|
| 14 |
+
|
| 15 |
+
# Test 2: Create MCPCall with 'mcp_server' key (new format)
|
| 16 |
+
call2 = MCPCall.model_validate({"mcp_server": "fmp", "tool_name": "get_company_profile"})
|
| 17 |
+
print(f"β Created MCPCall with 'mcp_server' key: {call2.mcp_server}")
|
| 18 |
+
|
| 19 |
+
# Test 3: Verify model_dump() uses 'mcp_server' field name
|
| 20 |
+
dumped = call1.model_dump()
|
| 21 |
+
assert "mcp_server" in dumped, "model_dump() should contain 'mcp_server' key"
|
| 22 |
+
assert dumped["mcp_server"] == "yahoo_finance"
|
| 23 |
+
print(f"β model_dump() contains 'mcp_server': {dumped['mcp_server']}")
|
| 24 |
+
|
| 25 |
+
# Test 4: Verify tool_name alias works
|
| 26 |
+
dumped2 = call1.model_dump()
|
| 27 |
+
assert "tool_name" in dumped2
|
| 28 |
+
assert dumped2["tool_name"] == "get_quote"
|
| 29 |
+
print(f"β model_dump() contains 'tool_name': {dumped2['tool_name']}")
|
| 30 |
+
|
| 31 |
+
print("\nβ
All MCPCall validation tests passed!\n")
|
| 32 |
+
|
| 33 |
+
|
| 34 |
+
def test_risk_metrics_formatting():
|
| 35 |
+
"""Test that format_risk_metrics properly formats nested risk data."""
|
| 36 |
+
print("Testing risk metrics formatting fix...\n")
|
| 37 |
+
|
| 38 |
+
# Sample risk data with nested structures
|
| 39 |
+
risk_data = {
|
| 40 |
+
"var": {
|
| 41 |
+
"confidence_level": 0.95,
|
| 42 |
+
"time_horizon": 1,
|
| 43 |
+
"value": -1119.58,
|
| 44 |
+
"percentage": 0.039
|
| 45 |
+
},
|
| 46 |
+
"cvar": {
|
| 47 |
+
"confidence_level": 0.95,
|
| 48 |
+
"value": -1409.97,
|
| 49 |
+
"percentage": 0.0491
|
| 50 |
+
},
|
| 51 |
+
"volatility": 0.393,
|
| 52 |
+
"sharpe_ratio": 0.839,
|
| 53 |
+
"sortino_ratio": 1.256,
|
| 54 |
+
"max_drawdown": -0.391
|
| 55 |
+
}
|
| 56 |
+
|
| 57 |
+
portfolio_value = 28700.00
|
| 58 |
+
|
| 59 |
+
# Format the metrics
|
| 60 |
+
formatted = format_risk_metrics(risk_data, portfolio_value)
|
| 61 |
+
|
| 62 |
+
# Verify key components are present
|
| 63 |
+
assert "Value at Risk (VaR)" in formatted, "VaR section missing"
|
| 64 |
+
assert "95% Confidence" in formatted, "Confidence level missing"
|
| 65 |
+
assert "$1,119.58" in formatted, "VaR value not formatted correctly"
|
| 66 |
+
assert "3.90%" in formatted, "VaR percentage not formatted correctly"
|
| 67 |
+
|
| 68 |
+
assert "Conditional Value at Risk (CVaR)" in formatted, "CVaR section missing"
|
| 69 |
+
assert "$1,409.97" in formatted, "CVaR value not formatted correctly"
|
| 70 |
+
assert "4.91%" in formatted, "CVaR percentage not formatted correctly"
|
| 71 |
+
|
| 72 |
+
assert "Risk-Adjusted Performance" in formatted, "Performance section missing"
|
| 73 |
+
assert "Annual Volatility: 39.30%" in formatted, "Volatility not formatted correctly"
|
| 74 |
+
assert "Sharpe Ratio: 0.839" in formatted, "Sharpe ratio missing"
|
| 75 |
+
assert "Sortino Ratio: 1.256" in formatted, "Sortino ratio missing"
|
| 76 |
+
assert "Maximum Drawdown: -39.10%" in formatted, "Max drawdown not formatted correctly"
|
| 77 |
+
|
| 78 |
+
# Verify star ratings
|
| 79 |
+
assert "βββ" in formatted, "Star ratings missing"
|
| 80 |
+
|
| 81 |
+
print("Formatted output:\n")
|
| 82 |
+
print(formatted)
|
| 83 |
+
print("\nβ
All risk metrics formatting tests passed!\n")
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
def test_star_ratings():
|
| 87 |
+
"""Test star rating function."""
|
| 88 |
+
print("Testing star rating system...\n")
|
| 89 |
+
|
| 90 |
+
# Sharpe ratio ratings
|
| 91 |
+
assert get_rating_stars(3.5, "sharpe") == "βββββ"
|
| 92 |
+
assert get_rating_stars(2.5, "sharpe") == "ββββ"
|
| 93 |
+
assert get_rating_stars(1.5, "sharpe") == "βββ"
|
| 94 |
+
assert get_rating_stars(0.839, "sharpe") == "ββ"
|
| 95 |
+
assert get_rating_stars(0.3, "sharpe") == "β"
|
| 96 |
+
print("β Sharpe ratio ratings work correctly")
|
| 97 |
+
|
| 98 |
+
# Sortino ratio ratings
|
| 99 |
+
assert get_rating_stars(3.5, "sortino") == "βββββ"
|
| 100 |
+
assert get_rating_stars(2.5, "sortino") == "ββββ"
|
| 101 |
+
assert get_rating_stars(1.256, "sortino") == "βββ"
|
| 102 |
+
print("β Sortino ratio ratings work correctly")
|
| 103 |
+
|
| 104 |
+
print("\nβ
All star rating tests passed!\n")
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
if __name__ == "__main__":
|
| 108 |
+
print("=" * 60)
|
| 109 |
+
print("Running Fix Verification Tests")
|
| 110 |
+
print("=" * 60)
|
| 111 |
+
print()
|
| 112 |
+
|
| 113 |
+
test_mcp_call_validation_alias()
|
| 114 |
+
test_risk_metrics_formatting()
|
| 115 |
+
test_star_ratings()
|
| 116 |
+
|
| 117 |
+
print("=" * 60)
|
| 118 |
+
print("All Tests Passed! β
")
|
| 119 |
+
print("=" * 60)
|