Reviewer: Cassandra Madison, Executive Director, Public Benefit Innovation Fund
Date: August 8, 2025
Application ID: PolicyEngine-Policy-Library-PBIF-2025
This application represents an exemplary fit for PBIF funding - addressing systemic barriers to benefit access through innovative AI-powered infrastructure while maintaining strong commitments to responsible development and community impact. The proposal demonstrates deep understanding of the problem space, realistic technical approach, and clear pathways to sustainable impact at scale. Recommend for funding with minor implementation enhancements.
| Dimension | Score | Rationale |
|---|---|---|
| Impact | 9/10 | Addresses systemic barriers affecting millions with measurable outcomes |
| Technical Feasibility | 9/10 | Proven team, realistic timeline, operational pilots demonstrate viability |
| Responsible AI | 8/10 | Strong framework with human oversight, needs community input enhancement |
| Strategic Fit | 9.5/10 | Perfect alignment with PBIF mission and innovation focus |
| Scalability | 9/10 | Exceptional potential for national and international replication |
Overall Score: 8.9/10
"Reducing administrative burden and improving access to public benefits"
- Directly addresses 18% document disappearance rate that creates barriers to benefit access
- Eliminates thousands of hours of administrative work maintaining broken links
- Provides permanent, reliable access to authoritative benefit documents
- Serves 160,000 people annually through partner organizations
"Supporting technology that increases efficiency and improves outcomes"
- AI-powered document extraction and preservation
- API infrastructure reduces duplicative document collection efforts
- 24pp improvement in LLM accuracy for benefit calculations
- Scales benefits information access without linear cost increases
"Fostering innovation in benefit delivery systems"
- Creates new category of policy document infrastructure
- Enables next generation of AI-powered benefit tools
- Establishes replicable model for government document preservation
- Catalyzes ecosystem-wide improvements in benefit access
This application exemplifies the type of innovative, technology-forward solutions PBIF was created to support:
- AI Integration: Sophisticated use of Claude/GPT-4 with appropriate human oversight
- Systems Thinking: Addresses infrastructure failure rather than symptoms
- Scalable Architecture: Technical approach supports exponential growth
- Open Source: Commitment to shared learning and replication
Quantitative Evidence:
- 18% URL death rate for 2019 benefit documents
- CaseText shutdown affecting thousands of legal references
- 160,000 people currently served by partner tools affected by broken links
Qualitative Evidence:
- Detailed understanding of impact on families, organizations, and AI systems
- Clear articulation of root causes (lack of preservation infrastructure)
- Strong partner validation of problem urgency
Research Foundation:
- Georgetown University and University of Michigan collaboration
- Atlanta Federal Reserve partnership validation
- Operational pilots demonstrating feasibility
The proposed solution directly addresses identified root causes:
- Document Disappearance → Permanent archival with version control
- Link Rot → Stable API with permanent source identifiers
- Fragmented Collections → Centralized, comprehensive coverage
- AI Inaccuracy → Authoritative source documents for training/validation
Responsible AI Framework:
- Human-in-the-loop validation of all AI-extracted documents
- Public documents only (no PII concerns)
- Complete transparency through open source development
- Audit trail with version history
Technical Sophistication:
- State-of-the-art LLMs (Claude/GPT-4) for intelligent extraction
- Proven architecture components (Git+LFS, OpenSearch, Browsertrix)
- RESTful API design with permanent identifiers
- Multiple output formats serving diverse use cases
Innovation Potential:
- Establishes new standard for government document preservation
- Creates foundation for next-generation AI policy tools
- Demonstrates model for responsible AI in government applications
Technical Scalability:
- Git-based architecture naturally distributes across jurisdictions
- API infrastructure supports exponential usage growth
- Document processing pipeline designed for parallel operation
- Open source enables community contributions and improvements
Operational Scalability:
- Proven team with experience managing complex technical projects
- Clear milestone progression from pilots to full coverage
- Partnership network already established for scaling
- Revenue model supports sustainable operations post-grant
Max Ghenis - CEO
- Proven track record: Founded PolicyEngine, former Google data scientist
- Domain expertise: MS Stanford, deep understanding of policy modeling
- Leadership experience: Successfully scaling technical organizations
Nikhil Woodruff - CTO
- Technical expertise: Lead engineer of PolicyEngine microsimulation models
- Implementation experience: Operational systems serving millions of users
- AI/ML background: Relevant experience for document processing challenges
Pavel Makarchuk - ML Engineer
- Specialized expertise: AI/ML development and implementation
- Startup experience: Understanding of rapid scaling and iteration
- Technical depth: Critical for sophisticated document extraction pipeline
Operational Pilots:
- us-nc-sources repository demonstrating git-based approach
- Atlanta Fed Policy Rules Database integration
- MyFriendBen production usage validates partner value proposition
Technical Credibility:
- PolicyEngine microsimulation models serve millions of users
- Existing API infrastructure demonstrates scaling experience
- Open source commitment with public development history
Year 1: $498,000 (within PBIF range of $500K-$2M)
| Category | Amount | Percentage | Assessment |
|---|---|---|---|
| Personnel | $405,000 | 81% | Appropriate for infrastructure development |
| Partner Grants | $60,000 | 12% | Good investment in ecosystem building |
| Infrastructure | $18,000 | 4% | May be conservative but reasonable for Year 1 |
| Contingency | $15,000 | 3% | Conservative risk management |
Cost per Person Served: $3.11 per person per year (498K ÷ 160K people) Infrastructure Value: Permanent public good serving future generations Ecosystem Impact: Enables countless downstream innovations impossible to quantify Leverage: $498K catalyzes transformation across entire benefits ecosystem
Post-Grant Revenue Model:
- API subscriptions for high-volume users ($500-5000/month)
- Enterprise contracts for custom integrations
- Government service agreements
- Continued foundation support for public good aspects
Break-Even Timeline:
- Month 18: Basic sustainability through API revenue
- Month 24: Full self-sufficiency with growth capacity
- Month 36: Surplus generation for expansion
Quantitative Metrics:
- Documents archived: 50,000 (Year 1) → 100,000 (Year 2)
- API usage: 100,000 calls/month (Year 1) → 1M calls/month (Year 2)
- People served: 80,000 (Year 1) → 160,000 (Year 2)
- Partner organizations: 15 (Year 1) → 30 (Year 2)
Impact Metrics:
- Time saved: 5,000 hours/year (Year 1) → 15,000 hours/year (Year 2)
- LLM accuracy improvement: 15pp (Year 1) → 24pp (Year 2)
- Document availability: 100% (both years)
Evaluation Methods:
- Automated metrics dashboard (quantitative tracking)
- Quarterly partner interviews (qualitative assessment)
- External evaluation by university partner (Year 2)
- Monthly reviews with adjustment capability
Monthly: Document retrieval rates, API usage, system performance Quarterly: Partner organization surveys on time saved and value delivered Annually: Beneficiary reach assessment, LLM accuracy benchmarking Continuous: Document availability monitoring, system health metrics
- Crawling Access: Some agencies may block automated crawling
- Mitigation: Respectful crawling practices, direct partnerships where possible
- Scale Performance: System performance degradation at scale
- Mitigation: Load testing, optimization, caching strategies
- Storage Costs: Document volume may exceed projections
- Mitigation: Tiered storage, cost monitoring, cloud optimization
- Team Scaling: May need additional personnel for full coverage
- Mitigation: Realistic hiring plan, contractor augmentation as needed
- Government Relations: Potential resistance to crawling activities
- Mitigation: Proactive outreach, emphasis on public benefit
- Competition: Commercial providers may develop competing solutions
- Mitigation: Open source approach, first-mover advantage, partner lock-in
- Adoption: Partners may be slow to integrate API
- Mitigation: Strong partner validation, integration support
- Sustainability: Revenue model may not develop as projected
- Mitigation: Conservative projections, multiple revenue streams
- Policy Changes: Government policies may restrict access
- Mitigation: Public documents focus, legal compliance emphasis
-
Community Engagement Enhancement
- Establish community advisory board including benefit recipients
- Add explicit community organization capacity building component
- Accelerate multilingual timeline or provide interim solutions
-
Government Relations Strategy
- Proactive outreach to agency IT and records management teams
- Develop formal partnership agreements where possible
- Create government relations budget line item
-
Technical Monitoring Enhancement
- Add detailed cost monitoring and alerting systems
- Implement comprehensive performance benchmarking framework
- Plan third-party security assessment before production
-
Research Integration
- Formal partnerships with policy research institutions
- Academic research initiatives using archived document collections
- Policy innovation analysis across jurisdictions
-
International Replication Planning
- Document replication methodology for other countries
- International partnership development
- Global standards development for policy document preservation
This application represents exactly the type of project PBIF was created to support:
- Systemic Impact: Addresses infrastructure failure affecting entire benefits ecosystem
- Innovation: Sophisticated AI integration with responsible development practices
- Scalability: Technical and operational architecture supports exponential growth
- Sustainability: Clear pathway to self-sufficiency with ongoing public benefit
- Catalytic Effect: Enables countless downstream innovations and improvements
The Policy Library has potential to transform:
- Benefit Access: Reliable document availability for millions of Americans
- Organizational Efficiency: Thousands of hours saved across hundreds of organizations
- AI Accuracy: Foundation for next generation of reliable AI policy tools
- Research Capacity: Unprecedented access to historical policy documents
- Global Standards: Replicable model for government document preservation worldwide
Strong confidence in successful execution based on:
- Proven Team: Track record of scaling technical infrastructure
- Operational Pilots: Demonstrated feasibility with real-world usage
- Partner Validation: Strong demand signals from existing partner network
- Technical Architecture: Sound approach using proven components
- Risk Management: Comprehensive identification and mitigation strategies
Funding Level: $498,000 (as requested) Funding Confidence: High (95%+) Strategic Priority: Tier 1 (highest funding priority)
Rationale: This application exemplifies PBIF's mission to reduce administrative burden and improve benefit access through innovative technology. The combination of urgent need, innovative solution, proven team, and transformational potential makes this a flagship investment opportunity.
Conditions for Funding:
- Monthly progress reviews during first 6 months
- Community advisory board establishment within 90 days
- Government relations strategy implementation within 120 days
- Mid-grant evaluation by external assessor
Expected Outcomes:
- 160,000 people with improved benefit access by Year 2
- 15,000+ hours saved annually across partner ecosystem
- 24pp improvement in AI-powered benefit calculations
- Replicable model for policy document infrastructure globally
This investment will create lasting infrastructure that serves the public good while demonstrating PBIF's commitment to catalytic, technology-forward solutions for systemic challenges in benefit delivery.
Total Recommendation Score: 8.9/10 - FUND (TIER 1 PRIORITY)